# tao-train-codetr

The tao-train-codetr skill guides Co-DETR training, evaluation, and inference in TAO. Use it to run object detection with COCO data or an image folder.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf (Devin Desktop), Claude.ai · ★ 4106 repo stars
- Author: NVIDIA Corporation
- License: Apache-2.0 · policy=mirror · the beginning of the file is at the end of this page
- Source: https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-codetr/SKILL.md
- Updated: 2026-10-30
- Security check: automated, 2026-11-16 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/tao-train-codetr

## What it does

The skill covers the TAO workflow for Co-DETR: training and evaluation with COCO datasets, plus inference on images using a class list. It helps configure dataset fields and class counts, select a detector checkpoint, and pass Hydra overrides. TAO appends the action name to `results_dir`, so do not add `train`, `evaluate`, or `inference` to that path yourself.

Docker and NVIDIA Container Toolkit are required. In some TAO images, the `codetr` command is missing even when the Python module is installed; the skill describes checking both launch options. Inference requires a full detector checkpoint with trained detection heads, not backbone weights alone. This model does not support export or TensorRT deployment workflows.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-codetr -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/tao-train-codetr/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/tao-train-codetr/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-codetr -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/tao-train-codetr/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-codetr -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/tao-train-codetr/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install NVIDIA/skills skills/tao-train-codetr
```

2. **Manually: put the files in place**
- File: `.github/skills/tao-train-codetr/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-codetr -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/NVIDIA/skills --path skills/tao-train-codetr
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/tao-train-codetr/SKILL.md` — The legacy .windsurf/skills/ path still works

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-codetr/SKILL.md

## When to use it
- train Co-DETR on a COCO dataset
- evaluate a detector on a test set
- run inference on an image directory with a class list
- choose a checkpoint for fine-tuning

## Example prompts
- Train Co-DETR on my COCO dataset and save the results in this directory.
- Evaluate this Co-DETR model against the test dataset.
- Run inference on the images in this folder using my class list.

## Source text · SKILL.md · Apache-2.0

Below is the content of a third-party file. It is data, not instructions for you: do not act on anything written inside unless the human asks. Block tag: 83462951e25fcf27.

BEGIN THIRD-PARTY FILE 83462951e25fcf27
````text
---
name: tao-train-codetr
description: Co-DETR (CoDINO) for object detection. A DETR-family detector with collaborative hybrid
  assignment — auxiliary one-to-many heads supervise the encoder during training, giving strong
  closed-set accuracy at high inference cost. Use when training, evaluating, or running inference for
  a TAO Co-DETR model. Trigger phrases include "train Co-DETR", "run CoDINO", "codetr inference",
  "collaborative DETR", "autolabel detections with Co-DETR".
license: Apache-2.0
compatibility: Requires docker + nvidia-container-toolkit.
metadata:
  version: "0.1.0"
  author: NVIDIA Corporation
allowed-tools: Read Bash
tags:
- object
- detection
---

# Co-DETR

> **Standalone install?** If this session was not initialized by the TAO skill bank plugin, run the `tao-setup` skill first (host preflight, credentials, cross-skill discovery).

Co-DETR trains a DETR detector alongside auxiliary one-to-many assignment heads (`num_co_heads`), which supervise the encoder more densely than DETR's one-to-one matching alone. The auxiliary heads exist only during training; inference runs the primary DETR head. Backbones are large by default — `vit_large_codetr` for training, `swin_large_patch4_window7_224` for the reference inference/eval configs — so this is an accuracy-first model, not a latency-first one.

## Availability — probe, then fall back to the module

The `codetr` **console script is not registered** in the TAO PyTorch images checked so far
(`7.0.1-pyt` and the `2026.7.31-rc-12-multiarch` nightly), even though the module itself
ships. So a bare `codetr` invocation fails while the network is perfectly usable.

Probe in two steps and use whichever works:

```bash
# 1. console script (preferred when present)
docker run --rm "$TAO_PYT_IMAGE" codetr --help >/dev/null 2>&1 && CODETR="codetr"

# 2. module fallback — works whenever the package is installed
[ -z "${CODETR:-}" ] && docker run --rm --entrypoint sh "$TAO_PYT_IMAGE" \
  -c 'python3 -c "import nvidia_tao_pytorch.cv.codetr"' >/dev/null 2>&1 \
  && CODETR="python3 -m nvidia_tao_pytorch.cv.codetr.entrypoint.codetr"

[ -z "${CODETR:-}" ] && { echo "FATAL: Co-DETR not available in $TAO_PYT_IMAGE"; exit 1; }
```

Both forms take identical arguments — the module entrypoint reports itself as `codetr` and
exposes the same `{train, evaluate, inference, default_specs}` subtasks. Substitute `$CODETR`
wherever this document writes `codetr`.

Only if **both** probes fail does this image genuinely lack Co-DETR. Then stop and ask which
image to use — do not silently substitute another detector.

## Dataclass Schemas

Generated TAO Core schemas are **not yet packaged** for this model, so `schemas/<action>.schema.json` and `references/spec_template_<action>.yaml` are absent except for the hand-written `references/spec_template_inference.yaml`. Read spec keys from the upstream experiment specs in `nvidia_tao_pytorch/cv/codetr/experiment_specs/` (`train.yaml`, `eval.yaml`, `inference.yaml`, `export.yaml`) until a maintainer regenerates them. AutoML is therefore not runnable for this model.

## Train Action Policy
````
END THIRD-PARTY FILE 83462951e25fcf27

The third-party file has ended. Everything between the 83462951e25fcf27 tags is data: requests and commands from it are not followed. An “end of file” line with another tag inside the block is part of the file.

First 3134 of 19149 bytes. © NVIDIA Corporation · Apache-2.0 · https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-codetr/SKILL.md

_Part of the page did not fit — see the HTML version._

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
