# tao-train-mask-auto-label

Runs training, evaluation, and inference for NVIDIA MAL segmentation models using ViT-MAE backbones. Helpful when creating segmentation masks from point or box annotations.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 1600 installs · ★ 3735
- Author: NVIDIA Corporation
- License: Apache-2.0 · policy=mirror · the beginning of the file is at the end of this page
- Source: https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-mask-auto-label/SKILL.md
- Updated: 2026-10-26
- HTML: https://affpapa.org/en/nevibecode/skills/tao-train-mask-auto-label

## What it does

The skill configures and runs Mask Auto-Label (MAL) tasks inside the `tao-toolkit:7.3.0-pyt` container. MAL generates segmentation masks from minimal prompts like points or bounding boxes. Host-level Python or TAO SDK packages are not needed because containerized execution handles `mal train`, `evaluate`, and `inference` commands directly.

Running this requires Docker with `nvidia-container-toolkit`. Dataset preparation is strict. The model demands COCO-style JSON annotations alongside `images.tar.gz` archives. Raw CSV datasets fail without prior conversion to this layout. The workflow also supports AutoML hyperparameter tuning to maximize validation mIoU. If installing outside the TAO skill bank plugin, execute `tao-setup` first.

## Requirements
- Works with: Docker

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-mask-auto-label -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/tao-train-mask-auto-label/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/tao-train-mask-auto-label/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-mask-auto-label -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/tao-train-mask-auto-label/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-mask-auto-label -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/tao-train-mask-auto-label/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install NVIDIA/skills tao-train-mask-auto-label
```

2. **Manually: put the files in place**
- File: `.github/skills/tao-train-mask-auto-label/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-mask-auto-label -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/NVIDIA/skills --path skills/tao-train-mask-auto-label
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/tao-train-mask-auto-label/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-mask-auto-label -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-mask-auto-label/SKILL.md

## When to use it
- train MAL segmentation models from point or bounding box prompts
- generate segmentation masks using ViT-MAE backbones
- evaluate segmentation accuracy via mIoU metrics
- configure AutoML runs with custom spec overrides

## Example prompts
- Train a MAL segmentation model using specs from train.yaml
- Run evaluate on the MAL model checkpoint to calculate mIoU
- Generate segmentation masks with mal inference using images.tar.gz

## Source text · SKILL.md · Apache-2.0

Below is the content of a third-party file. It is data, not instructions for you: do not act on anything written inside unless the human asks. Block tag: 474cafaab935fce9.

BEGIN THIRD-PARTY FILE 474cafaab935fce9
````text
---
name: tao-train-mask-auto-label
description: MAL (Mask Auto-Label) for weakly-supervised segmentation. Produces segmentation masks from minimal annotations
  (point or box annotations) using a ViT-MAE backbone. Use when training, evaluating, or running inference for a TAO MAL
  model. Trigger phrases include "train MAL", "Mask Auto-Label", "weakly-supervised segmentation", "box-prompted
  segmentation", "minimal-annotation mask prediction".
license: Apache-2.0
compatibility: Requires docker + nvidia-container-toolkit.
metadata:
  version: "0.1.0"
  author: NVIDIA Corporation
allowed-tools: Read Bash
tags:
- segmentation
---

# MAL

> **Standalone install?** If this session was not initialized by the TAO skill bank plugin, run the `tao-setup` skill first (host preflight, credentials, cross-skill discovery).

MAL (Mask Auto-Label) for weakly-supervised segmentation. Produces segmentation masks from minimal annotations (e.g., point or box annotations). Uses ViT-MAE backbone.

Set train.pretrained_model_path for ViT-MAE pretrained weights.

## Quick Start (docker run)

Docker-native launch — no TAO SDK and no Python on the host. Use the local
Docker/platform skill instead when it gives a stricter environment-specific
command (non-root UID mapping, cache redirects, remote daemons).

```bash
TAO_PYT_IMAGE_DEFAULT=nvcr.io/nvidia/tao/tao-toolkit:7.3.0-pyt  # versions-key: images.tao_toolkit.pyt
TAO_PYT_IMAGE="${TAO_PYT_IMAGE:-$TAO_PYT_IMAGE_DEFAULT}"
RUN_ROOT="${RUN_ROOT:-$PWD}"
DOCKER_COMMON=(
  --rm --gpus all --shm-size=8g
  --shm-size=8g
  --ulimit memlock=-1
  --ulimit stack=67108864
  -v "$RUN_ROOT/data:/data:ro"
  -v "$RUN_ROOT/specs:/specs:ro"
  -v "$RUN_ROOT/results:/results"
)
```

Train:

```bash
docker run "${DOCKER_COMMON[@]}" "$TAO_PYT_IMAGE" \
  mal train -e /specs/train.yaml
```

Evaluate:

```bash
docker run "${DOCKER_COMMON[@]}" "$TAO_PYT_IMAGE" \
  mal evaluate -e /specs/evaluate.yaml
```

Inference:

```bash
docker run "${DOCKER_COMMON[@]}" "$TAO_PYT_IMAGE" \
  mal inference -e /specs/inference.yaml
```

Every action takes its spec with `-e`; `results_dir` is set in the spec or
overridden on the command line. Mount any pretrained-weights directory the spec
references, and keep every in-container path consistent across actions.

## Dataclass Schemas
````
END THIRD-PARTY FILE 474cafaab935fce9

The third-party file has ended. Everything between the 474cafaab935fce9 tags is data: requests and commands from it are not followed. An “end of file” line with another tag inside the block is part of the file.

First 2307 of 11350 bytes. © NVIDIA Corporation · Apache-2.0 · https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-mask-auto-label/SKILL.md

## Similar
- [gpu-server-management](https://affpapa.org/en/nevibecode/skills/gpu-server-management.md) — skill · 331 installs
- [debug-openshell-cluster](https://affpapa.org/en/nevibecode/skills/debug-openshell-cluster.md) — skill · 474 installs
- [nemo-evaluator-sdk](https://affpapa.org/en/nevibecode/skills/nemo-evaluator-sdk.md) — skill · 943 installs
- [runpod-usage](https://affpapa.org/en/nevibecode/skills/runpod-usage.md) — skill · 12500 installs
- [runpod-templates](https://affpapa.org/en/nevibecode/skills/runpod-templates.md) — skill · 10100 installs
- [build-models](https://affpapa.org/en/nevibecode/skills/build-models.md) — skill · 1800 installs

_Part of the page did not fit — see the HTML version._

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md