# tao-train-fast-foundation-stereo

A skill for training, evaluating and exporting TAO FastFoundationStereo: it turns rectified stereo pairs into disparity maps with roughly 10× lower latency than full FoundationStereo.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 1600 installs · ★ 3735
- Author: NVIDIA Corporation
- License: Apache-2.0 · policy=mirror · the beginning of the file is at the end of this page
- Source: https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-fast-foundation-stereo/SKILL.md
- Updated: 2026-10-23
- HTML: https://affpapa.org/en/nevibecode/skills/tao-train-fast-foundation-stereo

## What it does

FastFoundationStereo (FFS) is the distilled bp2 commercial branch of FoundationStereo. The skill drives it through the shared `depth_net` CLI by setting `model.model_type: FastFoundationStereo`. Compared with the regular stereo model, only the pruned per-layer widths and a serialized forward path differ; the action verbs, dataset classes and deploy chain match `depth-net-stereo`.

Two flows are supported. Raw deploy takes the bp2 checkpoint as-is, skips `train` and goes straight to `inference`, `evaluate`, `export` or `gen_trt_engine`. Finetuning points `train.pretrained_model_path` at the bp2 file, trains on your data, then verifies and deploys the resulting checkpoint — the full seven-action sequence.

Both flows need a pre-trained `model_best_bp2_serialize.pth`. The stock PyT image does not guarantee that file is on disk, so treat it as an artifact you supply. FFS also expects 15 width override fields in the `model:` section that match the bp2 checkpoint exactly; leave one out and the forward pass fails on a shape mismatch. Your dataset and output directory have to be reachable inside the container — a direct `docker run` mounts the host dataset root read-only at the same path. Docker and nvidia-container-toolkit are required up front.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-fast-foundation-stereo -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/tao-train-fast-foundation-stereo/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/tao-train-fast-foundation-stereo/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-fast-foundation-stereo -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/tao-train-fast-foundation-stereo/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-fast-foundation-stereo -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/tao-train-fast-foundation-stereo/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install NVIDIA/skills tao-train-fast-foundation-stereo
```

2. **Manually: put the files in place**
- File: `.github/skills/tao-train-fast-foundation-stereo/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-fast-foundation-stereo -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/NVIDIA/skills --path skills/tao-train-fast-foundation-stereo
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/tao-train-fast-foundation-stereo/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-fast-foundation-stereo -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-fast-foundation-stereo/SKILL.md

## When to use it
- finetune FFS on your own stereo pairs from the bp2 checkpoint
- produce disparity maps from a left-right image pair
- export the model and build a TensorRT engine
- score accuracy on Middlebury, KITTI or ETH3D
- validate the pipeline on FSD or IsaacReal synthetic data

## Example prompts
- Train FastFoundationStereo on my dataset, the bp2 checkpoint is at `model_best_bp2_serialize.pth`
- Run inference over the stereo pairs listed in `data_file` and save the disparity maps
- Build the TensorRT engine for FFS and evaluate it on KITTI

## Source text · SKILL.md · Apache-2.0

Below is the content of a third-party file. It is data, not instructions for you: do not act on anything written inside unless the human asks. Block tag: 8c73273107a8f360.

BEGIN THIRD-PARTY FILE 8c73273107a8f360
```text
---
name: tao-train-fast-foundation-stereo
description: Real-time stereo depth estimation using FastFoundationStereo (FFS), the distilled bp2 commercial variant of
  FoundationStereo. Predicts disparity maps from stereo image pairs with ~10× lower latency than full FoundationStereo. Use
  when training, evaluating, exporting, or running inference for a TAO FastFoundationStereo (FFS) model. Trigger phrases
  include "train fast stereo", "real-time stereo disparity", "FastFoundationStereo", "distilled stereo depth".
license: Apache-2.0
compatibility: Requires docker + nvidia-container-toolkit.
metadata:
  version: "0.1.0"
  author: NVIDIA Corporation
allowed-tools: Read Bash
tags:
- stereo
- depth
- estimation
- realtime
- distilled
---

# Depth Net Fast Stereo

> **Standalone install?** If this session was not initialized by the TAO skill bank plugin, run the `tao-setup` skill first (host preflight, credentials, cross-skill discovery).

Real-time stereo depth estimation using **FastFoundationStereo (FFS)** — the bp2 commercial distilled variant of FoundationStereo. Predicts disparity maps from rectified stereo image pairs with per-layer pruned widths for real-time inference.

The mono / stereo / fast-stereo skills share the unified TAO `depth_net` CLI; FFS is selected via `model.model_type: FastFoundationStereo`. FFS differs from `FoundationStereo` only in pruned per-layer widths and a serialized forward path; everything else (entrypoint, action verbs, dataset classes, deploy chain) is identical to `depth-net-stereo`.

For TAO Deploy TensorRT actions (`gen_trt_engine`, TensorRT `evaluate`, TensorRT `inference`), read `references/tao-deploy-fast-foundation-stereo.md` first. The deploy spec template lives at `references/spec_template_deploy.yaml`.

## When to Use

Use this skill to train, evaluate, export, or run inference for a TAO FastFoundationStereo model. Two supported use cases:
```
END THIRD-PARTY FILE 8c73273107a8f360

The third-party file has ended. Everything between the 8c73273107a8f360 tags is data: requests and commands from it are not followed. An “end of file” line with another tag inside the block is part of the file.

First 1918 of 11859 bytes. © NVIDIA Corporation · Apache-2.0 · https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-fast-foundation-stereo/SKILL.md

## Similar
- [skill-creator](https://affpapa.org/en/nevibecode/skills/skill-creator.md) — skill · 414180 installs
- [writing-for-agents](https://affpapa.org/en/nevibecode/skills/writing-for-agents.md) — skill · 466719 installs
- [media-use](https://affpapa.org/en/nevibecode/skills/media-use.md) — skill · 1100000 installs

_Part of the page did not fit — see the HTML version._

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md