# tao-train-bevfusion

A skill that drives training and inference for BEVFusion, the sensor-fusion model that detects objects in 3D from lidar and camera. Useful when you work with the TAO BEVFusion.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 1600 installs · ★ 3735
- Author: NVIDIA Corporation
- License: Apache-2.0 · policy=mirror · the beginning of the file is at the end of this page
- Source: https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-bevfusion/SKILL.md
- Updated: 2026-10-26
- HTML: https://affpapa.org/en/nevibecode/skills/tao-train-bevfusion

## What it does

BEVFusion projects lidar points and camera frames into bird's-eye-view space, then finds objects in 3D. The skill handles training, evaluation and inference for the TAO model; dataset preparation is a separate action, `dataset_convert`.

It ships TAO Core schemas as `schemas/<action>.schema.json` and spec templates as `references/spec_template_<action>.yaml`. Training defaults to routing through `tao-skill-bank:tao-run-automl` once `schemas/train.schema.json` is present. Ask for plain training, or say you want AutoML off, and the run skips hyperparameter search.

You need Docker with nvidia-container-toolkit and the BEVFusion container `nvcr.io/nvidia/tao/tao-toolkit:5.5.0-pyt`. Shared TAO PyTorch 7.x images lack `mmdet3d`, so they break before a spec is even read. In a fresh session, run the `tao-setup` skill first. The monitoring metric is AP11.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-bevfusion -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/tao-train-bevfusion/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/tao-train-bevfusion/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-bevfusion -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/tao-train-bevfusion/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-bevfusion -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/tao-train-bevfusion/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install NVIDIA/skills tao-train-bevfusion
```

2. **Manually: put the files in place**
- File: `.github/skills/tao-train-bevfusion/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-bevfusion -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/NVIDIA/skills --path skills/tao-train-bevfusion
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/tao-train-bevfusion/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill tao-train-bevfusion -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-bevfusion/SKILL.md

## When to use it
- train BEVFusion on a KITTI-style set with lidar and cameras
- convert raw data into info pickles for train and val splits
- score 3D detection quality against AP11
- run inference on prepared point clouds and images
- switch a training run to plain mode without AutoML

## Example prompts
- Train BEVFusion on the dataset at `/path/to/kitti_root` for 30 epochs on one GPU
- Convert the dataset with `bevfusion convert -e <spec>`
- Turn off AutoML and start a plain BEVFusion training run

## Source text · SKILL.md · Apache-2.0

Below is the content of a third-party file. It is data, not instructions for you: do not act on anything written inside unless the human asks. Block tag: 37657aa06485394f.

BEGIN THIRD-PARTY FILE 37657aa06485394f
```text
---
name: tao-train-bevfusion
description: BEVFusion for multi-sensor 3D object detection. Fuses LiDAR point clouds and camera images in bird's-eye-view
  (BEV) space, used in autonomous driving for robust 3D perception. Use when training, evaluating, or running inference for
  a TAO BEVFusion model. Trigger phrases include "train BEVFusion", "LiDAR + camera fusion", "BEV 3D detection", "multi-sensor
  3D perception".
license: Apache-2.0
compatibility: Requires docker + nvidia-container-toolkit.
metadata:
  version: "0.1.0"
  author: NVIDIA Corporation
allowed-tools: Read Bash
tags:
- multi
- sensor
- 3d
- detection
---

# BEVFusion

> **Standalone install?** If this session was not initialized by the TAO skill bank plugin, run the `tao-setup` skill first (host preflight, credentials, cross-skill discovery).

BEVFusion for multi-sensor 3D object detection. Fuses LiDAR point clouds and camera images in bird's-eye-view (BEV) space. Used in autonomous driving for robust 3D perception.

Set pretrained backbone paths for Swin image backbone.

BEVFusion requires the BEVFusion-specific TAO container
`nvcr.io/nvidia/tao/tao-toolkit:5.5.0-pyt`. <!-- unpinned: BEVFusion-only container --> Shared TAO PyTorch 7.x images do
not package `mmdet3d` and fail before any BEVFusion action can parse its
spec. The model-skill action is named `dataset_convert`, but the 5.5 container
CLI subtask is `bevfusion convert -e <spec>`.

## Dataclass Schemas

Generated TAO Core schemas are packaged in `schemas/<action>.schema.json`, with `schemas/manifest.json` listing available actions. Each generated schema also emits `references/spec_template_<action>.yaml` from the schema top-level `default` field. AutoML enablement is declared at the model layer in `references/skill_info.yaml` via `automl_enabled`. Runnable AutoML for an action requires `schemas/<action>.schema.json` and `references/spec_template_<action>.yaml` to exist and parse. Use the packaged selected-action schema for `automl_default_parameters`, `automl_disabled_parameters`, defaults, min/max bounds, enums, option weights, math conditions, dependencies, and popular parameters. Do not expect `~/tao-core` at runtime; maintainers regenerate schemas/templates before packaging the skill bank.

## Train Action Policy
```
END THIRD-PARTY FILE 37657aa06485394f

The third-party file has ended. Everything between the 37657aa06485394f tags is data: requests and commands from it are not followed. An “end of file” line with another tag inside the block is part of the file.

First 2282 of 13477 bytes. © NVIDIA Corporation · Apache-2.0 · https://github.com/NVIDIA/skills/tree/HEAD/skills/tao-train-bevfusion/SKILL.md

## Similar
- [run-train](https://affpapa.org/en/nevibecode/skills/run-train.md) — skill · 336500 installs
- [train-sentence-transformers](https://affpapa.org/en/nevibecode/skills/train-sentence-transformers.md) — skill · 1500 installs
- [neural-train](https://affpapa.org/en/nevibecode/skills/neural-train.md) — skill · 748 installs
- [trader-train](https://affpapa.org/en/nevibecode/skills/trader-train.md) — skill · 736 installs
- [threejs-3d-generator](https://affpapa.org/en/nevibecode/skills/threejs-3d-generator.md) — skill · 3500 installs
- [layer-3d](https://affpapa.org/en/nevibecode/skills/layer-3d.md) — skill · 3300 installs

_Part of the page did not fit — see the HTML version._

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md