# databricks-ml-training

databricks-ml-training is a skill for training classifiers and regressors on Databricks with XGBoost, scikit-learn, LightGBM, or PyTorch, and tracking runs with MLflow.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode, Windsurf (Devin Desktop), Claude.ai · 354 installs · ★ 408 repo stars
- Author: databricks
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/databricks/databricks-agent-skills/tree/HEAD/skills/databricks-ml-training/SKILL.md
- Updated: 2026-10-24
- Security check: automated, 2026-11-16 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/databricks-ml-training

## What it does

This skill runs model training directly on a Databricks cluster instead of in the local Python process where your agent runs. Build classifiers and regressors with XGBoost, scikit-learn, LightGBM, or PyTorch, with MLflow tracking every run, parameter, and artifact. Models register to Unity Catalog and can then be scored in batch via Spark UDF or served real-time through `databricks-model-serving`.

Inside: hyperparameter search via Optuna, model versioning with movable aliases (`@prod`, `@challenger`), feature engineering backed by UC feature tables, point-in-time joins, materialized features, and streaming Kafka features. Custom PyFunc models are also supported. Requires Databricks CLI v0.294.0+. Training must always run as a serverless job on the cluster, never locally.

Do not use for endpoint management (`databricks-model-serving`) or MLflow-based model evaluation (`databricks-mlflow-evaluation`) — each has its own skill.

## Requirements
- Works with: Databricks

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add databricks/databricks-agent-skills --skill databricks-ml-training -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/databricks-ml-training/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/databricks-ml-training/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add databricks/databricks-agent-skills --skill databricks-ml-training -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/databricks-ml-training/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add databricks/databricks-agent-skills --skill databricks-ml-training -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/databricks-ml-training/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install databricks/databricks-agent-skills skills/databricks-ml-training
```

2. **Manually: put the files in place**
- File: `.github/skills/databricks-ml-training/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add databricks/databricks-agent-skills --skill databricks-ml-training -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/databricks/databricks-agent-skills --path skills/databricks-ml-training
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/databricks-ml-training/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add databricks/databricks-agent-skills --skill databricks-ml-training -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/databricks/databricks-agent-skills/tree/HEAD/skills/databricks-ml-training/SKILL.md

## When to use it
- train a classifier on Databricks tables with hyperparameter tuning
- register a model to Unity Catalog and swap production version via alias
- score all rows in a Delta table using a registered model as a Spark UDF
- build a feature pipeline with point-in-time joins and online store
- log training runs to MLflow with automatic code snapshot and metrics

## Example prompts
- Train an XGBoost classifier on `silver_turbine_features` using Optuna hyperparameter search and register it to UC
- Score the best model `ai_demo_gen.wind_farm.turbine_failure@prod` on all new records and write results to `gold_predictions`
- Create a feature view with rolling window aggregations and apply it during training with feature store backtracking

## Similar
- [databricks-expert](https://affpapa.org/en/nevibecode/skills/databricks-expert.md) — skill · 215 installs
- [data-engineering-patterns-fabric-databricks](https://affpapa.org/en/nevibecode/skills/data-engineering-patterns-fabric-databricks.md) — skill · 1600 installs
- [databricks-2025](https://affpapa.org/en/nevibecode/skills/databricks-2025.md) — skill · 168 installs
- [azure-databricks-platform-architect](https://affpapa.org/en/nevibecode/agents/azure-databricks-platform-architect.md) — subagent · ★ 6505 repo stars
- [databricks](https://affpapa.org/en/nevibecode/rules/databricks.md) — rule · ★ 3589 repo stars
- [databricks-cli](https://affpapa.org/en/nevibecode/plugins/databricks-cli.md) — plugin · ★ 1104 repo stars

From the same set: [databricks/databricks-agent-skills](https://affpapa.org/en/nevibecode/by/databricks/databricks-agent-skills.md) — 30 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
