# databricks-mlflow-evaluation

databricks-mlflow-evaluation — a skill for running mlflow.genai.evaluate(), building scorers and datasets, aligning judges with MemAlign, and watching production quality on Databricks.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode, Windsurf (Devin Desktop), Claude.ai · 355 installs · ★ 408 repo stars
- Author: databricks
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/databricks/databricks-agent-skills/tree/HEAD/skills/databricks-mlflow-evaluation/SKILL.md
- Updated: 2026-11-03
- Security check: automated, 2026-11-16 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/databricks-mlflow-evaluation

## What it does

The skill walks through the whole GenAI evaluation loop on Databricks. Three guides cover scorer design, dataset creation plus agent evaluation, and how to change agent behavior and confirm the improvement.

You need Databricks CLI 1.0.0 or newer. The parent skill databricks-core handles authentication and workspace discovery. Tracing is a separate concern — databricks-mlflow-tracing owns trace retrieval, observability, and trace-based debugging. Verify tracing first when your evaluation mode or scorers demand it.

For judge models, Databricks prefers `databricks:/system.ai.<model>` with `mlflow[databricks]>=3.16.0`. A 404 often means an older client that cannot parse the system model URI. Before touching credentials, check the installed client and the available `system.ai` model.

## Requirements
- Works with: Databricks

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/databricks-mlflow-evaluation/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/databricks-mlflow-evaluation/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/databricks-mlflow-evaluation/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/databricks-mlflow-evaluation/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install databricks/databricks-agent-skills skills/databricks-mlflow-evaluation
```

2. **Manually: put the files in place**
- File: `.github/skills/databricks-mlflow-evaluation/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/databricks/databricks-agent-skills --path skills/databricks-mlflow-evaluation
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/databricks-mlflow-evaluation/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/databricks/databricks-agent-skills/tree/HEAD/skills/databricks-mlflow-evaluation/SKILL.md

## When to use it
- run mlflow.genai.evaluate() on Databricks
- design scorers that match quality criteria
- build a dataset to evaluate an agent
- align a judge model using MemAlign
- verify an agent change actually improved results

## Example prompts
- Create a dataset and evaluate my agent using the MLflow evaluation guide
- Help me design scorers for answer quality
- Check tracing before running evaluation with these scorers

## Similar
- [mlflow-evaluation](https://affpapa.org/en/nevibecode/skills/mlflow-evaluation.md) — skill · 184 installs
- [databricks-expert](https://affpapa.org/en/nevibecode/skills/databricks-expert.md) — skill · 215 installs
- [mlflow-onboarding](https://affpapa.org/en/nevibecode/skills/mlflow-onboarding.md) — skill · 1300 installs
- [data-engineering-patterns-fabric-databricks](https://affpapa.org/en/nevibecode/skills/data-engineering-patterns-fabric-databricks.md) — skill · 1600 installs
- [databricks-2025](https://affpapa.org/en/nevibecode/skills/databricks-2025.md) — skill · 168 installs
- [azure-databricks-platform-architect](https://affpapa.org/en/nevibecode/agents/azure-databricks-platform-architect.md) — subagent · ★ 6505 repo stars

From the same set: [databricks/databricks-agent-skills](https://affpapa.org/en/nevibecode/by/databricks/databricks-agent-skills.md) — 30 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
