databricks-mlflow-evaluation — Claude Code skill for MLflow GenAI eval
databricks-mlflow-evaluation — a skill for running mlflow.genai.evaluate(), building scorers and datasets, aligning judges with MemAlign, and watching production quality on Databricks.
no license 355 installs ★ 408 updated 2026-11-03
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a claude-codeWhat databricks-mlflow-evaluation does
The skill walks through the whole GenAI evaluation loop on Databricks. Three guides cover scorer design, dataset creation plus agent evaluation, and how to change agent behavior and confirm the improvement.
You need Databricks CLI 1.0.0 or newer. The parent skill databricks-core handles authentication and workspace discovery. Tracing is a separate concern — databricks-mlflow-tracing owns trace retrieval, observability, and trace-based debugging. Verify tracing first when your evaluation mode or scorers demand it.
For judge models, Databricks prefers databricks:/system.ai.<model> with mlflow[databricks]>=3.16.0. A 404 often means an older client that cannot parse the system model URI. Before touching credentials, check the installed client and the available system.ai model.
What it helps with
- run mlflow.genai.evaluate() on Databricks
- design scorers that match quality criteria
- build a dataset to evaluate an agent
- align a judge model using MemAlign
- verify an agent change actually improved results
Example prompts for the agent
- Create a dataset and evaluate my agent using the MLflow evaluation guide
- Help me design scorers for answer quality
- Check tracing before running evaluation with these scorers
How to install databricks-mlflow-evaluation
1Skills CLI (Vercel)
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a claude-code2Manually: put the files in place
~/.claude/skills/databricks-mlflow-evaluation/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/databricks-mlflow-evaluation/
1Skills CLI (Vercel)
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a codex2Manually: put the files in place
.agents/skills/databricks-mlflow-evaluation/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a cursor2Manually: put the files in place
.cursor/skills/databricks-mlflow-evaluation/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install databricks/databricks-agent-skills skills/databricks-mlflow-evaluation2Manually: put the files in place
.github/skills/databricks-mlflow-evaluation/SKILL.md1Skills CLI (Vercel)
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a gemini-cli2Platform CLI
gemini skills install https://github.com/databricks/databricks-agent-skills --path skills/databricks-mlflow-evaluationManually: put the files in place
.devin/skills/databricks-mlflow-evaluation/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about databricks-mlflow-evaluation
How do I install databricks-mlflow-evaluation in Claude Code?
Run npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a claude-code. The skill file lands at ~/.claude/skills/databricks-mlflow-evaluation/SKILL.md.
What does databricks-mlflow-evaluation do?
It guides you through MLflow GenAI evaluation on Databricks: defining scorers, building datasets, aligning judges with MemAlign, improving prompts, and monitoring production quality.
How is it different from databricks-mlflow-tracing?
The tracing skill handles trace retrieval, observability, and trace-based debugging. This skill focuses on evaluation; when a failure needs trace-grounded diagnosis, you run the tracing skill first.