Skip to main content
skill CX Codex

databricks-mlflow-evaluation — Claude Code skill for MLflow GenAI eval

databricks-mlflow-evaluation — a skill for running mlflow.genai.evaluate(), building scorers and datasets, aligning judges with MemAlign, and watching production quality on Databricks.

no license 355 installs ★ 408 updated 2026-11-03

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a claude-code
Open on GitHub

What databricks-mlflow-evaluation does

The skill walks through the whole GenAI evaluation loop on Databricks. Three guides cover scorer design, dataset creation plus agent evaluation, and how to change agent behavior and confirm the improvement.

You need Databricks CLI 1.0.0 or newer. The parent skill databricks-core handles authentication and workspace discovery. Tracing is a separate concern — databricks-mlflow-tracing owns trace retrieval, observability, and trace-based debugging. Verify tracing first when your evaluation mode or scorers demand it.

For judge models, Databricks prefers databricks:/system.ai.<model> with mlflow[databricks]>=3.16.0. A 404 often means an older client that cannot parse the system model URI. Before touching credentials, check the installed client and the available system.ai model.

What it helps with

  • run mlflow.genai.evaluate() on Databricks
  • design scorers that match quality criteria
  • build a dataset to evaluate an agent
  • align a judge model using MemAlign
  • verify an agent change actually improved results

Example prompts for the agent

  • Create a dataset and evaluate my agent using the MLflow evaluation guide
  • Help me design scorers for answer quality
  • Check tracing before running evaluation with these scorers

How to install databricks-mlflow-evaluation

1Skills CLI (Vercel)

npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a claude-code

2Manually: put the files in place

~/.claude/skills/databricks-mlflow-evaluation/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/databricks-mlflow-evaluation/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about databricks-mlflow-evaluation

How do I install databricks-mlflow-evaluation in Claude Code?

Run npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation -a claude-code. The skill file lands at ~/.claude/skills/databricks-mlflow-evaluation/SKILL.md.

What does databricks-mlflow-evaluation do?

It guides you through MLflow GenAI evaluation on Databricks: defining scorers, building datasets, aligning judges with MemAlign, improving prompts, and monitoring production quality.

How is it different from databricks-mlflow-tracing?

The tracing skill handles trace retrieval, observability, and trace-based debugging. This skill focuses on evaluation; when a failure needs trace-grounded diagnosis, you run the tracing skill first.

Similar skills

All skills for AI and ML →

From the same set30

The whole set →
databricks-aibi-dashboards Builds Databricks AI/BI dashboards from the ground up: knows the Lakeview JSON layout and makes the agent run every SQL query through the CLI first. CX Codex +1 366installs databricks-apps-python databricks-apps-python — a skill for Claude Code, Codex, Cursor and other agents that adds Python backend support for Databricks Apps. CX Codex +1 363installs databricks-execution-compute databricks-execution-compute is a skill that runs code on Databricks and helps choose a compute mode for Spark scripts, longer jobs, and interactive clusters. CX Codex +1 359installs databricks-python-sdk Guidance for building with the Databricks Python SDK, Connect, CLI, and REST API. Use it when your code needs to work with Databricks services. CX Codex +1 358installs databricks-ml-training databricks-ml-training is a skill for training classifiers and regressors on Databricks with XGBoost, scikit-learn, LightGBM, or PyTorch, and tracking runs with MLflow. CX Codex +1 354installs databricks-lakeflow-connect databricks-lakeflow-connect — a skill for Claude Code builds and debugs Lakeflow Connect ingestion from SQL Server, PostgreSQL, MySQL, and Oracle into Unity Catalog. It covers CDC and query-based loads. CX Codex +1 348installs databricks-zerobus-ingest The databricks-zerobus-ingest skill helps route events, logs, CDC, and telemetry into Databricks Delta tables. It fits streaming setups that do not need a separate broker. CX Codex +1 331installs databricks-unstructured-pdf-generation databricks-unstructured-pdf-generation creates synthetic PDF files and paired evaluation questions locally, then copies them to Databricks Unity Catalog volumes for RAG testing. CX Codex +1 328installs

Project sponsors

Companies that keep this analytics open