mlflow-trace-metrics — Claude Code skill: install and usage
The mlflow-trace-metrics Claude Code skill queries aggregate trace, span, and assessment metrics from MLflow. Runs telemetry summaries for latency, token counts, and evaluation scores.
Apache-2.0 174 installs ★ 91 updated 2026-10-23
npx skills add mlflow/skills --skill mlflow-trace-metrics -a claude-codeWhat mlflow-trace-metrics does
The skill pulls aggregate operational telemetry from an open-source MLflow server. It groups latency percentiles, token usage totals, and assessment scores across specified experiments and time buckets.
Queries run through scripts/fetch_metrics.py with explicit parameters for servers, experiment IDs, metrics, and timestamps. Supported views include TRACES, SPANS, and ASSESSMENTS. The tool supports standard HTTP or HTTPS endpoints without extra authentication headers. Databricks-managed tracking and databricks:// profiles do not work with this helper.
If the server returns next_page_token, the helper reports partial data because it does not paginate subsequent records. Empty data points do not prove zero activity, since uninstrumented traces omit token counts.
What it helps with
- aggregate total token consumption over an experiment time window
- calculate P95 and average response latency across spans
- break down trace status counts to track failure occurrences
- evaluate average numeric scores from LLM assessment records
Example prompts for the agent
- Calculate total token sums for experiment 12 over the last 24 hours on server <SERVER>
- Fetch P95 and average latency grouped by trace_name on server <SERVER>
- Summarize assessment_value averages for experiment 3 using view ASSESSMENTS
How to install mlflow-trace-metrics
1Skills CLI (Vercel)
npx skills add mlflow/skills --skill mlflow-trace-metrics -a claude-code2Manually: put the files in place
~/.claude/skills/mlflow-trace-metrics/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/mlflow-trace-metrics/
1Skills CLI (Vercel)
npx skills add mlflow/skills --skill mlflow-trace-metrics -a codex2Manually: put the files in place
.agents/skills/mlflow-trace-metrics/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add mlflow/skills --skill mlflow-trace-metrics -a cursor2Manually: put the files in place
.cursor/skills/mlflow-trace-metrics/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install mlflow/skills mlflow-trace-metrics2Manually: put the files in place
.github/skills/mlflow-trace-metrics/SKILL.md1Skills CLI (Vercel)
npx skills add mlflow/skills --skill mlflow-trace-metrics -a gemini-cli2Platform CLI
gemini skills install https://github.com/mlflow/skills --path mlflow-trace-metricsManually: put the files in place
.devin/skills/mlflow-trace-metrics/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add mlflow/skills --skill mlflow-trace-metrics -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about mlflow-trace-metrics
How to install mlflow-trace-metrics in Claude Code?
Run npx skills add mlflow/skills --skill mlflow-trace-metrics -a claude-code. The definition lands in ~/.claude/skills/mlflow-trace-metrics/SKILL.md.
What does mlflow-trace-metrics do?
It queries aggregate metrics across MLflow traces, spans, and evaluation scores. It focuses on batch health summaries rather than single-trace inspection.
What are the requirements and limits of mlflow-trace-metrics?
It requires a reachable open-source MLflow server over HTTP or HTTPS and Python. It does not support Databricks-managed tracking, cannot attach auth headers, and does not fetch pages past next_page_token.
Source text
SKILL.md · Apache-2.0
--- name: mlflow-trace-metrics description: Query aggregate MLflow trace, span and assessment metrics for latency, tokens, counts and quality trends. Use for operational summaries rather than individual-trace diagnosis. --- # Query MLflow metrics Reuse server, experiment IDs and requested window. Select the exact metric, view, aggregation and grouping needed; token counts are not currency cost. ## Prerequisites For aggregate queries, read the [metrics API reference](references/api_reference.md). If the task also needs individual traces, read [trace retrieval](../mlflow-trace-retrieval/SKILL.md) before searching; if unavailable, use the [search guide](https://mlflow.org/docs/latest/genai/tracing/search-traces.md). ## Step 1: Check the client Resolve `scripts/fetch_metrics.py` relative to this skill; use `--help` for flags. The metrics endpoint and helper are not supported for Databricks-managed tracking. For OSS they accept HTTP(S), not `databricks://` profiles, and add no authentication headers. Use an existing supported authenticated client or report the blocker; never convert a Databricks profile to a workspace URL or request pasted tokens. ## Step 2: Choose metrics Select TRACES (default), SPANS or ASSESSMENTS using [the API reference](references/api_reference.md). Use COUNT for entities, SUM for tokens, AVG/P95 for distributions. Do not silently substitute metrics. ## Step 3: Query the time window Query explicit bounds: epoch milliseconds, timezone-qualified ISO timestamps, `now`, or relative values like `-24h`. Buckets are **seconds** and require both bounds. ```bash python <SKILL_DIR>/scripts/fetch_metrics.py -s <SERVER> -x <EXPERIMENT_ID> \ -m total_tokens -a SUM -t 3600 --start-time="-24h" --end-time=now -o json ``` Latency: `-m latency -a AVG,P95 -d trace_name`. Numeric quality: `-v ASSESSMENTS -m assessment_value -a AVG -d assessment_name`. Status counts: `-m trace_count -a COUNT -d trace_status`. Keep the same explicit window. ## Verification Inspect actual `metric_name`, dimensions and values; report exact experiment/window, metric, aggregation, units, filters and coverage. Empty `data_points` is not zero usage; absent token instrumentation can undercount. Quality summaries need per-scorer scales and valid/error/missing denominators: zero/false can be valid, scorer failures are not zero scores, and unrelated scorers should not be averaged. A `next_page_token` means partial results; the helper exposes it but cannot fetch subsequent pages. Use an existing pagination-capable client or disclose incompleteness. Counts by status are not an error rate: derive ERROR / all statuses only with a complete stated denominator. Distinguish request/auth failures from empty results. Aggregates need representative trace evidence for causal diagnosis. ## References [API reference](references/api_reference.md): wire fields, metric/view compatibility and metrics-specific filters.
© mlflow · Apache-2.0 · mlflow/skills