Skip to main content
skill CX Codex

mlflow-trace-metrics — Claude Code skill: install and usage

The mlflow-trace-metrics Claude Code skill queries aggregate trace, span, and assessment metrics from MLflow. Runs telemetry summaries for latency, token counts, and evaluation scores.

Apache-2.0 174 installs ★ 91 updated 2026-10-23

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add mlflow/skills --skill mlflow-trace-metrics -a claude-code
Open on GitHub

What mlflow-trace-metrics does

The skill pulls aggregate operational telemetry from an open-source MLflow server. It groups latency percentiles, token usage totals, and assessment scores across specified experiments and time buckets.

Queries run through scripts/fetch_metrics.py with explicit parameters for servers, experiment IDs, metrics, and timestamps. Supported views include TRACES, SPANS, and ASSESSMENTS. The tool supports standard HTTP or HTTPS endpoints without extra authentication headers. Databricks-managed tracking and databricks:// profiles do not work with this helper.

If the server returns next_page_token, the helper reports partial data because it does not paginate subsequent records. Empty data points do not prove zero activity, since uninstrumented traces omit token counts.

What it helps with

  • aggregate total token consumption over an experiment time window
  • calculate P95 and average response latency across spans
  • break down trace status counts to track failure occurrences
  • evaluate average numeric scores from LLM assessment records

Example prompts for the agent

  • Calculate total token sums for experiment 12 over the last 24 hours on server <SERVER>
  • Fetch P95 and average latency grouped by trace_name on server <SERVER>
  • Summarize assessment_value averages for experiment 3 using view ASSESSMENTS

How to install mlflow-trace-metrics

1Skills CLI (Vercel)

npx skills add mlflow/skills --skill mlflow-trace-metrics -a claude-code

2Manually: put the files in place

~/.claude/skills/mlflow-trace-metrics/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/mlflow-trace-metrics/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about mlflow-trace-metrics

How to install mlflow-trace-metrics in Claude Code?

Run npx skills add mlflow/skills --skill mlflow-trace-metrics -a claude-code. The definition lands in ~/.claude/skills/mlflow-trace-metrics/SKILL.md.

What does mlflow-trace-metrics do?

It queries aggregate metrics across MLflow traces, spans, and evaluation scores. It focuses on batch health summaries rather than single-trace inspection.

What are the requirements and limits of mlflow-trace-metrics?

It requires a reachable open-source MLflow server over HTTP or HTTPS and Python. It does not support Databricks-managed tracking, cannot attach auth headers, and does not fetch pages past next_page_token.

Source text

SKILL.md · Apache-2.0
---
name: mlflow-trace-metrics
description: Query aggregate MLflow trace, span and assessment metrics for latency, tokens, counts and quality trends. Use for operational summaries rather than individual-trace diagnosis.
---

# Query MLflow metrics

Reuse server, experiment IDs and requested window. Select the exact metric, view, aggregation and grouping needed; token counts are not currency cost.

## Prerequisites

For aggregate queries, read the [metrics API reference](references/api_reference.md). If the task also needs individual traces, read [trace retrieval](../mlflow-trace-retrieval/SKILL.md) before searching; if unavailable, use the [search guide](https://mlflow.org/docs/latest/genai/tracing/search-traces.md).

## Step 1: Check the client

Resolve `scripts/fetch_metrics.py` relative to this skill; use `--help` for flags. The metrics endpoint and helper are not supported for Databricks-managed tracking. For OSS they accept HTTP(S), not `databricks://` profiles, and add no authentication headers. Use an existing supported authenticated client or report the blocker; never convert a Databricks profile to a workspace URL or request pasted tokens.

## Step 2: Choose metrics

Select TRACES (default), SPANS or ASSESSMENTS using [the API reference](references/api_reference.md). Use COUNT for entities, SUM for tokens, AVG/P95 for distributions. Do not silently substitute metrics.

## Step 3: Query the time window

Query explicit bounds: epoch milliseconds, timezone-qualified ISO timestamps, `now`, or relative values like `-24h`. Buckets are **seconds** and require both bounds.

```bash
python <SKILL_DIR>/scripts/fetch_metrics.py -s <SERVER> -x <EXPERIMENT_ID> \
  -m total_tokens -a SUM -t 3600 --start-time="-24h" --end-time=now -o json
```

Latency: `-m latency -a AVG,P95 -d trace_name`. Numeric quality: `-v ASSESSMENTS -m assessment_value -a AVG -d assessment_name`. Status counts: `-m trace_count -a COUNT -d trace_status`. Keep the same explicit window.

## Verification

Inspect actual `metric_name`, dimensions and values; report exact experiment/window, metric, aggregation, units, filters and coverage. Empty `data_points` is not zero usage; absent token instrumentation can undercount. Quality summaries need per-scorer scales and valid/error/missing denominators: zero/false can be valid, scorer failures are not zero scores, and unrelated scorers should not be averaged.

A `next_page_token` means partial results; the helper exposes it but cannot fetch subsequent pages. Use an existing pagination-capable client or disclose incompleteness. Counts by status are not an error rate: derive ERROR / all statuses only with a complete stated denominator. Distinguish request/auth failures from empty results. Aggregates need representative trace evidence for causal diagnosis.

## References

[API reference](references/api_reference.md): wire fields, metric/view compatibility and metrics-specific filters.

© mlflow · Apache-2.0 · mlflow/skills

Similar skills

All skills for AI and ML →

From the same set17

The whole set →
mlflow-agent Routes MLflow requests to the right child skill for tracing, evaluation, debugging, or documentation. Use it when the task has no specific MLflow workflow selected. CX Codex +1 882installs sagemaker-mlflow sagemaker-mlflow connects the agent to SageMaker Managed MLflow via resource ARN (mlflow-app or mlflow-tracking-server), then hands off to other MLflow skills through MLFLOW_TRACKING_URI. CX Codex +1 394installs fix-agent-issue fix-agent-issue — a skill that walks the agent through EXPLORE → PLAN → IMPLEMENT → VERIFY so a behavior change rests on a trace and a test, not a guess. CX Codex +1 376installs build-a-scorer build-a-scorer helps design and iterate MLflow judges and scorers for LLM apps or RAG agents. It defines atomic criteria and picks the cheapest reliable check. CX Codex +1 339installs mlflow-evaluation The mlflow-evaluation Claude Code skill configures MLflow scorers and evaluates an agent, dataset, precomputed outputs, or traces. It checks GenAI quality and inspects evaluation results. CX Codex +1 184installs mlflow-trace-analysis The mlflow-trace-analysis Claude Code skill examines MLflow traces to explain agent failures, errors, and quality issues. It handles single traces and multi-turn sessions. CX Codex +1 178installs mlflow-trace-retrieval The mlflow-trace-retrieval Claude Code skill retrieves MLflow traces by ID or searches them with filters. It runs for trace lookups based on experiments, runs, users, time windows, and other fields. CX Codex +1 174installs mlflow Equips Claude Code with workflows for tracing, debugging, and evaluating GenAI apps using MLflow. Helpful when instrumenting LLMOps. ★ 91repo stars

Project sponsors

Companies that keep this analytics open