mlflow-trace-analysis — Claude Code skill: analysis and install
The mlflow-trace-analysis Claude Code skill examines MLflow traces to explain agent failures, errors, and quality issues. It handles single traces and multi-turn sessions.
Apache-2.0 178 installs ★ 91 updated 2026-10-23
npx skills add mlflow/skills --skill mlflow-trace-analysis -a claude-codeWhat mlflow-trace-analysis does
The skill traces observed behavior back to span inputs and outputs in MLflow data. It investigates errors, unexpected responses, quality concerns, latency, and failures that carry across conversation turns. It does not edit application code.
Analysis needs a trace or session ID and a configured backend. The skill fetches full JSON with mlflow traces get, then inspects the span tree, feedback, and error events against the expected behavior. For an unknown trace ID, use mlflow-trace-retrieval or consult the search guide and installed CLI help.
The findings distinguish observed evidence from plausible causes and unresolved questions; a hypothesis is not reported as a confirmed root cause. Reproduction is limited to one attempt through the usual instrumented entry point, and only when safe.
What it helps with
- locate the first span where an agent response diverged from expectations
- explain an error using span status and exception events
- trace how one failed turn affects later responses
- separate application failures from feedback-scoring errors
- inspect latency across nested and overlapping spans
Example prompts for the agent
- Analyze trace
TRACE_IDand find where the agent response diverged from expectations. - Follow session
SESSION_IDto see how a failure in one turn affected later replies. - Review feedback and spans in trace
TRACE_ID; separate evidence from possible causes.
How to install mlflow-trace-analysis
1Skills CLI (Vercel)
npx skills add mlflow/skills --skill mlflow-trace-analysis -a claude-code2Manually: put the files in place
~/.claude/skills/mlflow-trace-analysis/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/mlflow-trace-analysis/
1Skills CLI (Vercel)
npx skills add mlflow/skills --skill mlflow-trace-analysis -a codex2Manually: put the files in place
.agents/skills/mlflow-trace-analysis/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add mlflow/skills --skill mlflow-trace-analysis -a cursor2Manually: put the files in place
.cursor/skills/mlflow-trace-analysis/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install mlflow/skills mlflow-trace-analysis2Manually: put the files in place
.github/skills/mlflow-trace-analysis/SKILL.md1Skills CLI (Vercel)
npx skills add mlflow/skills --skill mlflow-trace-analysis -a gemini-cli2Platform CLI
gemini skills install https://github.com/mlflow/skills --path mlflow-trace-analysisManually: put the files in place
.devin/skills/mlflow-trace-analysis/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add mlflow/skills --skill mlflow-trace-analysis -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about mlflow-trace-analysis
How do I install mlflow-trace-analysis in Claude Code?
Run npx skills add mlflow/skills --skill mlflow-trace-analysis -a claude-code. The skill file is installed at ~/.claude/skills/mlflow-trace-analysis/SKILL.md.
What does mlflow-trace-analysis do?
It examines MLflow traces and multi-turn sessions, connecting span inputs and outputs with errors, unexpected responses, quality issues, and latency. It provides diagnostic findings without changing application code.
What does mlflow-trace-analysis need to run?
It needs a trace or session ID and a configured backend. To find an unknown trace ID, use mlflow-trace-retrieval or the search guide and installed MLflow CLI help.
Source text
SKILL.md · Apache-2.0
---
name: mlflow-trace-analysis
description: Analyze MLflow traces, agent failures, and multi-turn chat sessions. Explain errors, unexpected output, quality issues, latency, or cross-turn propagation without editing application code.
---
# Analyze MLflow traces
Use the trace ID or session ID, configured backend, reported behavior and expected outcome. Reuse traces already fetched. For a multi-turn conversation, read [session analysis](references/session-analysis.md).
## Prerequisites
Before searching for an unknown trace ID, read [trace retrieval](../mlflow-trace-retrieval/SKILL.md) for filters, time bounds and pagination. Reuse an already fetched trace. If that skill is not installed, use the [search guide](https://mlflow.org/docs/latest/genai/tracing/search-traces.md) and installed CLI help.
Prefer existing evidence. Only when no usable trace exists, reproduce once through the normal instrumented entry point when safe; otherwise report the missing evidence. Record HUMAN feedback only when authorized and attached to an identified trace.
## Step 1: Fetch the trace
Fetch full JSON once to a file; large piped CLI output can be truncated by the execution tool. `get` needs no `--output` flag. Avoid `--extract-fields` for nested span diagnosis.
```bash
mlflow traces get --trace-id <TRACE_ID> > /tmp/trace.json
```
## Step 2: Inspect the span tree
Summarize state, assessments and parent links:
```bash
jq '{state: .info.state,
spans: [.data.spans[] | {span_id, parent_span_id, name, status}],
error_spans: [.data.spans[] | select(.status.code == "STATUS_CODE_ERROR") | .name],
assessments: .info.assessments}' /tmp/trace.json
```
## Step 3: Read feedback
Read feedback **source, value, top-level rationale, validity and scope** before forming a hypothesis. HUMAN feedback describes the reported experience; corroborate it with spans. LLM_JUDGE/CODE feedback may be wrong or misconfigured. `feedback.error` means scoring failed, not that the application failed; exclude errored/overridden judgments from quality conclusions. Genuine zero/false values remain valid. Expectations are desired-output labels, not scores.
## Step 4: Follow inputs and outputs
Follow actual inputs and outputs from root to the first divergence, then downstream effects. Raw MLflow span I/O is usually serialized JSON in `attributes["mlflow.spanInputs"]` / `attributes["mlflow.spanOutputs"]`; parse it before interpreting it. If the root is an empty wrapper, inspect descendants. For other OTel sources or versions, inspect raw keys instead of assuming this schema.
## Step 5: Check errors and performance
Check exception events and status messages. `OK` means execution success, not correct output; `STATUS_CODE_UNSET` is not itself an error. For performance, distinguish enclosing wrappers, overlapping child work and uninstrumented gaps; repeated names alone do not prove retries. Inspect recorded token usage rather than inventing cost or assuming a token spike proves injection.
## Step 6: Check the source
Correlate the decisive span's observed data with relevant source/configuration when available. Function names are clues; autologged names follow framework conventions. Separate observed facts, plausible causes and missing evidence. Do not edit code, create scorers or run an evaluation for diagnosis alone.
See the [refund-policy evidence table](references/session-analysis.md#worked-example-refund-policy-propagation) for a retrieval miss, unsupported answer and later propagation.
## Verification
Report observed versus expected behavior, trace and decisive span IDs, the evidence chain, confidence and the smallest likely remedy. Confirm claims against actual span I/O and feedback semantics, not previews alone. If evidence is missing, say what would resolve the uncertainty; do not promote a hypothesis to a confirmed root cause.
## References
- Fields, units, CLI versus SDK representation, OTel attributes or assessment interpretation: [trace structure](references/trace-structure.md).
- Bounded batches, recurring failures or component bottlenecks: [cross-trace analysis](references/cross-trace-analysis.md).
- Multi-turn propagation: [session analysis](references/session-analysis.md).
- A user-requested implementation: pass diagnosis and trace ID to `mlflow-agent-improvement` when installed; investigation itself stops at evidence and recommendations.
© mlflow · Apache-2.0 · mlflow/skills