comparator — Claude Code subagent: what it does, how to install
comparator runs blind comparisons between two AI outputs against the original prompt. Identifies the stronger result without knowing which skill produced each file.
no license ★ 406 updated 2026-06-25
.claude/agents/comparator.mdWhat comparator does
The subagent receives paths to two outputs, the initial task prompt, and optional assertion checks. It inspects outputs as plain files or directory trees. Crucially, it never sees which system generated artifact A or B.
Evaluation uses two distinct scales from 1 to 5: content quality and document structure. The agent checks correctness, completeness, schema accuracy, and overall layout. If you pass specific assertions, it calculates pass rates as secondary validation points. Draws remain an exception.
All findings write out to comparison.json or a chosen target path. You get numerical scores on a 10-point scale alongside explicit lists of strengths and flaws for both candidates.
What it helps with
- pick the stronger output between two AI runs
- build a task-specific grading rubric for content and structure
- score candidate files against a checklist of assertions
- generate a structured JSON summary with strengths and weaknesses
Example prompts for the agent
- Compare artifacts in
output_aandoutput_bagainst the goal inprompt.txt - Run a blind evaluation between
report_a.mdandreport_b.mdand save the verdict tocomparison.json - Judge outputs A and B with expectations loaded from
checklist.json
How to install comparator
Manually: put the files in place
.claude/agents/comparator.mdFor all projects use ~/.claude/agents/. Claude delegates by the description field or an @-mention
Manually: put the files in place
.codex/agents/comparator.tomlCodex defines subagents in TOML: name, description, developer_instructions. Move the Markdown text into developer_instructions by hand. For all projects use ~/.codex/agents/
Manually: put the files in place
.cursor/agents/comparator.mdCursor also reads .claude/agents/
Step by step, with commands for each platform: Claude Code subagents: how to create one, call it and keep context clean
Questions about comparator
How do I install comparator in Claude Code?
Place the subagent file at .claude/agents/comparator.md inside your workspace directory.
What does comparator do?
It provides blind evaluation between two alternative outputs created for the same prompt. It scores content and structure separately to select the winning artifact without bias.
Where does comparator store evaluation results?
It writes the output details to comparison.json or a designated custom file path in JSON format.