# comparator

comparator runs blind comparisons between two AI outputs against the original prompt. Identifies the stronger result without knowing which skill produced each file.

- subagent · `type=agent` · Claude Code, Codex, Cursor · ★ 406 repo stars
- Author: syahiidkamil
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/syahiidkamil/Software-Engineer-AI-Agent-Atlas/tree/HEAD/.claude/skills/skill-creator/agents/comparator.md
- Updated: 2026-06-25
- Security check: automated, 2026-11-16 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/agents/comparator

## What it does

The subagent receives paths to two outputs, the initial task prompt, and optional assertion checks. It inspects outputs as plain files or directory trees. Crucially, it never sees which system generated artifact A or B.

Evaluation uses two distinct scales from 1 to 5: content quality and document structure. The agent checks correctness, completeness, schema accuracy, and overall layout. If you pass specific assertions, it calculates pass rates as secondary validation points. Draws remain an exception.

All findings write out to `comparison.json` or a chosen target path. You get numerical scores on a 10-point scale alongside explicit lists of strengths and flaws for both candidates.

## Install

### Claude Code (native platform)

**Manually: put the files in place**
- File: `.claude/agents/comparator.md` — For all projects use ~/.claude/agents/. Claude delegates by the description field or an @-mention

### Codex

**Manually: put the files in place**
- File: `.codex/agents/comparator.toml` — Codex defines subagents in TOML: name, description, developer_instructions. Move the Markdown text into developer_instructions by hand. For all projects use ~/.codex/agents/

### Cursor

**Manually: put the files in place**
- File: `.cursor/agents/comparator.md` — Cursor also reads .claude/agents/

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/syahiidkamil/Software-Engineer-AI-Agent-Atlas/tree/HEAD/.claude/skills/skill-creator/agents/comparator.md

## When to use it
- pick the stronger output between two AI runs
- build a task-specific grading rubric for content and structure
- score candidate files against a checklist of assertions
- generate a structured JSON summary with strengths and weaknesses

## Example prompts
- Compare artifacts in `output_a` and `output_b` against the goal in `prompt.txt`
- Run a blind evaluation between `report_a.md` and `report_b.md` and save the verdict to `comparison.json`
- Judge outputs A and B with expectations loaded from `checklist.json`

## Similar
- [bmad-eval](https://affpapa.org/en/nevibecode/skills/bmad-eval.md) — skill · 9100 installs
- [llm-eval-harness](https://affpapa.org/en/nevibecode/skills/llm-eval-harness.md) — skill · 438 installs
- [principle-explain-the-number](https://affpapa.org/en/nevibecode/skills/principle-explain-the-number.md) — skill · 9200 installs
- [eval-runner](https://affpapa.org/en/nevibecode/skills/eval-runner.md) — skill · ★ 54 repo stars
- [eval-judge](https://affpapa.org/en/nevibecode/agents/eval-judge.md) — subagent · ★ 42780 repo stars
- [perf-baseline-manager](https://affpapa.org/en/nevibecode/skills/perf-baseline-manager.md) — skill · 2300 installs

From the same set: [syahiidkamil/software-engineer-ai-agent-atlas](https://affpapa.org/en/nevibecode/by/syahiidkamil/software-engineer-ai-agent-atlas.md) — 26 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
