# grader

grader assesses agent execution transcripts and generated files against expectations, flags superficial passes, and critiques test assertions.

- subagent · `type=agent` · Claude Code, Codex, Cursor · ★ 406 repo stars
- Author: syahiidkamil
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/syahiidkamil/Software-Engineer-AI-Agent-Atlas/tree/HEAD/.claude/skills/skill-creator/agents/grader.md
- Updated: 2026-06-25
- Security check: automated, 2026-11-16 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/agents/grader

## What it does

The subagent audits task runs by comparing expected outcomes against an execution transcript and generated artifacts. It takes three parameters: a list of assertions, a path to the Markdown log, and an output directory. Instead of trusting what the executor claimed in the transcript, it opens and inspects output files directly. A test fails whenever an artifact exists with hollow or mismatched content, preventing fake passes.

Beyond prewritten checks, `grader` extracts unstated claims about process, facts, and output quality from the execution log and verifies them against the artifacts. If `user_notes.md` is present in the output folder, the agent reads executor uncertainties and notes them in the review. All ratings, tool call statistics, and timing benchmarks are saved into `grading.json`.

The burden of proof stays on the assertion. Absent clear evidence, the verdict defaults to `FAIL`. The subagent also reviews the eval definitions themselves, pointing out assertions that are too lenient to catch failures.

## Install

### Claude Code (native platform)

**Manually: put the files in place**
- File: `.claude/agents/grader.md` — For all projects use ~/.claude/agents/. Claude delegates by the description field or an @-mention

### Codex

**Manually: put the files in place**
- File: `.codex/agents/grader.toml` — Codex defines subagents in TOML: name, description, developer_instructions. Move the Markdown text into developer_instructions by hand. For all projects use ~/.codex/agents/

### Cursor

**Manually: put the files in place**
- File: `.cursor/agents/grader.md` — Cursor also reads .claude/agents/

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/syahiidkamil/Software-Engineer-AI-Agent-Atlas/tree/HEAD/.claude/skills/skill-creator/agents/grader.md

## When to use it
- score task results against explicit expectations
- catch superficial test passes with invalid contents
- verify implicit factual and process claims from logs
- pinpoint gaps and weak checks in evaluation suites
- generate a structured `grading.json` report with timing metrics

## Example prompts
- Review the execution log in `transcript.md` and verify outputs in `run_artifacts/`
- Grade this task run against my checklist and write the result to `grading.json`
- Audit the latest session transcript and suggest improvements for weak eval assertions

## Similar
- [bmad-eval](https://affpapa.org/en/nevibecode/skills/bmad-eval.md) — skill · 9100 installs
- [eval-harness](https://affpapa.org/en/nevibecode/skills/eval-harness.md) — skill · 6500 installs
- [adk-evals](https://affpapa.org/en/nevibecode/skills/adk-evals.md) — skill · 1700 installs
- [inngest-agent-evals](https://affpapa.org/en/nevibecode/skills/inngest-agent-evals.md) — skill · 635 installs
- [simulations](https://affpapa.org/en/nevibecode/skills/simulations.md) — skill · 161 installs
- [creating-skills](https://affpapa.org/en/nevibecode/skills/osmontero-creating-skills.md) — skill · 101 installs

From the same set: [syahiidkamil/software-engineer-ai-agent-atlas](https://affpapa.org/en/nevibecode/by/syahiidkamil/software-engineer-ai-agent-atlas.md) — 26 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
