# eval-grader

eval-grader — a grader for skill-eval assertions that returns PASS, FAIL or PARTIAL on each, with one line of evidence. Handy when a reviewer's write-up needs a verdict from something that did not produce it.

- subagent · `type=agent` · Claude Code, Codex, Cursor · ★ 155 repo stars
- Author: serpro69
- License: Elastic-2.0 · policy=link-only · description only; the file stays with the author
- Source: https://github.com/serpro69/claude-toolbox/tree/HEAD/klaude-plugin/agents/eval-grader.md
- Updated: 2026-10-29
- Security check: automated, 2026-11-16 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/agents/eval-grader

## What it does

Hand it a reviewer's write-up plus a list of assertions and it answers with a verdict on each one: PASS, FAIL or PARTIAL, and a single line pointing at the text that settled it. The agent had no hand in producing what it grades, and it never opens the live fixtures, so the grade cannot quietly drift toward the source of truth.

Component mode is the default. Only the output text counts there, and execution traces do not upgrade a grade. Workflow mode has to be named by the controller: the supplied rubric and manifest load first, sealed evidence after that. An unrecognised mode is an input error, and the agent asks for a corrected one rather than guessing.

\`Read` is the only tool, kept to two jobs: fetching a reviewer-output file the harness left on disk, and re-reading its own agent file or playbook. Paths such as `test-files/`, `klaude-plugin/profiles/**` and `SKILL.md` stay shut, because opening them brings rubric leakage back in. A claim the output text cannot support turns into PARTIAL.

## Install

### Claude Code (native platform)

1. **Plugin marketplace**
In the agent chat (not a shell):
```text
/plugin marketplace add serpro69/claude-toolbox
```
In the agent chat (not a shell):
```text
/plugin install kk@claude-toolbox
```

2. **Manually: put the files in place**
- File: `.claude/agents/eval-grader.md` — For all projects use ~/.claude/agents/. Claude delegates by the description field or an @-mention

### Codex

**Manually: put the files in place**
- File: `.codex/agents/eval-grader.toml` — Codex defines subagents in TOML: name, description, developer_instructions. Move the Markdown text into developer_instructions by hand. For all projects use ~/.codex/agents/

### Cursor

**Manually: put the files in place**
- File: `.cursor/agents/eval-grader.md` — Cursor also reads .claude/agents/

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/serpro69/claude-toolbox/tree/HEAD/klaude-plugin/agents/eval-grader.md

## When to use it
- Grade a skill eval's assertions one by one
- Get a verdict from an agent that did not write the output
- Review a reviewer's text without touching the fixtures
- Work through a sealed evidence package in workflow mode
- Spot which behaviour a reviewer's output fails to show

## Example prompts
- Grade the reviewer output against these assertions and give me PASS, FAIL or PARTIAL for each, with one line of evidence.
- Grading mode: workflow — load the rubric and manifest first, then open only the evidence files listed there.
- Check the checklist-loading assertion: if there's no quoted trigger, mark it PARTIAL.

## Similar
- [skill-grader](https://affpapa.org/en/nevibecode/skills/skill-grader.md) — skill · 147 installs
- [google-agents-cli-eval](https://affpapa.org/en/nevibecode/skills/google-agents-cli-eval.md) — skill · 586000 installs
- [bmad-eval](https://affpapa.org/en/nevibecode/skills/bmad-eval.md) — skill · 9100 installs
- [eval-harness](https://affpapa.org/en/nevibecode/skills/eval-harness.md) — skill · 6500 installs
- [llm-eval-harness](https://affpapa.org/en/nevibecode/skills/llm-eval-harness.md) — skill · 438 installs
- [eval-judge](https://affpapa.org/en/nevibecode/agents/eval-judge.md) — subagent · ★ 42780 repo stars

From the same set: [serpro69/claude-toolbox](https://affpapa.org/en/nevibecode/by/serpro69/claude-toolbox.md) — 30 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
