# agent-evaluation

Framework for testing agents, prompts, and skills using multi-dimensional scoring rubrics and LLM-as-judge prompts.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 1300 installs · ★ 1787
- Author: NeoLabHQ
- License: GPL-3.0 · policy=link-only · description only; the file stays with the author
- Source: https://github.com/NeoLabHQ/context-engineering-kit/tree/HEAD/skills/agent-evaluation/SKILL.md
- Updated: 2026-08-26
- HTML: https://affpapa.org/en/nevibecode/skills/neolabhq-agent-evaluation

## What it does

This skill outlines an evaluation workflow for autonomous coding agents and prompt sets. Agents follow non-deterministic execution paths toward a goal, so step-by-step assertions usually fail. The skill shifts evaluation toward end outcomes using a weighted rubric: instruction following (0.30), output completeness (0.25), tool efficiency (0.20), reasoning quality (0.15), and response coherence (0.10).

It provides an LLM-as-judge template and references research showing that placing justification before the score improves evaluation reliability by 15-25%. Relying strictly on automated judges leaves silent hallucinations undetected on edge cases. Manual spot-checking remains necessary for subtle context drift.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add NeoLabHQ/context-engineering-kit --skill agent-evaluation -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/agent-evaluation/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/agent-evaluation/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add NeoLabHQ/context-engineering-kit --skill agent-evaluation -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/agent-evaluation/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add NeoLabHQ/context-engineering-kit --skill agent-evaluation -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/agent-evaluation/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install NeoLabHQ/context-engineering-kit agent-evaluation
```

2. **Manually: put the files in place**
- File: `.github/skills/agent-evaluation/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add NeoLabHQ/context-engineering-kit --skill agent-evaluation -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/NeoLabHQ/context-engineering-kit --path skills/agent-evaluation
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/agent-evaluation/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add NeoLabHQ/context-engineering-kit --skill agent-evaluation -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/NeoLabHQ/context-engineering-kit/tree/HEAD/skills/agent-evaluation/SKILL.md

## When to use it
- evaluate agent outputs against a weighted multi-factor rubric
- measure prompt and context engineering revisions
- set up structured LLM-as-judge evaluation prompts
- catch subtle failures across prolonged conversational contexts

## Example prompts
- Evaluate the agent output using the weighted rubric with evidence before scores
- Compare two agent prompts based on instruction following and tool efficiency
- Generate an LLM-as-judge prompt to assess agent reasoning and output completeness

## Similar
- [improve-codebase-architecture](https://affpapa.org/en/nevibecode/skills/improve-codebase-architecture.md) — skill · 1163035 installs
- [design-taste-frontend](https://affpapa.org/en/nevibecode/skills/design-taste-frontend.md) — skill · 678700 installs
- [tdd](https://affpapa.org/en/nevibecode/skills/tdd.md) — skill · 1141062 installs
- [skill-creator](https://affpapa.org/en/nevibecode/skills/skill-creator.md) — skill · 414180 installs
- [agent-browser](https://affpapa.org/en/nevibecode/skills/vercel-labs-agent-browser.md) — skill · 1200000 installs
- [prototype](https://affpapa.org/en/nevibecode/skills/prototype.md) — skill · 986201 installs

## Other versions with the same name
- [agent-evaluation · sickn33](https://affpapa.org/en/nevibecode/skills/sickn33-agent-evaluation.md) — 971 installs
- [agent-evaluation · mlflow](https://affpapa.org/en/nevibecode/skills/mlflow-agent-evaluation.md) — 1100 installs

From the same set: [neolabhq/context-engineering-kit](https://affpapa.org/en/nevibecode/by/neolabhq/context-engineering-kit.md) — 64 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md