# agent-evaluation

Evaluate an agent’s behavior, capabilities, and consistency across runs. Use the skill when you need to test an agent beyond a single benchmark result.

- skill · `type=skill` · Claude Code · 378 installs
- Author: davila7
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/davila7/claude-code-templates/tree/HEAD/cli-tool/components/skills/ai-research/agent-evaluation
- Updated: 2026-10-24
- HTML: https://affpapa.org/en/nevibecode/skills/agent-evaluation

## What it does

This skill outlines ways to assess LLM agents through behavioral checks, capability assessments, reliability measures, and regression tests. Repeating tests and examining how results vary can reveal issues that a single successful run would miss. A strong benchmark result can still fail to reflect performance on practical tasks.

The approaches include checking behavioral invariants and actively probing for failure cases. The skill also covers flaky tests, metric gaming, and test data leaking into training or prompts. Testing fundamentals and knowledge of LLMs are prerequisites; without them, choosing suitable checks and interpreting results will be difficult.

## Install

### Claude Code (native platform)

1. **claude-code-templates**
```bash
npx claude-code-templates@latest --skill ai-research/agent-evaluation
```

2. **Skills CLI (Vercel)**
```bash
npx skills add davila7/claude-code-templates --skill agent-evaluation -a claude-code
```

3. **Manually: put the files in place**
- File: `~/.claude/skills/agent-evaluation/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/agent-evaluation/

### Codex

**Manually: put the files in place**
- File: `.agents/skills/agent-evaluation/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

**Manually: put the files in place**
- File: `.cursor/skills/agent-evaluation/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install davila7/claude-code-templates agent-evaluation
```

2. **Manually: put the files in place**
- File: `.github/skills/agent-evaluation/SKILL.md`

### Gemini CLI

**Platform CLI**
```bash
gemini skills install https://github.com/davila7/claude-code-templates --path cli-tool/components/skills/ai-research/agent-evaluation
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/agent-evaluation/SKILL.md` — The legacy .windsurf/skills/ path still works

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/davila7/claude-code-templates/tree/HEAD/cli-tool/components/skills/ai-research/agent-evaluation

## When to use it
- check whether an agent follows behavioral requirements across runs
- assess an agent’s capabilities on different tasks
- detect inconsistent results between test runs
- find cases where an agent optimizes for a metric instead of the task
- review tests for data leakage into training or prompts

## Example prompts
- Check whether the agent follows its behavioral requirements across repeated runs.
- Draft an evaluation plan for the agent’s capabilities on different tasks.
- Review the agent tests for unstable results, metric gaming, and data leakage.

## Similar
- [improve-codebase-architecture](https://affpapa.org/en/nevibecode/skills/improve-codebase-architecture.md) — skill · 1163035 installs
- [design-taste-frontend](https://affpapa.org/en/nevibecode/skills/design-taste-frontend.md) — skill · 678700 installs
- [tdd](https://affpapa.org/en/nevibecode/skills/tdd.md) — skill · 1141062 installs
- [skill-creator](https://affpapa.org/en/nevibecode/skills/skill-creator.md) — skill · 414180 installs
- [agent-browser](https://affpapa.org/en/nevibecode/skills/vercel-labs-agent-browser.md) — skill · 1200000 installs
- [prototype](https://affpapa.org/en/nevibecode/skills/prototype.md) — skill · 986201 installs

## Other versions with the same name
- [agent-evaluation · sickn33](https://affpapa.org/en/nevibecode/skills/sickn33-agent-evaluation.md) — 971 installs
- [agent-evaluation · neolabhq](https://affpapa.org/en/nevibecode/skills/neolabhq-agent-evaluation.md) — 1300 installs
- [agent-evaluation · mlflow](https://affpapa.org/en/nevibecode/skills/mlflow-agent-evaluation.md) — 1100 installs

From the same set: [davila7/claude-code-templates](https://affpapa.org/en/nevibecode/by/davila7/claude-code-templates.md) — 495 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md