# model-evaluator

This subagent helps compare models for a task, plan benchmarks, and investigate quality regressions after deployment. Use it when a choice needs measurable criteria.

- subagent · `type=agent` · Claude Code · 1061 installs
- Author: davila7
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/davila7/claude-code-templates/tree/HEAD/cli-tool/components/agents/ai-specialists/model-evaluator.md
- Updated: 2026-10-24
- HTML: https://affpapa.org/en/nevibecode/agents/model-evaluator

## What it does

model-evaluator helps assess models against task-specific measures such as output quality, latency, and cost. It can shape a benchmark, select metrics, and review results. For a post-deployment check, it compares current performance with a stored baseline.

Before recommending or testing a model, the agent uses WebSearch to verify current model names and IDs. WebFetch is available for detailed pricing and leaderboard pages. It first asks for success criteria, budget, latency and throughput targets, data constraints, and candidate models. A recommendation without those requirements would be premature.

In Claude Code, the agent can use Read, Write, Edit, Bash, Glob, Grep, WebSearch, and WebFetch. To install with the listed method, run `npx claude-code-templates@latest --agent ai-specialists/model-evaluator`; the agent file is `.claude/agents/model-evaluator.md`.

## Install

### Claude Code (native platform)

1. **claude-code-templates**
```bash
npx claude-code-templates@latest --agent ai-specialists/model-evaluator
```

2. **Manually: put the files in place**
- File: `.claude/agents/model-evaluator.md` — For all projects use ~/.claude/agents/. Claude delegates by the description field or an @-mention

### Codex

**Manually: put the files in place**
- File: `.codex/agents/model-evaluator.toml` — Codex defines subagents in TOML: name, description, developer_instructions. Move the Markdown text into developer_instructions by hand. For all projects use ~/.codex/agents/

### Cursor

**Manually: put the files in place**
- File: `.cursor/agents/model-evaluator.md` — Cursor also reads .claude/agents/

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/davila7/claude-code-templates/tree/HEAD/cli-tool/components/agents/ai-specialists/model-evaluator.md

## When to use it
- compare candidate models against task requirements
- design a benchmark and choose evaluation metrics
- check whether a model update caused quality regressions
- weigh model quality against inference cost

## Example prompts
- Compare models for support-ticket summaries using my budget and latency targets.
- Plan a benchmark for code-generation models and choose suitable evaluation metrics.
- Compare current results with the baseline and find which inputs regressed.

## Similar
- [prompt-engineer](https://affpapa.org/en/nevibecode/agents/voltagent-community-prompt-engineer.md) — subagent · 16078 installs
- [prompt-engineer](https://affpapa.org/en/nevibecode/agents/prompt-engineer.md) — subagent · 16078 installs
- [ai-engineer](https://affpapa.org/en/nevibecode/agents/voltagent-community-ai-engineer.md) — subagent · 7687 installs
- [ai-engineer](https://affpapa.org/en/nevibecode/agents/ai-engineer.md) — subagent · 7687 installs
- [search-specialist](https://affpapa.org/en/nevibecode/agents/search-specialist.md) — subagent · 6886 installs
- [business-analyst](https://affpapa.org/en/nevibecode/agents/business-analyst.md) — subagent · 3855 installs

From the same set: [davila7/claude-code-templates](https://affpapa.org/en/nevibecode/by/davila7/claude-code-templates.md) — 495 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md