# ae-evaluation

The ae-evaluation Claude Code skill manages Agent Evaluation tasks through ae-cli. It configures and runs agent or preset-output evaluations, then helps inspect their results.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode, Windsurf (Devin Desktop), Claude.ai · 126 installs · ★ 23 repo stars
- Author: ThinkingAIAgenticEngine
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/ThinkingAIAgenticEngine/ae-cli/tree/HEAD/skills/ae-evaluation/SKILL.md
- Updated: 2026-10-22
- Security check: automated, 2026-11-17 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/ae-evaluation

## What it does

The skill manages evaluation datasets and assets on Agent Evaluation. It configures models, credentials and integrations, starts evaluation runs, and analyzes their results. It also handles recurring schedules and resource access.

Remote evaluation requires the online platform and an `AE_EVAL_ACCESS_TOKEN`. In an AE sandbox, the CLI obtains a short-lived session automatically; set `AE_EVAL_TE_PROJECT_ID` when the sandbox token can reach multiple TE projects. Outside a sandbox, a missing token means no network request is sent. Commands use the current Agent Evaluation host by default; select another with `--host`.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/ae-evaluation/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/ae-evaluation/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/ae-evaluation/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/ae-evaluation/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install ThinkingAIAgenticEngine/ae-cli skills/ae-evaluation
```

2. **Manually: put the files in place**
- File: `.github/skills/ae-evaluation/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/ThinkingAIAgenticEngine/ae-cli --path skills/ae-evaluation
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/ae-evaluation/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/ThinkingAIAgenticEngine/ae-cli/tree/HEAD/skills/ae-evaluation/SKILL.md

## When to use it
- create and publish an evaluation dataset
- configure a model and an agent under test
- run an evaluation and inspect its results
- schedule recurring evaluation runs

## Example prompts
- Create an evaluation dataset from this CSV file.
- Configure the agent and run an evaluation against the selected dataset.
- Show the results and traces from the latest run.

## Similar
- [agent-evaluation](https://affpapa.org/en/nevibecode/skills/agent-evaluation.md) — skill · 378 installs
- [evaluation](https://affpapa.org/en/nevibecode/skills/evaluation.md) — skill · 293 installs
- [evaluation-design](https://affpapa.org/en/nevibecode/skills/evaluation-design.md) — skill · ★ 1627 repo stars
- [gsd-eval-review](https://affpapa.org/en/nevibecode/skills/gsd-eval-review.md) — skill · 570 installs
- [ai-eval-playbook](https://affpapa.org/en/nevibecode/skills/ai-eval-playbook.md) — skill · 118 installs
- [agentic-eval](https://affpapa.org/en/nevibecode/skills/agentic-eval.md) — skill · 10213 installs

From the same set: [thinkingaiagenticengine/ae-cli](https://affpapa.org/en/nevibecode/by/thinkingaiagenticengine/ae-cli.md) — 25 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
