ae-evaluation — Claude Code skill: agent evaluation and setup
The ae-evaluation Claude Code skill manages Agent Evaluation tasks through ae-cli. It configures and runs agent or preset-output evaluations, then helps inspect their results.
no license 126 installs ★ 23 updated 2026-10-22
npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a claude-codeWhat ae-evaluation does
The skill manages evaluation datasets and assets on Agent Evaluation. It configures models, credentials and integrations, starts evaluation runs, and analyzes their results. It also handles recurring schedules and resource access.
Remote evaluation requires the online platform and an AE_EVAL_ACCESS_TOKEN. In an AE sandbox, the CLI obtains a short-lived session automatically; set AE_EVAL_TE_PROJECT_ID when the sandbox token can reach multiple TE projects. Outside a sandbox, a missing token means no network request is sent. Commands use the current Agent Evaluation host by default; select another with --host.
What it helps with
- create and publish an evaluation dataset
- configure a model and an agent under test
- run an evaluation and inspect its results
- schedule recurring evaluation runs
Example prompts for the agent
- Create an evaluation dataset from this CSV file.
- Configure the agent and run an evaluation against the selected dataset.
- Show the results and traces from the latest run.
How to install ae-evaluation
1Skills CLI (Vercel)
npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a claude-code2Manually: put the files in place
~/.claude/skills/ae-evaluation/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/ae-evaluation/
1Skills CLI (Vercel)
npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a codex2Manually: put the files in place
.agents/skills/ae-evaluation/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a cursor2Manually: put the files in place
.cursor/skills/ae-evaluation/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install ThinkingAIAgenticEngine/ae-cli skills/ae-evaluation2Manually: put the files in place
.github/skills/ae-evaluation/SKILL.md1Skills CLI (Vercel)
npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a gemini-cli2Platform CLI
gemini skills install https://github.com/ThinkingAIAgenticEngine/ae-cli --path skills/ae-evaluationManually: put the files in place
.devin/skills/ae-evaluation/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about ae-evaluation
How do I install ae-evaluation in Claude Code?
Run npx skills add ThinkingAIAgenticEngine/ae-cli --skill ae-evaluation -a claude-code. The skill file is at ~/.claude/skills/ae-evaluation/SKILL.md.
What does ae-evaluation do?
It manages Agent Evaluation assets, configures and runs evaluations for agents or preset outputs, and helps analyze run results.
What does ae-evaluation need to run?
It requires access to the online Agent Evaluation platform and an AE_EVAL_ACCESS_TOKEN. In an AE sandbox, the CLI obtains a session token automatically; use AE_EVAL_TE_PROJECT_ID to select a TE project when the token can access several.