evaluate-plugin — Claude skill: audit a plugin, get a report
Walks through a local Claude plugin and tells you what to fix first. Useful before you compare versions.
npx skills add openai/plugins --skill evaluate-plugin -a codexWhat evaluate-plugin does
The skill targets a plugin root that contains .codex-plugin/plugin.json. A plain request like "evaluate this plugin" is the default entrypoint: it runs plugin-eval start <plugin-root> --request "<user request>" --format markdown so you can see the routed local path.
Then comes plugin-eval analyze <plugin-root> --format markdown. Read the Fix First block before digging into manifest findings, nested skill findings, or code coverage. When a plugin holds several skills, the skill names the strongest and the weakest one outright.
Want measured usage instead of a static read? Switch to the benchmark flow: plugin-eval init-benchmark <plugin-root>, then plugin-eval benchmark <plugin-root> --dry-run. Trend data comes from plugin-eval compare before.json after.json. One catch: none of this runs without plugin-eval on your machine, and the HTML report is a separate step — plugin-eval report result.json --format html --output ./plugin-eval-report.html.
What it helps with
- audit a local Claude plugin and get a structured report
- find out why a plugin scored the way it did
- pick the first thing to fix
- compare two plugin versions from JSON output
- produce an HTML report for the team
Example prompts for the agent
- Evaluate this plugin.
- What should I fix first?
- Help me benchmark this plugin.
How to install evaluate-plugin
1Skills CLI (Vercel)
npx skills add openai/plugins --skill evaluate-plugin -a codex2Manually: put the files in place
.agents/skills/evaluate-plugin/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add openai/plugins --skill evaluate-plugin -a claude-code2Manually: put the files in place
~/.claude/skills/evaluate-plugin/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/evaluate-plugin/
Manually: put the files in place
.cursor/skills/evaluate-plugin/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install openai/plugins evaluate-plugin2Manually: put the files in place
.github/skills/evaluate-plugin/SKILL.mdPlatform CLI
gemini skills install https://github.com/openai/plugins --path plugins/plugin-eval/skills/evaluate-pluginManually: put the files in place
.devin/skills/evaluate-plugin/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: Codex skills: how to install, where they live, how to call them
Questions about evaluate-plugin
How do I install evaluate-plugin in Claude?
Use the skills CLI: npx skills add openai/plugins --skill evaluate-plugin, then the platform-specific form npx skills add openai/plugins --skill evaluate-plugin -a codex. The skill file lands at .agents/skills/evaluate-plugin/SKILL.md.
What does evaluate-plugin do?
It runs a local Claude plugin through plugin-eval, reporting on the manifest, nested skills and code coverage, and leads with a Fix First block of priorities.
How is analysis different from benchmarking?
plugin-eval analyze gives a static report on the plugin, while benchmarking measures usage: plugin-eval init-benchmark <plugin-root> first, then plugin-eval benchmark <plugin-root> --dry-run.
Source text
SKILL.md · MIT
--- name: evaluate-plugin description: Evaluate a local Codex plugin in engineer-friendly language. Use when the user says "evaluate this plugin", "audit this plugin", "why did this score that way", "what should I fix first", "help me benchmark this plugin", or asks for a plugin-wide report before comparing versions. --- # Evaluate Plugin Use this skill when the target is a plugin root with `.codex-plugin/plugin.json`. ## Workflow 1. Treat "Evaluate this plugin." as the default entrypoint. 2. If the request comes in as natural chat language, use `plugin-eval start <plugin-root> --request "<user request>" --format markdown` first so the user sees the routed local path. 3. Run `plugin-eval analyze <plugin-root> --format markdown`. 4. Read `Fix First` before drilling into manifest findings, nested skill findings, and code or coverage details. 5. If the plugin contains multiple skills, summarize the strongest and weakest ones explicitly. 6. If the user wants measured usage, switch to "Help me benchmark this plugin." and use the starter benchmark flow. 7. If the user wants trend data, compare two JSON outputs with `plugin-eval compare`. ## Chat Requests To Recognize - `Evaluate this plugin.` - `Audit this plugin.` - `Why did this score that way?` - `What should I fix first?` - `Help me benchmark this plugin.` - `What should I run next?` ## Commands ```bash plugin-eval start <plugin-root> --request "Evaluate this plugin." --format markdown plugin-eval analyze <plugin-root> --format markdown plugin-eval start <plugin-root> --request "What should I run next?" --format markdown plugin-eval compare before.json after.json plugin-eval report result.json --format html --output ./plugin-eval-report.html plugin-eval init-benchmark <plugin-root> plugin-eval benchmark <plugin-root> --dry-run ``` ## Reference - `../../references/chat-first-workflows.md`
© OpenAI Codex · MIT · openai/plugins
