Scorecard — Claude Code MCP server: testing LLM apps
Scorecard — an MCP server that tests and evaluates LLM apps: scenario checks, a running view of model behaviour and performance numbers, all from inside your AI client.
no license updated 2026-10-29
Open on GitHubWhat Scorecard does
Scorecard is an MCP server for evaluating LLM apps: scenario tests plus a running view of how the model behaves. It earns its place once the app is built and you need to know what it actually answers.
Benchmarks for performance sit alongside the tests and monitoring. The server is listed for Claude, Codex, Cursor, GitHub Copilot, Windsurf (Devin Desktop), Cline, Zed and Gemini CLI.
docker is behind it. Stay in one client and never switch, and the rest of that list does nothing for you.
What it helps with
- run an LLM app through structured test scenarios
- keep watching model behaviour after a change ships
- benchmark performance and compare runs
- evaluate an LLM app before release
Example prompts for the agent
- Run my LLM service through
Scorecardtests and list what broke. - Pull the performance numbers from
Scorecardfor the latest build. - Start
Scorecardmonitoring on the model responses and tell me what shifted today.
How to install Scorecard
Install
The MCP registry has no launch command for it: see the repository README
Step by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more
Questions about Scorecard
What does Scorecard do?
It runs an LLM app through structured tests, watches how the model behaves and collects performance numbers. All of that happens inside the AI client the server is connected to.
Which clients support Scorecard?
Claude, Codex, Cursor, GitHub Copilot, Windsurf (Devin Desktop), Cline, Zed and Gemini CLI.
Is Scorecard a plugin or an MCP server?
It is an MCP server from docker. It plugs into an AI client and exposes testing, monitoring and benchmarking tools for LLM apps.