Skip to main content
MCP server CX Codex

Scorecard — Claude Code MCP server: testing LLM apps

Scorecard — an MCP server that tests and evaluates LLM apps: scenario checks, a running view of model behaviour and performance numbers, all from inside your AI client.

no license updated 2026-10-29

Open on GitHub

What Scorecard does

Scorecard is an MCP server for evaluating LLM apps: scenario tests plus a running view of how the model behaves. It earns its place once the app is built and you need to know what it actually answers.

Benchmarks for performance sit alongside the tests and monitoring. The server is listed for Claude, Codex, Cursor, GitHub Copilot, Windsurf (Devin Desktop), Cline, Zed and Gemini CLI.

docker is behind it. Stay in one client and never switch, and the rest of that list does nothing for you.

What it helps with

  • run an LLM app through structured test scenarios
  • keep watching model behaviour after a change ships
  • benchmark performance and compare runs
  • evaluate an LLM app before release

Example prompts for the agent

  • Run my LLM service through Scorecard tests and list what broke.
  • Pull the performance numbers from Scorecard for the latest build.
  • Start Scorecard monitoring on the model responses and tell me what shifted today.

How to install Scorecard

Install

The MCP registry has no launch command for it: see the repository README

Step by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more

Questions about Scorecard

What does Scorecard do?

It runs an LLM app through structured tests, watches how the model behaves and collects performance numbers. All of that happens inside the AI client the server is connected to.

Which clients support Scorecard?

Claude, Codex, Cursor, GitHub Copilot, Windsurf (Devin Desktop), Cline, Zed and Gemini CLI.

Is Scorecard a plugin or an MCP server?

It is an MCP server from docker. It plugs into an AI client and exposes testing, monitoring and benchmarking tools for LLM apps.

Similar MCP servers

All MCP servers for Testing and QA →

Project sponsors

Companies that keep this analytics open