evalview-mcp — Claude Code MCP server for agent testing
Capture an AI agent’s current behavior and compare later runs against it. Useful for spotting changes to tool calls or answer quality after an update.
claude mcp add evalview-mcp -e OPENAI_API_KEY=<OPENAI_API_KEY> -- uvx evalviewWhat evalview-mcp does
EvalView records an agent run as a baseline, then checks later runs against it. The comparison covers tool names, parameters, and call order, so a prompt edit that changes the agent’s actions can show up even when the final response looks acceptable.
Tool and sequence comparisons run offline without an API key. You can add an LLM judge to score output quality, while running the agent itself may still incur provider charges. Setting up the server in Claude Code uses OPENAI_API_KEY, so have that key ready.
What it helps with
- compare agent runs with a saved baseline
- spot changes in tool-call order
- catch lower answer quality after a prompt edit
- check agent behavior before merging changes
Example prompts for the agent
- Use
evalview-mcpto compare my agent’s latest run with its saved baseline. - Check whether the agent changed its tools or the order of its calls.
- Assess whether the agent’s answers lost quality after my last change.
How to install evalview-mcp
Platform CLI
claude mcp add evalview-mcp -e OPENAI_API_KEY=<OPENAI_API_KEY> -- uvx evalviewReplace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from evalview <command>
Platform CLI
codex mcp add evalview-mcp --env OPENAI_API_KEY=<OPENAI_API_KEY> -- uvx evalviewReplace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from evalview <command>
Config file
{
"mcpServers": {
"evalview-mcp": {
"command": "uvx",
"args": [
"evalview"
],
"env": {
"OPENAI_API_KEY": "<OPENAI_API_KEY>"
}
}
}
}Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from evalview <command>
Config file
{
"servers": {
"evalview-mcp": {
"type": "stdio",
"command": "uvx",
"args": [
"evalview"
],
"env": {
"OPENAI_API_KEY": "<OPENAI_API_KEY>"
}
}
}
}Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from evalview <command>
Platform CLI
gemini mcp add -e OPENAI_API_KEY=<OPENAI_API_KEY> evalview-mcp uvx evalviewReplace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from evalview <command>
Step by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more
Questions about evalview-mcp
How do I install evalview-mcp in Claude Code?
Run claude mcp add evalview-mcp -e OPENAI_API_KEY=<OPENAI_API_KEY> -- uvx evalview, replacing <OPENAI_API_KEY> with your key.
What does evalview-mcp do?
It helps record an agent’s behavior as a baseline and compare later runs against it, including tool calls and answer quality.
Does checking an agent require an API key?
Comparing tool calls and their order works without one. An LLM judge for output-quality scoring needs an API key, and running the agent may incur provider charges.
