Okareo — Claude Code MCP server: evaluate voice agents
Brings Okareo's evaluation tools into your editor: scenarios, test runs, quality checks and multi-turn simulations of voice agents.
claude mcp add --transport http okareo https://tools.okareo.com/mcpWhat Okareo does
Okareo publishes its evaluation features as MCP tools. From the editor you can build a scenario out of input and result rows, register a generation model, run a quality test against a scenario with chosen checks, and read the results back. A separate set of tools covers simulations, where a target can be a generation model, a custom endpoint, or a voice service such as OpenAI, Deepgram or Twilio.
The hosted endpoint is the shortest route. Point your MCP config at https://tools.okareo.com/mcp; on the first connection the assistant opens a browser for Okareo sign-in, and the client keeps the token afterwards, so nothing secret lands in .mcp.json. Older clients and headless setups fall back to a bearer header with your API key, and the env-var form is the better choice there.
Running the server locally makes sense for airgapped machines or work on the repo itself. That path wants Python 3.10–3.12 and OKAREO_API_KEY set in your shell, then uvx okareo-mcp, or pip install okareo-mcp if uv is missing. With access to several Okareo organizations, the hosted endpoint also exposes list_tenants and switch_tenant; locally you clear authentication and sign in again as the account you need.
What it helps with
- turn input and result rows into a named scenario
- score a model against a scenario with a set of checks
- pull the transcript of one conversation from a run
- re-score a finished run against different checks
- set up a voice target for a multi-turn simulation
Example prompts for the agent
- Save a scenario
checkout_flowfrom these rows and tell me how many it holds - Run a test of the registered model against that scenario and show the summary
- Generate a check from the description “answer never invents prices” and save it
How to install Okareo
Platform CLI: remote server
claude mcp add --transport http okareo https://tools.okareo.com/mcpPlatform CLI: remote server
codex mcp add okareo --url https://tools.okareo.com/mcpConfig file
{
"mcpServers": {
"okareo": {
"url": "https://tools.okareo.com/mcp"
}
}
}Config file
{
"servers": {
"okareo": {
"type": "http",
"url": "https://tools.okareo.com/mcp"
}
}
}Platform CLI: remote server
gemini mcp add -t http okareo https://tools.okareo.com/mcpStep by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more
Questions about Okareo
How do I install Okareo in Claude Code?
Run claude mcp add --transport http okareo https://tools.okareo.com/mcp, then reload the assistant. The first connection opens a browser for Okareo sign-in.
What does Okareo do?
It gives the assistant Okareo tools for scenarios, generation models, quality checks and multi-turn simulations, all callable from the editor.
Do I need Python for the hosted endpoint?
No. Python 3.10–3.12 is only needed for local install modes; the remote endpoint runs without it.
