# Okareo

Brings Okareo's evaluation tools into your editor: scenarios, test runs, quality checks and multi-turn simulations of voice agents.

- MCP server · `type=mcp` · Claude Code, Codex, Cursor, GitHub Copilot, Windsurf (Devin Desktop), Cline, Zed, Gemini CLI
- Author: okareo-ai
- License: Apache-2.0 · policy=link-only · description only; the file stays with the author
- Source: https://github.com/okareo-ai/okareo-mcp
- Updated: 2026-10-15
- HTML: https://affpapa.org/en/nevibecode/mcp/okareo

## What it does

Okareo publishes its evaluation features as MCP tools. From the editor you can build a scenario out of input and result rows, register a generation model, run a quality test against a scenario with chosen checks, and read the results back. A separate set of tools covers simulations, where a target can be a generation model, a custom endpoint, or a voice service such as OpenAI, Deepgram or Twilio.

The hosted endpoint is the shortest route. Point your MCP config at `https://tools.okareo.com/mcp`; on the first connection the assistant opens a browser for Okareo sign-in, and the client keeps the token afterwards, so nothing secret lands in `.mcp.json`. Older clients and headless setups fall back to a bearer header with your API key, and the env-var form is the better choice there.

Running the server locally makes sense for airgapped machines or work on the repo itself. That path wants Python 3.10–3.12 and `OKAREO_API_KEY` set in your shell, then `uvx okareo-mcp`, or `pip install okareo-mcp` if `uv` is missing. With access to several Okareo organizations, the hosted endpoint also exposes `list_tenants` and `switch_tenant`; locally you clear authentication and sign in again as the account you need.

## Requirements
- Remote server: `https://tools.okareo.com/mcp`

## Install

### Claude Code (native platform)

**Platform CLI: remote server**
```bash
claude mcp add --transport http okareo https://tools.okareo.com/mcp
```

### Codex

**Platform CLI: remote server**
```bash
codex mcp add okareo --url https://tools.okareo.com/mcp
```

### Cursor

**Config file**
File `.cursor/mcp.json`:
```json
{
    "mcpServers": {
        "okareo": {
            "url": "https://tools.okareo.com/mcp"
        }
    }
}
```

### GitHub Copilot

**Config file**
File `.vscode/mcp.json`:
```json
{
    "servers": {
        "okareo": {
            "type": "http",
            "url": "https://tools.okareo.com/mcp"
        }
    }
}
```

### Gemini CLI

**Platform CLI: remote server**
```bash
gemini mcp add -t http okareo https://tools.okareo.com/mcp
```

## When to use it
- turn input and result rows into a named scenario
- score a model against a scenario with a set of checks
- pull the transcript of one conversation from a run
- re-score a finished run against different checks
- set up a voice target for a multi-turn simulation

## Example prompts
- Save a scenario `checkout_flow` from these rows and tell me how many it holds
- Run a test of the registered model against that scenario and show the summary
- Generate a check from the description “answer never invents prices” and save it

## Similar
- [gaia-architecture-comparison](https://affpapa.org/en/nevibecode/skills/gaia-architecture-comparison.md) — skill · 432 installs
- [ai-evals](https://affpapa.org/en/nevibecode/skills/ai-evals.md) — skill · 2100 installs
- [debugging-livekit-agents](https://affpapa.org/en/nevibecode/skills/debugging-livekit-agents.md) — skill · 399 installs
- [agent-evaluation](https://affpapa.org/en/nevibecode/skills/neolabhq-agent-evaluation.md) — skill · 1300 installs
- [create-assistant](https://affpapa.org/en/nevibecode/skills/create-assistant.md) — skill · 1100 installs
- [agent-observability-eval-bootstrap](https://affpapa.org/en/nevibecode/skills/agent-observability-eval-bootstrap.md) — skill · 547 installs

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md