# metrillm

Benchmarks local models from Ollama or LM Studio for speed and quality, then says whether your hardware handles them. Works from any MCP client.

- MCP server · `type=mcp` · Claude Code, Codex, Cursor, GitHub Copilot, Windsurf (Devin Desktop), Cline, Zed, Gemini CLI · ★ 8
- Author: MetriLLM
- License: Apache-2.0 · policy=link-only · description only; the file stays with the author
- Source: https://github.com/MetriLLM/metrillm/tree/HEAD/mcp
- Updated: 2026-06-02
- HTML: https://affpapa.org/en/nevibecode/mcp/metrillm

## What it does

The server hands your agent four tools for local LLMs. `list_models` shows what's installed in the runtime, and `run_benchmark` times a chosen model and tests its output quality. Earlier runs are stored on disk and come back through `get_results`. Pass `perfOnly: true` when you only want speed numbers. A finished run returns a verdict, an overall score, tokens per second, time to first token and memory use.

\`share_result` takes an absolute path to a result JSON and uploads it to the public MetriLLM leaderboard. That sends data off your machine, so call it on purpose. Self-hosted or staging setups can redirect it with `METRILLM_SUPABASE_URL`, `METRILLM_SUPABASE_ANON_KEY` and `METRILLM_PUBLIC_RESULT_BASE_URL`.

You need Node.js 20 or newer, a running Ollama (`ollama serve`) or LM Studio with its Local Server switched on, and at least one model pulled there. The `runtime` parameter defaults to `ollama`. MLX, llama.cpp and vLLM are only planned for now.

## Requirements
- Needs: Node.js (commands run via npx)

## Install

### Claude Code (native platform)

**Platform CLI**
```bash
claude mcp add metrillm -- npx -y metrillm-mcp
```

### Codex

**Platform CLI**
```bash
codex mcp add mcp -- npx -y metrillm-mcp
```

### Cursor

**Config file**
File `.cursor/mcp.json`:
```json
{
    "mcpServers": {
        "mcp": {
            "command": "npx",
            "args": [
                "-y",
                "metrillm-mcp"
            ]
        }
    }
}
```

### GitHub Copilot

**Config file**
File `.vscode/mcp.json`:
```json
{
    "servers": {
        "mcp": {
            "type": "stdio",
            "command": "npx",
            "args": [
                "-y",
                "metrillm-mcp"
            ]
        }
    }
}
```

### Gemini CLI

**Platform CLI**
```bash
gemini mcp add mcp npx -y metrillm-mcp
```

## When to use it
- measure tokens per second for a model running in Ollama
- check whether your machine copes with a given local model
- compare speed across models without running the quality tests
- pull up results from earlier benchmark runs
- publish a result to the public leaderboard

## Example prompts
- List the models available in Ollama through metrillm
- Run a metrillm benchmark on `llama3.2:3b`
- Use `run_benchmark` on `llama3.2:3b` with `perfOnly` set to true

## Similar
- [MCP Toolbox for Databases](https://affpapa.org/en/nevibecode/mcp/googleapis-mcp-toolbox-for-databases.md) — MCP server · ★ 16730
- [labelhead-artist-momentum](https://affpapa.org/en/nevibecode/mcp/labelhead-artist-momentum.md) — MCP server · ★ 107701
- [Agent Skills Search Server](https://affpapa.org/en/nevibecode/mcp/agent-skills-search-server.md) — MCP server · ★ 26673
- [Skyvern](https://affpapa.org/en/nevibecode/mcp/skyvern.md) — MCP server · ★ 23283
- [Nhost](https://affpapa.org/en/nevibecode/mcp/nhost.md) — MCP server · ★ 9369
- [PraisonAI](https://affpapa.org/en/nevibecode/mcp/praisonai.md) — MCP server · ★ 9298

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md