metrillm — Claude Code MCP server: what it does, how to install
Benchmarks local models from Ollama or LM Studio for speed and quality, then says whether your hardware handles them. Works from any MCP client.
claude mcp add metrillm -- npx -y metrillm-mcpWhat metrillm does
The server hands your agent four tools for local LLMs. list_models shows what's installed in the runtime, and run_benchmark times a chosen model and tests its output quality. Earlier runs are stored on disk and come back through get_results. Pass perfOnly: true when you only want speed numbers. A finished run returns a verdict, an overall score, tokens per second, time to first token and memory use.
share_result takes an absolute path to a result JSON and uploads it to the public MetriLLM leaderboard. That sends data off your machine, so call it on purpose. Self-hosted or staging setups can redirect it with METRILLM_SUPABASE_URL, METRILLM_SUPABASE_ANON_KEY and METRILLM_PUBLIC_RESULT_BASE_URL.
You need Node.js 20 or newer, a running Ollama (ollama serve) or LM Studio with its Local Server switched on, and at least one model pulled there. The runtime parameter defaults to ollama. MLX, llama.cpp and vLLM are only planned for now.
What it helps with
- measure tokens per second for a model running in Ollama
- check whether your machine copes with a given local model
- compare speed across models without running the quality tests
- pull up results from earlier benchmark runs
- publish a result to the public leaderboard
Example prompts for the agent
- List the models available in Ollama through metrillm
- Run a metrillm benchmark on
llama3.2:3b - Use
run_benchmarkonllama3.2:3bwithperfOnlyset to true
How to install metrillm
Platform CLI
claude mcp add metrillm -- npx -y metrillm-mcpPlatform CLI
codex mcp add mcp -- npx -y metrillm-mcpConfig file
{
"mcpServers": {
"mcp": {
"command": "npx",
"args": [
"-y",
"metrillm-mcp"
]
}
}
}Config file
{
"servers": {
"mcp": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"metrillm-mcp"
]
}
}
}Platform CLI
gemini mcp add mcp npx -y metrillm-mcpStep by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more
Questions about metrillm
How do I install metrillm in Claude Code?
Run claude mcp add metrillm -- npx -y metrillm-mcp. Node.js 20 or newer has to be installed.
What does metrillm do?
It benchmarks a local model in Ollama or LM Studio and returns a verdict, an overall score, tokens per second and memory use. It also lists your models and fetches earlier results.
Which runtimes does metrillm support?
Ollama and LM Studio. MLX, llama.cpp and vLLM are planned. Ollama needs ollama serve running, and LM Studio needs its Local Server enabled.
