# token-compressor

Shortens prompts by 40–60% using local models and guards against semantic drift with vector similarity checks before sending text to LLMs.

- MCP server · `type=mcp` · Claude Code, Codex, Cursor, GitHub Copilot, Windsurf (Devin Desktop), Cline, Zed, Gemini CLI · ★ 12
- Author: base76-research-lab
- License: MIT · policy=link-only · description only; the file stays with the author
- Source: https://github.com/base76-research-lab/token-compressor
- Updated: 2026-03-08
- HTML: https://affpapa.org/en/nevibecode/mcp/token-compressor

## What it does

The server cuts prompt token size through a two-step local workflow. First, a local model cleans out conversational padding and redundancy while keeping negations and conditional clauses intact.

Next, an embedding check computes cosine similarity between the draft and the source text. If the score falls below the 0.85 threshold, the pipeline discards the rewrite and returns original text unchanged. Requests under 80 tokens bypass the process entirely.

Without a local Ollama daemon hosting `llama3.2:1b` and `nomic-embed-text`, every tool execution will fail. The package also requires Python 3.10+ alongside `numpy` and `ollama`.

## Requirements
- Needs: Python and uv (commands run via uvx)

## Install

### Claude Code (native platform)

**Platform CLI**
```bash
claude mcp add token-compressor -- uvx token-compressor-mcp
```
uvx runs the command named after the package; if the README names another one, use uvx --from token-compressor-mcp <command>

### Codex

**Platform CLI**
```bash
codex mcp add token-compressor -- uvx token-compressor-mcp
```
uvx runs the command named after the package; if the README names another one, use uvx --from token-compressor-mcp <command>

### Cursor

**Config file**
File `.cursor/mcp.json`:
```json
{
    "mcpServers": {
        "token-compressor": {
            "command": "uvx",
            "args": [
                "token-compressor-mcp"
            ]
        }
    }
}
```
uvx runs the command named after the package; if the README names another one, use uvx --from token-compressor-mcp <command>

### GitHub Copilot

**Config file**
File `.vscode/mcp.json`:
```json
{
    "servers": {
        "token-compressor": {
            "type": "stdio",
            "command": "uvx",
            "args": [
                "token-compressor-mcp"
            ]
        }
    }
}
```
uvx runs the command named after the package; if the README names another one, use uvx --from token-compressor-mcp <command>

### Gemini CLI

**Platform CLI**
```bash
gemini mcp add token-compressor uvx token-compressor-mcp
```
uvx runs the command named after the package; if the README names another one, use uvx --from token-compressor-mcp <command>

## When to use it
- cut LLM prompt token consumption
- strip wordy boilerplate from system prompts
- keep conditional rules intact during prompt rewrites
- prevent semantic drift using embedding score checks

## Example prompts
- Run `compress_prompt` on this draft instruction and check the token count
- Shorten this long workflow context via token-compressor before submitting

## Similar
- [REA](https://affpapa.org/en/nevibecode/mcp/rea.md) — MCP server · ★ 392262
- [GitHub](https://affpapa.org/en/nevibecode/mcp/github.md) — MCP server · ★ 33989
- [Netdata](https://affpapa.org/en/nevibecode/mcp/netdata.md) — MCP server · ★ 81163
- [browser-use](https://affpapa.org/en/nevibecode/mcp/browser-use.md) — MCP server · ★ 119231
- [Codebase Memory](https://affpapa.org/en/nevibecode/mcp/codebase-memory.md) — MCP server · ★ 47577
- [mcp-server](https://affpapa.org/en/nevibecode/mcp/mcp-server.md) — MCP server · ★ 43902

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md