Skip to main content
plugin

llm-transpile — Claude Code plugin: what it does, how to install

llm-transpile compresses Markdown, HTML, and plain text files whenever Claude Code reads them. Keeps context clean without manual preparation steps.

Apache-2.0 ★ 9 updated 2026-10-24

Install into Claude Code · Plugin marketplace All methods and platforms ↓
/plugin marketplace add anthropics/claude-plugins-community
/plugin install llm-transpile@claude-community
Open on GitHub

What llm-transpile does

The plugin intercepts document reads. It turns Markdown, HTML, and plain text into a structured bridge format. The parser preserves headings, lists, and code blocks while linearizing tables into key-value pairs. Repetitive domain words get mapped to compact symbols backed by a dictionary header.

Compression automatically steps through four escalating stages as token budgets shrink. For plain text files, syntactic overhead can slightly increase token usage in lighter modes, so the strongest savings occur on HTML documents and deep Markdown trees.

Built with Rust, the tool integrates through a PreToolUse hook. Claude Code registers the hook automatically upon starting the next session.

What it helps with

  • cut context consumption when reading project documentation
  • strip markup and styling noise from HTML pages
  • linearize Markdown tables into compact key-value structures
  • replace recurring terminology with dictionary tokens

Example prompts for the agent

  • "Read docs/api.md and summarize the authentication flow."
  • "Inspect index.html and outline the core interface elements."

How to install llm-transpile

Plugin marketplace

/plugin marketplace add anthropics/claude-plugins-community
/plugin install llm-transpile@claude-community

Step by step, with commands for each platform: Claude Code plugins: marketplaces, installing, and building your own

Questions about llm-transpile

How to install llm-transpile in Claude Code?

Run /plugin marketplace add anthropics/claude-plugins-community, then run /plugin install llm-transpile@claude-community. The PreToolUse hook registers automatically on your next session start.

What does llm-transpile do?

It intercepts file read events in Claude Code and converts documents into a compressed intermediate format. This preserves context budget for actual programming tasks.

How does adaptive compression work?

The transpiler escalates through four stages as the available token window narrows. Common terms are moved into a dictionary header to minimize repetitive token overhead.

Source text

SKILL.md · Apache-2.0
<div align="center">
<h1>llm-transpile</h1> 

<p align="center">
  <a href="https://github.com/epicsagas/llm-transpile/stargazers"><img alt="Stars" src="https://img.shields.io/github/stars/epicsagas/llm-transpile?style=for-the-badge&labelColor=0d1117&color=ffd700&logo=github&logoColor=white" /></a>
  <a href="https://github.com/epicsagas/llm-transpile/network/members"><img alt="Forks" src="https://img.shields.io/github/forks/epicsagas/llm-transpile?style=for-the-badge&labelColor=0d1117&color=2ecc71&logo=github&logoColor=white" /></a>
  <a href="https://github.com/epicsagas/llm-transpile/issues"><img alt="Issues" src="https://img.shields.io/github/issues/epicsagas/llm-transpile?style=for-the-badge&labelColor=0d1117&color=ff6b6b&logo=github&logoColor=white" /></a>
  <a href="https://github.com/epicsagas/llm-transpile/commits/main"><img alt="Last commit" src="https://img.shields.io/github/last-commit/epicsagas/llm-transpile?style=for-the-badge&labelColor=0d1117&color=58a6ff&logo=git&logoColor=white" /></a>
</p>
<p align="center">
  <a href="https://crates.io/crates/llm-transpile"><img alt="Crates.io" src="https://img.shields.io/crates/v/llm-transpile?style=for-the-badge&labelColor=0d1117&color=fc8d62&logo=rust&logoColor=white" /></a>
  <a href="https://docs.rs/llm-transpile"><img alt="docs.rs" src="https://img.shields.io/docsrs/llm-transpile?style=for-the-badge&labelColor=0d1117&color=8e44ad&logo=docsdotrs&logoColor=white" /></a>
  <a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-Apache--2.0-3fb950?style=for-the-badge&labelColor=0d1117" /></a>
  <a href="https://blog.rust-lang.org/"><img alt="Rust" src="https://img.shields.io/badge/rust-1.92+-d73a49?style=for-the-badge&labelColor=0d1117&logo=rust&logoColor=white" /></a>
  <a href="https://buymeacoffee.com/epicsaga"><img alt="Buy Me a Coffee" src="https://img.shields.io/badge/buy_me_a_coffee-FFDD00?style=for-the-badge&labelColor=0d1117&logo=buymeacoffee&logoColor=black" /></a>
</p>

**Token-optimized document transpiler for LLM pipelines**

[한국어](docs/i18n/README.ko.md) · [日本語](docs/i18n/README.ja.md) · [中文](docs/i18n/README.zh.md) · [Español](docs/i18n/README.es.md) · [Français](docs/i18n/README.fr.md) · [Deutsch](docs/i18n/README.de.md) · [Português](docs/i18n/README.pt.md) · [Русский](docs/i18n/README.ru.md) · [العربية](docs/i18n/README.ar.md) · [हिन्दी](docs/i18n/README.hi.md)

</div>

Raw documents (Markdown, HTML, plain text) → structured bridge format `<D>?<H><B>` — with adaptive compression that keeps you under token budget.

---

<details>
<summary>Table of Contents</summary>

- [Why](#why)
- [Installation](#installation)
- [Updating](#updating)
- [CLI Usage](#cli-usage)
- [Usage Statistics](#usage-statistics)
- [Benchmarking](#benchmarking)
- [Library Usage](#library-usage)
- [Output Format](#output-format)
- [Fidelity Levels](#fidelity-levels)
- [Adaptive Compression](#adaptive-compression)
- [Input Formats](#input-formats)
- [Error Handling](#error-handling)
- [Performance](#performance)
- [Contributing](#contributing)
- [License](#license)

</details>

---

## Why

LLMs perform better when context is clean and dense. This library handles the mechanical work:

| | Feature | Why it matters |
|--|---------|----------------|
| 🏗️ | **Structural parsing** | Markdown/HTML/plain text → typed IR nodes (headings, paragraphs, tables, lists, code blocks) |
| 📉 | **Adaptive compression** | Automatically escalates through 4 stages as token budget fills up |
| 🔣 | **Symbol substitution** | Repeated domain terms → Unicode PUA characters, decoded by `<D>` dictionary header |
| 📊 | **Table linearization** | Markdown tables → compact `Key:Val` (≤5 rows) or pipe-separated rows for larger tables |
| 🌊 | **Streaming output** | Tokio stream delivers the first chunk immediately, minimizing TTFT |

### Benchmarks

48 documents, 3 formats, 15 languages — Apple M-series, `--release` build. Numbers below are measured with the **real `cl100k` BPE tokenizer** (not the self-referential heuristic — see the analysis). Full methodology and token-honesty breakdown: [`docs/EVALUATION.md`](docs/EVALUATION.md)

| Format | Semantic reduction | Compressed reduction | Lossless word coverage | Throughput |
|--------|-------------------:|--------------------:|----------------------:|-----------:|
| Markdown | 27.4% | 69.4% | 99.0% | — |
| HTML | 98.7% | 99.3% | 99.0% | — |
| PlainText | −3.5% | 30.4% | 99.0% | — |
| **Overall (BPE)** | **81.5%** | **91.8%** | **99.0%** | **~1,070 tok/ms** |

> ⚠️ The overall figure is dominated by HTML markup stripping. **Markdown 27.4% is the genuine compression rate.** PlainText is net-negative in Semantic mode due to structural overhead. See [`docs/EVALUATION.md`](docs/EVALUATION.md) for the per-format reality.

> HTML reduction reflects markup overhead removal (nav, scripts, styles), not prose compression alone.

---

## Installation

### Claude Code

```
/plugin marketplace add epicsagas/plugins
/plugin install transpile@epicsagas
```

Auto-installs the binary and seeds the PreToolUse hook on next session start — no additional setup required.

### Codex CLI

```bash
codex plugin marketplace add epicsagas/plugins
```

The PreToolUse hook is registered automatically — no further steps needed.

### macOS / Linux

```bash
brew install epicsagas/tap/llm-transpile
```

No Homebrew? Use the installer script:

```bash
curl --proto '=https' --tlsv1.2 -LsSf \
  https://github.com/epicsagas/llm-transpile/releases/latest/download/install.sh | sh
```

### Windows

```powershell
irm https://github.com/epicsagas/llm-transpile/releases/latest/download/install.ps1 | iex
```

### Via Rust toolchain

```bash
cargo binstall llm-transpile   # pre-built binary (fast)
cargo install llm-transpile    # build from source
```

### After installing

Configure tool integrations:

```bash
transpile install
```

`transpile install` launches an interactive wizard that detects and configures whichever tools are installed:

| Tool | Integration method | What it does |
|------|--------------------|--------------|
| **Antigravity** | `SKILL.md` | LLM auto-invokes `transpile` on document file extensions |
| **Cursor** | `.mdc` rule (`alwaysApply`) | Triggers `transpile` before reading document files |
| **OpenCode** | `SKILL.md` | LLM auto-invokes `transpile` on document file extensions |
| **Cline** | `SKILL.md` | LLM auto-invokes `transpile` on document file extensions |

All tools use a skill file that teaches the LLM to run `TRANSPILE_AGENT=<agent> transpile --input <file>` automatically — no size check needed, extension alone triggers it.

**Selective install / uninstall**

```bash
transpile install antigravity cursor    # specific tools only
transpile install --all                 # everything at once
transpile install --dry-run             # preview what would change
transpile install --list                # show status of all integrations

transpile uninstall cursor         # remove one
transpile uninstall --all          # remove everything
transpile uninstall --dry-run      # preview removals
```

### Library (Rust crate)

```toml
[dependencies]
llm-transpile = "0.1"
```

Requires **Rust 1.92+**.

**Antigravity (Gemini CLI)**

```bash
agy plugins install https://github.com/epicsagas/llm-transpile
```

Auto-installs the plugin (hooks) and registers it on next session start.

---

## Updating

| Method | Command |
|--------|---------|
| Homebrew | `brew upgrade llm-transpile` |
| curl / PowerShell installer | Re-run the install command above |
| cargo binstall | `cargo binstall llm-transpile@latest` |
| cargo install | `cargo install llm-transpile@latest` |

```bash
transpile --version
```

---

## CLI Usage

```
transpile [OPTIONS]

Options:
  -i, --input <FILE>       Input file path (reads from stdin if omitted)
  -f, --format <FORMAT>    Input format: markdown | html | plaintext  [default: markdown]
                           Auto-detected from file extension when --input is used
  -l, --fidelity <LEVEL>   Compression level: lossless | semantic | compressed  [default: semantic]
  -b, --budget <N>         Token budget upper limit (unlimited if omitted)
  -c, --count              Print only the input token count, then exit
  -j, --json               Output as JSON {input_tok, output_tok, reduction_pct, content}
  -q, --quiet              Suppress the stats line on stderr
      --stats              Print stats line to stdout after content (single-stream capture)
  -h, --help               Print help
  -V, --version            Print version
```

**Examples**

```bash
# Convert a Markdown file (format auto-detected from .md extension)
transpile --input doc.md

# Read from stdin — clean stdout, stats on stderr
cat doc.html | transpile --format html --fidelity compressed --budget 1024

# Pipe cleanly — suppress stats entirely
transpile --input doc.md --quiet | send_to_llm_api

# Check token count without converting
transpile --input doc.md --count

# JSON output for scripts and pipelines
transpile --input doc.md --json | jq '.reduction_pct'

# Capture content + stats in one stream (stdout)
transpile --input doc.md --stats > output_with_stats.txt

# Lossless — no compression, full content preserved (legal/audit docs)
transpile --input contract.md --fidelity lossless

# Aggressive compression into a 512-token budget
transpile --input article.md --fidelity compressed --budget 512
```

> Stats (`[273 → 150 tok  45.1% reduction]`) are written to **stderr** by default, so stdout stays clean for piping. Use `--quiet` to suppress, or `--stats` to redirect to stdout.

---

## Usage Statistics

Every `transpile` invocation automatically appends a record to `~/.agents/transpile/stats/YYYY-MM-DD.jsonl`.

### ASCII table

```bash
transpile stats show                # today
transpile stats show --days 7       # last N days
transpile stats show --agent claude # filter by agent
```

Example output:

```
transpile stats — last 7 days

  Date          Agent         Calls   Input tok  Output tok    Saved  Reduction
  ──────────────────────────────────────────────────────────────────────────
  2026-05-18                    238   4 999 355   4 248 769  750 586      15.0%
  2026-05-19                    390   1 577 739   1 463 504  114 235       7.2%
  2026-05-20                    288   2 148 207   1 836 916  311 291      14.5%
  2026-05-21                     99     635 313     544 709   90 604      14.3%
  2026-05-22                    299   8 328 530   7 732 860  595 670       7.2%
  2026-05-23                    418  15 939 148  13 501 134  2 438 014      15.3%
  2026-05-24                    186   3 313 950   2 782 467  531 483      16.0%
  ──────────────────────────────────────────────────────────────────────────
  Total                        1919  36 942 242  32 110 359  4 831 883      13.1%
```

### HTML dashboard

```bash
transpile stats report                 # opens in browser (default: last 7 days)
transpile stats report --days 30       # last 30 days
transpile stats report --no-open       # generate without opening
transpile stats report --out /tmp/custom.html
```

> Reports are generated at `~/.agents/transpile/reports/` by default. Override with `--out`.

**JSONL record fields**

| Field | Type | Description |
|-------|------|-------------|
| `ts` | ISO 8601 | Timestamp of the invocation |
| `agent` | string | Tool that triggered the call (`claude`, `antigravity`, `codex`, `opencode`) |
| `file` | string | Input file path (empty when reading from stdin) |
| `format` | string | `markdown`, `html`, or `plaintext` |
| `fidelity` | string | `lossless`, `semantic`, or `compressed` |
| `input_tok` | integer | Token count before transpilation |
| `output_tok` | integer | Token count after transpilation |
| `reduction_pct` | float | Percentage of tokens saved |
| `saved` | integer | Absolute tokens saved (`input_tok − output_tok`) |

**`TRANSPILE_AGENT` environment variable**

The `agent` field is populated from the `TRANSPILE_AGENT` environment variable. Each integration sets this automatically (`claude`, `antigravity`, `codex`, `opencode`, `cursor`). You can also set it manually:

```bash
TRANSPILE_AGENT=claude transpile --input doc.md
```

### Benchmarking

```bash
# Run benchmarks against a directory of test files
transpile bench run --dataset ./eval                    # generates JSONL log
transpile bench run --dataset ./eval --report           # run + open HTML report
transpile bench report                                 # regenerate report from logs
```

The HTML benchmark report includes:

- **KPI cards** — semantic reduction, compressed reduction, throughput (tok/ms), word coverage, total input tokens, run count
- **7 charts** — reduction trend over time, throughput per run, semantic vs throughput scatter, box plot per format, format distribution, token size histogram, word coverage donut
- **Runs table** — per-run summary with aggregate metrics
- **Records table** — per-file detail with filter by format, run, and filename
- **Theme toggle** — dark / light mode with persistent preference
- **Bilingual** — auto-detects Korean locale; manual 한/EN toggle

---

## Library Usage

### Synchronous

```rust
use llm_transpiler::{transpile, FidelityLevel, InputFormat};

let md = r#"
# Software License Agreement

This agreement is made between Licensor and Licensee.

| Item     | Cost  |
|----------|-------|
| Base fee | $800  |
| Support  | $200  |
"#;

let output = transpile(md, InputFormat::Markdown, FidelityLevel::Semantic, Some(4096))?;
println!("{}", output);
```

### Streaming (Tokio)

```rust
use llm_transpiler::{transpile_stream, FidelityLevel, InputFormat};
use futures::StreamExt;

let mut stream = transpile_stream(input, InputFormat::Markdown, FidelityLevel::Semantic, 4096).await;

while let Some(chunk) = stream.next().await {
    let chunk = chunk?;
    print!("{}", chunk.content);
    if chunk.is_final { break; }
}
```

### Token count estimate

```rust
let n = llm_transpiler::token_count("Hello, world!");
```

---

## Output Format

```
<D>                  ← Symbol dictionary (omitted when no substitutions occur)
{sym}=repeated-term
</D>
<H>                  ← YAML-like metadata header
t: document title
s: one-line summary
k: [keyword1, keyword2]
</H>
<B>                  ← Document body (compressed + substituted)
...content...
</B>
```

The `<D>` block uses Unicode Private Use Area characters (`U+E000–U+F8FF`) as compact symbol handles, avoiding collision with visible text patterns. The dictionary supports up to **6,400 unique terms** per document.

---

## Fidelity Levels

| Level | Typical use case | Compression applied |
|-------|-----------------|---------------------|
| `Lossless` | Legal / audit documents | None — original content guaranteed |
| `Semantic` | General RAG pipelines | Stopword removal + low-importance pruning |
| `Compressed` | Summarization, tight budgets | Maximum compression, first-sentence extraction |

---

## Adaptive Compression

The compressor monitors budget usage in real time and escalates automatically:

| Budget usage | Stage | What happens |
|---|---|---|
| 0–60% | `StopwordOnly` | English/Korean stopwords stripped |
| 60–80% | `PruneLowImportance` | Bottom 20% of paragraphs by importance score removed |
| 80–95% | `DeduplicateAndLinearize` | Duplicate sentences removed; tables linearized |
| 95%+ | `MaxCompression` | Each paragraph truncated to first sentence |

> `Lossless` mode bypasses all compression stages unconditionally.

During streaming, when budget usage crosses 80%, remaining nodes are automatically switched to `Compressed` mode.

---

## Input Formats

| `InputFormat` | Parser |
|---|---|
| `Markdown` | [pulldown-cmark](https://crates.io/crates/pulldown-cmark) — CommonMark + GFM tables |
| `Html` | ammonia sanitization → tag stripping → plain text pipeline |
| `PlainText` | Blank-line paragraph splitting |

---

## Error Handling

```rust
use llm_transpiler::TranspileError;

match transpile(input, format, fidelity, budget) {
    Ok(output) => { /* use output */ }
    Err(TranspileError::Parse(msg))          => eprintln!("parse failed: {msg}"),
    Err(TranspileError::SymbolOverflow(e))   => eprintln!("too many unique terms: {e}"),
    Err(TranspileError::LosslessModeViolation) => eprintln!("compression in lossless mode"),
    Err(e)                                   => eprintln!("error: {e}"),
}
```

---

## Performance

Measured on release build (`cargo build --release`), Apple M-series, 48 documents across Markdown / HTML / PlainText. All reduction figures are measured with the **real `cl100k` BPE tokenizer** (not the self-referential heuristic). See [`docs/EVALUATION.md`](docs/EVALUATION.md) for the full methodology and per-format breakdown.

| Metric | Measured | Notes |
|--------|----------|-------|
| Throughput (Markdown-only peak) | **10,975 tok/ms** | ≈75× faster than Python parsing baseline; single-format peak |
| Throughput (dataset aggregate) | **~1,070 tok/ms** | Weighted across all 48 docs / 3 formats (BPE) — see Benchmarks table |
| Semantic reduction | **27.4%** (Markdown) | Genuine compression rate; within the 15–30% target band |
| Compressed reduction | **69.4%** (Markdown) | Budget-adaptive, guaranteed ≥ PruneLowImportance |
| Lossless word coverage | **99.0% avg** | Across all formats and languages |
| HTML reduction | **98.7%** | Reflects markup overhead removal (nav/scripts/styles) |
| Multilingual support | 15 languages tested | AR/DE/ES/FR/HI/IT/JA/KO/NL/PL/PT/RU/SV/TR/ZH — 99.0% avg word coverage |

Run the evaluation suite yourself:

```bash
make eval          # structured JSON (BPE + heuristic; consumed by `epic eval`)
make eval-report   # human-readable per-file table + summary
```

Full per-file breakdown, methodology, and the token-honesty analysis: [`docs/EVALUATION.md`](docs/EVALUATION.md) (한국어: [`docs/i18n/EVALUATION.ko.md`](docs/i18n/EVALUATION.ko.md))

---

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for full guidelines. PRs welcome — check open issues labeled `good first issue`.

---

## License

Apache-2.0 — see [LICENSE](LICENSE).

© epicsagas · Apache-2.0 · epicsagas/llm-transpile

Similar plugins

All plugins for Agent setup: memory, context, orchestration →

Project sponsors

Companies that keep this analytics open