turboquant-pytorch — Claude Code skill: what it does, how to install
Implements two-stage vector quantization in PyTorch to compress LLM KV cache down to 2–4 bits. Preserves attention scores while cutting memory consumption.
npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a claude-codeWhat turboquant-pytorch does
This skill guides the AI assistant in setting up TurboQuant for transformer models. The method shrinks key and value caches using orthogonal random rotations, Lloyd-Max scalar quantizers, and 1-bit QJL residual sign projections. It targets inner product precision rather than reconstructing exact raw vectors.
The project layout includes ready-to-use building blocks like TurboQuantMSE and TurboQuantCompressorV2. During testing on Qwen2.5-3B with an 8K context window, 3-bit quantization reduces memory use from 289 MB to 58 MB. Keys yield unbiased attention scores without expanding back to full precision.
Codebook generation relies on scipy, and full model validation demands a CUDA GPU with at least 6GB of VRAM.
What it helps with
- Compress transformer KV cache down to 3-bit representation
- Generate Lloyd-Max scalar quantization codebooks with SciPy
- Calculate asymmetric attention scores directly on compressed key vectors
- Apply QJL 1-bit residual signs to keep dot product estimates unbiased
Example prompts for the agent
- Implement a 3-bit KV cache compression layer using TurboQuantCompressorV2
- Generate a Lloyd-Max codebook for head dimension 128 using turboquant
- Show how to compute attention weights over compressed keys without full dequantization
How to install turboquant-pytorch
1Skills CLI (Vercel)
npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a claude-code2Manually: put the files in place
~/.claude/skills/turboquant-pytorch/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/turboquant-pytorch/
1Skills CLI (Vercel)
npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a codex2Manually: put the files in place
.agents/skills/turboquant-pytorch/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a cursor2Manually: put the files in place
.cursor/skills/turboquant-pytorch/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install reason-machines/trending-skills turboquant-pytorch2Manually: put the files in place
.github/skills/turboquant-pytorch/SKILL.md1Skills CLI (Vercel)
npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a gemini-cli2Platform CLI
gemini skills install https://github.com/reason-machines/trending-skills --path skills/turboquant-pytorchManually: put the files in place
.devin/skills/turboquant-pytorch/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about turboquant-pytorch
How to install turboquant-pytorch in Claude Code?
Run npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a claude-code. This imports the skill files into your assistant setup.
What does turboquant-pytorch do?
It provides PyTorch implementations of two-stage vector quantization for LLM context buffers. The technique compresses keys and values down to 2–4 bits while retaining attention accuracy.
What hardware and packages are required?
Basic algorithms require torch>=2.0 and scipy. Testing against Qwen2.5-3B requires a CUDA GPU with at least 6GB VRAM, along with transformers, accelerate, and bitsandbytes.
