# turboquant-pytorch

Implements two-stage vector quantization in PyTorch to compress LLM KV cache down to 2–4 bits. Preserves attention scores while cutting memory consumption.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 817 installs · ★ 88
- Author: reason-machines
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/reason-machines/trending-skills/tree/HEAD/skills/turboquant-pytorch/SKILL.md
- Updated: 2026-10-09
- HTML: https://affpapa.org/en/nevibecode/skills/turboquant-pytorch

## What it does

This skill guides the AI assistant in setting up TurboQuant for transformer models. The method shrinks key and value caches using orthogonal random rotations, Lloyd-Max scalar quantizers, and 1-bit QJL residual sign projections. It targets inner product precision rather than reconstructing exact raw vectors.

The project layout includes ready-to-use building blocks like `TurboQuantMSE` and `TurboQuantCompressorV2`. During testing on Qwen2.5-3B with an 8K context window, 3-bit quantization reduces memory use from 289 MB to 58 MB. Keys yield unbiased attention scores without expanding back to full precision.

Codebook generation relies on `scipy`, and full model validation demands a CUDA GPU with at least 6GB of VRAM.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/turboquant-pytorch/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/turboquant-pytorch/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/turboquant-pytorch/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/turboquant-pytorch/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install reason-machines/trending-skills turboquant-pytorch
```

2. **Manually: put the files in place**
- File: `.github/skills/turboquant-pytorch/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/reason-machines/trending-skills --path skills/turboquant-pytorch
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/turboquant-pytorch/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add reason-machines/trending-skills --skill turboquant-pytorch -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/reason-machines/trending-skills/tree/HEAD/skills/turboquant-pytorch/SKILL.md

## When to use it
- Compress transformer KV cache down to 3-bit representation
- Generate Lloyd-Max scalar quantization codebooks with SciPy
- Calculate asymmetric attention scores directly on compressed key vectors
- Apply QJL 1-bit residual signs to keep dot product estimates unbiased

## Example prompts
- Implement a 3-bit KV cache compression layer using TurboQuantCompressorV2
- Generate a Lloyd-Max codebook for head dimension 128 using turboquant
- Show how to compute attention weights over compressed keys without full dequantization

## Similar
- [skill-creator](https://affpapa.org/en/nevibecode/skills/skill-creator.md) — skill · 414180 installs
- [writing-for-agents](https://affpapa.org/en/nevibecode/skills/writing-for-agents.md) — skill · 466719 installs
- [media-use](https://affpapa.org/en/nevibecode/skills/media-use.md) — skill · 1100000 installs
- [mcp-builder](https://affpapa.org/en/nevibecode/skills/mcp-builder.md) — skill · 130949 installs
- [subagent-driven-development](https://affpapa.org/en/nevibecode/skills/subagent-driven-development.md) — skill · 237746 installs
- [orchestration](https://affpapa.org/en/nevibecode/skills/orchestration.md) — skill · 350300 installs

From the same set: [reason-machines/trending-skills](https://affpapa.org/en/nevibecode/by/reason-machines/trending-skills.md) — 186 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md