# emerging-techniques-speculative-decoding

A skill that walks your agent through speculative decoding, Medusa and lookahead decoding to speed up token generation by 1.5–3.6×.

- skill · `type=skill` · Claude Code · 59 installs
- Author: Orchestra Research
- License: MIT · policy=link-only · description only; the file stays with the author
- Source: https://github.com/davila7/claude-code-templates/tree/HEAD/cli-tool/components/skills/ai-research/emerging-techniques-speculative-decoding
- Updated: 2026-10-24
- HTML: https://affpapa.org/en/nevibecode/skills/emerging-techniques-speculative-decoding

## What it does

Three ways to make an LLM answer faster are covered here: a small draft model that proposes tokens for a larger one to verify, Medusa with extra prediction heads, and lookahead decoding built on Jacobi iteration. The agent gets Python snippets: loading target and draft models with `transformers`, calling `medusa_generate` with `posterior_threshold` and `posterior_alpha`, and setting up `LookaheadDecoding` with `window_size`, `ngram_size` and `guess_size`.

The algorithm is spelled out too. The draft model guesses K tokens, the target model scores all of them in a single forward pass, and matching ones are kept. Medusa builds a tree of candidates, and the skill separates Medusa-1, where only the heads are trained, from Medusa-2, where the base model is fine-tuned as well.

Examples rely on `transformers` and `torch`. Medusa and lookahead decoding each need their own repository cloned and installed with `pip install -e .`, so without those dependencies the snippets will not run.

## Install

### Claude Code (native platform)

1. **claude-code-templates**
```bash
npx claude-code-templates@latest --skill ai-research/emerging-techniques-speculative-decoding
```

2. **Skills CLI (Vercel)**
```bash
npx skills add davila7/claude-code-templates --skill emerging-techniques-speculative-decoding -a claude-code
```

3. **Manually: put the files in place**
- File: `~/.claude/skills/emerging-techniques-speculative-decoding/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/emerging-techniques-speculative-decoding/

### Codex

**Manually: put the files in place**
- File: `.agents/skills/emerging-techniques-speculative-decoding/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

**Manually: put the files in place**
- File: `.cursor/skills/emerging-techniques-speculative-decoding/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install davila7/claude-code-templates emerging-techniques-speculative-decoding
```

2. **Manually: put the files in place**
- File: `.github/skills/emerging-techniques-speculative-decoding/SKILL.md`

### Gemini CLI

**Platform CLI**
```bash
gemini skills install https://github.com/davila7/claude-code-templates --path cli-tool/components/skills/ai-research/emerging-techniques-speculative-decoding
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/emerging-techniques-speculative-decoding/SKILL.md` — The legacy .windsurf/skills/ path still works

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/davila7/claude-code-templates/tree/HEAD/cli-tool/components/skills/ai-research/emerging-techniques-speculative-decoding

## When to use it
- speed up token generation without retraining the model
- cut latency in a chatbot or code generator
- pick between a draft model, Medusa and lookahead decoding
- run inference on limited hardware
- understand how the target model verifies candidates in parallel

## Example prompts
- Show me how to pass a draft model to `generate` via `assistant_model`
- Compare Medusa-1 and Medusa-2 on speed and quality
- Set up `LookaheadDecoding` with `window_size=15` and `ngram_size=5`

## Similar
- [jetson-speculative-decoding](https://affpapa.org/en/nevibecode/skills/jetson-speculative-decoding.md) — skill · 1100 installs
- [dflash-mlx-speculative-decoding](https://affpapa.org/en/nevibecode/skills/dflash-mlx-speculative-decoding.md) — skill · 443 installs
- [anti-reversing-techniques](https://affpapa.org/en/nevibecode/skills/anti-reversing-techniques.md) — skill · 10900 installs
- [building-with-medusa](https://affpapa.org/en/nevibecode/skills/building-with-medusa.md) — skill · 4600 installs
- [storefront-best-practices](https://affpapa.org/en/nevibecode/skills/storefront-best-practices.md) — skill · 4600 installs
- [waf-bypass-techniques](https://affpapa.org/en/nevibecode/skills/waf-bypass-techniques.md) — skill · 3800 installs

From the same set: [davila7/claude-code-templates](https://affpapa.org/en/nevibecode/by/davila7/claude-code-templates.md) — 361 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md