# emerging-techniques-long-context

Guides Claude Code in expanding transformer context windows using RoPE, YaRN, ALiBi, and position interpolation. Useful when working with sequences from 32k to over 128k tokens.

- skill · `type=skill` · Claude Code · 107 installs
- Author: Orchestra Research
- License: MIT · policy=link-only · description only; the file stays with the author
- Source: https://github.com/davila7/claude-code-templates/tree/HEAD/cli-tool/components/skills/ai-research/emerging-techniques-long-context
- Updated: 2026-10-24
- HTML: https://affpapa.org/en/nevibecode/skills/emerging-techniques-long-context

## What it does

This skill supplies Claude Code with reference architectures and implementation snippets to scale context limits in transformer models. It covers positional techniques such as RoPE, YaRN, ALiBi, and position interpolation. These approaches let you adapt architectures like LLaMA to handle documents from 32k to past 128k tokens without training from scratch.

Reference implementations include PyTorch code and Hugging Face snippets. These scripts run on PyTorch. You get self-contained modules for rotary frequencies, tensor rotation, and slope calculation for ALiBi attention masks. It also demonstrates how to configure linear or dynamic `rope_scaling` directly on existing model configs.

Executing these methods in your project requires `torch` and `transformers`. Handling long sequences in practice also depends on installing `flash-attn` with `--no-build-isolation` to avoid quadratic memory overhead.

## Install

### Claude Code (native platform)

1. **claude-code-templates**
```bash
npx claude-code-templates@latest --skill ai-research/emerging-techniques-long-context
```

2. **Skills CLI (Vercel)**
```bash
npx skills add davila7/claude-code-templates --skill emerging-techniques-long-context -a claude-code
```

3. **Manually: put the files in place**
- File: `~/.claude/skills/emerging-techniques-long-context/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/emerging-techniques-long-context/

### Codex

**Manually: put the files in place**
- File: `.agents/skills/emerging-techniques-long-context/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

**Manually: put the files in place**
- File: `.cursor/skills/emerging-techniques-long-context/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install davila7/claude-code-templates emerging-techniques-long-context
```

2. **Manually: put the files in place**
- File: `.github/skills/emerging-techniques-long-context/SKILL.md`

### Gemini CLI

**Platform CLI**
```bash
gemini skills install https://github.com/davila7/claude-code-templates --path cli-tool/components/skills/ai-research/emerging-techniques-long-context
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/emerging-techniques-long-context/SKILL.md` — The legacy .windsurf/skills/ path still works

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/davila7/claude-code-templates/tree/HEAD/cli-tool/components/skills/ai-research/emerging-techniques-long-context

## When to use it
- extend LLaMA context window to 32k or 128k tokens
- implement RoPE rotary embeddings using native PyTorch
- add ALiBi linear bias slopes directly to attention matrices
- configure linear or dynamic `rope_scaling` in Hugging Face models

## Example prompts
- Configure rope_scaling for LLaMA-2-7b to scale the context to 32768 tokens using linear interpolation.
- Implement a PyTorch RotaryEmbedding class to add RoPE to custom attention layers.
- Generate an ALiBi bias matrix for 8 attention heads with sequence length 2048.

## Similar
- [skill-creator](https://affpapa.org/en/nevibecode/skills/skill-creator.md) — skill · 414180 installs
- [writing-for-agents](https://affpapa.org/en/nevibecode/skills/writing-for-agents.md) — skill · 466719 installs
- [media-use](https://affpapa.org/en/nevibecode/skills/media-use.md) — skill · 1100000 installs
- [mcp-builder](https://affpapa.org/en/nevibecode/skills/mcp-builder.md) — skill · 130949 installs
- [subagent-driven-development](https://affpapa.org/en/nevibecode/skills/subagent-driven-development.md) — skill · 237746 installs
- [orchestration](https://affpapa.org/en/nevibecode/skills/orchestration.md) — skill · 350300 installs

From the same set: [davila7/claude-code-templates](https://affpapa.org/en/nevibecode/by/davila7/claude-code-templates.md) — 804 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md