emerging-techniques-long-context — Claude Code skill
Guides Claude Code in expanding transformer context windows using RoPE, YaRN, ALiBi, and position interpolation. Useful when working with sequences from 32k to over 128k tokens.
npx claude-code-templates@latest --skill ai-research/emerging-techniques-long-contextWhat emerging-techniques-long-context does
This skill supplies Claude Code with reference architectures and implementation snippets to scale context limits in transformer models. It covers positional techniques such as RoPE, YaRN, ALiBi, and position interpolation. These approaches let you adapt architectures like LLaMA to handle documents from 32k to past 128k tokens without training from scratch.
Reference implementations include PyTorch code and Hugging Face snippets. These scripts run on PyTorch. You get self-contained modules for rotary frequencies, tensor rotation, and slope calculation for ALiBi attention masks. It also demonstrates how to configure linear or dynamic rope_scaling directly on existing model configs.
Executing these methods in your project requires torch and transformers. Handling long sequences in practice also depends on installing flash-attn with --no-build-isolation to avoid quadratic memory overhead.
What it helps with
- extend LLaMA context window to 32k or 128k tokens
- implement RoPE rotary embeddings using native PyTorch
- add ALiBi linear bias slopes directly to attention matrices
- configure linear or dynamic
rope_scalingin Hugging Face models
Example prompts for the agent
- Configure rope_scaling for LLaMA-2-7b to scale the context to 32768 tokens using linear interpolation.
- Implement a PyTorch RotaryEmbedding class to add RoPE to custom attention layers.
- Generate an ALiBi bias matrix for 8 attention heads with sequence length 2048.
How to install emerging-techniques-long-context
1claude-code-templates
npx claude-code-templates@latest --skill ai-research/emerging-techniques-long-context2Skills CLI (Vercel)
npx skills add davila7/claude-code-templates --skill emerging-techniques-long-context -a claude-code3Manually: put the files in place
~/.claude/skills/emerging-techniques-long-context/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/emerging-techniques-long-context/
Manually: put the files in place
.agents/skills/emerging-techniques-long-context/SKILL.mdFor all projects use ~/.agents/skills/
Manually: put the files in place
.cursor/skills/emerging-techniques-long-context/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install davila7/claude-code-templates emerging-techniques-long-context2Manually: put the files in place
.github/skills/emerging-techniques-long-context/SKILL.mdPlatform CLI
gemini skills install https://github.com/davila7/claude-code-templates --path cli-tool/components/skills/ai-research/emerging-techniques-long-contextManually: put the files in place
.devin/skills/emerging-techniques-long-context/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about emerging-techniques-long-context
How do I install emerging-techniques-long-context in Claude Code?
Run npx claude-code-templates@latest --skill ai-research/emerging-techniques-long-context. This command adds the long-context skill template directly to Claude Code.
What does emerging-techniques-long-context do?
It gives Claude Code architectural guides and code templates for expanding transformer context lengths. The instructions cover RoPE, YaRN, ALiBi, and context interpolation configs.
What Python libraries are required to run the code?
Core examples require torch and transformers. Custom implementations use einops and rotary-embedding-torch, while sequence acceleration requires flash-attn.
