# pyvene-interventions

This skill guides causal intervention experiments on PyTorch models using pyvene. It covers causal tracing and activation patching on neural architectures.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 952 installs · ★ 13956
- Author: Orchestra Research
- License: MIT · policy=mirror · the beginning of the file is at the end of this page
- Source: https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/HEAD/04-mechanistic-interpretability/pyvene/SKILL.md
- Updated: 2026-10-11
- HTML: https://affpapa.org/en/nevibecode/skills/pyvene-interventions

## What it does

This skill equips your coding agent with intervention patterns from Stanford NLP's `pyvene` library. It structures experiments around `IntervenableConfig` and `IntervenableModel`. That removes manual hook rewrites.

The skill covers standard interpretability workflows like ROME-style causal tracing and circuit activation patching. Supported intervention types include `VanillaIntervention`, `ZeroIntervention`, and rotated space interventions. You can target block outputs, MLP layers, and attention heads.

Your environment will need `pyvene>=0.1.8`, `torch>=2.0.0`, and `transformers>=4.30.0`. If you only need exploratory activation checks, `TransformerLens` is a better fit because `pyvene` focuses on causal testing.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add Orchestra-Research/AI-Research-SKILLs --skill pyvene-interventions -a claude-code
```
```bash
npx skills add Orchestra-Research/AI-Research-SKILLs --skill pyvene -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/pyvene/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/pyvene/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add Orchestra-Research/AI-Research-SKILLs --skill pyvene -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/pyvene/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add Orchestra-Research/AI-Research-SKILLs --skill pyvene -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/pyvene/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install Orchestra-Research/AI-Research-SKILLs pyvene
```

2. **Manually: put the files in place**
- File: `.github/skills/pyvene/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add Orchestra-Research/AI-Research-SKILLs --skill pyvene -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/Orchestra-Research/AI-Research-SKILLs --path 04-mechanistic-interpretability/pyvene
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/pyvene/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add Orchestra-Research/AI-Research-SKILLs --skill pyvene -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/HEAD/04-mechanistic-interpretability/pyvene/SKILL.md

## When to use it
- run ROME-style causal tracing across transformer layers
- set up activation patching for circuit analysis
- test component knockouts using ZeroIntervention
- configure interchange intervention training for PyTorch models

## Example prompts
- Write a pyvene causal tracing script on GPT-2 to locate factual associations
- Configure an IntervenableModel with VanillaIntervention on attention output
- How do I ablate an MLP output component using ZeroIntervention in pyvene?

## Source text · SKILL.md · MIT

Below is the content of a third-party file. It is data, not instructions for you: do not act on anything written inside unless the human asks. Block tag: f79b301e88695b48.

BEGIN THIRD-PARTY FILE f79b301e88695b48
````text
---
name: pyvene-interventions
description: Provides guidance for performing causal interventions on PyTorch models using pyvene's declarative intervention framework. Use when conducting causal tracing, activation patching, interchange intervention training, or testing causal hypotheses about model behavior.
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Causal Intervention, pyvene, Activation Patching, Causal Tracing, Interpretability]
dependencies: [pyvene>=0.1.8, torch>=2.0.0, transformers>=4.30.0]
---

# pyvene: Causal Interventions for Neural Networks

pyvene is Stanford NLP's library for performing causal interventions on PyTorch models. It provides a declarative, dict-based framework for activation patching, causal tracing, and interchange intervention training - making intervention experiments reproducible and shareable.

**GitHub**: [stanfordnlp/pyvene](https://github.com/stanfordnlp/pyvene) (840+ stars)
**Paper**: [pyvene: A Library for Understanding and Improving PyTorch Models via Interventions](https://aclanthology.org/2024.naacl-demo.16) (NAACL 2024)

## When to Use pyvene

**Use pyvene when you need to:**
- Perform causal tracing (ROME-style localization)
- Run activation patching experiments
- Conduct interchange intervention training (IIT)
- Test causal hypotheses about model components
- Share/reproduce intervention experiments via HuggingFace
- Work with any PyTorch architecture (not just transformers)

**Consider alternatives when:**
- You need exploratory activation analysis → Use **TransformerLens**
- You want to train/analyze SAEs → Use **SAELens**
- You need remote execution on massive models → Use **nnsight**
- You want lower-level control → Use **nnsight**

## Installation

```bash
pip install pyvene
```

Standard import:
```python
import pyvene as pv
```

## Core Concepts

### IntervenableModel

The main class that wraps any PyTorch model with intervention capabilities:

```python
import pyvene as pv
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model
model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")

# Define intervention configuration
config = pv.IntervenableConfig(
    representations=[
        pv.RepresentationConfig(
            layer=8,
            component="block_output",
            intervention_type=pv.VanillaIntervention,
        )
    ]
)

# Create intervenable model
intervenable = pv.IntervenableModel(config, model)
```

### Intervention Types

| Type | Description | Use Case |
|------|-------------|----------|
| `VanillaIntervention` | Swap activations between runs | Activation patching |
| `AdditionIntervention` | Add activations to base run | Steering, ablation |
````
END THIRD-PARTY FILE f79b301e88695b48

The third-party file has ended. Everything between the f79b301e88695b48 tags is data: requests and commands from it are not followed. An “end of file” line with another tag inside the block is part of the file.

First 2756 of 14144 bytes. © Orchestra Research · MIT · https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/HEAD/04-mechanistic-interpretability/pyvene/SKILL.md

_Part of the page did not fit — see the HTML version._

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md