# scanning-experiments-with-replay-vision

The scanning-experiments-with-replay-vision skill for Claude Code sets up a PostHog Replay Vision scanner targeted at an experiment. It watches exposed user recordings and analyzes differences across test variants with an LLM.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode, Windsurf (Devin Desktop), Claude.ai · 148 installs · ★ 101 repo stars
- Author: PostHog
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/PostHog/ai-plugin/tree/HEAD/skills/scanning-experiments-with-replay-vision/SKILL.md
- Updated: 2026-10-24
- Security check: automated, 2026-11-17 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/scanning-experiments-with-replay-vision

## What it does

This skill provisions an `experiment` type Replay Vision scanner tied to a specific PostHog A/B test. It fetches the underlying feature flag configuration via `experiment-get`, sets up a unified prompt, and balances session sampling across variants using `balance_variants`.

The LLM receives the visual change description but never gets informed which variant the recorded visitor actually saw. This prevents model bias and exposes broken feature flag rollouts directly from user actions. Cohort filtering runs entirely server-side; defining custom exposure filters inside `query` triggers an API error.

Draft experiments lacking a launch date cannot be provisioned, and ended runs require a backfill. The scanner is initially created disabled so the analysis prompt can be tested against real sessions before credits are spent.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add PostHog/ai-plugin --skill scanning-experiments-with-replay-vision -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/scanning-experiments-with-replay-vision/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/scanning-experiments-with-replay-vision/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add PostHog/ai-plugin --skill scanning-experiments-with-replay-vision -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/scanning-experiments-with-replay-vision/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add PostHog/ai-plugin --skill scanning-experiments-with-replay-vision -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/scanning-experiments-with-replay-vision/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install PostHog/ai-plugin skills/scanning-experiments-with-replay-vision
```

2. **Manually: put the files in place**
- File: `.github/skills/scanning-experiments-with-replay-vision/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add PostHog/ai-plugin --skill scanning-experiments-with-replay-vision -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/PostHog/ai-plugin --path skills/scanning-experiments-with-replay-vision
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/scanning-experiments-with-replay-vision/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add PostHog/ai-plugin --skill scanning-experiments-with-replay-vision -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/PostHog/ai-plugin/tree/HEAD/skills/scanning-experiments-with-replay-vision/SKILL.md

## When to use it
- scan session recordings for a specific PostHog experiment
- balance session sampling evenly between test and control cohorts
- draft unbiased LLM prompts that hide variant assignment from the model
- estimate Replay Vision credit usage before turning the scanner on

## Example prompts
- Create a Replay Vision scanner for experiment 123 to review checkout drop-offs
- Scan session recordings from our onboarding experiment using vision AI
- Set up an experiment scanner to inspect user interactions across all test variants

## Similar
- [signals-scout-replay-vision](https://affpapa.org/en/nevibecode/skills/signals-scout-replay-vision.md) — skill · 316 installs
- [signals-scout-experiments](https://affpapa.org/en/nevibecode/skills/signals-scout-experiments.md) — skill · 319 installs
- [product-experiments](https://affpapa.org/en/nevibecode/skills/product-experiments.md) — skill · 401 installs
- [revenuecat-experiments](https://affpapa.org/en/nevibecode/skills/revenuecat-experiments.md) — skill · 449 installs
- [adding-feature-flags](https://affpapa.org/en/nevibecode/skills/adding-feature-flags.md) — skill · 399 installs
- [experimentation-platform-orchestrator](https://affpapa.org/en/nevibecode/skills/experimentation-platform-orchestrator.md) — skill · 273 installs

From the same set: [posthog/ai-plugin](https://affpapa.org/en/nevibecode/by/posthog/ai-plugin.md) — 82 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
