Skip to main content
plugin

watch-video · marcinsufa — Claude Code plugin: video analysis and how to install

watch-video is a Claude Code plugin that turns videos into timestamped frames, a Whisper transcript, and reports in three formats.

MIT ★ 9 updated 2026-10-24

Install into Claude Code · Plugin marketplace All methods and platforms ↓
/plugin marketplace add anthropics/claude-plugins-community
/plugin install watch-video@claude-community
Open on GitHub

What watch-video does

One command starts the pipeline: ffmpeg extracts JPEG frames, Whisper (local faster-whisper, Groq, or OpenAI) transcribes audio, deduplication drops near-identical frames but keeps narrated moments, optional Tesseract OCR reads on-screen text. Output: report.md with timestamps, report.html with base64-embedded frames, and report.docx. Supports ~1,500 sites via yt-dlp, fetches Jira attachments directly via API token. Local Whisper runs without API keys, hosted models cost a few cents per video.

Requires Python 3.10+ and ffmpeg in PATH. URL mode needs yt-dlp, local Whisper needs faster-whisper, deduplication needs Pillow and imagehash. Diarization (speaker separation) via whisperx downloads ~500 MB–1 GB of pyannote.audio weights and requires a HuggingFace token with access to three gated repos. Windows Defender may slow the first MCP server run by 2–3 minutes — the Claude Code plugin path avoids this.

What it helps with

  • Extract all key moments from a video with text and timestamps
  • Get a YouTube or Loom transcript for analysis
  • Parse a Jira attachment without manual download
  • Build a report with frames and quotes for a document or presentation
  • Batch-process videos through CLI in a CI pipeline

Example prompts for the agent

  • Watch https://youtu.be/… and tell me what it's about
  • Analyze PROJ-1234 and find the bug on screen
  • Extract all frames from demo.mp4 with transcript and save to report.html

How to install watch-video

Plugin marketplace

/plugin marketplace add anthropics/claude-plugins-community
/plugin install watch-video@claude-community

Step by step, with commands for each platform: Claude Code plugins: marketplaces, installing, and building your own

Questions about watch-video

How to install watch-video in Claude Code?

Run /plugin marketplace add anthropics/claude-plugins-community, then /plugin install watch-video@claude-community. After that, tell Claude: "Watch <URL> and tell me what's in it."

What does watch-video do?

Plugin downloads video (local file, YouTube, Loom, Vimeo, TikTok, ~1,500 sites, or Jira attachment), cuts frames via ffmpeg, transcribes audio with Whisper, removes duplicate frames, optionally OCRs on-screen text with Tesseract, and produces a report in Markdown, HTML, and DOCX with timestamps.

What dependencies does watch-video require?

Python 3.10+ and ffmpeg are required. URL mode needs yt-dlp, local Whisper needs faster-whisper, deduplication needs Pillow and imagehash, OCR needs pytesseract and the Tesseract binary. Hosted Whisper (Groq, OpenAI, Deepgram) requires an API key, diarization needs a HuggingFace token and ~1 GB of pyannote.audio weights.

Source text

SKILL.md · MIT
# claude-watch-video

> **Give Claude eyes and ears for any video** — local files, public URLs (YouTube, Loom, Vimeo, TikTok…), or Jira attachments. Pipeline runs locally on your machine; produces a paste-ready Markdown evidence bundle.

[![smoketest](https://github.com/MarcinSufa/claude-watch-video/actions/workflows/smoketest.yml/badge.svg)](https://github.com/MarcinSufa/claude-watch-video/actions/workflows/smoketest.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/)
[![Platform](https://img.shields.io/badge/platform-windows%20%7C%20macOS%20%7C%20linux-lightgrey.svg)](#prerequisites-any-path)
[![Plugin](https://img.shields.io/badge/Claude%20Code-plugin-purple.svg)](https://docs.claude.com/en/docs/claude-code/plugins)

Turn *"watch PROJ-1234 and tell me what broke"* into a single command. The skill downloads the video, extracts keyframes with ffmpeg, transcribes audio with local or hosted Whisper, deduplicates near-identical frames while preserving narrated moments, optionally OCRs on-screen text, and writes a paste-ready `report.md`. Costs **$0 to a few cents** per video. See [what it costs](#what-it-costs) for the breakdown.

---

## What it produces

A 5:30 FOMC press conference distilled to one moment, picked by Claude against the prompt *"summarize the rate decision, inflation outlook, and rate-path forecast"*:

> **00:22 — the rate decision**
>
> <img src="docs/images/fomc/00-22-rate-decision.jpg" width="420" alt="Powell at 00:22 announcing the rate hold">
>
> *"Today, the FOMC decided to leave our policy rate unchanged. We see the current stance of monetary policy as appropriate to promote progress toward our maximum employment..."* Committee judges current policy stance appropriate for the dual mandate.

That's one card from a real `highlights.md` rendered inline. The pipeline also produces a full `report.md` evidence bundle (transcript + all timestamped frames), `report.html`, `report.docx`, and machine-readable `highlights.json`. **65 seconds end-to-end, $0** with local Whisper. See the [Powell FOMC walkthrough](docs/walkthrough-fomc.md) for the full artifacts, dedup metrics, and per-step timings.

---

## Install — pick one

### Claude Code (recommended)

```bash
/plugin marketplace add MarcinSufa/claude-watch-video
/plugin install watch-video@claude-watch-video
```

Then ask Claude: *"Watch https://youtu.be/… and tell me what's in it."* or *"Watch PROJ-1234 and identify the bug."*

Works on every Claude Code surface — CLI, IDE extensions, desktop. The plugin auto-installs the same underlying skill. To install it by hand instead, clone the repo to `~/.claude/skills/watch-video/`: [`SKILL.md`](SKILL.md) sits at the repo root and finds its scripts through `${CLAUDE_SKILL_DIR}` either way.

### CLI direct (CI / scripting / power users)

```bash
git clone https://github.com/MarcinSufa/claude-watch-video.git
python3 claude-watch-video/scripts/watch_video.py "<url>" --workdir /tmp/test --dedup --verbose
```

Zero-host, zero-MCP, fastest path on any platform. Produces `transcript.md`, `frames/`, `report.md` / `.html` / `.docx`. Good for CI pipelines, batch processing, or driving the analysis yourself.

### Other MCP hosts (Claude Desktop, Cursor, Cline, Codex CLI, …)

An MCP server install is available — full instructions in [`mcp-server/README.md`](mcp-server/README.md).

> ⚠️ **Honest warning:** on Windows + Claude Desktop the first run is **slow (~2-3 minutes)** because Windows Defender scans the freshly-spawned Python subprocess. The Claude Code plugin path above bypasses that entirely, and the CLI bypasses MCP entirely. **If you have the choice, use one of those.** The MCP path exists for environments where neither is an option.

> **Codex CLI / Agent Skills note:** OpenAI's [openai/skills](https://github.com/openai/skills) framework presents skills as portable Agent Skills, but the install mechanism is still evolving. For Codex CLI today, **the CLI direct path above works without any agent-side install** (just `python3 scripts/watch_video.py ...` from your shell). If you specifically need MCP integration, the MCP path is available; if a future Codex skill-installer convention lands, this README will be updated.

### Prerequisites (any path)

Required: **Python 3.10+** and **`ffmpeg`** (which bundles `ffprobe`).

Install ffmpeg per platform:

| OS | Command |
|---|---|
| Linux (Debian/Ubuntu) | `sudo apt install ffmpeg` |
| Linux (Fedora/RHEL) | `sudo dnf install ffmpeg` |
| macOS | `brew install ffmpeg` |
| Windows | `winget install Gyan.FFmpeg` |

Commands below use `python3`. On Windows use `py` (or `python` if it is a real interpreter, not the Microsoft Store alias).

Optional (unlock features as you need them):

| Adds | Install |
|---|---|
| URL mode (YouTube, Loom, …) | `pip install --user yt-dlp` |
| Local transcription (`--whisper local`) | `pip install --user faster-whisper 'av<19'` (PyAV 19 breaks faster-whisper 1.2.x: `TypeError ... 'metadata_errors'`) |
| Smart dedup (`--dedup`) | `pip install --user Pillow imagehash` |
| OCR (`--ocr`) | `pip install --user pytesseract Pillow` + Tesseract binary ([references/setup.md](references/setup.md), [OCR notes](references/ocr-dedup.md#on-screen-text-search-with---ocr)) |
| Hosted Whisper (Groq/OpenAI/Deepgram) | API key — env var or `~/.watch-video/credentials.json` |
| **Local diarization (`--whisper whisperx`)** | `pip install pyannote.audio` (the underlying lib; ~500 MB-1 GB of model weights cached on first run) + HF token (`HF_TOKEN` env or `hf_token` in credentials.json) + accept terms at **three** gated repos: [pyannote/speaker-diarization-3.1](https://huggingface.co/pyannote/speaker-diarization-3.1), [pyannote/segmentation-3.0](https://huggingface.co/pyannote/segmentation-3.0), and [pyannote/speaker-diarization-community-1](https://huggingface.co/pyannote/speaker-diarization-community-1) |
| LLM highlights (`--highlights-prompt`) | Anthropic/OpenAI/Groq/DeepSeek/Gemini API key |
| Jira auto-fetch + opt-in posting | Atlassian token at `~/.atlassian-token/credentials.json` |

---

## Pipeline

```
            input
              │
              ▼
            fetch          (URL → yt-dlp · path → as-is · Jira key → REST API)
              │
              ▼
            probe          (ffprobe metadata + audio volume detection)
             / \
            /   \
        frames   transcribe   (captions · local Whisper · Groq · OpenAI · Deepgram+diarization)
            \   /
             \ /
            dedup           (perceptual hash + transcript-aware protection)
              │
              ▼
             ocr            (Tesseract on kept frames; optional)
              │
              ▼
           report           (report.md / .html / .docx)
              │
              ╎  (opt-in, never default)
              ▼
         Jira comment
```

Every step is independently cached — re-run with a tweaked flag and only the affected tail of the pipeline executes (~0.25 s for a no-op re-run vs ~30 s cold).

---

## What it costs

> ⚠️ The numbers below have caused confusion. The first table is what **this skill costs**. The second is what **alternatives charge** for the same job. They are not the same thing.

### What this skill costs (per ~40-min video)

| Setup | Cost | Time |
|---|---|---|
| **Pipeline only** (no agent reads the artifacts) | **$0** | ~15 s |
| Pipeline + Claude Haiku 4.5 reads the artifacts | **~$0.04** | ~4-5 min |
| Pipeline + Claude Sonnet 4.6 reads the artifacts | ~$0.16 | ~4-5 min |

The pipeline runs locally — no API calls for the download, frame extraction, transcription (with `--whisper local`, the default), dedup, OCR, or report generation. Video data never leaves your computer.

The only token cost is when an *agent* reads the generated artifacts. Hosted transcription (Groq/OpenAI/Deepgram) is optional and costs a fraction of a cent per minute.

### What alternatives charge (for context)

Same 40-min video, same output goal (transcript + visual context + structured analysis):

| Alternative tool | Cost | Where the video goes |
|---|---|---|
| Gemini 3 Flash native video upload | ~$0.30 | Google |
| Gemini 3 Pro native video upload | ~$0.80 | Google |
| OpenAI Whisper + GPT-4o vision DIY | ~$4.50 | OpenAI |
| Microsoft Video Indexer (advanced) | ~$8 | Microsoft |
| Anthropic Claude Haiku raw video upload | ~$17 | Anthropic |
| Anthropic Claude Sonnet raw video upload | ~$66 | Anthropic |
| Anthropic Claude Opus raw video upload | ~$331 | Anthropic |

The Anthropic raw-upload column is the cleanest apples-to-apples on output quality (same Claude model reads the same content) — and the 30 fps frame tokenization makes that path uneconomic by ~400-8,000×. By preprocessing locally and feeding only the transcript + deduped frames into the agent, this skill replaces hundreds of dollars of frame tokens with a $0 pipeline + a few cents of structured text.

Full per-tier breakdown + replication commands: [docs/cost-study-atlassian-video.md](docs/cost-study-atlassian-video.md).

---

## Features

- **Multi-source input** — local file path, public URL (`yt-dlp` supports 1500+ sites), Jira issue key (`PROJ-1234`), or `auto` (newest video in `~/Downloads/`)
- **Six transcription modes** — `captions` (free from YouTube VTTs), `local` faster-whisper (offline, default), `groq` Whisper-large-v3, `openai` Whisper-1, `deepgram` Nova-3 (hosted diarization, ~$0.0043/min), and `whisperx` (**free + offline** diarization via local Whisper + pyannote.audio directly; ~500 MB-1 GB of model weights cached on first run). Both diarization providers produce `**S0**` / `**S1**` paragraph labels + a `speakers.json` summary; relabel anonymous speakers with real names via `scripts/relabel_speakers.py`.
- **Smart frame dedup** — pHash + temporal protection + transcript-aware keep rules. ~50% token reduction on screen recordings *without* losing the moment the user typed the wrong value
- **OCR you can grep** — Tesseract on kept frames; `grep -i "unload" ocr.txt` answers "when did the user enter 90?" in milliseconds
- **Per-step cache** — re-run with a tweaked flag and only the affected tail of the pipeline executes
- **Paste-ready evidence bundle** — `report.md` interleaves transcript paragraphs with frame thumbnails; drop into Jira / PR / design-review doc as-is
- **Bulk mode** — process a sprint's worth of bug tickets in one command
- **Opt-in Jira posting** — explicit `--post-to-jira` + confirmation prompt; never default, never silent. See [safety model](#safety-model)
- **LLM highlights** — six providers (Anthropic / OpenAI / Groq / DeepSeek / Gemini / generic openai-compat) for picking the moments that match your prompt

---

## End-to-end walkthroughs

Two real runs with all artifacts and timings captured verbatim — separate docs so they don't bloat this page:

- 📊 **[Walkthrough — Powell's FOMC statement](docs/walkthrough-fomc.md)** — 5:30 Federal Reserve press conference distilled to 5 quantitative highlights. **/usr/bin/bash** (local Whisper), **65s** end-to-end. Demonstrates: continuous-narration source, zero-cost transcription, structured macro analysis.
- 🆕 **[Walkthrough — Claude Code release-notes](docs/walkthrough-claude-code-release.md)** — 54s product release video reduced to 5 actionable workflow changes. **$0**, **29s** end-to-end, **44%** dedup reduction. Demonstrates: fast-cut B-roll, screen-recording context, agent-readable timestamps.

---

## Use cases

- **Bug triage from Jira screen recordings** — `python3 scripts/watch_video.py PROJ-1234 --dedup --ocr` gives you the transcript + key frames + on-screen text in under a minute. Add `--post-to-jira` to attach the analysis back to the ticket
- **Sprint retro on bug videos** — `python3 scripts/watch_batch.py --jira-jql "project = PROJ AND labels = video-bug AND created >= -7d"` processes a week's videos at once
- **Researching public content** — point at any YouTube/Loom URL; captions-first means free + sub-5-second on most YouTube videos
- **Podcast / multi-speaker transcription** — `--whisper deepgram` gives anonymous speaker labels (`S0`, `S1`) and a `speakers.json` summary you can later relabel with real names
- **Compliance / privacy-first** — default `--whisper local` runs entirely on your machine; nothing is uploaded
- **CI integration** — the CLI is pipeline-friendly: shell out from a GitHub Actions / GitLab CI / Jenkins step on test-failure videos, parse `meta.json` for the result. Per-step caching means re-runs after a flag tweak are near-instant.

---

## Examples

```bash
# Most common: full-auto Jira workflow
python3 scripts/watch_video.py PROJ-1234 --dedup --ocr

# YouTube clip with bumped resolution for tiny on-screen text
python3 scripts/watch_video.py "https://youtu.be/abc" --resolution 1280

# Scope to a 10-second window of a long video
python3 scripts/watch_video.py PROJ-1234 --start 0:30 --end 0:40

# Fast cold-start with hosted Whisper
python3 scripts/watch_video.py PROJ-1234 --whisper groq

# Multi-speaker podcast with hosted diarization (Deepgram; ~$0.0043/min)
python3 scripts/watch_video.py "https://youtu.be/joe-vs-naval" --whisper deepgram

# Same, but local + offline (WhisperX recipe; free; ~1 GB first-run model download)
# Implemented via faster-whisper + pyannote.audio directly (no whisperx package
# wrapper, which has stale dep pins broken on Python 3.14). Requires HF token
# + accepting terms on three gated pyannote models. See references/whisper-providers.md.
python3 scripts/watch_video.py "https://youtu.be/joe-vs-naval" --whisper whisperx

# Relabel anonymous speakers with real names (v2.3.1+; works for both providers)
# Read speakers.json first to see who said what; then:
python3 scripts/relabel_speakers.py /tmp/watch-joe-vs-naval \
  --names "S0=Joe Rogan,S1=Naval Ravikant"
# transcript.md and report.md/.html/.docx are atomically rewritten in place.

# LLM-driven highlight selection (default model is Claude Haiku 4.5)
python3 scripts/watch_video.py PROJ-1234 \
  --highlights-prompt "highlight only bug-related parts"

# Post analysis back to the ticket (opt-in, confirmation prompt)
python3 scripts/watch_video.py PROJ-1234 --dedup --ocr --post-to-jira

# Bulk: process a sprint's worth of bug tickets
python3 scripts/watch_batch.py \
  --jira-jql "project = PROJ AND labels = video-bug AND created >= -7d" \
  --dedup --ocr
```

Every flag is documented in [docs/configuration.md](docs/configuration.md).

---

## Safety model

The skill ships with safety guardrails for the one action that leaves the machine — posting to Jira:

- **`--post-to-jira` is opt-in.** Default behavior never writes anything anywhere.
- **Interactive confirmation prompt.** When `--post-to-jira` is set, the CLI prints the planned comment and asks for `y/N` before sending. The default is `N`.
- **`--post-to-jira-dry-run`** previews the comment without sending — for sanity-checking the formatting in CI or before a real run.
- **Local writes are confined.** Pipeline artifacts go to the workdir, and the MCP `fetch_jira_attachment` tool only accepts an `outdir` under the tmp root — a ticket cannot steer downloads at your home directory. Attachment filenames are flattened to a single safe path component before use.
- **No unsolicited Jira writes from any context.** Skill / plugin / MCP / direct API: all paths require explicit per-invocation consent. The MCP `post_to_jira` tool defaults to `confirm=False` (dry-run); MCP hosts MUST surface the planned action to the user before passing `confirm=True`.

---

## Architecture

CLI scripts are the canonical implementation. Everything else (plugin, skill, MCP server) is a thin adapter that calls these scripts under the hood.

```
scripts/                  ← canonical implementation
├── watch_video.py        ← top-level orchestrator
├── fetch.py              ← URL / file / Jira input
├── probe.py              ← ffprobe metadata + audio analysis
├── frames.py             ← ffmpeg keyframe extraction
├── transcribe.py         ← captions / local / Groq / OpenAI / Deepgram
├── dedup.py              ← pHash + transcript-aware protection
├── ocr.py                ← Tesseract on kept frames
├── report.py             ← report.md / .html / .docx generation
├── highlights.py         ← LLM-driven moment picker (6 providers)
└── post_to_jira.py       ← opt-in, confirmation-gated Jira posting

mcp-server/server.py      ← thin async wrapper over the CLI scripts
```

Each step writes to disk atomically (staged then `os.replace`'d). Per-step fingerprint cache; re-run with a tweaked flag and only the affected tail executes. Workdir is a contract — every artifact has a known filename and shape ([references/meta-schema.md → file layout](references/meta-schema.md#workspace)).

---

## Versioning + Roadmap

Latest release: **v2.3.2** ([changelog](https://github.com/MarcinSufa/claude-watch-video/releases)) — adds `--whisper whisperx`, a free + offline local-diarization alternative to Deepgram. Same `speakers.json` schema, so `relabel_speakers.py` works against it unchanged.

What's queued:
- **v2.4.0+** — OCR cross-correlation for screen-recording name overlays (auto-label speakers from Zoom/Teams/Meet name tags, eliminating the manual relabel step entirely on those sources)

Full roadmap: [ROADMAP.md](ROADMAP.md).

---

## License

MIT — see [LICENSE](LICENSE). Free for any use, commercial OK, just keep the copyright notice.

## Acknowledgements

Built on [ffmpeg](https://ffmpeg.org/), [yt-dlp](https://github.com/yt-dlp/yt-dlp), [faster-whisper](https://github.com/SYSTRAN/faster-whisper), [Tesseract](https://github.com/tesseract-ocr/tesseract), [Pillow](https://pillow.readthedocs.io/), [imagehash](https://github.com/JohannesBuchner/imagehash), [python-docx](https://github.com/python-openxml/python-docx), and the [Anthropic SDK](https://github.com/anthropics/anthropic-sdk-python). LLM highlight picking optionally uses [OpenAI](https://github.com/openai/openai-python), [Groq](https://groq.com/), [DeepSeek](https://api.deepseek.com), or [Google Gemini](https://ai.google.dev/) via their OpenAI-compatibility endpoints. Speaker diarization via [Deepgram Nova-3](https://deepgram.com).

© MarcinSufa · MIT · MarcinSufa/claude-watch-video

Similar plugins

All plugins for Agent setup: memory, context, orchestration →

watch-video by other authors comes first.

watch-video CodePawl The watch-video Claude Code plugin examines videos and tutorial channels through their frames, edits, audio, transcripts, and speaking style. It turns a video into a remake recipe or a channel into a skill with a required verification pass. ★ 350repo stars video-analysis Starchild-ai-agent video-analysis inspects video files using native multimodal processing or frame sampling with Whisper speech transcription. CX Codex +1 1,800installs video-understand calesthio video-understand — a skill for Claude Code, Codex, Cursor and other agents that pulls frames with ffmpeg and transcribes audio with Whisper entirely on your machine. No external keys, no network calls. CX Codex +1 1,500installs claude-real-video-for-agents HUANGCHIHHUNGLeo claude-real-video-for-agents extracts scene-aware keyframes, removes duplicates, and transcribes audio so coding agents can analyze video files. CX Codex +1 589installs Video Processor disler Video Processor — a skill that handles video files from the agent: pull the audio track, convert to mp4 or webm, transcribe speech with Whisper. Useful when video files show up in a project. CX Codex +1 ★ 1,573repo stars Video Transcriber nhatvu148 Video Transcriber turns videos from links or local files into text with whisper.cpp, running on your own machine. Handy when a coding agent needs the spoken words, not the picture. CX Codex +1 ★ 20repo stars claude-real-video HUANGCHIHHUNGLeo claude-real-video is a Claude Code skill that extracts keyframes and speech transcripts from video. It activates when someone asks about a video URL or a local file. CX Codex +1 672installs

Project sponsors

Companies that keep this analytics open