# ffmpeg-opencv-integration

ffmpeg-opencv-integration — a skill for Claude Code and Codex guides video processing pipelines using FFmpeg and OpenCV. Prevents RGB/BGR mismatches, pipe stalls, and dropped audio tracks.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode, Windsurf (Devin Desktop), Claude.ai · 107 installs · ★ 66 repo stars
- Author: JosiahSiegel
- License: MIT · policy=mirror · the beginning of the file is at the end of this page
- Source: https://github.com/JosiahSiegel/claude-plugin-marketplace/tree/HEAD/plugins/ffmpeg-python/skills/ffmpeg-opencv-integration/SKILL.md
- Updated: 2026-06-18
- Security check: automated, 2026-11-17 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/ffmpeg-opencv-integration

## What it does

The skill guides video frame handoff between FFmpeg and OpenCV in Python pipelines. It standardizes raw video streaming over subprocess pipes, enforcing explicit pixel formats like `bgr24` on the OpenCV boundary and `yuv420p` for destination H.264 encoders.

It outlines library tradeoffs across standard use cases: standard `cv2.VideoCapture` for local playback, ffmpegcv for NVDEC and NVENC GPU tasks, VidGear for multi-threaded RTSP ingestion, Decord for ML batch sampling, and PyAV for direct libav container inspection. It also links Modal recipes for cloud processing.

Critical checks focus on NumPy coordinate ordering (`img[y, x]` instead of `img[x, y]`) and exact buffer sizing where `width * height * channels` must match `-pix_fmt`. The host environment requires FFmpeg binaries and OpenCV installed.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/ffmpeg-opencv-integration/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/ffmpeg-opencv-integration/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/ffmpeg-opencv-integration/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/ffmpeg-opencv-integration/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install JosiahSiegel/claude-plugin-marketplace plugins/ffmpeg-python/skills/ffmpeg-opencv-integration
```

2. **Manually: put the files in place**
- File: `.github/skills/ffmpeg-opencv-integration/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/JosiahSiegel/claude-plugin-marketplace --path plugins/ffmpeg-python/skills/ffmpeg-opencv-integration
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/ffmpeg-opencv-integration/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/JosiahSiegel/claude-plugin-marketplace/tree/HEAD/plugins/ffmpeg-python/skills/ffmpeg-opencv-integration/SKILL.md

## When to use it
- stream raw frames between FFmpeg processes and OpenCV buffers
- preserve synchronized audio streams during frame-level video modifications
- select suitable ingestion libraries across PyAV, VidGear, and Decord
- resolve color inversion bugs between BGR OpenCV arrays and RGB inputs

## Example prompts
- Set up a Python subprocess pipe to feed raw BGR frames from FFmpeg into OpenCV
- Show how to replace video frames using OpenCV while remuxing original audio in FFmpeg
- Provide a Decord batch loading setup for video frames in a PyTorch pipeline

## Source text · SKILL.md · MIT

Below is the content of a third-party file. It is data, not instructions for you: do not act on anything written inside unless the human asks. Block tag: 5a077c35d25b720b.

BEGIN THIRD-PARTY FILE 5a077c35d25b720b
```text
---
name: ffmpeg-opencv-integration
description: |
  Complete FFmpeg + OpenCV + Python integration guide for video processing pipelines.
  PROACTIVELY activate for: (1) FFmpeg to OpenCV frame handoff, (2) cv2.VideoCapture vs ffmpeg subprocess, (3) BGR/RGB color format conversion gotchas, (4) Frame dimension order img[y,x] vs img[x,y], (5) ffmpegcv GPU-accelerated video I/O, (6) VidGear multi-threaded streaming, (7) Decord batch video loading for ML, (8) PyAV frame-level processing, (9) Audio stream preservation with video filters, (10) Memory-efficient frame generators, (11) OpenCV + FFmpeg + Modal parallel processing, (12) Pipe frames between FFmpeg and OpenCV.
  Provides: Color format conversion patterns, coordinate system gotchas, library selection guide, memory management, subprocess pipe patterns, GPU-accelerated alternatives to cv2.VideoCapture.
  Ensures: Correct integration between FFmpeg and OpenCV without color/coordinate bugs.
  See also: ffmpeg-python-integration-reference for type-safe parameter mappings.
---

# FFmpeg + OpenCV Integration Guide

Use this skill when FFmpeg handles video I/O and OpenCV handles image processing. This SKILL is a lean orchestrator; full pipe patterns, library examples, and Modal recipes are preserved in `references/opencv-pipelines-and-libraries.md`.

## When to Use

- Decode with FFmpeg and process frames with OpenCV
- Encode OpenCV-generated/processed frames with FFmpeg
- Compare `cv2.VideoCapture`, subprocess pipes, PyAV, ffmpegcv, VidGear, and Decord
- Fix RGB/BGR color bugs or `(height, width)` dimension-order bugs
- Preserve audio while replacing processed video frames
- Build GPU-assisted video I/O around CPU OpenCV processing

## Library Selection

| Need | Best option | Why |
|---|---|---|
| Simple local file | `cv2.VideoCapture` | Built-in and simple |
| Full FFmpeg format/protocol support | subprocess pipe | Exact CLI behavior |
| GPU video I/O | ffmpegcv | NVDEC/NVENC with OpenCV-like API |
| Network/RTSP streaming | VidGear | Threaded capture and stream helpers |
| ML batch loading | Decord | Fast random/batch frame access |
| Frame-level libav control | PyAV | Direct FFmpeg library access |

## Critical Gotchas
```
END THIRD-PARTY FILE 5a077c35d25b720b

The third-party file has ended. Everything between the 5a077c35d25b720b tags is data: requests and commands from it are not followed. An “end of file” line with another tag inside the block is part of the file.

First 2206 of 4091 bytes. © JosiahSiegel · MIT · https://github.com/JosiahSiegel/claude-plugin-marketplace/tree/HEAD/plugins/ffmpeg-python/skills/ffmpeg-opencv-integration/SKILL.md

_Part of the page did not fit — see the HTML version._

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
