ffmpeg-opencv-integration — Claude Code skill: what it does
ffmpeg-opencv-integration — a skill for Claude Code and Codex guides video processing pipelines using FFmpeg and OpenCV. Prevents RGB/BGR mismatches, pipe stalls, and dropped audio tracks.
MIT 107 installs ★ 66 updated 2026-06-18
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a claude-codeWhat ffmpeg-opencv-integration does
The skill guides video frame handoff between FFmpeg and OpenCV in Python pipelines. It standardizes raw video streaming over subprocess pipes, enforcing explicit pixel formats like bgr24 on the OpenCV boundary and yuv420p for destination H.264 encoders.
It outlines library tradeoffs across standard use cases: standard cv2.VideoCapture for local playback, ffmpegcv for NVDEC and NVENC GPU tasks, VidGear for multi-threaded RTSP ingestion, Decord for ML batch sampling, and PyAV for direct libav container inspection. It also links Modal recipes for cloud processing.
Critical checks focus on NumPy coordinate ordering (img[y, x] instead of img[x, y]) and exact buffer sizing where width * height * channels must match -pix_fmt. The host environment requires FFmpeg binaries and OpenCV installed.
What it helps with
- stream raw frames between FFmpeg processes and OpenCV buffers
- preserve synchronized audio streams during frame-level video modifications
- select suitable ingestion libraries across PyAV, VidGear, and Decord
- resolve color inversion bugs between BGR OpenCV arrays and RGB inputs
Example prompts for the agent
- Set up a Python subprocess pipe to feed raw BGR frames from FFmpeg into OpenCV
- Show how to replace video frames using OpenCV while remuxing original audio in FFmpeg
- Provide a Decord batch loading setup for video frames in a PyTorch pipeline
How to install ffmpeg-opencv-integration
1Skills CLI (Vercel)
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a claude-code2Manually: put the files in place
~/.claude/skills/ffmpeg-opencv-integration/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/ffmpeg-opencv-integration/
1Skills CLI (Vercel)
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a codex2Manually: put the files in place
.agents/skills/ffmpeg-opencv-integration/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a cursor2Manually: put the files in place
.cursor/skills/ffmpeg-opencv-integration/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install JosiahSiegel/claude-plugin-marketplace plugins/ffmpeg-python/skills/ffmpeg-opencv-integration2Manually: put the files in place
.github/skills/ffmpeg-opencv-integration/SKILL.md1Skills CLI (Vercel)
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a gemini-cli2Platform CLI
gemini skills install https://github.com/JosiahSiegel/claude-plugin-marketplace --path plugins/ffmpeg-python/skills/ffmpeg-opencv-integrationManually: put the files in place
.devin/skills/ffmpeg-opencv-integration/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about ffmpeg-opencv-integration
How to install ffmpeg-opencv-integration in Claude Code?
Run npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a claude-code. The file is stored at ~/.claude/skills/ffmpeg-opencv-integration/SKILL.md.
What is ffmpeg-opencv-integration?
It is an orchestration guide for Python video pipelines, supplying patterns for raw frame streaming, format conversions, and library selection across OpenCV and FFmpeg.
What dependencies are required for these pipelines?
You need system FFmpeg binaries, Python with NumPy and OpenCV, plus optional packages like Decord, ffmpegcv, or PyAV depending on the chosen pipeline.
Source text
SKILL.md · MIT
---
name: ffmpeg-opencv-integration
description: |
Complete FFmpeg + OpenCV + Python integration guide for video processing pipelines.
PROACTIVELY activate for: (1) FFmpeg to OpenCV frame handoff, (2) cv2.VideoCapture vs ffmpeg subprocess, (3) BGR/RGB color format conversion gotchas, (4) Frame dimension order img[y,x] vs img[x,y], (5) ffmpegcv GPU-accelerated video I/O, (6) VidGear multi-threaded streaming, (7) Decord batch video loading for ML, (8) PyAV frame-level processing, (9) Audio stream preservation with video filters, (10) Memory-efficient frame generators, (11) OpenCV + FFmpeg + Modal parallel processing, (12) Pipe frames between FFmpeg and OpenCV.
Provides: Color format conversion patterns, coordinate system gotchas, library selection guide, memory management, subprocess pipe patterns, GPU-accelerated alternatives to cv2.VideoCapture.
Ensures: Correct integration between FFmpeg and OpenCV without color/coordinate bugs.
See also: ffmpeg-python-integration-reference for type-safe parameter mappings.
---
# FFmpeg + OpenCV Integration Guide
Use this skill when FFmpeg handles video I/O and OpenCV handles image processing. This SKILL is a lean orchestrator; full pipe patterns, library examples, and Modal recipes are preserved in `references/opencv-pipelines-and-libraries.md`.
## When to Use
- Decode with FFmpeg and process frames with OpenCV
- Encode OpenCV-generated/processed frames with FFmpeg
- Compare `cv2.VideoCapture`, subprocess pipes, PyAV, ffmpegcv, VidGear, and Decord
- Fix RGB/BGR color bugs or `(height, width)` dimension-order bugs
- Preserve audio while replacing processed video frames
- Build GPU-assisted video I/O around CPU OpenCV processing
## Library Selection
| Need | Best option | Why |
|---|---|---|
| Simple local file | `cv2.VideoCapture` | Built-in and simple |
| Full FFmpeg format/protocol support | subprocess pipe | Exact CLI behavior |
| GPU video I/O | ffmpegcv | NVDEC/NVENC with OpenCV-like API |
| Network/RTSP streaming | VidGear | Threaded capture and stream helpers |
| ML batch loading | Decord | Fast random/batch frame access |
| Frame-level libav control | PyAV | Direct FFmpeg library access |
## Critical Gotchas
1. **OpenCV is BGR.** FFmpeg/PyAV/PIL/Decord often produce RGB. Convert explicitly.
2. **NumPy dimensions are `(height, width, channels)`.** Pixel access is `img[y, x]`, not `img[x, y]`.
3. **Video filters can drop audio.** Preserve or remux original audio intentionally.
4. **Release resources.** Always close `VideoCapture`, pipes, writers, and PyAV containers.
5. **Rawvideo pipes require exact frame size.** `width * height * channels` must match `-pix_fmt`.
## Minimal Pipe Patterns
FFmpeg to OpenCV using BGR frames:
```python
cmd = [
"ffmpeg", "-i", input_path,
"-f", "rawvideo", "-pix_fmt", "bgr24", "-"
]
```
OpenCV to FFmpeg using BGR frames:
```python
cmd = [
"ffmpeg", "-y",
"-f", "rawvideo", "-vcodec", "rawvideo",
"-s", f"{width}x{height}", "-pix_fmt", "bgr24",
"-r", str(fps), "-i", "-",
"-c:v", "libx264", "-preset", "fast", "-crf", "23",
"-pix_fmt", "yuv420p", output_path
]
```
## Core Workflow
1. Probe input dimensions, fps, duration, and audio streams.
2. Pick the I/O library based on the selection table.
3. Lock pixel format at boundaries (`bgr24` for OpenCV pipes; `yuv420p` for final H.264 output).
4. Process frames in generators/batches; avoid loading full videos unless they are small.
5. Preserve or re-encode audio explicitly.
6. Verify output duration, fps, resolution, codec, and A/V sync.
## Reference Map
- `references/opencv-pipelines-and-libraries.md` - Full preserved reference: color/dimension gotchas, cleanup patterns, FFmpeg-to-OpenCV and OpenCV-to-FFmpeg pipes, bidirectional pipeline, ffmpegcv, VidGear, Decord, PyAV, Modal.com examples, GPU pipeline, cheat sheets, sources.
## Related Skills
- `ffmpeg-python-integration-reference` - Type-safe parameters, colors, time units
- `ffmpeg-pyav-integration` - PyAV API details
- `ffmpeg-hardware-acceleration` - GPU decode/encode and filter pipelines
© JosiahSiegel · MIT · JosiahSiegel/claude-plugin-marketplace