Skip to main content
skill CX Codex

ffmpeg-opencv-integration — Claude Code skill: what it does

ffmpeg-opencv-integration — a skill for Claude Code and Codex guides video processing pipelines using FFmpeg and OpenCV. Prevents RGB/BGR mismatches, pipe stalls, and dropped audio tracks.

MIT 107 installs ★ 66 updated 2026-06-18

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a claude-code
Open on GitHub

What ffmpeg-opencv-integration does

The skill guides video frame handoff between FFmpeg and OpenCV in Python pipelines. It standardizes raw video streaming over subprocess pipes, enforcing explicit pixel formats like bgr24 on the OpenCV boundary and yuv420p for destination H.264 encoders.

It outlines library tradeoffs across standard use cases: standard cv2.VideoCapture for local playback, ffmpegcv for NVDEC and NVENC GPU tasks, VidGear for multi-threaded RTSP ingestion, Decord for ML batch sampling, and PyAV for direct libav container inspection. It also links Modal recipes for cloud processing.

Critical checks focus on NumPy coordinate ordering (img[y, x] instead of img[x, y]) and exact buffer sizing where width * height * channels must match -pix_fmt. The host environment requires FFmpeg binaries and OpenCV installed.

What it helps with

  • stream raw frames between FFmpeg processes and OpenCV buffers
  • preserve synchronized audio streams during frame-level video modifications
  • select suitable ingestion libraries across PyAV, VidGear, and Decord
  • resolve color inversion bugs between BGR OpenCV arrays and RGB inputs

Example prompts for the agent

  • Set up a Python subprocess pipe to feed raw BGR frames from FFmpeg into OpenCV
  • Show how to replace video frames using OpenCV while remuxing original audio in FFmpeg
  • Provide a Decord batch loading setup for video frames in a PyTorch pipeline

How to install ffmpeg-opencv-integration

1Skills CLI (Vercel)

npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a claude-code

2Manually: put the files in place

~/.claude/skills/ffmpeg-opencv-integration/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/ffmpeg-opencv-integration/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about ffmpeg-opencv-integration

How to install ffmpeg-opencv-integration in Claude Code?

Run npx skills add JosiahSiegel/claude-plugin-marketplace --skill ffmpeg-opencv-integration -a claude-code. The file is stored at ~/.claude/skills/ffmpeg-opencv-integration/SKILL.md.

What is ffmpeg-opencv-integration?

It is an orchestration guide for Python video pipelines, supplying patterns for raw frame streaming, format conversions, and library selection across OpenCV and FFmpeg.

What dependencies are required for these pipelines?

You need system FFmpeg binaries, Python with NumPy and OpenCV, plus optional packages like Decord, ffmpegcv, or PyAV depending on the chosen pipeline.

Source text

SKILL.md · MIT
---
name: ffmpeg-opencv-integration
description: |
  Complete FFmpeg + OpenCV + Python integration guide for video processing pipelines.
  PROACTIVELY activate for: (1) FFmpeg to OpenCV frame handoff, (2) cv2.VideoCapture vs ffmpeg subprocess, (3) BGR/RGB color format conversion gotchas, (4) Frame dimension order img[y,x] vs img[x,y], (5) ffmpegcv GPU-accelerated video I/O, (6) VidGear multi-threaded streaming, (7) Decord batch video loading for ML, (8) PyAV frame-level processing, (9) Audio stream preservation with video filters, (10) Memory-efficient frame generators, (11) OpenCV + FFmpeg + Modal parallel processing, (12) Pipe frames between FFmpeg and OpenCV.
  Provides: Color format conversion patterns, coordinate system gotchas, library selection guide, memory management, subprocess pipe patterns, GPU-accelerated alternatives to cv2.VideoCapture.
  Ensures: Correct integration between FFmpeg and OpenCV without color/coordinate bugs.
  See also: ffmpeg-python-integration-reference for type-safe parameter mappings.
---

# FFmpeg + OpenCV Integration Guide

Use this skill when FFmpeg handles video I/O and OpenCV handles image processing. This SKILL is a lean orchestrator; full pipe patterns, library examples, and Modal recipes are preserved in `references/opencv-pipelines-and-libraries.md`.

## When to Use

- Decode with FFmpeg and process frames with OpenCV
- Encode OpenCV-generated/processed frames with FFmpeg
- Compare `cv2.VideoCapture`, subprocess pipes, PyAV, ffmpegcv, VidGear, and Decord
- Fix RGB/BGR color bugs or `(height, width)` dimension-order bugs
- Preserve audio while replacing processed video frames
- Build GPU-assisted video I/O around CPU OpenCV processing

## Library Selection

| Need | Best option | Why |
|---|---|---|
| Simple local file | `cv2.VideoCapture` | Built-in and simple |
| Full FFmpeg format/protocol support | subprocess pipe | Exact CLI behavior |
| GPU video I/O | ffmpegcv | NVDEC/NVENC with OpenCV-like API |
| Network/RTSP streaming | VidGear | Threaded capture and stream helpers |
| ML batch loading | Decord | Fast random/batch frame access |
| Frame-level libav control | PyAV | Direct FFmpeg library access |

## Critical Gotchas

1. **OpenCV is BGR.** FFmpeg/PyAV/PIL/Decord often produce RGB. Convert explicitly.
2. **NumPy dimensions are `(height, width, channels)`.** Pixel access is `img[y, x]`, not `img[x, y]`.
3. **Video filters can drop audio.** Preserve or remux original audio intentionally.
4. **Release resources.** Always close `VideoCapture`, pipes, writers, and PyAV containers.
5. **Rawvideo pipes require exact frame size.** `width * height * channels` must match `-pix_fmt`.

## Minimal Pipe Patterns

FFmpeg to OpenCV using BGR frames:

```python
cmd = [
    "ffmpeg", "-i", input_path,
    "-f", "rawvideo", "-pix_fmt", "bgr24", "-"
]
```

OpenCV to FFmpeg using BGR frames:

```python
cmd = [
    "ffmpeg", "-y",
    "-f", "rawvideo", "-vcodec", "rawvideo",
    "-s", f"{width}x{height}", "-pix_fmt", "bgr24",
    "-r", str(fps), "-i", "-",
    "-c:v", "libx264", "-preset", "fast", "-crf", "23",
    "-pix_fmt", "yuv420p", output_path
]
```

## Core Workflow

1. Probe input dimensions, fps, duration, and audio streams.
2. Pick the I/O library based on the selection table.
3. Lock pixel format at boundaries (`bgr24` for OpenCV pipes; `yuv420p` for final H.264 output).
4. Process frames in generators/batches; avoid loading full videos unless they are small.
5. Preserve or re-encode audio explicitly.
6. Verify output duration, fps, resolution, codec, and A/V sync.

## Reference Map

- `references/opencv-pipelines-and-libraries.md` - Full preserved reference: color/dimension gotchas, cleanup patterns, FFmpeg-to-OpenCV and OpenCV-to-FFmpeg pipes, bidirectional pipeline, ffmpegcv, VidGear, Decord, PyAV, Modal.com examples, GPU pipeline, cheat sheets, sources.

## Related Skills

- `ffmpeg-python-integration-reference` - Type-safe parameters, colors, time units
- `ffmpeg-pyav-integration` - PyAV API details
- `ffmpeg-hardware-acceleration` - GPU decode/encode and filter pipelines

© JosiahSiegel · MIT · JosiahSiegel/claude-plugin-marketplace

Similar skills

All skills for AI and ML →

From the same set96

The whole set →
hook-development hook-development — a practical guide to building Claude Code hooks, from checks before a tool runs to tests after file changes. Use it to configure hook behavior or investigate why one does not fire. CX Codex +1 112installs ffmpeg-video-analysis The ffmpeg-video-analysis Claude Code skill uses FFmpeg filters to inspect video and measure quality. It activates for checks such as black or frozen frames, blur, scene changes, and crop detection. CX Codex +1 111installs viral-video-platform-specs The viral-video-platform-specs Claude Code skill selects export settings for social video platforms. It activates for upload specs, aspect-ratio changes, and multi-platform exports. CX Codex +1 108installs ffmpeg-glitch-distortion-effects ffmpeg-glitch-distortion-effects — a Claude Code skill for FFmpeg glitch effects and video distortion. It activates for datamosh, VHS looks, color shifts, pixelation, and related edits. CX Codex +1 107installs unity-ui-development unity-ui-development guides Unity interface creation across UGUI and UI Toolkit. Provides recipes for UXML structure, USS styling, RectTransform anchors, and runtime data binding. CX Codex +1 107installs viral-video-platform-algorithms The viral-video-platform-algorithms Claude Code skill explains TikTok, YouTube, Instagram, and Facebook algorithms. It activates for platform strategy, video reach, hashtags, and posting schedules. CX Codex +1 106installs azure-ml-foundry-workspace azure-ml-foundry-workspace guides Azure Machine Learning and Azure AI Foundry setup, from workspace networking to compute and model endpoints. Use it to plan configuration or investigate deployment logs. CX Codex +1 105installs ffmpeg-animation-timing-reference The ffmpeg-animation-timing-reference Claude Code skill explains timing units in FFmpeg and ASS/SSA subtitles. It helps select animation, transition, and caption durations and check audio-video sync. CX Codex +1 104installs

Project sponsors

Companies that keep this analytics open