К основному содержанию
скилл CX Codex

video-pipeline — скилл для Claude Code: видео от запроса до MP4

Скилл video-pipeline для Claude Code описывает путь от запроса до MP4 и распределяет работу между HyperFrames и Remotion. Запускается при создании видео или настройке очереди рендеринга.

MIT ★ 147 обновлён 2026-11-13

Установить в Claude Code · Через Skills CLI (Vercel) Все способы и платформы ↓
npx skills add notivn/AIEVv2 --skill video-pipeline -a claude-code
Открыть на GitHub

Что делает video-pipeline

video-pipeline задаёт этапы видеопроизводства: подготовку сцен в HyperFrames, сборку ролика в Remotion, проверку черновика и финальный экспорт. Список сцен и параметры проекта хранятся в meta.json; Remotion читает их оттуда, а промежуточные MP4 связывают два движка.

Перед сборкой нужно проверить кадры сцен через ffmpeg, а перед финальным рендером — запустить QC через API на localhost:6869. Проваленные проверки блокируют финальную сборку, пока черновик не исправлен и QC не пройден повторно. Для публикации скрипт генерирует субтитры и метаданные; если монтаж изменил транскрипт, нужна его финальная версия в assets/transcript.final.json.

Какие задачи решает

  • разложить производство видео на этапы от запроса до MP4
  • собрать сцены и исходные видео по данным meta.json
  • проверить черновик на ошибки изображения и звука перед финальным рендером
  • подготовить субтитры и пакет публикации для TikTok, YouTube и Facebook

Примеры запросов агенту

  • Создай видео по этому запросу и проведи его через весь конвейер до MP4.
  • Настрой очередь рендеринга для проекта и собирай список сцен из meta.json.
  • Проверь черновик через QC, исправь проваленные проверки и подготовь финальный ролик.

Как установить video-pipeline

1Через Skills CLI (Vercel)

npx skills add notivn/AIEVv2 --skill video-pipeline -a claude-code

2Вручную: положить файлы

~/.claude/skills/video-pipeline/SKILL.md

Папка скилла целиком; для одного проекта — .claude/skills/video-pipeline/

По шагам, с командами для каждой платформы: Как установить скилл в Claude Code: пять способов

Вопросы о video-pipeline

Как установить video-pipeline в Claude Code?

Выполни npx skills add notivn/AIEVv2 --skill video-pipeline -a claude-code. Файл скилла появится по пути ~/.claude/skills/video-pipeline/SKILL.md.

Что делает video-pipeline?

Описывает рабочий процесс от создания проекта и сцен до сборки MP4, проверок качества и подготовки файлов для публикации. HyperFrames создаёт сцены, Remotion собирает ролик.

Что нужно для работы video-pipeline?

В процессе используются HyperFrames, Remotion и ffmpeg. Для QC и публикации скрипт обращается к серверу по адресу localhost:6869.

Исходный текст

SKILL.md · MIT
---
name: video-pipeline
description: End-to-end video production workflow for the AI Edit Video system - from the user request to the MP4 in outputs/, coordinating HyperFrames (scenes) and Remotion (assembly). Read this when starting any video or when building the backend render queue.
---

# Video Pipeline - from request to MP4

## Engine roles (never swap them)

| Task | Engine | Why |
|---|---|---|
| Scene motion graphics: kinetic typography, karaoke captions, count-ups, data tables, callouts, shaders | **HyperFrames** | HTML + GSAP is its strength, and the Vietnamese skills already ship the fixes |
| Stitching scenes + source footage, transitions between scenes, mixing audio + sound effects, full-video caption overlay, final export | **Remotion** | Programmatic assembly, `<Sequence>`/`<Audio>`/`<OffthreadVideo>` |

How the two engines talk to each other: **intermediate MP4 files** in `video-projects/<name>/renders/`. HyperFrames does not know Remotion exists and vice versa - the only shared thing is the manifest.

## Project lifecycle

### 1. Initialize
- Create `video-projects/<kebab-case-name>/` with `index.html`, `compositions/`, `assets/`, `renders/`, `hyperframes.json`, `meta.json`.
- `meta.json` is the central manifest:

```json
{
  "id": "tiktok-paper-gpt5",
  "name": "Mổ xẻ paper GPT-5",
  "width": 1080, "height": 1920, "fps": 30,
  "status": "draft",
  "scenes": [
    { "id": "s01-hook",   "src": "compositions/s01-hook.html",   "durationInFrames": 90,  "render": "renders/s01-hook.mp4" },
    { "id": "s02-talking","srcVideo": "assets/talking-head.mp4", "from": 2.5, "to": 14.0 }
  ],
  "audio": {
    "voice": "assets/voice.mp3",
    "sfx": [ { "file": "assets/sound-effects/whoosh-01.mp3", "atFrame": 88 } ]
  },
  "output": null
}
```

- `scenes[]` is the contract between the two engines: a scene with `src` is rendered by HyperFrames; a scene with `srcVideo` is footage used as-is. Remotion reads this file to assemble - **never hardcode the scene list in the Remotion code**.
- **The user edits `meta.json` too.** The dashboard has a video editor (timeline + live Remotion preview) that writes `scenes`, `audio`, `captions`, `subtitles`, `subtitleStyle` and `overlays` straight into this file - between your turns, or before your session even started. So, every single time:
  1. **Read `meta.json` again right before you change it** - never trust the copy you read earlier in the session.
  2. **Edit in place** (the `Edit` tool on the exact keys you are changing). **Never `Write` the whole file back from memory**: that silently throws away whatever the user just dragged, trimmed or retyped.
  3. **Keep every field you do not recognise**, at any depth (extra keys on a scene, a cue, `audio`...). They belong to the user, the editor or the system.
  The server snapshots the timeline when your session starts and ends (`video-projects/<id>/.history/`), so the user can always roll back - but a clobbered edit is still lost work for them.

### 2. Build the scenes (HyperFrames)
- Copy `assets/brand/brand-tokens.css` into the project, write compositions following the `window.__timelines` convention.
- Vietnamese videos: apply the verified fixes (gradient text losing diacritics -> render text as real elements + use `background-clip` correctly; check that every diacritic is present on the first/last frame of each reveal).
- Lint clean before rendering: `npx hyperframes lint`.
- Render a draft of each scene: `npx hyperframes render --quality draft --output renders/<scene>.draft.mp4`.

### 3. Frame verification (MANDATORY, before assembly)
```bash
ffmpeg -ss <hero-moment-seconds> -i renders/<scene>.draft.mp4 -frames:v 1 verify/<scene>.png
```
Inspect every image: does the Vietnamese text keep all its diacritics? Is any text overflowing the edges? Any unexpected white/black frames? Is a face cropped? If anything is wrong, fix the scene - do not move on.

### 4. Assembly (Remotion) - see the `remotion-assemble` skill for details
- The Remotion composition reads `meta.json`, builds `<Sequence>` entries from `scenes[]`, and inserts sfx at `atFrame`.
- Render a full draft -> review it in the web UI -> approve.

### 5. Automated QC (MANDATORY, blocks final)
```bash
curl -s -X POST http://localhost:6869/api/projects/<id>/qc -H "content-type: application/json" -d "{}"
```
The server measures the latest draft with ffmpeg and returns `{ report: { status, checks[] } }`:

| check | meaning | what to do on fail |
|---|---|---|
| `resolution` | dimensions/fps do not match `meta.json` | re-render with the correct settings |
| `loudness` | far off -14 LUFS | adjust the mix volume, do not fix it by pulling the peak |
| `truepeak` | clipping (> -0.5 dBTP) | lower the source gain, see "SFX loudness and lead silence" in the `noti-tiktok-vn` skill (voice headroom before layering SFX) |
| `blackframes` | black frames in the MIDDLE of the video (leading/trailing fades are ignored) | a gap at a scene change - fix `transitionOverlap`/`durationInFrames` |
| `freeze` | frozen picture >= 2s | the scene is missing animation, or the footage is broken |
| `tail-silence` | silent tail > 1.2s | trim the tail |
| `av-duration` | picture and sound differ by > 0.5s | wrong total `durationInFrames` |
| `safe-area` | ALWAYS passes, returns `frames` = images with the obstructed bands outlined in red | **you MUST Read every image** and judge for yourself; if any text sits in the red zone, pull it inward, see the `key-layout` skill |

- `status: "fail"` -> the server rejects the `assemble-final` job with **409 QC_REQUIRED / QC_FAILED**. Fix the root cause, re-render the draft, run QC again. Do not use `force: true` to dodge it unless the user explicitly asks.
- `status: "warn"` -> consider fixing, not blocking.
- The final report must state the QC results and how each fail/warn check was handled.

### 6. Final
- Re-render the HyperFrames scenes at `--quality standard`.
- Remotion final render -> `outputs/<project>-v<N>.mp4`.
- Update `meta.json`: `status: "done"`, `output: "outputs/..."`. The web UI reads status from here.

### 7. Thumbnail + publish package
```bash
curl -s -X POST http://localhost:6869/api/projects/<id>/thumbnail -H "content-type: application/json" -d "{\"title\":\"...\",\"frameAt\":12}"
curl -s -X POST http://localhost:6869/api/projects/<id>/publish   -H "content-type: application/json" -d "{}"
```
`publish` generates `.srt`/`.vtt` from the transcript and has the AI write the title/description/hashtags for TikTok, YouTube and Facebook following the Style Design. **Precondition**: the transcript must be the FINAL one. If it was cut/remapped, write the final version to `assets/transcript.final.json` (the transcript loader prefers this file) - otherwise the subtitles will drift out of sync with the cut video.

## Source kind: footage, article or text request

`brief.sourceKind` says what the video is built from (the "Nguồn video" card on the project page). The edit prompt always prints a "Nguồn của video" section for it:
- `video` (default) - edit the uploaded files in `assets/`.
- `article` - no footage. The article text is in `assets/source-article.md` (fetched from `brief.sourceUrl`; if the file is missing, `POST /api/projects/:id/source/article` first). Build a motion graphics video that retells its key points.
- `request` - no footage. `brief.sourceText` is the user's request/content; build a motion graphics video for it. Never invent figures, names or quotes it does not give.

Voice-over for the two no-footage kinds is `brief.narrationMode` (the prompt prints it in all three directions):
- `none` - no voice, no transcript: auto-cut does not apply and karaoke has no speech to sync to.
- `tts` - write the narration script, then `POST /api/projects/:id/narration {"script": ...}` and poll the task. The server reads it in EXACTLY `brief.narrationVoice` (the body cannot pick a voice - the user chose it), writes `assets/narration.wav` + `assets/narration.transcript.json` and sets `meta.audio.voice`.
- `file` - the user's recording `brief.narrationFile`: `POST /narration {}` only transcribes it. The recording is the timeline and the main content.
With a voice, the voice is the timeline: video length = voice length, scenes and karaoke follow the transcript, music ducks under it. Images come from AI illustrations only when `brief.autoIllustrations` is on (the server refuses otherwise) - off means text + HTML/SVG graphics + images already in `assets/`. Prefer `noti-tiktok-full-text` (vertical) or `noti-youtube-edit` mode B (landscape) over talking-head skills.

## Finish mode: stop at the draft, or go all the way to final

`brief.finishMode` decides where an AI edit session stops. The edit prompt always states the mode, in both directions - follow the prompt over the step list above when they disagree.

| `finishMode` | Session goal | Where you stop | Done means |
|---|---|---|---|
| `"draft"` (default) | `draft` | after step 5 (QC) | a NEW `assemble-draft` **job** finished during this run, and QC ran on that exact `outputs/<id>-draft.mp4` |
| `"final"` | `final` | after step 7 | `outputs/<id>-v<N>.mp4` exists and `meta.json` has `status: "done"` + `output` |

In `draft` mode:
- **Never render final**: no `assemble-final` / `scene-final` job, no `--quality standard` CLI render, no thumbnail, no publish package, do not set `status: "done"` or `output` - even though steps 6-7 above say so. The user reviews the draft, edits it by hand, and clicks "Xuất final" themselves.
- The final full draft must come from a queued job (`POST /api/projects/<id>/editor/render` with `{"quality":"draft"}` queues the stale scene drafts + `assemble-draft`). A draft rendered by hand with the CLI has no job row, so the server gate never sees it and keeps auto-resuming the session.
- Run QC on that draft. A `fail` still has to be fixed (re-render the draft, QC again) - the gate only checks that QC ran, but the user will hit `QC_FAILED` on export.
- End with a report: what was built, the QC result, and that the draft is waiting for review in the editor.

The batch flows (short clips, aspect-ratio repurposing, Text to video, Auto cut) still open `final` sessions, whatever the child project's brief says.

### Handoff from the editor ("Giao cho AI")

`POST /api/projects/<id>/editor/handoff {instruction, finishMode?}` opens a new session (default goal `draft`) that **continues from the current timeline**. The prompt opens with a "BÀN GIAO TỪ TRÌNH CHỈNH SỬA" block. The user has already edited by hand, so:
- Re-read `meta.json` first and before every edit. Treat what is there as approved.
- **Keep every manual edit**: scene order/length/`from`/`to`, cues, sfx, music, and the v2 fields `layers[]` (with `animation`, `keyframes`), `fit`, `transform`, `crop`, `color`, `volume`, `fadeInSec`/`fadeOutSec`, `speed`, `transition`. Change only what the instruction needs. Unknown fields stay.
- Do not redo brief steps that are already done (auto-cut already applied - check `assets/auto-trim-report.json` / `transcript.cut.json`; subtitles already there; music already chosen). Do the missing part plus the instruction.
- Do not delete and rebuild existing HyperFrames compositions; edit only the scenes that need it.
- The server snapshots an `ai-before` revision before you start, so the user can undo your whole run in one click. That is no excuse for sloppy edits.

**The user may edit between turns.** In any session, the editor can write `meta.json` while you are idle (between turns, or while you wait for a job). Never keep `meta.json` in memory across a wait: Read it again right before each Edit, edit in place, never Write the whole file from memory.

## Render queue rules (backend)

1. **Every render goes through the queue** - including the ones Claude runs by hand. Jobs are written to SQLite: `id, projectId, type (scene-draft|scene-final|assemble-draft|assemble-final|image-gen), status, progress, log, startedAt, finishedAt`.
2. Jobs run **in parallel up to `QUEUE_CONCURRENCY`** (default 2, configurable via env; set 1 on weak machines). Safety constraint: two jobs **from the same project never run at the same time** (to avoid trampling renders/meta) - parallelism only happens across different projects.
3. Progress: parse the CLI stdout (both engines print frame progress) -> update the DB -> push SSE to the web UI.
4. Failed jobs: keep the complete log in the DB, show it in the UI, and **do not auto-retry more than once**.
5. Draft always comes before final. The backend rejects a final job if the project has no successful draft for the current version of the scenes.

## Sound effects

- Shared library: `assets/sound-effects/`, every file has an entry in `assets/sound-effects/library.json` (`file`, `tags`, `durationMs`, and a `description` written in Vietnamese).
- When using one in a video: copy it into `video-projects/<name>/assets/sound-effects/` then declare it in `meta.json` - a project must contain all of its own assets (re-rendering must not depend on a library that may change).
- The Sound Effects page in the web UI reads `library.json`, plays previews inline, and allows uploading new files (the backend updates the json).

## Speeding up renders (this machine has a GTX 1660 - verified 2026-07)

The number one reason the pipeline burns an entire hour: **re-rendering a draft of the WHOLE video after every small edit**. The rules:

1. **Verify layout/text with `npx hyperframes snapshot` or `inspect`** (a few seconds) instead of rendering
   a full draft (tens of minutes). Render a full draft only ONCE, when every snapshot is good, and final ONCE.
2. **Small edit -> re-render only what changed**: if you edited one scene, re-render that scene (`render -c <scene>`),
   do not re-render the whole composition. The split-scene + Remotion-assemble pipeline is built exactly for this.
3. **HyperFrames speed flags** (the backend queue adds them automatically; when running by hand you MUST remember them):
   `-w 8 --browser-gpu` for every render; add `--gpu` (NVENC) for drafts. Final keeps CPU encoding
   (libx264) for maximum quality. Measured in practice: ~30% faster on short scenes, more on long ones.
4. **Transcription runs only ONCE** - if `transcript.json` already exists, reuse it, NEVER transcribe again.
5. **Remotion speed flags** (when running `npx remotion render/still` by hand you MUST add):
   `--concurrency 8 --gl angle`. WITHOUT `--gl angle`, Remotion's Chrome renders with the
   software renderer (SwANGLE) - the CPU carries 100% while the GPU idles (symptom: Task Manager CPU ~95%,
   GPU ~5%). On Linux use `--gl angle-egl` instead of `angle`.

## Known issues (verified in production 2026-07)

- **`meta.json` must respect the data-type contract** (the web UI reads it directly - a wrong type crashes the page):
  `output` is a path **STRING** (e.g. `"outputs/<id>-v1.mp4"`), NOT an object. Extra metadata
  (duration, quality, renderedAt, ...) goes into a separate `outputInfo` field if needed. `scenes[].durationInFrames`
  is a number; sfx are placed with `atFrame` (a frame number) as in the schema at the top of this file.

- **Rendering a single scene**: use the `-c` flag, not a positional argument: `npx hyperframes render -c compositions/s01.html --quality draft --output renders/s01.draft.mp4`. A `<template>` sub-composition must be referenced by index.html through `data-composition-src` for `-c` to be able to render it.
- **Words running together on per-word reveals**: the HyperFrames pipeline swallows the whitespace between inline-block `<span>`s - separate words with `margin: 0 0.14em` on `.word`, do not rely on whitespace in the HTML.
- **The `sub_timeline_readiness_timeout` warning** when rendering a template file with `-c`: the render still comes out correct (best-effort) but wastes another 45s waiting - acceptable for drafts; if you want it gone entirely, render through index.html.
- **A TS comment containing the glob path `*/`** (e.g. `video-projects/*/meta.json`) closes the block comment early -> a baffling compile error. Write `video-projects/<id>/meta.json` instead.
- **Vietnamese JSON sent with curl -d from Git Bash on Windows loses its diacritics** (verified 2026-08: the
  thumbnail title arrived at the server as "B?n th? kh?ch..." and rendered broken). The console codepage mangles
  UTF-8 before curl sends it. For any API body containing Vietnamese, POST from Node instead:
  `node -e "fetch(url, { method: 'POST', headers: {'content-type': 'application/json'}, body: JSON.stringify({...}) })"`
  (or write the JSON to a file and use `curl --data-binary @file`).

## Agent sandbox (what the policy blocks)

Every tool call from the in-app agent passes through `apps/server/src/agentPolicy.ts` (a PreToolUse hook) before it runs. The prompt always carries untrusted text (transcripts, imported articles, file descriptions), so these are refused no matter what the prompt says:

- **Network**: only the local API (`http://localhost:6869`) - `curl` to any other host, `wget`, `Invoke-WebRequest`, proxies/`--resolve`, external URLs inside `node -e`/scripts, and UNC paths. Brand logos come from `POST /api/brand-logos`, images from `/api/illustrations`; never download them yourself.
- **Installs**: `npm install/i/ci/exec`, `npx <anything except hyperframes/remotion/tsc/tsx>` (or with `@version`/`-y`), `pip`/`python -m pip`, `winget`/`choco`. Missing dependency = ask the user to run Doctor.
- **Secrets**: `.env`, `~/.claude/.credentials.json`, `~/.ssh`... API keys are not in your environment either - the server holds them.
- **Self-escalation**: writing `.claude/settings*.json`, `.mcp.json`, `agentPolicy.ts`, `.git/hooks`; user-only APIs (`/api/connections`, `/api/tunnel`, `/api/update`, `/api/health`, `/api/chat`, ...); WebFetch/WebSearch and MCP tools.

A refusal comes back as the tool error with the reason - change approach, do not try to get around it. To allow something new, edit the lists at the top of `agentPolicy.ts` (a human change, never the agent's).

## Checklist before reporting completion

- [ ] If the brief enables autoCut: the cut was done per the `auto-cut` skill, a second silencedetect pass verified the cut version, and the report states the seconds/segments removed
- [ ] Every scene passed frame verification, Vietnamese text keeps all its diacritics
- [ ] Audio is in sync at the start/middle/end (check 3 random points)
- [ ] Sound effects land on the right frames, and their volume does not bury the voice (sfx ~10dB below voice)
- [ ] The output matches the dimensions/fps in `meta.json`
- [ ] Automated QC ran on the draft with no remaining `fail` checks (the report spells out any leftover warn checks)
- [ ] `finishMode: "final"` only: `meta.json` has updated `status` + `output`
- [ ] `finishMode: "final"` only: the thumbnail was generated and Read to verify the diacritics are complete
- [ ] `finishMode: "final"` only: the publish package was generated (`.srt`/`.vtt` + metadata), using the FINAL post-cut transcript
- [ ] `finishMode: "draft"`: the draft came from an `assemble-draft` job of this run, QC ran on it, and nothing final was rendered
- [ ] Any new lesson learned was written into the relevant skill

© notivn · MIT · notivn/AIEVv2

Похожие скиллы

Все скиллы по теме «ИИ и ML» →
video coreyhaines31 Скилл video для Claude Code и Cursor помогает проектировать и создавать видео через нейросети или код. Он выбирает подходящий стек, пишет код для генерации и составляет промпты к моделям. CX Codex +1 98 700установок yyl-remotion-video ttfake92-lab Скилл yyl-remotion-video для Claude Code и других агентов: берёт статью, сценарий или заметки и собирает проект Remotion, рендерит mp4 без звука. CX Codex +1 549установок video-edit calesthio Скилл video-edit для Claude Code обрабатывает локальные видеофайлы через ffmpeg: нарезает ролики, меняет пропорции под соцсети и сжимает дорожки. CX Codex +1 1 600установок ffmpeg digitalsamba Готовые команды FFmpeg для конвертации, сжатия, обрезки видео и подготовки ассетов к проектам на Remotion. CX Codex +1 16 500установок reverse-any-video heygen-com Скилл reverse-any-video для Claude Code воссоздает любое видео в редактируемый проект HyperFrames с покадровой сверкой по оригиналу. CX Codex +1 180 600установок review-video ShuaiMXu Скилл review-video для Claude Code проверяет и чинит продуктовые или промо-видео по исходному проекту, таймкодам изменений и независимой ревизии таймлайна. Запускается при видеопроизводстве или разборе багов монтажа. CX Codex +1 ★ 529звёзд репо

Из того же набора15

Весь набор →
make-a-video make-a-video — скилл для Claude Code, который ведёт от идеи к готовому MP4 через HyperFrames. Он собирает требования, готовит сценарный план и проверяет видео перед финальным рендером. CX Codex +1 ★ 147звёзд репо auto-cut Скилл auto-cut для Claude Code убирает тишину, оговорки и повторы дублей из видео с говорящей головой до монтажа. Включается по флагу autoCut в брифе или при жалобах на паузы. CX Codex +1 ★ 147звёзд репо ai-illustrations Скилл ai-illustrations для Claude Code добавляет сгенерированные изображения в видеомонтаж. Он включается, когда в задании разрешены AI-иллюстрации или запрошены изображения для видео. CX Codex +1 ★ 147звёзд репо remotion-assemble Скилл remotion-assemble для Claude Code настраивает сборку видео из сцен HyperFrames, футажей, голоса и звуков по файлу meta.json. Инструкции включаются при редактировании кода в каталоге engines/remotion. CX Codex +1 ★ 147звёзд репо noti-tiktok-full-text Скилл noti-tiktok-full-text для Claude Code собирает вертикальные TikTok-видео с разбором научных статей через HyperFrames. Включается по стенограмме, иллюстрациям и сканам страниц PDF. CX Codex +1 ★ 147звёзд репо short-form-video Скилл short-form-video для Claude Code помогает собирать и править вертикальные ролики в Hyperframes с говорящей головой, графикой и караоке-субтитрами. Запускается по запросам о TikTok, Reels, Shorts и пересборке короткого видео. CX Codex +1 ★ 147звёзд репо noti-tiktok-vn Скилл noti-tiktok-vn для Claude Code монтирует вертикальные видео 9:16 для TikTok через HyperFrames. Собирает говорящую голову, караоке-субтитры, зумы и типографику по стилю Noti.vn в готовый MP4. CX Codex +1 ★ 147звёзд репо noti-youtube-edit Скилл noti-youtube-edit для Claude Code собирает горизонтальные YouTube-видео из вьетнамского клипа или расшифровки. Есть режим с говорящей головой и режим текстового объяснения. CX Codex +1 ★ 147звёзд репо

Спонсоры проекта

Компании, которые помогают держать аналитику открытой