video-pipeline — скилл для Claude Code: видео от запроса до MP4
Скилл video-pipeline для Claude Code описывает путь от запроса до MP4 и распределяет работу между HyperFrames и Remotion. Запускается при создании видео или настройке очереди рендеринга.
MIT ★ 147 обновлён 2026-11-13
npx skills add notivn/AIEVv2 --skill video-pipeline -a claude-codeЧто делает video-pipeline
video-pipeline задаёт этапы видеопроизводства: подготовку сцен в HyperFrames, сборку ролика в Remotion, проверку черновика и финальный экспорт. Список сцен и параметры проекта хранятся в meta.json; Remotion читает их оттуда, а промежуточные MP4 связывают два движка.
Перед сборкой нужно проверить кадры сцен через ffmpeg, а перед финальным рендером — запустить QC через API на localhost:6869. Проваленные проверки блокируют финальную сборку, пока черновик не исправлен и QC не пройден повторно. Для публикации скрипт генерирует субтитры и метаданные; если монтаж изменил транскрипт, нужна его финальная версия в assets/transcript.final.json.
Какие задачи решает
- разложить производство видео на этапы от запроса до MP4
- собрать сцены и исходные видео по данным
meta.json - проверить черновик на ошибки изображения и звука перед финальным рендером
- подготовить субтитры и пакет публикации для TikTok, YouTube и Facebook
Примеры запросов агенту
- Создай видео по этому запросу и проведи его через весь конвейер до MP4.
- Настрой очередь рендеринга для проекта и собирай список сцен из
meta.json. - Проверь черновик через QC, исправь проваленные проверки и подготовь финальный ролик.
Как установить video-pipeline
1Через Skills CLI (Vercel)
npx skills add notivn/AIEVv2 --skill video-pipeline -a claude-code2Вручную: положить файлы
~/.claude/skills/video-pipeline/SKILL.mdПапка скилла целиком; для одного проекта — .claude/skills/video-pipeline/
1Через Skills CLI (Vercel)
npx skills add notivn/AIEVv2 --skill video-pipeline -a codex2Вручную: положить файлы
.agents/skills/video-pipeline/SKILL.mdДля всех проектов — ~/.agents/skills/
1Через Skills CLI (Vercel)
npx skills add notivn/AIEVv2 --skill video-pipeline -a cursor2Вручную: положить файлы
.cursor/skills/video-pipeline/SKILL.mdCursor читает и .agents/skills/, и .claude/skills/
1GitHub CLI
gh skill install notivn/AIEVv2 .claude/skills/video-pipeline2Вручную: положить файлы
.github/skills/video-pipeline/SKILL.md1Через Skills CLI (Vercel)
npx skills add notivn/AIEVv2 --skill video-pipeline -a gemini-cli2Командой платформы
gemini skills install https://github.com/notivn/AIEVv2 --path .claude/skills/video-pipelineВручную: положить файлы
.devin/skills/video-pipeline/SKILL.mdСтарый путь .windsurf/skills/ тоже читается
Claude.ai
Упакуйте папку в zip и загрузите в Settings → Capabilities → Skills
По шагам, с командами для каждой платформы: Как установить скилл в Claude Code: пять способов
Вопросы о video-pipeline
Как установить video-pipeline в Claude Code?
Выполни npx skills add notivn/AIEVv2 --skill video-pipeline -a claude-code. Файл скилла появится по пути ~/.claude/skills/video-pipeline/SKILL.md.
Что делает video-pipeline?
Описывает рабочий процесс от создания проекта и сцен до сборки MP4, проверок качества и подготовки файлов для публикации. HyperFrames создаёт сцены, Remotion собирает ролик.
Что нужно для работы video-pipeline?
В процессе используются HyperFrames, Remotion и ffmpeg. Для QC и публикации скрипт обращается к серверу по адресу localhost:6869.
Исходный текст
SKILL.md · MIT
---
name: video-pipeline
description: End-to-end video production workflow for the AI Edit Video system - from the user request to the MP4 in outputs/, coordinating HyperFrames (scenes) and Remotion (assembly). Read this when starting any video or when building the backend render queue.
---
# Video Pipeline - from request to MP4
## Engine roles (never swap them)
| Task | Engine | Why |
|---|---|---|
| Scene motion graphics: kinetic typography, karaoke captions, count-ups, data tables, callouts, shaders | **HyperFrames** | HTML + GSAP is its strength, and the Vietnamese skills already ship the fixes |
| Stitching scenes + source footage, transitions between scenes, mixing audio + sound effects, full-video caption overlay, final export | **Remotion** | Programmatic assembly, `<Sequence>`/`<Audio>`/`<OffthreadVideo>` |
How the two engines talk to each other: **intermediate MP4 files** in `video-projects/<name>/renders/`. HyperFrames does not know Remotion exists and vice versa - the only shared thing is the manifest.
## Project lifecycle
### 1. Initialize
- Create `video-projects/<kebab-case-name>/` with `index.html`, `compositions/`, `assets/`, `renders/`, `hyperframes.json`, `meta.json`.
- `meta.json` is the central manifest:
```json
{
"id": "tiktok-paper-gpt5",
"name": "Mổ xẻ paper GPT-5",
"width": 1080, "height": 1920, "fps": 30,
"status": "draft",
"scenes": [
{ "id": "s01-hook", "src": "compositions/s01-hook.html", "durationInFrames": 90, "render": "renders/s01-hook.mp4" },
{ "id": "s02-talking","srcVideo": "assets/talking-head.mp4", "from": 2.5, "to": 14.0 }
],
"audio": {
"voice": "assets/voice.mp3",
"sfx": [ { "file": "assets/sound-effects/whoosh-01.mp3", "atFrame": 88 } ]
},
"output": null
}
```
- `scenes[]` is the contract between the two engines: a scene with `src` is rendered by HyperFrames; a scene with `srcVideo` is footage used as-is. Remotion reads this file to assemble - **never hardcode the scene list in the Remotion code**.
- **The user edits `meta.json` too.** The dashboard has a video editor (timeline + live Remotion preview) that writes `scenes`, `audio`, `captions`, `subtitles`, `subtitleStyle` and `overlays` straight into this file - between your turns, or before your session even started. So, every single time:
1. **Read `meta.json` again right before you change it** - never trust the copy you read earlier in the session.
2. **Edit in place** (the `Edit` tool on the exact keys you are changing). **Never `Write` the whole file back from memory**: that silently throws away whatever the user just dragged, trimmed or retyped.
3. **Keep every field you do not recognise**, at any depth (extra keys on a scene, a cue, `audio`...). They belong to the user, the editor or the system.
The server snapshots the timeline when your session starts and ends (`video-projects/<id>/.history/`), so the user can always roll back - but a clobbered edit is still lost work for them.
### 2. Build the scenes (HyperFrames)
- Copy `assets/brand/brand-tokens.css` into the project, write compositions following the `window.__timelines` convention.
- Vietnamese videos: apply the verified fixes (gradient text losing diacritics -> render text as real elements + use `background-clip` correctly; check that every diacritic is present on the first/last frame of each reveal).
- Lint clean before rendering: `npx hyperframes lint`.
- Render a draft of each scene: `npx hyperframes render --quality draft --output renders/<scene>.draft.mp4`.
### 3. Frame verification (MANDATORY, before assembly)
```bash
ffmpeg -ss <hero-moment-seconds> -i renders/<scene>.draft.mp4 -frames:v 1 verify/<scene>.png
```
Inspect every image: does the Vietnamese text keep all its diacritics? Is any text overflowing the edges? Any unexpected white/black frames? Is a face cropped? If anything is wrong, fix the scene - do not move on.
### 4. Assembly (Remotion) - see the `remotion-assemble` skill for details
- The Remotion composition reads `meta.json`, builds `<Sequence>` entries from `scenes[]`, and inserts sfx at `atFrame`.
- Render a full draft -> review it in the web UI -> approve.
### 5. Automated QC (MANDATORY, blocks final)
```bash
curl -s -X POST http://localhost:6869/api/projects/<id>/qc -H "content-type: application/json" -d "{}"
```
The server measures the latest draft with ffmpeg and returns `{ report: { status, checks[] } }`:
| check | meaning | what to do on fail |
|---|---|---|
| `resolution` | dimensions/fps do not match `meta.json` | re-render with the correct settings |
| `loudness` | far off -14 LUFS | adjust the mix volume, do not fix it by pulling the peak |
| `truepeak` | clipping (> -0.5 dBTP) | lower the source gain, see "SFX loudness and lead silence" in the `noti-tiktok-vn` skill (voice headroom before layering SFX) |
| `blackframes` | black frames in the MIDDLE of the video (leading/trailing fades are ignored) | a gap at a scene change - fix `transitionOverlap`/`durationInFrames` |
| `freeze` | frozen picture >= 2s | the scene is missing animation, or the footage is broken |
| `tail-silence` | silent tail > 1.2s | trim the tail |
| `av-duration` | picture and sound differ by > 0.5s | wrong total `durationInFrames` |
| `safe-area` | ALWAYS passes, returns `frames` = images with the obstructed bands outlined in red | **you MUST Read every image** and judge for yourself; if any text sits in the red zone, pull it inward, see the `key-layout` skill |
- `status: "fail"` -> the server rejects the `assemble-final` job with **409 QC_REQUIRED / QC_FAILED**. Fix the root cause, re-render the draft, run QC again. Do not use `force: true` to dodge it unless the user explicitly asks.
- `status: "warn"` -> consider fixing, not blocking.
- The final report must state the QC results and how each fail/warn check was handled.
### 6. Final
- Re-render the HyperFrames scenes at `--quality standard`.
- Remotion final render -> `outputs/<project>-v<N>.mp4`.
- Update `meta.json`: `status: "done"`, `output: "outputs/..."`. The web UI reads status from here.
### 7. Thumbnail + publish package
```bash
curl -s -X POST http://localhost:6869/api/projects/<id>/thumbnail -H "content-type: application/json" -d "{\"title\":\"...\",\"frameAt\":12}"
curl -s -X POST http://localhost:6869/api/projects/<id>/publish -H "content-type: application/json" -d "{}"
```
`publish` generates `.srt`/`.vtt` from the transcript and has the AI write the title/description/hashtags for TikTok, YouTube and Facebook following the Style Design. **Precondition**: the transcript must be the FINAL one. If it was cut/remapped, write the final version to `assets/transcript.final.json` (the transcript loader prefers this file) - otherwise the subtitles will drift out of sync with the cut video.
## Source kind: footage, article or text request
`brief.sourceKind` says what the video is built from (the "Nguồn video" card on the project page). The edit prompt always prints a "Nguồn của video" section for it:
- `video` (default) - edit the uploaded files in `assets/`.
- `article` - no footage. The article text is in `assets/source-article.md` (fetched from `brief.sourceUrl`; if the file is missing, `POST /api/projects/:id/source/article` first). Build a motion graphics video that retells its key points.
- `request` - no footage. `brief.sourceText` is the user's request/content; build a motion graphics video for it. Never invent figures, names or quotes it does not give.
Voice-over for the two no-footage kinds is `brief.narrationMode` (the prompt prints it in all three directions):
- `none` - no voice, no transcript: auto-cut does not apply and karaoke has no speech to sync to.
- `tts` - write the narration script, then `POST /api/projects/:id/narration {"script": ...}` and poll the task. The server reads it in EXACTLY `brief.narrationVoice` (the body cannot pick a voice - the user chose it), writes `assets/narration.wav` + `assets/narration.transcript.json` and sets `meta.audio.voice`.
- `file` - the user's recording `brief.narrationFile`: `POST /narration {}` only transcribes it. The recording is the timeline and the main content.
With a voice, the voice is the timeline: video length = voice length, scenes and karaoke follow the transcript, music ducks under it. Images come from AI illustrations only when `brief.autoIllustrations` is on (the server refuses otherwise) - off means text + HTML/SVG graphics + images already in `assets/`. Prefer `noti-tiktok-full-text` (vertical) or `noti-youtube-edit` mode B (landscape) over talking-head skills.
## Finish mode: stop at the draft, or go all the way to final
`brief.finishMode` decides where an AI edit session stops. The edit prompt always states the mode, in both directions - follow the prompt over the step list above when they disagree.
| `finishMode` | Session goal | Where you stop | Done means |
|---|---|---|---|
| `"draft"` (default) | `draft` | after step 5 (QC) | a NEW `assemble-draft` **job** finished during this run, and QC ran on that exact `outputs/<id>-draft.mp4` |
| `"final"` | `final` | after step 7 | `outputs/<id>-v<N>.mp4` exists and `meta.json` has `status: "done"` + `output` |
In `draft` mode:
- **Never render final**: no `assemble-final` / `scene-final` job, no `--quality standard` CLI render, no thumbnail, no publish package, do not set `status: "done"` or `output` - even though steps 6-7 above say so. The user reviews the draft, edits it by hand, and clicks "Xuất final" themselves.
- The final full draft must come from a queued job (`POST /api/projects/<id>/editor/render` with `{"quality":"draft"}` queues the stale scene drafts + `assemble-draft`). A draft rendered by hand with the CLI has no job row, so the server gate never sees it and keeps auto-resuming the session.
- Run QC on that draft. A `fail` still has to be fixed (re-render the draft, QC again) - the gate only checks that QC ran, but the user will hit `QC_FAILED` on export.
- End with a report: what was built, the QC result, and that the draft is waiting for review in the editor.
The batch flows (short clips, aspect-ratio repurposing, Text to video, Auto cut) still open `final` sessions, whatever the child project's brief says.
### Handoff from the editor ("Giao cho AI")
`POST /api/projects/<id>/editor/handoff {instruction, finishMode?}` opens a new session (default goal `draft`) that **continues from the current timeline**. The prompt opens with a "BÀN GIAO TỪ TRÌNH CHỈNH SỬA" block. The user has already edited by hand, so:
- Re-read `meta.json` first and before every edit. Treat what is there as approved.
- **Keep every manual edit**: scene order/length/`from`/`to`, cues, sfx, music, and the v2 fields `layers[]` (with `animation`, `keyframes`), `fit`, `transform`, `crop`, `color`, `volume`, `fadeInSec`/`fadeOutSec`, `speed`, `transition`. Change only what the instruction needs. Unknown fields stay.
- Do not redo brief steps that are already done (auto-cut already applied - check `assets/auto-trim-report.json` / `transcript.cut.json`; subtitles already there; music already chosen). Do the missing part plus the instruction.
- Do not delete and rebuild existing HyperFrames compositions; edit only the scenes that need it.
- The server snapshots an `ai-before` revision before you start, so the user can undo your whole run in one click. That is no excuse for sloppy edits.
**The user may edit between turns.** In any session, the editor can write `meta.json` while you are idle (between turns, or while you wait for a job). Never keep `meta.json` in memory across a wait: Read it again right before each Edit, edit in place, never Write the whole file from memory.
## Render queue rules (backend)
1. **Every render goes through the queue** - including the ones Claude runs by hand. Jobs are written to SQLite: `id, projectId, type (scene-draft|scene-final|assemble-draft|assemble-final|image-gen), status, progress, log, startedAt, finishedAt`.
2. Jobs run **in parallel up to `QUEUE_CONCURRENCY`** (default 2, configurable via env; set 1 on weak machines). Safety constraint: two jobs **from the same project never run at the same time** (to avoid trampling renders/meta) - parallelism only happens across different projects.
3. Progress: parse the CLI stdout (both engines print frame progress) -> update the DB -> push SSE to the web UI.
4. Failed jobs: keep the complete log in the DB, show it in the UI, and **do not auto-retry more than once**.
5. Draft always comes before final. The backend rejects a final job if the project has no successful draft for the current version of the scenes.
## Sound effects
- Shared library: `assets/sound-effects/`, every file has an entry in `assets/sound-effects/library.json` (`file`, `tags`, `durationMs`, and a `description` written in Vietnamese).
- When using one in a video: copy it into `video-projects/<name>/assets/sound-effects/` then declare it in `meta.json` - a project must contain all of its own assets (re-rendering must not depend on a library that may change).
- The Sound Effects page in the web UI reads `library.json`, plays previews inline, and allows uploading new files (the backend updates the json).
## Speeding up renders (this machine has a GTX 1660 - verified 2026-07)
The number one reason the pipeline burns an entire hour: **re-rendering a draft of the WHOLE video after every small edit**. The rules:
1. **Verify layout/text with `npx hyperframes snapshot` or `inspect`** (a few seconds) instead of rendering
a full draft (tens of minutes). Render a full draft only ONCE, when every snapshot is good, and final ONCE.
2. **Small edit -> re-render only what changed**: if you edited one scene, re-render that scene (`render -c <scene>`),
do not re-render the whole composition. The split-scene + Remotion-assemble pipeline is built exactly for this.
3. **HyperFrames speed flags** (the backend queue adds them automatically; when running by hand you MUST remember them):
`-w 8 --browser-gpu` for every render; add `--gpu` (NVENC) for drafts. Final keeps CPU encoding
(libx264) for maximum quality. Measured in practice: ~30% faster on short scenes, more on long ones.
4. **Transcription runs only ONCE** - if `transcript.json` already exists, reuse it, NEVER transcribe again.
5. **Remotion speed flags** (when running `npx remotion render/still` by hand you MUST add):
`--concurrency 8 --gl angle`. WITHOUT `--gl angle`, Remotion's Chrome renders with the
software renderer (SwANGLE) - the CPU carries 100% while the GPU idles (symptom: Task Manager CPU ~95%,
GPU ~5%). On Linux use `--gl angle-egl` instead of `angle`.
## Known issues (verified in production 2026-07)
- **`meta.json` must respect the data-type contract** (the web UI reads it directly - a wrong type crashes the page):
`output` is a path **STRING** (e.g. `"outputs/<id>-v1.mp4"`), NOT an object. Extra metadata
(duration, quality, renderedAt, ...) goes into a separate `outputInfo` field if needed. `scenes[].durationInFrames`
is a number; sfx are placed with `atFrame` (a frame number) as in the schema at the top of this file.
- **Rendering a single scene**: use the `-c` flag, not a positional argument: `npx hyperframes render -c compositions/s01.html --quality draft --output renders/s01.draft.mp4`. A `<template>` sub-composition must be referenced by index.html through `data-composition-src` for `-c` to be able to render it.
- **Words running together on per-word reveals**: the HyperFrames pipeline swallows the whitespace between inline-block `<span>`s - separate words with `margin: 0 0.14em` on `.word`, do not rely on whitespace in the HTML.
- **The `sub_timeline_readiness_timeout` warning** when rendering a template file with `-c`: the render still comes out correct (best-effort) but wastes another 45s waiting - acceptable for drafts; if you want it gone entirely, render through index.html.
- **A TS comment containing the glob path `*/`** (e.g. `video-projects/*/meta.json`) closes the block comment early -> a baffling compile error. Write `video-projects/<id>/meta.json` instead.
- **Vietnamese JSON sent with curl -d from Git Bash on Windows loses its diacritics** (verified 2026-08: the
thumbnail title arrived at the server as "B?n th? kh?ch..." and rendered broken). The console codepage mangles
UTF-8 before curl sends it. For any API body containing Vietnamese, POST from Node instead:
`node -e "fetch(url, { method: 'POST', headers: {'content-type': 'application/json'}, body: JSON.stringify({...}) })"`
(or write the JSON to a file and use `curl --data-binary @file`).
## Agent sandbox (what the policy blocks)
Every tool call from the in-app agent passes through `apps/server/src/agentPolicy.ts` (a PreToolUse hook) before it runs. The prompt always carries untrusted text (transcripts, imported articles, file descriptions), so these are refused no matter what the prompt says:
- **Network**: only the local API (`http://localhost:6869`) - `curl` to any other host, `wget`, `Invoke-WebRequest`, proxies/`--resolve`, external URLs inside `node -e`/scripts, and UNC paths. Brand logos come from `POST /api/brand-logos`, images from `/api/illustrations`; never download them yourself.
- **Installs**: `npm install/i/ci/exec`, `npx <anything except hyperframes/remotion/tsc/tsx>` (or with `@version`/`-y`), `pip`/`python -m pip`, `winget`/`choco`. Missing dependency = ask the user to run Doctor.
- **Secrets**: `.env`, `~/.claude/.credentials.json`, `~/.ssh`... API keys are not in your environment either - the server holds them.
- **Self-escalation**: writing `.claude/settings*.json`, `.mcp.json`, `agentPolicy.ts`, `.git/hooks`; user-only APIs (`/api/connections`, `/api/tunnel`, `/api/update`, `/api/health`, `/api/chat`, ...); WebFetch/WebSearch and MCP tools.
A refusal comes back as the tool error with the reason - change approach, do not try to get around it. To allow something new, edit the lists at the top of `agentPolicy.ts` (a human change, never the agent's).
## Checklist before reporting completion
- [ ] If the brief enables autoCut: the cut was done per the `auto-cut` skill, a second silencedetect pass verified the cut version, and the report states the seconds/segments removed
- [ ] Every scene passed frame verification, Vietnamese text keeps all its diacritics
- [ ] Audio is in sync at the start/middle/end (check 3 random points)
- [ ] Sound effects land on the right frames, and their volume does not bury the voice (sfx ~10dB below voice)
- [ ] The output matches the dimensions/fps in `meta.json`
- [ ] Automated QC ran on the draft with no remaining `fail` checks (the report spells out any leftover warn checks)
- [ ] `finishMode: "final"` only: `meta.json` has updated `status` + `output`
- [ ] `finishMode: "final"` only: the thumbnail was generated and Read to verify the diacritics are complete
- [ ] `finishMode: "final"` only: the publish package was generated (`.srt`/`.vtt` + metadata), using the FINAL post-cut transcript
- [ ] `finishMode: "draft"`: the draft came from an `assemble-draft` job of this run, QC ran on it, and nothing final was rendered
- [ ] Any new lesson learned was written into the relevant skill
© notivn · MIT · notivn/AIEVv2