Skip to main content
skill CX Codex

remotion-assemble — Claude Code skill: what it does, how to install

remotion-assemble is a Claude Code skill that stitches HyperFrames scenes, footage, voiceovers, and SFX into a video via meta.json. It activates when editing files inside engines/remotion.

MIT ★ 147 updated 2026-11-13

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add notivn/AIEVv2 --skill remotion-assemble -a claude-code
Open on GitHub

What remotion-assemble does

remotion-assemble configures the final video timeline using metadata from meta.json. Remotion acts purely as an assembly pipeline, combining pre-rendered HyperFrames MP4s, raw footage, speech, and sound effects. Motion graphics stay strictly within HyperFrames rather than Remotion code.

Inside engines/remotion, the Assemble.tsx composition builds the timeline while manifest.ts checks properties with Zod. The schema enforces bounds for transforms, crop values, playback speeds from 0.25 to 4, and CSS color presets. Any out-of-range value triggers a 400 INVALID_TIMELINE error on the server and stops the render job.

What it helps with

  • assemble pre-rendered video clips and voiceover tracks
  • configure timeline transitions and text layers in meta.json
  • edit layout and video compositions in engines/remotion
  • align sound effects with frame timestamps

Example prompts for the agent

  • Assemble the video timeline in engines/remotion using meta.json
  • Add a fade transition and a video PiP overlay to the current scenes manifest

How to install remotion-assemble

1Skills CLI (Vercel)

npx skills add notivn/AIEVv2 --skill remotion-assemble -a claude-code

2Manually: put the files in place

~/.claude/skills/remotion-assemble/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/remotion-assemble/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about remotion-assemble

How to install remotion-assemble in Claude Code?

Run npx skills add notivn/AIEVv2 --skill remotion-assemble -a claude-code in your terminal.

What is the main purpose of remotion-assemble?

It provides guidelines for stitching pre-rendered HyperFrames MP4 files, audio clips, and overlays into a finished video project.

What causes a 400 INVALID_TIMELINE error in remotion-assemble?

The error occurs when timeline values in meta.json violate the ranges defined in the Zod schema, which halts the rendering process.

Source text

SKILL.md · MIT
---
name: remotion-assemble
description: How to use Remotion as the assembly layer - read the project's meta.json and stitch the rendered HyperFrames scenes + footage + voice + sound effects into the finished video. Read this when writing/editing code in engines/remotion or when assembling the timeline for a video.
---

# Remotion Assemble - building the timeline from meta.json

In this system Remotion **does exactly one thing**: assemble the pieces that already exist (scene MP4s rendered by HyperFrames, source footage, voice, sound effects) into the final video. Do not build motion graphics in Remotion - that is HyperFrames' job.

## Structure of engines/remotion

```
engines/remotion/
├── package.json          ← remotion, @remotion/cli, @remotion/renderer
├── remotion.config.ts
└── src/
    ├── Root.tsx               ← registers 3 compositions: "Assemble" + "Poster" + "Thumbnail"
    ├── Assemble.tsx           ← master composition: read manifest → build timeline
    ├── Poster.tsx             ← static poster composition (image projects)
    ├── Thumbnail.tsx          ← static video-thumbnail composition (frame card + title)
    ├── manifest.ts            ← load + validate video props (zod)
    ├── duration.ts            ← scene length incl. `speed` (shared with the web editor; server twin in timeline.ts)
    ├── color.ts               ← timeline color adjust → CSS filter (`colorFilterCss`, `ColorDefs`, preset approximations)
    ├── layerGeometry.ts       ← `layerBox()` - layer position/size/animation/keyframes (shared with the editor's drag frame)
    ├── posterManifest.ts      ← load + validate poster props (zod)
    ├── thumbnailManifest.ts   ← load + validate thumbnail props (zod)
    ├── brandFonts.ts          ← load brand fonts via staticFile (offline)
    ├── media.tsx              ← media URL resolver (staticFile by default; the dashboard preview swaps it)
    ├── index.ts               ← entry point registering Root
    └── components/
        ├── SceneClip.tsx      ← <OffthreadVideo>/<Img> for one scene/footage clip
        ├── Transition.tsx     ← scene transitions inside transitionOverlap (fade, slide, wipe, zoom, blur, none)
        ├── LayerTrack.tsx     ← overlay layers (image / video PiP / text / shape)
        ├── ColorBox.tsx       ← wraps a scene/layer with its color adjust + vignette
        ├── SfxTrack.tsx       ← places <Audio> at sfx[].atFrame
        ├── MusicTrack.tsx     ← background music: loop + fade + deterministic ducking
        ├── CaptionTrack.tsx   ← karaoke captions from word timestamps
        ├── HighlightTrack.tsx ← main/related key bands (key-layout)
        └── vietnameseFont.ts  ← @font-face Inter vietnamese subset
```

Manifest fields beyond scenes/captions/overlays (see `manifest.ts`):
- `watermark` - the Style Design corner logo, stamped by the SERVER (`jobs/assemble.ts`), never by the AI; `null` = the style has no logo.
- `audio.music` - background music `{ file, volume, duckVolume, speech: [[startSec,endSec],...] }` with deterministic ducking over speech ranges (see the `background-music` skill); `null` = no music.
- `scenes[].zoom` - camera move on footage `{ origin?, keys: [{frame, scale, ease?}] }`; `frame` counts from the START OF THE SCENE, applied to the wrapper div (never the video element).
- `audio.sfx[].mediaStart` - seconds skipped at the head of the sfx file (lead silence trim, see noti-tiktok-vn "SFX loudness and lead silence").

### Editor v2 fields (all optional - absent = the old behaviour)

Full contract: `docs/EDITOR-V2-PLAN.md` section 1. The server validator (`apps/server/src/timeline.ts`) rejects anything out of range with `400 INVALID_TIMELINE`, and the zod schema has the same ranges, so an out-of-range value written by hand **kills the render**. Stay inside these ranges.

**Scene** (`scenes[]`):

| Field | Range | Meaning |
|---|---|---|
| `fit` | `"cover"` (default) \| `"contain"` | how the source fills the frame |
| `transform` | `{ x, y: -1..1, scale: 0.1..5, rotate: -180..180, flipH, flipV }` | x/y = OFFSET as a fraction of the frame side (x = 1 moves a full width); scale multiplies the existing `zoom` |
| `crop` | `{ top, right, bottom, left }` each 0..0.45 | crops the SOURCE, then fits the rest (`object-view-box`, no black bars) |
| `color` | ColorAdjust (below) | color on the timeline - does not touch the file |
| `volume` | 0..2 (footage) | `muted: true` still wins |
| `fadeInSec` / `fadeOutSec` | 0..10 (footage) | audio fade of the clip's own sound |
| `speed` | 0.25..4 (footage) | playback rate. **Length = (round(to·fps) − round(from·fps)) / speed** when `durationInFrames` is absent - never compute it yourself with a different formula; the helper is `duration.ts` (server twin `sceneDurationFrames`). `durationInFrames` WINS over `speed`: if the scene has one, rewrite it in the same edit (and pin `to`) or the clip ends early (black tail) / gets cut |
| `transition` | `{ type }` | transition INTO THE NEXT scene, played inside this scene's `transitionOverlap`. Types: `fade` (default, the old crossfade) `slide-left` `slide-right` `slide-up` `slide-down` (push) `wipe-left` `wipe-right` `zoom-in` `zoom-out` `blur` `none`. No overlap = hard cut whatever the type |

**ColorAdjust** (scene `color` and layer `color`): `preset` = one of the 14 ids in `apps/server/src/color.ts` (approximated with CSS), `brightness` `contrast` `saturation` `temperature` `tint` -1..1 (0 = unchanged, saturation -1 = black and white, temperature + = warm), `hue` -180..180°, `vignette` 0..1, `blur` 0..20 px. This is a look on the timeline, not the ffmpeg asset grade of the `color-grading` skill - for HDR/log footage still delog the file first.

**Layers** (top-level `layers[]`) - drawn above the scenes, below highlight/captions/watermark; later entries are on top:

```jsonc
"layers": [
  { "id": "title", "kind": "text", "from": 30, "durationInFrames": 90,
    "text": "Dòng 1\nDòng 2", "width": 0.8, "transform": { "y": -0.6 },
    "style": { "fontSizePx": 84, "color": "#ffffff", "backgroundColor": "rgba(0,0,0,0.6)", "paddingPx": 24, "radiusPx": 20 },
    "animation": { "in": "pop", "out": "fade", "durationSec": 0.4 } },
  { "id": "pip", "kind": "video", "src": "assets/broll.mp4", "mediaStart": 2,
    "from": 120, "durationInFrames": 150, "width": 0.4,
    "transform": { "x": 0.5, "y": 0.55 }, "radiusPx": 24, "muted": true },
  { "id": "logo-openai", "kind": "image", "src": "assets/openai.svg", "aspect": 1, "from": 0, "durationInFrames": 60,
    "keyframes": [ { "frame": 0, "scale": 0.8, "ease": "out" }, { "frame": 20, "scale": 1 } ] }
]
```

- `kind`: `image` | `video` (both need `src`, a project-relative path like every other media path - staged by `jobs/assemble.ts`) | `text` | `shape` (a colored box: `style.backgroundColor`, `aspect` w/h default 16/9).
- `from` / `durationInFrames` are ABSOLUTE frames, like cues. `keyframes[].frame` is RELATIVE to `from`.
- Position: `transform.x/y` is the CENTRE of the box, -1..1 from the frame centre (0,0 = middle, 1 = edge) - different from a scene's `transform`. `width` is a fraction of the frame width (0.05..2, default 0.5); height follows the file's aspect (image/video), `aspect` (shape) or the text.
- Text: `style.fontFamily` is one of the SubtitleTrack ids (`vietnamese` default, `sans`, `serif`, `mono`) - Vietnamese diacritics are safe; `fontSizePx` is in composition pixels.
- `animation.in/out`: `none` `fade` `slide-up` `slide-down` `slide-left` `slide-right` `pop` `zoom`; `durationSec` default 0.4 (capped at half the layer on short layers).
- `mediaAspect` (layer) and `mediaDurationSec` (sfx) are filled by the server/preview at build time (ffprobe) - never write them into `meta.json`. ffprobe often cannot measure an SVG (depends on the ffmpeg build), so for an SVG image layer set `aspect` (w/h) yourself, otherwise the box falls back to 16/9.
- Geometry is computed by `layerBox()` in `layerGeometry.ts`; the editor's drag frame uses the same function, so never re-implement it in a component.
- Logo rule still applies: a brand logo layer must come from `assets/brand-logos/` copied into the project - and never add the Style Design corner logo as a layer (the server stamps it).

**Audio fades**: `audio.sfx[].fadeInSec/fadeOutSec` (0..10, fade-out counts back from the END of the file), `audio.music.fadeInSec/fadeOutSec` (override the 0.5 s / 1 s defaults; 0 = no fade).

## Core principles

1. **A single data-driven `Assemble` composition.** Everything comes from the project's `meta.json`, passed through `defaultProps`/`inputProps`. Adding a new video means no Remotion code changes, only a new manifest.

```bash
npx remotion render Assemble \
  --props="<abs>/video-projects/<name>/props.resolved.json" \
  --output="../../outputs/<name>-v1.mp4" \
  --concurrency 8 --gl angle
```

> ⚠️ Asset paths in the props must be `staging/...` - the backend stages assets into
> `engines/remotion/public/staging/` via hardlinks and then writes `props.resolved.json`.
> **NEVER render straight from `meta.json`** (its paths are relative to the project
> folder, and Remotion cannot resolve those through `staticFile`).
>
> `--gl angle` is MANDATORY on a machine with a GPU - without it Remotion renders with the software renderer,
> CPU at 100% and GPU at 5%. For drafts add `--crf 28 --x264-preset veryfast`. (The backend queue adds these flags automatically.)

2. **Dimensions/fps come from the manifest** - use `calculateMetadata` to set `width/height/fps/durationInFrames` dynamically from the props, never hardcode them in `Root.tsx`.

3. **Embedded video uses `<OffthreadVideo>`**, not `<Video>` - it is more stable server-side and lands on the right frame. Whatever fps a HyperFrames scene was rendered at, the manifest must declare that exact fps; an fps mismatch between scene and composition is the number one source of stuttering.

4. **Timeline = accumulating `durationInFrames`:**

```tsx
let from = 0;
scenes.map((s) => {
  const seq = (
    <Sequence key={s.id} from={from} durationInFrames={s.durationInFrames}>
      <SceneClip scene={s} />
    </Sequence>
  );
  from += s.durationInFrames - (s.transitionOverlap ?? 0);
  return seq;
});
```

When a transition has an overlap, subtract that overlap while accumulating - forgetting to subtract leaves a black gap between scenes.

5. **Audio:**
   - Voice: one `<Audio src={voice}>` running from frame 0 - the voice is the sync backbone, scenes follow the voice and never the other way round.
   - Sound effects: each `sfx[]` entry becomes one `<Sequence from={atFrame}><Audio volume={0.3}/></Sequence>`. Default sfx volume is 0.3 (~10dB below the voice); override it in the manifest with the `volume` field if needed.
   - Do not normalize/mix manually with FFmpeg after rendering - mix inside Remotion so the draft sounds like the final.

6. **Caption and overlay tracks live in `meta.json`, and their timestamps are in FRAMES.**

   `CaptionTrack` and `HighlightTrack` read `props.captions` / `props.overlays` **only**. They never
   open a transcript file. If you leave those arrays empty there are no captions, no matter how good
   the transcript is.

```jsonc
// meta.json
"captions": [
  { "from": 90, "durationInFrames": 75,
    "words": [ { "text": "Trí", "start": 90, "end": 96 },
               { "text": "tuệ", "start": 96, "end": 104, "hi": true } ] }
],
"overlays": [
  { "from": 0, "durationInFrames": 150, "tier": "main", "accent": "hot",
    "kicker": "AI", "parts": [ { "t": "Dựng video" }, { "t": "bằng AI", "hi": true } ] }
]
```

   > ⚠️ **The unit changes here.** A transcript stores `start`/`end` in **seconds**; `captions[].words[]`
   > stores them as **absolute composition frames** (`second * fps`, counted from the start of the whole
   > video, not from the start of the cue). Passing seconds through produces captions that all flash in
   > the first half-second — and nothing errors, so it is easy to ship by accident.
   >
   > `hi: true` marks a highlighted word. `tier: "main"` renders a top band, `"sub"` a bottom pill that
   > lifts itself when captions are present.
   >
   > Any cue (caption, subtitle, overlay) may carry `"transform": { "x", "y", "scale" }` - an offset from
   > the track's default anchor, `x`/`y` as a fraction of the frame width/height (-1..1), `scale`
   > 0.3..3. The user sets it by dragging the cue on the editor's preview. **Leave it alone** when you
   > rewrite a cue's text or timing; only change it when asked to move or resize the text.

7. **Every scene needs `durationInFrames` unless it is a `srcVideo` scene with both `from` and `to`.**
   That is the only case Remotion infers a length for; an image or HyperFrames scene without an explicit
   `durationInFrames` **throws at render time**. This bites hardest on videos with no source footage
   (text-to-video, image-only explainers), where nothing has a natural duration to infer from.

8. **`meta.json` is shared with a human editor.** The dashboard's video editor previews the `Assemble`
   composition live in the browser (`@remotion/player`) and saves the timeline keys (`scenes`, `audio`,
   `captions`, `subtitles`, `subtitleStyle`, `overlays`, `layers`) straight back into `meta.json`. When you assemble
   or adjust the timeline: **re-read `meta.json` right before each change, change it in place with `Edit`,
   never `Write` the whole file from memory, and keep every field you do not recognise** (the schema uses
   `z.looseObject` precisely so extra fields survive). Paths stay relative to the project folder with no
   `..` - the editor's server-side validator rejects anything else, and so does staging.

9. **Asset paths**: Remotion code only loads through `staticFile()` - the backend stages assets into `engines/remotion/public/staging/<project>/` (hardlink) and writes the `staging/...` path into `props.resolved.json`; Remotion never reads an absolute path. This runs on Windows - always use `path.join` on the backend, never string concatenation.

   Inside the `Assemble` tree, get that `staticFile` through **`useMediaSrc()`** (`src/media.tsx`), never by calling `staticFile()` directly. The dashboard editor runs the same `Assemble` in `@remotion/player`, where there is no Remotion `public/`: it wraps the tree in `MediaResolverProvider` and maps project-relative paths to `/media/video-projects/<id>/...` and `fonts/...` to `/media/remotion-fonts/...`. A direct `staticFile()` (or one evaluated at module scope, like the old `vietnameseFontFaceCss` constant) renders fine through the CLI but 404s in the preview. The default resolver IS `staticFile`, so CLI renders produce the same frames as before. Poster/Thumbnail/brandFonts only render through the CLI and keep `staticFile`.

## Draft vs Final

| | Draft | Final |
|---|---|---|
| Command | `--crf 28 --x264-preset veryfast` (+ `--concurrency 8 --gl angle`) | defaults (crf 18) |
| Scene input | `renders/*.draft.mp4` | `renders/*.mp4` (quality standard) |
| Purpose | review pacing, sync, transitions | publishing |

The backend picks the scene input set based on the job type - the Remotion code does not distinguish draft from final, it only takes the paths from the manifest.

## Known issues & how to avoid them

- **Stutter/freeze at a scene boundary**: the scene fps differs from the composition fps, or `durationInFrames` in the manifest does not match the real length of the MP4. Check with `ffprobe -show_streams <file>` before assembling.
- **Audio drifting out of sync toward the end**: a VBR voice mp3 -> convert it to CBR/WAV before putting it in the manifest (`ffmpeg -i voice.mp3 -ar 48000 voice.wav`).
- **Color mismatch between HyperFrames scenes and footage**: both engines render through Chromium so they usually match; if they do not, check whether the footage has an unusual color space tag (`bt709` is the standard) - transcode the footage to bt709 first.
- **Render hanging on Windows**: usually caused by Vietnamese characters/spaces in an asset file path - name asset files in ASCII kebab-case right from the import step.
- **One black frame at every footage cut**: a `srcVideo` scene sized `round((to-from)*fps)` while `OffthreadVideo` plays `round(to*fps) - round(from*fps)` frames - the two differ by one (from=1.02, to=3.98 @30fps: 89 vs 88). Always round EACH END, never the difference; `resolveSceneDurationInFrames` and `frameOf` in `jobs/assemble.ts` must stay in lockstep with `SceneClip`.
- **Sped-up footage (`speed` > 1) going black for the second half of the scene**: since Remotion 4.0.537 (the pinned version) the `Sequence` inside `OffthreadVideo` already divides `(endAt - startFrom)` by `playbackRate` itself, so `endAt` must be the plain SOURCE frame `round(to * fps)`. Converting it again (`startFrom + ceil((endAt - startFrom) / speed)`, which 4.0.500 needed for `speed` < 1) divides twice: speed 2 on a 10 s clip gives a 150-frame scene whose video stops at frame 75. Check `node_modules/remotion/dist/cjs/video/OffthreadVideo.js` again whenever you bump Remotion.
- **Whole render dies on one tiny cue** (`inputRange must be strictly monotonically increasing`): a fade written as `[0, 4, max(5, last-4), last]` is not increasing when a cue is 6 frames or less (a translated cue clamped near the end of the video is enough). Use `fadeInOut()` from `components/fade.ts` for any new track instead of hand-writing the range.
- **Rendering on a machine with no network**: Remotion tries to download Chrome from `remotion.media`. Pass `--browser-executable=<local chrome/headless_shell>` instead.

## Remotion license (keep in mind when scaling up)

Remotion is free for individuals and companies of <= 3 people; beyond that you need a Company License. This note is for planning growth - it does not affect the current stage.

© notivn · MIT · notivn/AIEVv2

Similar skills

All skills for Frontend and UI →

From the same set15

The whole set →
offline-voice offline-voice is a Claude Code skill that guides the setup of VieNeu-TTS narration and voice cloning. Activates when working on /api/tts, /api/voices, or narration debugging. CX Codex +1 ★ 147repo stars make-a-video make-a-video is a Claude Code skill that turns a concept into a finished MP4 with HyperFrames. It gathers requirements, builds a storyboard, and checks the preview before the final render. CX Codex +1 ★ 147repo stars auto-cut auto-cut is a Claude Code skill that cuts silences, stutters, and repeated takes from talking-head footage before editing. It activates when the autoCut brief setting is enabled. CX Codex +1 ★ 147repo stars ai-illustrations ai-illustrations is a Claude Code skill that adds generated images to a video edit. It activates when the brief enables AI illustrations or requests them for a video. CX Codex +1 ★ 147repo stars video-pipeline video-pipeline is a Claude Code skill that guides video production from a request to an MP4 using HyperFrames and Remotion. It activates when starting a video or building a render queue. CX Codex +1 ★ 147repo stars noti-tiktok-full-text noti-tiktok-full-text is a Claude Code skill that builds vertical TikTok explainer videos from AI papers with HyperFrames. Runs when you provide a transcript, paper page scans, and figures. CX Codex +1 ★ 147repo stars short-form-video short-form-video is a Claude Code skill for building and revising vertical Hyperframes videos with a talking head, motion graphics, and karaoke captions. It activates for TikTok, Reels, Shorts, or requests to retime an existing short. CX Codex +1 ★ 147repo stars noti-tiktok-vn noti-tiktok-vn is a Claude Code skill that builds vertical 9:16 TikTok videos using HyperFrames and GSAP. It animates talking-head footage, karaoke captions, kinetic text, and camera punch-ins. CX Codex +1 ★ 147repo stars

Project sponsors

Companies that keep this analytics open