Skip to main content
skill CX Codex

video-to-game — Claude Code skill: what it does, how to install

The video-to-game Claude Code skill turns gameplay footage or a game concept into a Three.js browser game with assets generated through fal. It also records a playtest and compares game frames with the source clip.

MIT ★ 686 updated 2026-10-29

Install into Claude Code · Plugin marketplace All methods and platforms ↓
/plugin marketplace add lovisdotio/video-to-game
/plugin install video-to-game@video-to-game
Open on GitHub

What video-to-game does

The skill breaks a video into frames, a color palette, and cuts, then guides the build of a Three.js game from that material. It generates images, 3D models, sound effects, and music through fal, while logging model calls. Included scripts analyze footage, compare frames, and record a playtest.

The game supports keyboard and gamepad input. A keyboard-driven scenario records a run and identifies its liveliest stretch. The scripts require Python 3.9+, ffmpeg, and Node 18+ with playwright-core; asset generation also requires fal-client and the FAL_KEY environment variable. The game uses Three.js rather than Unity, Unreal, or Godot.

What it helps with

  • turn gameplay footage into a browser game
  • inspect a clip’s frames, palette, and cuts
  • generate game assets from video frames through fal
  • compare game screenshots with the source clip
  • record a playtest and locate its liveliest stretch

Example prompts for the agent

  • Turn gameplay.mp4 into a playable Three.js browser game.
  • Analyze gameplay.mp4 for frames and color palette, then build a game from it.
  • Record a playtest and compare its frames with gameplay.mp4.

How to install video-to-game

1Plugin marketplace

/plugin marketplace add lovisdotio/video-to-game
/plugin install video-to-game@video-to-game

This installs the whole video-to-game plugin with all its skills

2Skills CLI (Vercel)

npx skills add lovisdotio/video-to-game --skill video-to-game -a claude-code

3Manually: put the files in place

~/.claude/skills/video-to-game/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/video-to-game/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about video-to-game

How do I install video-to-game in Claude Code?

Add the marketplace with /plugin marketplace add lovisdotio/video-to-game, then install the skill with /plugin install video-to-game@video-to-game.

What does video-to-game do?

It guides the conversion of gameplay footage or a concept into a Three.js game, creates assets through fal, and records a playtest with frame comparisons.

What does video-to-game need to run?

It needs Python 3.9+, ffmpeg, Node 18+ with playwright-core, plus fal-client and the FAL_KEY environment variable for asset generation.

Source text

SKILL.md · MIT
---
name: video-to-game
description: >-
  Turns a gameplay video (generated by a video model such as MiniMax H3 Max or Seedance, or any clip) into a real,
  playable WebGL game in Three.js whose every image, 3D model, sound and music track is generated with fal, then
  proves it: side-by-side frames of the video and the game, and a scenario that plays the game with the keyboard
  and films its best seconds. Ships scripts to study the video (frames, palette, cuts), run and log fal models,
  record a playtest at the real frame rate, compare game and video, and find the liveliest window of a take.
  Use when the user wants a video, an AI gameplay clip or a game concept "turned into a game", "made playable",
  "recreated in the browser" or "rebuilt in Three.js/WebGL". Not for game engines (Unity, Unreal, Godot) or for
  copying existing commercial games.
license: MIT
metadata:
  hermes:
    tags:
      - threejs
      - webgl
      - game-development
      - video-to-game
      - fal
      - generative-assets
    category: development
---

# Video to Game

The video is the game design document, and you cannot watch it. A video model is good at what you are bad at:
the look of a game, its camera, its one memorable mechanic. Your job is to reverse-engineer that clip into a
game someone can play, with assets generated to match it, and to prove it with pictures, not adjectives.

## Workflow

1. **Generate or take the video.** One original concept, one mechanic you can see, a third-person or side camera,
   no HUD text. `prompts/gameplay-video-prompts.md` (repo root) has the recipe and 31 prompts that worked.
2. **Study it.** `python3 scripts/study_video.py gameplay.mp4 -o study/` saves a frame every 0.25 s, a contact sheet,
   the palette and the cuts. Open the sheet, then the frames. Write the spec before any code:
   camera (distance, height, lens), hero, the one core mechanic, world layout, palette, light, materials, mood,
   and the key moment the video sells. See `references/study-the-video.md`.
3. **Generate every asset with fal**, one job at a time, from the video's own frames as references:
   concept sheet, background removal, image to 3D, rigging, textures, parallax layers, sound effects, music.
   `python3 scripts/fal_asset.py <endpoint> --arg k=v --file k=path -o assets/` runs a model and logs it.
   Nothing drawn by hand, nothing downloaded elsewhere. See `references/fal-assets.md`.
4. **Build the game** from `templates/game.html`: Three.js only, one import map, a real loop (controls, goal,
   progression, feedback, end, restart), keyboard and gamepad, `?autostart` and `?seed=N`, `window.__game` state.
   See `references/game-loop.md`.
5. **Match the look.** Palette, fog, light, grade, bloom, vignette and grain are part of the spec from day one,
   not polish. Compare a game frame with a video frame at fixed spots after every change:
   `python3 scripts/side_by_side.py gameplay.mp4 shot.png -o pairs/ --times 2,5,8`.
   See `references/look-and-post.md`.
6. **Play it yourself.** Write `play.mjs` from `templates/play.mjs`: keyboard only, reading `window.__game`.
   `node scripts/record_play.mjs "http://127.0.0.1:8000/?autostart" play.mjs -o take.mp4` films it at its real
   frame rate; `python3 scripts/best_window.py take.mp4 --length 10` finds the liveliest stretch; look at those
   frames. Fix what a player would hit: falls, walls, dead seconds, unreadable moments. See `references/playtest.md`.
7. **Deliver:** `index.html` + `assets/` + `README.md` (concept, controls, every fal asset with its endpoint),
   the side-by-side pairs, the best take, and an honest list of what differs from the video.

Scripts need Python 3.9+, ffmpeg, Node 18+ with `playwright-core` for the recorder, and `pip install fal-client`
with `FAL_KEY` set for `fal_asset.py`. Run them from this skill's directory or with full paths.

## Hard rules

- **The game looks like its video.** Same camera, palette, light and materials. Every look decision is checked
  on a side-by-side pair, at the same three spots each time. "Looks right" without a pair image is not a check.
- **The video's mechanic is the game.** If the clip shows a girl turning the world upside down, the game is about
  turning the world upside down, not a generic runner with her model in it.
- **Every asset comes from fal**, generated from the video's frames as references. Log each call (file, endpoint).
  One job at a time: look at each result before asking for the next.
- **No text on screen** beyond a minimal title and one controls line. The world explains itself.
- **Original worlds only.** No characters, logos, names or level designs from existing games or films.
- **Post-processing is in the spec**: tone mapping, one grade, bloom, fog for depth, vignette, grain. Keep 60 fps
  on a laptop; 30 fps is acceptable only for a much better image.
- **Calm the world, let the game read.** One harmonized palette, no acid colors; dense but grouped; hero and
  interactive things stand out on a quiet background.
- **Play before you ship.** A recorded take with zero console errors, no fall, no dead second, the key moment
  of the video visible.

## Reference routing

| File | Read when |
|---|---|
| `references/study-the-video.md` | starting: frames, palette, camera, mechanic, the spec to write |
| `references/fal-assets.md` | generating assets: which model for what, the character pipeline, prompts that work, costs |
| `references/game-loop.md` | building: structure, input, camera, state contract, multiplayer, performance |
| `references/look-and-post.md` | matching the video's look: grade, fog, bloom, density, hierarchy, the side-by-side routine |
| `references/playtest.md` | playing your own game: scenarios, recording, best window, what to fix |

Templates: `templates/game.html` (Three.js skeleton with the post stack, input, state contract),
`templates/play.mjs` (a keyboard playtest scenario).

## Output expectations

- Say what was checked and how: which pairs, which take, fps, console errors.
- List the fal calls: count per endpoint and total cost.
- List honestly what differs from the video (a simplified mechanic, a missing effect), never "identical".

© Lovis Odin · MIT · lovisdotio/video-to-game

Similar skills

All skills for Game dev and 3D: Unity, Unreal, Godot, Blender →

From the same set3

Project sponsors

Companies that keep this analytics open