video-to-game — Claude Code skill: what it does, how to install
The video-to-game Claude Code skill turns gameplay footage or a game concept into a Three.js browser game with assets generated through fal. It also records a playtest and compares game frames with the source clip.
MIT ★ 686 updated 2026-10-29
/plugin marketplace add lovisdotio/video-to-game
/plugin install video-to-game@video-to-gameWhat video-to-game does
The skill breaks a video into frames, a color palette, and cuts, then guides the build of a Three.js game from that material. It generates images, 3D models, sound effects, and music through fal, while logging model calls. Included scripts analyze footage, compare frames, and record a playtest.
The game supports keyboard and gamepad input. A keyboard-driven scenario records a run and identifies its liveliest stretch. The scripts require Python 3.9+, ffmpeg, and Node 18+ with playwright-core; asset generation also requires fal-client and the FAL_KEY environment variable. The game uses Three.js rather than Unity, Unreal, or Godot.
What it helps with
- turn gameplay footage into a browser game
- inspect a clip’s frames, palette, and cuts
- generate game assets from video frames through fal
- compare game screenshots with the source clip
- record a playtest and locate its liveliest stretch
Example prompts for the agent
- Turn
gameplay.mp4into a playable Three.js browser game. - Analyze
gameplay.mp4for frames and color palette, then build a game from it. - Record a playtest and compare its frames with
gameplay.mp4.
How to install video-to-game
1Plugin marketplace
/plugin marketplace add lovisdotio/video-to-game/plugin install video-to-game@video-to-gameThis installs the whole video-to-game plugin with all its skills
2Skills CLI (Vercel)
npx skills add lovisdotio/video-to-game --skill video-to-game -a claude-code3Manually: put the files in place
~/.claude/skills/video-to-game/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/video-to-game/
1Skills CLI (Vercel)
npx skills add lovisdotio/video-to-game --skill video-to-game -a codex2Manually: put the files in place
.agents/skills/video-to-game/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add lovisdotio/video-to-game --skill video-to-game -a cursor2Manually: put the files in place
.cursor/skills/video-to-game/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install lovisdotio/video-to-game skills/video-to-game2Manually: put the files in place
.github/skills/video-to-game/SKILL.md1Skills CLI (Vercel)
npx skills add lovisdotio/video-to-game --skill video-to-game -a gemini-cli2Platform CLI
gemini skills install https://github.com/lovisdotio/video-to-game --path skills/video-to-gameManually: put the files in place
.devin/skills/video-to-game/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about video-to-game
How do I install video-to-game in Claude Code?
Add the marketplace with /plugin marketplace add lovisdotio/video-to-game, then install the skill with /plugin install video-to-game@video-to-game.
What does video-to-game do?
It guides the conversion of gameplay footage or a concept into a Three.js game, creates assets through fal, and records a playtest with frame comparisons.
What does video-to-game need to run?
It needs Python 3.9+, ffmpeg, Node 18+ with playwright-core, plus fal-client and the FAL_KEY environment variable for asset generation.
Source text
SKILL.md · MIT
---
name: video-to-game
description: >-
Turns a gameplay video (generated by a video model such as MiniMax H3 Max or Seedance, or any clip) into a real,
playable WebGL game in Three.js whose every image, 3D model, sound and music track is generated with fal, then
proves it: side-by-side frames of the video and the game, and a scenario that plays the game with the keyboard
and films its best seconds. Ships scripts to study the video (frames, palette, cuts), run and log fal models,
record a playtest at the real frame rate, compare game and video, and find the liveliest window of a take.
Use when the user wants a video, an AI gameplay clip or a game concept "turned into a game", "made playable",
"recreated in the browser" or "rebuilt in Three.js/WebGL". Not for game engines (Unity, Unreal, Godot) or for
copying existing commercial games.
license: MIT
metadata:
hermes:
tags:
- threejs
- webgl
- game-development
- video-to-game
- fal
- generative-assets
category: development
---
# Video to Game
The video is the game design document, and you cannot watch it. A video model is good at what you are bad at:
the look of a game, its camera, its one memorable mechanic. Your job is to reverse-engineer that clip into a
game someone can play, with assets generated to match it, and to prove it with pictures, not adjectives.
## Workflow
1. **Generate or take the video.** One original concept, one mechanic you can see, a third-person or side camera,
no HUD text. `prompts/gameplay-video-prompts.md` (repo root) has the recipe and 31 prompts that worked.
2. **Study it.** `python3 scripts/study_video.py gameplay.mp4 -o study/` saves a frame every 0.25 s, a contact sheet,
the palette and the cuts. Open the sheet, then the frames. Write the spec before any code:
camera (distance, height, lens), hero, the one core mechanic, world layout, palette, light, materials, mood,
and the key moment the video sells. See `references/study-the-video.md`.
3. **Generate every asset with fal**, one job at a time, from the video's own frames as references:
concept sheet, background removal, image to 3D, rigging, textures, parallax layers, sound effects, music.
`python3 scripts/fal_asset.py <endpoint> --arg k=v --file k=path -o assets/` runs a model and logs it.
Nothing drawn by hand, nothing downloaded elsewhere. See `references/fal-assets.md`.
4. **Build the game** from `templates/game.html`: Three.js only, one import map, a real loop (controls, goal,
progression, feedback, end, restart), keyboard and gamepad, `?autostart` and `?seed=N`, `window.__game` state.
See `references/game-loop.md`.
5. **Match the look.** Palette, fog, light, grade, bloom, vignette and grain are part of the spec from day one,
not polish. Compare a game frame with a video frame at fixed spots after every change:
`python3 scripts/side_by_side.py gameplay.mp4 shot.png -o pairs/ --times 2,5,8`.
See `references/look-and-post.md`.
6. **Play it yourself.** Write `play.mjs` from `templates/play.mjs`: keyboard only, reading `window.__game`.
`node scripts/record_play.mjs "http://127.0.0.1:8000/?autostart" play.mjs -o take.mp4` films it at its real
frame rate; `python3 scripts/best_window.py take.mp4 --length 10` finds the liveliest stretch; look at those
frames. Fix what a player would hit: falls, walls, dead seconds, unreadable moments. See `references/playtest.md`.
7. **Deliver:** `index.html` + `assets/` + `README.md` (concept, controls, every fal asset with its endpoint),
the side-by-side pairs, the best take, and an honest list of what differs from the video.
Scripts need Python 3.9+, ffmpeg, Node 18+ with `playwright-core` for the recorder, and `pip install fal-client`
with `FAL_KEY` set for `fal_asset.py`. Run them from this skill's directory or with full paths.
## Hard rules
- **The game looks like its video.** Same camera, palette, light and materials. Every look decision is checked
on a side-by-side pair, at the same three spots each time. "Looks right" without a pair image is not a check.
- **The video's mechanic is the game.** If the clip shows a girl turning the world upside down, the game is about
turning the world upside down, not a generic runner with her model in it.
- **Every asset comes from fal**, generated from the video's frames as references. Log each call (file, endpoint).
One job at a time: look at each result before asking for the next.
- **No text on screen** beyond a minimal title and one controls line. The world explains itself.
- **Original worlds only.** No characters, logos, names or level designs from existing games or films.
- **Post-processing is in the spec**: tone mapping, one grade, bloom, fog for depth, vignette, grain. Keep 60 fps
on a laptop; 30 fps is acceptable only for a much better image.
- **Calm the world, let the game read.** One harmonized palette, no acid colors; dense but grouped; hero and
interactive things stand out on a quiet background.
- **Play before you ship.** A recorded take with zero console errors, no fall, no dead second, the key moment
of the video visible.
## Reference routing
| File | Read when |
|---|---|
| `references/study-the-video.md` | starting: frames, palette, camera, mechanic, the spec to write |
| `references/fal-assets.md` | generating assets: which model for what, the character pipeline, prompts that work, costs |
| `references/game-loop.md` | building: structure, input, camera, state contract, multiplayer, performance |
| `references/look-and-post.md` | matching the video's look: grade, fog, bloom, density, hierarchy, the side-by-side routine |
| `references/playtest.md` | playing your own game: scenarios, recording, best window, what to fix |
Templates: `templates/game.html` (Three.js skeleton with the post stack, input, state contract),
`templates/play.mjs` (a keyboard playtest scenario).
## Output expectations
- Say what was checked and how: which pairs, which take, fps, console errors.
- List the fal calls: count per endpoint and total cost.
- List honestly what differs from the video (a simplified mechanic, a missing effect), never "identical".
© Lovis Odin · MIT · lovisdotio/video-to-game