Skip to main content
skill CX Codex

capy-video-gen-skill — Claude Code skill

capy-video-gen-skill — a Claude Code skill that transforms scripts and ideas into full-length videos where character faces remain consistent throughout. Use it when you need to create multi-shot video from text, a story, or a brief concept.

MIT ★ 138 updated 2026-09-03

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add happycapy-ai/Happycapy-skills --skill capy-video-gen-skill -a claude-code
Open on GitHub

What capy-video-gen-skill does

ViMax is a multi-stage pipeline that parses a script, extracts characters with their features, generates three-angle portraits for each one, designs a shot-by-shot storyboard, renders each frame with face recognition prioritized, produces video clips, and assembles the final result. Across 257 experiments, face identification accuracy improved 70% (face distance dropped from 0.74 to 0.22).

You need an AI_GATEWAY_API_KEY to run it — HappyCapy pre-fills this. Inside, ViMax uses image generation (Gemini Flash) and video generation (Veo), and the reference image selector always ranks face identity first. The skill accepts two inputs: either a finished screenplay (script2video) or a brief idea (idea2video, which auto-generates a script).

For stable faces, write detailed character descriptions (ethnicity, age, hair colour and texture, eye shape, marks, jawline). Include at least one close-up shot per character. If you have real photos, place them in the working directory and pass them as a reference registry.

What it helps with

  • generate video from a written screenplay where all character faces stay identical
  • break down a concept into video clips with consistent character faces
  • assemble multi-shot video with control over framing and camera movement
  • produce character portraits in multiple angles for face recognition
  • select best-rendered frames while prioritising face identity

Example prompts for the agent

  • Create a video from this screenplay with three characters, each with a distinct face throughout
  • Generate a full video from the idea of two friends meeting, no more than five shots
  • Take the description of a girl in a red jacket meeting a friend, render it with close-up detail on faces

How to install capy-video-gen-skill

1Skills CLI (Vercel)

npx skills add happycapy-ai/Happycapy-skills --skill capy-video-gen-skill -a claude-code

2Manually: put the files in place

~/.claude/skills/capy-video-gen-skill/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/capy-video-gen-skill/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about capy-video-gen-skill

How do I install capy-video-gen-skill in Claude Code?

Run npx skills add happycapy-ai/Happycapy-skills --skill capy-video-gen-skill -a claude-code, then open ~/.claude/skills/capy-video-gen-skill/SKILL.md. For Codex use .codex/skills/capy-video-gen-skill/SKILL.md, for Cursor use .cursor/skills/capy-video-gen-skill/SKILL.md.

What does capy-video-gen-skill do?

It converts a screenplay or idea into video where all character faces remain identical across every shot. The pipeline extracts characters from text, renders their portraits, designs a frame-by-frame storyboard, generates each frame with face identity prioritized, and assembles the final video.

What improves face accuracy in the output?

Write detailed character descriptions in your screenplay: ethnicity, hair colour, eye shape, distinguishing marks, jawline structure. Include at least one tight close-up of each character. If you have real photos, put them in the working directory and pass them as a reference registry.

Source text

SKILL.md · MIT
---
name: capy-video-gen-skill
description: Multi-shot AI video generation pipeline with face identity consistency. Converts scripts or ideas into complete videos using character extraction, storyboarding, frame generation, and video assembly. 300 experiments validated, 70% face distance improvement. Use when the user asks to create a video from a script, story, idea, or wants multi-shot video with consistent characters.
allowed-tools: Bash, Read, Write, Edit
---

# Capy Video Gen Skill - Script-to-Video Pipeline

Generate complete multi-shot videos from scripts or ideas with consistent character faces across all scenes. Built for HappyCapy AI Gateway. 300 experiments validated, 70% face distance improvement.

## Overview

ViMax converts text scripts into full videos through an automated pipeline:
1. Extract characters from script with detailed physical features
2. Generate front/side/back character portraits
3. Design shot-by-shot storyboard
4. Decompose each shot into first_frame, last_frame, and motion descriptions
5. Build camera tree for shot relationships
6. Generate frames with reference image selection (face identity as top priority)
7. Generate video clips from frames
8. Concatenate into final video

## Installation Location

The ViMax pipeline code is at: `/home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax/`

All commands must be run from this directory using the venv:
```bash
cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax
```

## Prerequisites

- `AI_GATEWAY_API_KEY` environment variable (auto-configured in HappyCapy)
- Python venv at `.venv/` (already set up)

## Quick Start

### Script-to-Video

Edit the script, requirements, and style in the entry script, then run:

```bash
cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax
.venv/bin/python main_happycapy_script2video.py
```

### Idea-to-Video

For generating from a brief idea (auto-generates script first):

```bash
cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax
.venv/bin/python main_happycapy_idea2video.py
```

## Programmatic Usage

```python
import asyncio
from langchain.chat_models import init_chat_model
from tools.render_backend import RenderBackend
from utils.config_loader import load_config
from pipelines.script2video_pipeline import Script2VideoPipeline

config = load_config("configs/happycapy_script2video.yaml")
chat_model = init_chat_model(**config["chat_model"]["init_args"])
backend = RenderBackend.from_config(config)

pipeline = Script2VideoPipeline(
    chat_model=chat_model,
    image_generator=backend.image_generator,
    video_generator=backend.video_generator,
    working_dir=config["working_dir"],
)

# Run the pipeline
asyncio.run(pipeline(
    script="Your script here...",
    user_requirement="No more than 8 shots total.",
    style="Cinematic, warm lighting"
))
```

## Pipelines

### Script2VideoPipeline
- Input: A formatted screenplay/script with character dialogue and scene descriptions
- Output: Concatenated video at `{working_dir}/final_video.mp4`
- Config: `configs/happycapy_script2video.yaml`

### Idea2VideoPipeline
- Input: A brief idea/concept (1-3 paragraphs)
- Output: Auto-generates a script, then produces video
- Config: `configs/happycapy_idea2video.yaml`

## Configuration

HappyCapy configs at `configs/happycapy_script2video.yaml`:

```yaml
chat_model:
  init_args:
    model: gpt-4.1
    model_provider: openai
    api_key: ${AI_GATEWAY_API_KEY}
    base_url: https://ai-gateway.happycapy.ai/api/v1/openai/v1

image_generator:
  class_path: tools.ImageGeneratorHappyCapyAPI
  init_args:
    api_key: ${AI_GATEWAY_API_KEY}
    model: google/gemini-3.1-flash-image-preview

video_generator:
  class_path: tools.VideoGeneratorHappyCapyAPI
  init_args:
    api_key: ${AI_GATEWAY_API_KEY}
    model: google/veo-3.1-generate-preview

working_dir: .working_dir/script2video
```

## Key Components

### Agents (AI Processing)

| Agent | File | Purpose |
|-------|------|---------|
| CharacterExtractor | `agents/character_extractor.py` | Extract characters with static/dynamic features from script |
| CharacterPortraitsGenerator | `agents/character_portraits_generator.py` | Generate front/side/back portraits for each character |
| StoryboardArtist | `agents/storyboard_artist.py` | Design shot-by-shot storyboard with first/last frames and motion |
| ReferenceImageSelector | `agents/reference_image_selector.py` | Select best reference images for each frame (face identity #1 priority) |
| CameraImageGenerator | `agents/camera_image_generator.py` | Build camera trees and generate transition videos |
| BestImageSelector | `agents/best_image_selector.py` | Select best generated image from candidates |
| Screenwriter | `agents/screenwriter.py` | Generate scripts from ideas |

### Tools (Generation Backends)

| Tool | File | Purpose |
|------|------|---------|
| ImageGeneratorHappyCapyAPI | `tools/image_generator_happycapy_api.py` | Image generation via HappyCapy Gateway (Gemini) |
| VideoGeneratorHappyCapyAPI | `tools/video_generator_happycapy_api.py` | Video generation via HappyCapy Gateway (Veo) |
| RenderBackend | `tools/render_backend.py` | Factory for instantiating generators from config |

### Interfaces (Data Models)

- `CharacterInScene` - Character with identifier, static_features, dynamic_features
- `ShotDescription` - Shot with ff_desc, lf_desc, motion_desc, variation_type
- `Camera` - Camera with parent-child relationships
- `Frame` - Frame with shot_idx, frame_type, visible characters
- `ImageOutput` / `VideoOutput` - Generation outputs with save methods

## Face Identity Consistency (CRITICAL)

This pipeline includes face identity improvements validated through 257 experiments (70% improvement in face distance, from 0.74 to 0.22):

### Built-In Protections

1. **Reference Image Selector**: Face identity is the #1 priority when selecting reference images. The front-view portrait is always included when a character's face is visible.

2. **Character Portraits**: Enhanced prompts generate identity-critical details (exact nose shape, eye spacing, jawline, distinguishing marks) for cross-scene recognition.

3. **Video Prompt Face Lock**: Every video generation prompt is prepended with a face identity instruction requiring the character's face to remain identical to the starting frame throughout the clip.

### Best Practices When Using ViMax

- **Hyper-detailed character descriptions**: Include ethnicity, age, hair texture/style/color, eye shape, facial hair, glasses, skin tone, build, and distinguishing marks in your script's character introductions
- **Extreme close-up shots**: Include at least one extreme close-up per character to anchor identity
- **Consistent lighting**: Specify similar lighting across scenes to prevent face drift
- **User-provided reference photos**: Place photos in the working directory and pass them as `character_portraits_registry` to skip AI portrait generation

### What Does NOT Work

- Complex prompt engineering (viseme morphing, phoneme anchoring) does not improve face identity
- Simple, direct prompts with detailed physical descriptions outperform clever prompts
- Lip-sync to external audio is NOT possible (Veo generates its own internal audio)

See `FACE_IDENTITY_GUIDE.md` in the ViMax directory for full details.

## Output Structure

After a run, the working directory contains:

```
.working_dir/script2video/
  characters.json                      # Extracted characters
  character_portraits_registry.json    # Portrait paths registry
  character_portraits/                 # Generated portraits
    0_CharacterName/
      front.png
      side.png
      back.png
  storyboard.json                     # Shot descriptions
  camera_tree.json                    # Camera relationships
  shots/
    0/
      shot_description.json
      first_frame.png
      last_frame.png (if medium/large variation)
      video.mp4
    1/
      ...
  final_video.mp4                     # Final concatenated output
```

## Customization

### Using Your Own Reference Photos

To use real photos instead of AI-generated portraits:

```python
# Build a portrait registry pointing to your photos
character_portraits_registry = {
    "Alice": {
        "front": {"path": "/path/to/alice_front.png", "description": "Front view of Alice"},
        "side": {"path": "/path/to/alice_side.png", "description": "Side view of Alice"},
        "back": {"path": "/path/to/alice_back.png", "description": "Back view of Alice"},
    }
}

# Pass to pipeline (skips portrait generation)
await pipeline(
    script=script,
    user_requirement=user_requirement,
    style=style,
    character_portraits_registry=character_portraits_registry,
)
```

### Changing Models

Edit the YAML config to use different models:
- Image: `google/gemini-3.1-flash-image-preview` (recommended for face identity)
- Video: `google/veo-3.1-generate-preview` (recommended) or `openai/sora-2`
- Chat: `gpt-4.1` (recommended) or any OpenAI-compatible model

## Troubleshooting

### "No module named 'tools'" or similar import errors
Run from the ViMax root directory:
```bash
cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax
.venv/bin/python main_happycapy_script2video.py
```

### API rate limit errors
Reduce `max_requests_per_minute` in the YAML config.

### Face identity drift in generated videos
- Add more physical detail to character descriptions in your script
- Use user-provided reference photos instead of AI-generated portraits
- Include extreme close-up shots for important characters
- Keep lighting consistent across scenes

© happycapy-ai · MIT · happycapy-ai/Happycapy-skills

Similar skills

All skills for AI and ML →

From the same set41

The whole set →
agent-0-strategic-planner The agent-0-strategic-planner subagent ranks GitHub issues by user impact and feasibility. Use it to find meaningful contribution opportunities instead of spending effort on cosmetic fixes. CX Codex ★ 138repo stars claude-code-templates Installs and drives the claude-code-templates CLI: a catalog of agents, slash commands, MCP servers, hooks and project templates for Claude, plus session monitoring. CX Codex +1 ★ 138repo stars 360-panorama-viewer 360-panorama-viewer builds a standalone HTML file with 360° panoramas rendered via Three.js. Useful for creating interactive VR galleries from local or generated equirectangular images. CX Codex +1 ★ 138repo stars agent-8-review-responder The agent-8-review-responder subagent tracks PR reviews and CI checks, handles simple feedback, and replies to reviewers. It escalates security concerns and changes that need human judgment. CX Codex ★ 138repo stars reddit-post-writer reddit-post-writer drafts authentic Reddit posts via phased emotional state simulation and checks against AI writing habits. CX Codex +1 ★ 138repo stars film-creator film-creator turns a text idea or image into a short film by writing a screenplay, planning scenes, and generating video. Use it to shape a concept into a cinematic story. CX Codex +1 ★ 138repo stars olive-rose-pptx olive-rose-pptx generates presentations in Olive Rose Salon brand language: dark olive-brown backgrounds, pale pink headlines, high-fashion poster aesthetic. Each slide reads as an exhibition poster, not a report. CX Codex +1 ★ 138repo stars mobile-design mobile-design enforces touch-first principles and performance standards for iOS and Android apps. Keeps AI from generating desktop-style patterns on mobile. CX Codex +1 ★ 138repo stars

Project sponsors

Companies that keep this analytics open