Skip to main content
skill CX Codex

offline-voice — Claude Code skill: what it does, how to install

offline-voice is a Claude Code skill that guides the setup of VieNeu-TTS narration and voice cloning. Activates when working on /api/tts, /api/voices, or narration debugging.

MIT ★ 147 updated 2026-11-13

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add notivn/AIEVv2 --skill offline-voice -a claude-code
Open on GitHub

What offline-voice does

Documents the on-device VieNeu-TTS v3 Turbo synthesis setup alongside Gemini API narration. The local engine handles Vietnamese narration with English code-switching and clones voices from 3–8 seconds of reference audio.

Enforces two isolated setup tiers. Preset synthesis only demands pip install vieneu on ONNX Runtime taking ~30 MB without Torch. Voice cloning requires torch and torchaudio, pulling an extra 2–3 GB. Initial generation downloads roughly 1 GB of weights from Hugging Face.

Supplies verified fixes for dependency traps: monkeypatching torchaudio.load with soundfile to prevent TorchCodec loading failures on static FFmpeg, pinning av>=11,<17 for faster-whisper, and strict positional parsing for voice metadata labels.

What it helps with

  • configure local VieNeu-TTS synthesis without API charges
  • bypass TorchCodec import errors during voice cloning
  • resolve PyAV version conflicts in faster-whisper transcriptions
  • parse Vietnamese voice metadata without confusing gender and region

Example prompts for the agent

  • Fix the 503 LOAD_FAILED error on the VieNeu local narration worker.
  • How do I set up voice cloning without forcing full Torch on preset speech users?
  • Apply the soundfile monkeypatch for torchaudio reference loading.

How to install offline-voice

1Skills CLI (Vercel)

npx skills add notivn/AIEVv2 --skill offline-voice -a claude-code

2Manually: put the files in place

~/.claude/skills/offline-voice/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/offline-voice/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about offline-voice

How to install offline-voice in Claude Code?

Run npx skills add notivn/AIEVv2 --skill offline-voice -a claude-code. The file will be placed in ~/.claude/skills/offline-voice/SKILL.md.

What does offline-voice do?

It gives instructions for on-device VieNeu-TTS integration, worker runtime handling, and workarounds for torchaudio and audio library bugs.

What is required to run offline-voice?

Basic synthesis requires Python and vieneu running via ONNX Runtime. Cloning additionally requires torch, torchaudio, and 1 GB of disk cache for Hugging Face model files.

Source text

SKILL.md · MIT
---
name: offline-voice
description: How the on-device narration engine (VieNeu-TTS) and voice cloning work in this system - install tiers, the worker protocol, and the verified traps around torchaudio, voice metadata parsing and duration. Read when working on Text to video narration, the Voices page, the /api/voices or /api/tts endpoints, or when a user reports that offline voices or voice cloning fail to install or sound wrong.
---

# Offline voice - VieNeu-TTS and voice cloning

The system has **two narration engines running side by side**. They are deliberately not merged.

| | `gemini` | `vieneu` |
|---|---|---|
| Where it runs | Google API | The user's own machine |
| Cost | Per call | Free |
| Needs network | Yes | No |
| Voices | 30 preset | 14 Vietnamese preset + unlimited cloned |
| Voice cloning | **No** (API only accepts preset names) | **Yes** |
| Speed | A few seconds | RTF ~1.0 (about real time) |
| Setup | `GEMINI_API_KEY` | `pip install vieneu` |

Keep both. Do not "simplify" by deleting one: a weak machine reads at roughly real time, which is unacceptable for a long script, while Gemini costs money on every preview click.

## Why VieNeu and not F5-TTS / XTTS / Chatterbox

Checked against the requirement "Vietnamese + English, cloning, and shippable in an open-source repo":

- **Chatterbox** (MIT, excellent English) has **no Vietnamese**. Disqualified on language.
- **F5-TTS** code is MIT but the **weights are CC-BY-NC**, and every Vietnamese finetune is CC-BY-NC-SA. Disqualified on licence.
- **viXTTS / XTTS-v2** inherit Coqui's CPML non-commercial licence. Disqualified on licence.
- **VietTTS** (dangvansam) is Apache-2.0 code but the installer is Linux-only, and its weights are CC-BY-NC too.
- **VieNeu-TTS** is Apache 2.0 on the code and on the **v3 Turbo weights** (`pnnbao-ump/VieNeu-TTS-v3-Turbo`, trained from scratch on ~10k hours, not a finetune), plus Apache 2.0 on its codec dependency `OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano`. Vietnamese-first with English code-switching, torch-free on CPU via ONNX, clones from 3-8 seconds of reference audio.

If a future engine is evaluated, check the **weights** licence separately from the code licence. That is where every other candidate failed.

**Licence trap inside the package itself.** `vieneu/assets/voices.json` carries `"license": "CC BY-NC 4.0"` and `"Model and voices are for non-commercial use only"`. That notice is scoped to **that file**, which is the **v2** preset pack: 6 voices named `Binh, Tuyen, Vinh, Doan, Ly, Ngoc`, default `Binh`. We run `mode="v3turbo"`, which reads `voices_v3_turbo.json` (14 Vietnamese voices, default `Phạm Tuyên`). The two sets have **zero overlap** - verified by comparing the key sets - so nothing on our path is NC-licensed. If anyone ever switches the engine mode away from `v3turbo`, or a preset named `Binh`/`Tuyen`/`Vinh`/`Doan`/`Ly`/`Ngoc` shows up in the picker, the commercial position changes and must be rechecked.

## Install tiers - keep them separate

Two independent levels. Never collapse them into one check:

1. **Speech only**: `pip install vieneu`. Torch-free (ONNX Runtime), ~30 MB of wheels. This is enough for all 14 preset voices.
2. **Cloning**: additionally `pip install torch torchaudio`, 2-3 GB.

`start/doctor.mjs` reports these as `vieneu` and `vieneu-clone`, and only asks about torch once `vieneu` is present. Forcing a 2-3 GB torch download on someone who only wants offline narration is the mistake to avoid.

The first synthesis after a server start downloads ~1 GB of model from HuggingFace and takes 30s; subsequent process starts take ~15s to load. Always surface this in the UI, otherwise the first click looks like a hang.

## Known issues

**Symptom:** every transcription (video projects, Text to video, narration) fails with `TypeError: open() got an unexpected keyword argument 'metadata_errors'` from `faster_whisper/audio.py`.
**Cause:** PyAV 17+ dropped `metadata_errors`, but faster-whisper 1.2.x still passes it to `av.open()` (verified 2026-10: `av` 19.0.1 + `faster-whisper` 1.2.1). An unpinned `pip install faster-whisper` pulls the newest `av`.
**Fix:** `.runtime/venv/Scripts/python.exe -m pip install "av>=11,<17"` (16.1.0 verified). `start/doctor.mjs` now pins it in the faster-whisper fix, but a venv that already has av 19 must be downgraded by hand - the doctor's probe only checks `import faster_whisper`, which still succeeds.

**Symptom:** every VieNeu call (narration, Text to video, voice preview) returns `503 LOAD_FAILED` for the rest of the server's life, yet the same Python loads `Vieneu()` and synthesizes fine from the command line.
**Cause:** one failed model load (seen once as `External data path escapes model directory` while the ~1GB model was still being fetched on first use) set `loadFailure` in `ttsLocal.ts`. That flag overrode every later probe, and the only way to clear it was `probeLocalEngine(true)` - which no route called. An agent mid-edit had no way out but asking the user to restart the backend.
**Fix:** the flag now expires after 2 minutes (the next call retries the real load), and `GET /api/tts/engines?refresh=1` - the Voices page "Kiểm tra lại" button - clears it immediately. To check a machine by hand, run the worker with the server's env (`HF_HOME` and `HUGGINGFACE_HUB_CACHE` both = `.runtime/models`) and send `{"id":1,"cmd":"synth","text":"Xin chào","voice":"Mai Anh","out":"<abs path>.wav"}` on stdin.

**Symptom:** Cloning fails with `ImportError: TorchCodec is required for load_with_torchcodec`, or `RuntimeError: Could not load libtorchcodec`.
**Cause:** `add_voice()` reads the reference through `torchaudio.load()`. From torchaudio 2.9 every backend routes through `torchcodec`, which needs FFmpeg **shared libraries** - the usual Windows FFmpeg builds are static, so it can never load. Installing `torchcodec` does not fix it. Passing `backend="soundfile"` does not fix it either (verified: all three backends raise the same error).
**Fix:** monkeypatch `torchaudio.load` to use `soundfile` **before importing vieneu**. `soundfile` is already a vieneu dependency, so this adds nothing to install. Guard it in try/except so that a machine without torch still gets preset voices.

```python
import torch, torchaudio, soundfile as sf
def _load(uri, *a, **kw):
    data, sr = sf.read(str(uri), dtype="float32", always_2d=True)
    return torch.from_numpy(data.T).contiguous(), sr
torchaudio.load = _load
```

**Symptom:** Voices are filed under the wrong region, or every male voice looks southern.
**Cause:** `list_preset_voices()` returns labels shaped `Name — <Nam|Nữ> · <Bắc|Trung|Nam> · Phong cách <...>`. Field 1 is **gender**, field 2 is **region**, and the word **"Nam" means both "male" and "southern"**.
**Fix:** parse strictly by position, never by searching for the substring "Nam". Regression cases: `Thái Sơn` = gender nam / region nam; `Mai Anh` = gender nu / region bac.

**Symptom:** Cloned voices vanish after a restart, or a stray voice appears in the preset list and leaks into vieneu's own error messages.
**Cause:** `tts.save_voices()` (and `add_voice(..., save=True)`) rewrites `vieneu/assets/voices_v3_turbo.json` inside site-packages, permanently adding your test voice as a 15th "preset". This happened for real during development.
**Fix:** never call either. The store lives at `assets/voices/` (`library.json` plus `<id>/ref.wav`) and voices are re-registered into the worker on demand with `add_voice(..., save=False)`. Registration costs ~3s and is lost whenever the worker restarts, so track what the live process actually has. As defence in depth the worker drops any preset whose description does not parse into `gender · region · style`, and `listLocalVoices()` drops presets colliding with a store id - a contaminated install still yields a clean 14. To repair one by hand, delete the offending key from `voices_v3_turbo.json` or `pip install --force-reinstall vieneu`.

**Symptom:** A storytelling voice reads like a newsreader.
**Cause:** `infer()` has its own `style=` argument defaulting to `"tu_nhien"`, which is independent of the voice you picked.
**Fix:** look up each preset's own `style` (`get_preset_voice(name)` returns `{gender, style, description, ...}` as real fields) and pass it into `infer()`. Only **region** has to be parsed out of the description string.

**Symptom:** Timeline drifts against the narration.
**Cause:** estimating duration from character count.
**Fix:** every duration is measured with ffprobe, exactly as on the Gemini path. This is not engine-specific advice - see the `video-pipeline` skill.

## Architecture notes

- `apps/server/python/vieneu_worker.py` is a long-lived worker speaking JSON-lines on stdin/stdout. **stdout is protocol only**; all logging goes to stderr. It must `sys.stdout.reconfigure(encoding="utf-8")` first thing or Vietnamese corrupts on the Windows cp1252 console.
- `ping` must answer **without loading the model** - it backs the availability check.
- One worker, requests serialised (the model is not re-entrant), 10-minute idle shutdown because it holds ~1 GB of RAM.
- Output is already **48 kHz mono**, which equals the pipeline's `OUT_SAMPLE_RATE`, so no resampling. It still goes through the same `TRIM_FILTER` and gap-insertion path as Gemini in `synthScript()` - two engines with two joining strategies would drift audibly at the seams.
- The Perth watermark is left at its default (on). It is inaudible and is the responsible default for cloned speech.

## Reference audio guidance for users

3-8 seconds, clean speech, no background music, no noise. Longer is **not** better - the model only uses the head, so anything past ~8s is discarded. Below 3s the embedding is too weak and the clone stops resembling the speaker.

© notivn · MIT · notivn/AIEVv2

Similar skills

All skills for AI and ML →

From the same set15

The whole set →
skill-authoring skill-authoring is a Claude Code skill that sets the rules for creating and revising skills for AI Edit Video. It activates on requests to write a skill or capture a lesson after a video is finished. CX Codex +1 ★ 147repo stars color-grading color-grading is a Claude Code skill that handles video color grading in the AI Edit Video pipeline. Runs when processing HDR or log clips, or upon a Chỉnh màu request. CX Codex +1 ★ 147repo stars background-music background-music is a Claude Code skill that selects a soundtrack and configures automatic ducking under speech in Remotion videos. It runs when a brief requests background music or an edit asks to add or adjust it. CX Codex +1 ★ 147repo stars key-layout key-layout is a Claude Code skill that places a video's main topic at the top and related points above the captions. It activates when keyLayoutEnabled is on in the brief or the edit prompt includes “Bố cục Key: BẬT”. CX Codex +1 ★ 147repo stars make-a-video make-a-video is a Claude Code skill that turns a concept into a finished MP4 with HyperFrames. It gathers requirements, builds a storyboard, and checks the preview before the final render. CX Codex +1 ★ 147repo stars auto-cut auto-cut is a Claude Code skill that cuts silences, stutters, and repeated takes from talking-head footage before editing. It activates when the autoCut brief setting is enabled. CX Codex +1 ★ 147repo stars ai-illustrations ai-illustrations is a Claude Code skill that adds generated images to a video edit. It activates when the brief enables AI illustrations or requests them for a video. CX Codex +1 ★ 147repo stars remotion-assemble remotion-assemble is a Claude Code skill that stitches HyperFrames scenes, footage, voiceovers, and SFX into a video via meta.json. It activates when editing files inside engines/remotion. CX Codex +1 ★ 147repo stars

Project sponsors

Companies that keep this analytics open