offline-voice — скилл для Claude Code: что делает и как установить
Скилл offline-voice для Claude Code объясняет настройку локального движка озвучки VieNeu-TTS и клонирования голоса. Включается при отладке маршрутов /api/tts и озвучки видео.
MIT ★ 147 обновлён 2026-11-13
npx skills add notivn/AIEVv2 --skill offline-voice -a claude-codeЧто делает offline-voice
Описывает устройство локального синтеза речи VieNeu-TTS v3 Turbo и клонирования по 3–8 секундам сэмпла, разделяя бесплатный офлайн-движок и вызовы к облачному API Gemini.
Разделяет установку на два независимых уровня. Для воспроизведения пресетов нужен пакет pip install vieneu на базе ONNX Runtime без Torch, что занимает около 30 МБ. Клонирование голоса требует дополнительной установки пакетов torch и torchaudio весом 2–3 ГБ. Первый запуск синтеза скачивает веса модели объёмом около 1 ГБ из Hugging Face.
Содержит готовые решения известных системных сбоев: подмену torchaudio.load через библиотеку soundfile для обхода ошибки TorchCodec на статических сборках FFmpeg, ограничение версии av>=11,<17 для работы faster-whisper и позиционный парсинг региональных меток во избежание омонимов.
Какие задачи решает
- настроить локальную озвучку VieNeu-TTS без облачных API
- обойти ошибку TorchCodec при клонировании голоса
- устранить конфликт библиотек PyAV и faster-whisper
- разобрать метаданные голосов без путаницы региона и пола
Примеры запросов агенту
- Почему генерация голоса VieNeu возвращает ошибку 503 LOAD_FAILED?
- Как настроить клонирование голоса в /api/tts без установки пакетов Torch для всех пользователей?
- Примени monkeypatch для torchaudio через soundfile при загрузке референсного аудио.
Как установить offline-voice
1Через Skills CLI (Vercel)
npx skills add notivn/AIEVv2 --skill offline-voice -a claude-code2Вручную: положить файлы
~/.claude/skills/offline-voice/SKILL.mdПапка скилла целиком; для одного проекта — .claude/skills/offline-voice/
1Через Skills CLI (Vercel)
npx skills add notivn/AIEVv2 --skill offline-voice -a codex2Вручную: положить файлы
.agents/skills/offline-voice/SKILL.mdДля всех проектов — ~/.agents/skills/
1Через Skills CLI (Vercel)
npx skills add notivn/AIEVv2 --skill offline-voice -a cursor2Вручную: положить файлы
.cursor/skills/offline-voice/SKILL.mdCursor читает и .agents/skills/, и .claude/skills/
1GitHub CLI
gh skill install notivn/AIEVv2 .claude/skills/offline-voice2Вручную: положить файлы
.github/skills/offline-voice/SKILL.md1Через Skills CLI (Vercel)
npx skills add notivn/AIEVv2 --skill offline-voice -a gemini-cli2Командой платформы
gemini skills install https://github.com/notivn/AIEVv2 --path .claude/skills/offline-voiceВручную: положить файлы
.devin/skills/offline-voice/SKILL.mdСтарый путь .windsurf/skills/ тоже читается
Claude.ai
Упакуйте папку в zip и загрузите в Settings → Capabilities → Skills
По шагам, с командами для каждой платформы: Как установить скилл в Claude Code: пять способов
Вопросы об offline-voice
Как установить offline-voice в Claude Code?
Выполните команду npx skills add notivn/AIEVv2 --skill offline-voice -a claude-code. Скилл запишется по пути ~/.claude/skills/offline-voice/SKILL.md.
Что делает offline-voice?
Предоставляет агенту правила настройки движка VieNeu-TTS, протокол воркера синтеза и фиксы для совместимости torchaudio и PyAV.
Что нужно для работы offline-voice?
Для пресетов требуется библиотека vieneu с ONNX Runtime. Для клонирования нужны пакеты torch, torchaudio и около 1 ГБ места на диске для загрузки весов модели.
Исходный текст
SKILL.md · MIT
---
name: offline-voice
description: How the on-device narration engine (VieNeu-TTS) and voice cloning work in this system - install tiers, the worker protocol, and the verified traps around torchaudio, voice metadata parsing and duration. Read when working on Text to video narration, the Voices page, the /api/voices or /api/tts endpoints, or when a user reports that offline voices or voice cloning fail to install or sound wrong.
---
# Offline voice - VieNeu-TTS and voice cloning
The system has **two narration engines running side by side**. They are deliberately not merged.
| | `gemini` | `vieneu` |
|---|---|---|
| Where it runs | Google API | The user's own machine |
| Cost | Per call | Free |
| Needs network | Yes | No |
| Voices | 30 preset | 14 Vietnamese preset + unlimited cloned |
| Voice cloning | **No** (API only accepts preset names) | **Yes** |
| Speed | A few seconds | RTF ~1.0 (about real time) |
| Setup | `GEMINI_API_KEY` | `pip install vieneu` |
Keep both. Do not "simplify" by deleting one: a weak machine reads at roughly real time, which is unacceptable for a long script, while Gemini costs money on every preview click.
## Why VieNeu and not F5-TTS / XTTS / Chatterbox
Checked against the requirement "Vietnamese + English, cloning, and shippable in an open-source repo":
- **Chatterbox** (MIT, excellent English) has **no Vietnamese**. Disqualified on language.
- **F5-TTS** code is MIT but the **weights are CC-BY-NC**, and every Vietnamese finetune is CC-BY-NC-SA. Disqualified on licence.
- **viXTTS / XTTS-v2** inherit Coqui's CPML non-commercial licence. Disqualified on licence.
- **VietTTS** (dangvansam) is Apache-2.0 code but the installer is Linux-only, and its weights are CC-BY-NC too.
- **VieNeu-TTS** is Apache 2.0 on the code and on the **v3 Turbo weights** (`pnnbao-ump/VieNeu-TTS-v3-Turbo`, trained from scratch on ~10k hours, not a finetune), plus Apache 2.0 on its codec dependency `OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano`. Vietnamese-first with English code-switching, torch-free on CPU via ONNX, clones from 3-8 seconds of reference audio.
If a future engine is evaluated, check the **weights** licence separately from the code licence. That is where every other candidate failed.
**Licence trap inside the package itself.** `vieneu/assets/voices.json` carries `"license": "CC BY-NC 4.0"` and `"Model and voices are for non-commercial use only"`. That notice is scoped to **that file**, which is the **v2** preset pack: 6 voices named `Binh, Tuyen, Vinh, Doan, Ly, Ngoc`, default `Binh`. We run `mode="v3turbo"`, which reads `voices_v3_turbo.json` (14 Vietnamese voices, default `Phạm Tuyên`). The two sets have **zero overlap** - verified by comparing the key sets - so nothing on our path is NC-licensed. If anyone ever switches the engine mode away from `v3turbo`, or a preset named `Binh`/`Tuyen`/`Vinh`/`Doan`/`Ly`/`Ngoc` shows up in the picker, the commercial position changes and must be rechecked.
## Install tiers - keep them separate
Two independent levels. Never collapse them into one check:
1. **Speech only**: `pip install vieneu`. Torch-free (ONNX Runtime), ~30 MB of wheels. This is enough for all 14 preset voices.
2. **Cloning**: additionally `pip install torch torchaudio`, 2-3 GB.
`start/doctor.mjs` reports these as `vieneu` and `vieneu-clone`, and only asks about torch once `vieneu` is present. Forcing a 2-3 GB torch download on someone who only wants offline narration is the mistake to avoid.
The first synthesis after a server start downloads ~1 GB of model from HuggingFace and takes 30s; subsequent process starts take ~15s to load. Always surface this in the UI, otherwise the first click looks like a hang.
## Known issues
**Symptom:** every transcription (video projects, Text to video, narration) fails with `TypeError: open() got an unexpected keyword argument 'metadata_errors'` from `faster_whisper/audio.py`.
**Cause:** PyAV 17+ dropped `metadata_errors`, but faster-whisper 1.2.x still passes it to `av.open()` (verified 2026-10: `av` 19.0.1 + `faster-whisper` 1.2.1). An unpinned `pip install faster-whisper` pulls the newest `av`.
**Fix:** `.runtime/venv/Scripts/python.exe -m pip install "av>=11,<17"` (16.1.0 verified). `start/doctor.mjs` now pins it in the faster-whisper fix, but a venv that already has av 19 must be downgraded by hand - the doctor's probe only checks `import faster_whisper`, which still succeeds.
**Symptom:** every VieNeu call (narration, Text to video, voice preview) returns `503 LOAD_FAILED` for the rest of the server's life, yet the same Python loads `Vieneu()` and synthesizes fine from the command line.
**Cause:** one failed model load (seen once as `External data path escapes model directory` while the ~1GB model was still being fetched on first use) set `loadFailure` in `ttsLocal.ts`. That flag overrode every later probe, and the only way to clear it was `probeLocalEngine(true)` - which no route called. An agent mid-edit had no way out but asking the user to restart the backend.
**Fix:** the flag now expires after 2 minutes (the next call retries the real load), and `GET /api/tts/engines?refresh=1` - the Voices page "Kiểm tra lại" button - clears it immediately. To check a machine by hand, run the worker with the server's env (`HF_HOME` and `HUGGINGFACE_HUB_CACHE` both = `.runtime/models`) and send `{"id":1,"cmd":"synth","text":"Xin chào","voice":"Mai Anh","out":"<abs path>.wav"}` on stdin.
**Symptom:** Cloning fails with `ImportError: TorchCodec is required for load_with_torchcodec`, or `RuntimeError: Could not load libtorchcodec`.
**Cause:** `add_voice()` reads the reference through `torchaudio.load()`. From torchaudio 2.9 every backend routes through `torchcodec`, which needs FFmpeg **shared libraries** - the usual Windows FFmpeg builds are static, so it can never load. Installing `torchcodec` does not fix it. Passing `backend="soundfile"` does not fix it either (verified: all three backends raise the same error).
**Fix:** monkeypatch `torchaudio.load` to use `soundfile` **before importing vieneu**. `soundfile` is already a vieneu dependency, so this adds nothing to install. Guard it in try/except so that a machine without torch still gets preset voices.
```python
import torch, torchaudio, soundfile as sf
def _load(uri, *a, **kw):
data, sr = sf.read(str(uri), dtype="float32", always_2d=True)
return torch.from_numpy(data.T).contiguous(), sr
torchaudio.load = _load
```
**Symptom:** Voices are filed under the wrong region, or every male voice looks southern.
**Cause:** `list_preset_voices()` returns labels shaped `Name — <Nam|Nữ> · <Bắc|Trung|Nam> · Phong cách <...>`. Field 1 is **gender**, field 2 is **region**, and the word **"Nam" means both "male" and "southern"**.
**Fix:** parse strictly by position, never by searching for the substring "Nam". Regression cases: `Thái Sơn` = gender nam / region nam; `Mai Anh` = gender nu / region bac.
**Symptom:** Cloned voices vanish after a restart, or a stray voice appears in the preset list and leaks into vieneu's own error messages.
**Cause:** `tts.save_voices()` (and `add_voice(..., save=True)`) rewrites `vieneu/assets/voices_v3_turbo.json` inside site-packages, permanently adding your test voice as a 15th "preset". This happened for real during development.
**Fix:** never call either. The store lives at `assets/voices/` (`library.json` plus `<id>/ref.wav`) and voices are re-registered into the worker on demand with `add_voice(..., save=False)`. Registration costs ~3s and is lost whenever the worker restarts, so track what the live process actually has. As defence in depth the worker drops any preset whose description does not parse into `gender · region · style`, and `listLocalVoices()` drops presets colliding with a store id - a contaminated install still yields a clean 14. To repair one by hand, delete the offending key from `voices_v3_turbo.json` or `pip install --force-reinstall vieneu`.
**Symptom:** A storytelling voice reads like a newsreader.
**Cause:** `infer()` has its own `style=` argument defaulting to `"tu_nhien"`, which is independent of the voice you picked.
**Fix:** look up each preset's own `style` (`get_preset_voice(name)` returns `{gender, style, description, ...}` as real fields) and pass it into `infer()`. Only **region** has to be parsed out of the description string.
**Symptom:** Timeline drifts against the narration.
**Cause:** estimating duration from character count.
**Fix:** every duration is measured with ffprobe, exactly as on the Gemini path. This is not engine-specific advice - see the `video-pipeline` skill.
## Architecture notes
- `apps/server/python/vieneu_worker.py` is a long-lived worker speaking JSON-lines on stdin/stdout. **stdout is protocol only**; all logging goes to stderr. It must `sys.stdout.reconfigure(encoding="utf-8")` first thing or Vietnamese corrupts on the Windows cp1252 console.
- `ping` must answer **without loading the model** - it backs the availability check.
- One worker, requests serialised (the model is not re-entrant), 10-minute idle shutdown because it holds ~1 GB of RAM.
- Output is already **48 kHz mono**, which equals the pipeline's `OUT_SAMPLE_RATE`, so no resampling. It still goes through the same `TRIM_FILTER` and gap-insertion path as Gemini in `synthScript()` - two engines with two joining strategies would drift audibly at the seams.
- The Perth watermark is left at its default (on). It is inaudible and is the responsible default for cloned speech.
## Reference audio guidance for users
3-8 seconds, clean speech, no background music, no noise. Longer is **not** better - the model only uses the head, so anything past ~8s is discarded. Below 3s the embedding is too weak and the clone stops resembling the speaker.
© notivn · MIT · notivn/AIEVv2