multimodal-whisper — Claude Code skill: speech recognition
A Claude skill that wires up Whisper: transcribe audio in 99 languages, translate to English, and detect the spoken language.
npx claude-code-templates@latest --skill ai-research/multimodal-whisperWhat multimodal-whisper does
The skill gives your agent access to OpenAI's Whisper speech recognition model. It transcribes audio and video, translates speech into English, and identifies the language of a recording. Useful when a podcast, meeting or interview needs to become text inside the project.
Six model sizes ship with it, from tiny at 39M parameters to large at 1550M. For a balance of speed and quality the docs point to turbo; base is fine for prototyping. You get word-level timestamps, an initial_prompt for technical terms and proper nouns, and temperature fallback when confidence drops. Output can be txt, srt, vtt or json.
You need Python 3.8–3.11, the openai-whisper package and ffmpeg before anything runs. GPU processing is much faster than CPU, so match the model size to your VRAM. Whisper can hallucinate on long recordings, which makes a read-through worth the time.
What it helps with
- turn a podcast or interview into text
- generate srt subtitles for a video
- translate a foreign-language recording into English
- detect which language a speaker is using
- pull notes out of a noisy meeting recording
Example prompts for the agent
- Transcribe
audio.mp3with theturbomodel and save the text - Generate srt subtitles from
video.mp4in English - Translate
spanish.mp3to English with word timestamps
How to install multimodal-whisper
1claude-code-templates
npx claude-code-templates@latest --skill ai-research/multimodal-whisper2Skills CLI (Vercel)
npx skills add davila7/claude-code-templates --skill multimodal-whisper -a claude-code3Manually: put the files in place
~/.claude/skills/multimodal-whisper/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/multimodal-whisper/
Manually: put the files in place
.agents/skills/multimodal-whisper/SKILL.mdFor all projects use ~/.agents/skills/
Manually: put the files in place
.cursor/skills/multimodal-whisper/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install davila7/claude-code-templates multimodal-whisper2Manually: put the files in place
.github/skills/multimodal-whisper/SKILL.mdPlatform CLI
gemini skills install https://github.com/davila7/claude-code-templates --path cli-tool/components/skills/ai-research/multimodal-whisperManually: put the files in place
.devin/skills/multimodal-whisper/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about multimodal-whisper
How do I install multimodal-whisper in Claude?
Through the templates: npx claude-code-templates@latest --skill ai-research/multimodal-whisper. Or via the skills manager: npx skills add davila7/claude-code-templates --skill multimodal-whisper -a claude-code.
What does multimodal-whisper do?
It hooks Whisper into your agent for speech recognition: transcription across 99 languages, translation to English, and language identification. It handles podcasts, video and meeting recordings.
What do I need to run it?
Python 3.8–3.11, the openai-whisper package and ffmpeg. Larger models need VRAM: large takes about 10 GB.
