deepgram-core-workflow-b — Claude Code skill for live transcription
The deepgram-core-workflow-b skill outlines real-time transcription with Deepgram over WebSocket. Use it when adding live captions, voice controls, or speech features to an app.
MIT 1 installs ★ 2,935 updated 2026-10-29
npx skills add jeremylongshore/tons-of-skills-marketplace --skill deepgram-core-workflow-b -a claude-codeWhat deepgram-core-workflow-b does
The skill walks through using @deepgram/sdk to stream audio to Deepgram and handle interim and final transcript events. It also covers utterance-end detection and speaker diarization. A reconnect pattern and an SSE endpoint for browser clients are included.
You need @deepgram/sdk and a configured DEEPGRAM_API_KEY. Audio can come from a microphone, a file stream, or browser audio sent over WebSocket. Microphone capture relies on Sox's rec command, so that path will fail unless Sox is installed. The workflow also advises testing with non-sensitive speech and logging state changes rather than audio, API keys, or full participant speech.
What it helps with
- stream audio to Deepgram over WebSocket
- handle interim and final transcript events
- group transcript words by speaker
- reconnect after a connection error
- send live transcript events to browser clients through SSE
Example prompts for the agent
- Set up live transcription with Deepgram WebSocket in this project.
- Add microphone capture with
recand handle final transcript events. - Group the final transcript by speaker and reconnect after a dropped connection.
How to install deepgram-core-workflow-b
1Skills CLI (Vercel)
npx skills add jeremylongshore/tons-of-skills-marketplace --skill deepgram-core-workflow-b -a claude-code2Manually: put the files in place
~/.claude/skills/deepgram-core-workflow-b/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/deepgram-core-workflow-b/
1Skills CLI (Vercel)
npx skills add jeremylongshore/tons-of-skills-marketplace --skill deepgram-core-workflow-b -a codex2Manually: put the files in place
.agents/skills/deepgram-core-workflow-b/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add jeremylongshore/tons-of-skills-marketplace --skill deepgram-core-workflow-b -a cursor2Manually: put the files in place
.cursor/skills/deepgram-core-workflow-b/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install jeremylongshore/tons-of-skills-marketplace skills/.curated/deepgram-core-workflow-b2Manually: put the files in place
.github/skills/deepgram-core-workflow-b/SKILL.md1Skills CLI (Vercel)
npx skills add jeremylongshore/tons-of-skills-marketplace --skill deepgram-core-workflow-b -a gemini-cli2Platform CLI
gemini skills install https://github.com/jeremylongshore/tons-of-skills-marketplace --path skills/.curated/deepgram-core-workflow-bManually: put the files in place
.devin/skills/deepgram-core-workflow-b/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add jeremylongshore/tons-of-skills-marketplace --skill deepgram-core-workflow-b -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about deepgram-core-workflow-b
How do I install deepgram-core-workflow-b in Claude Code?
Run npx skills add jeremylongshore/tons-of-skills-marketplace --skill deepgram-core-workflow-b -a claude-code. The skill file is at ~/.claude/skills/deepgram-core-workflow-b/SKILL.md.
What does deepgram-core-workflow-b do?
It guides setup for live transcription through Deepgram WebSocket, including transcript events and speaker diarization. It also covers reconnect handling and an SSE route for browser clients.
What do I need to capture microphone audio?
Install @deepgram/sdk, configure DEEPGRAM_API_KEY, and install Sox. The microphone example captures audio with rec.
Source text
SKILL.md · MIT
--- name: deepgram-core-workflow-b description: 'Implement real-time streaming transcription with Deepgram WebSocket. Use when building live transcription, voice interfaces, real-time captioning, or voice AI applications. Trigger: "deepgram streaming", "real-time transcription", "live transcription", "websocket transcription", "voice streaming", "deepgram live". ' allowed-tools: Read, Write, Edit, Bash(npm:*), Bash(pip:*), Grep version: 1.13.0 license: MIT author: Jeremy Longshore <[email protected]> tags: - saas - deepgram - voice-ai - transcription - streaming - websocket compatibility: Designed for Claude Code --- # Deepgram Core Workflow B: Live Streaming Transcription ## Examples Open a development streaming session with a non-sensitive test utterance, verify connection/authentication, partial/final transcript state, timeout, and close behavior. Enforce the session's data/consent policy and log a correlation ID plus state transitions—not audio bytes, API keys, or full participant speech. ## Overview Real-time streaming transcription using Deepgram's WebSocket API. The SDK manages the WebSocket connection via `listen.live()`. Covers microphone capture, interim/final result handling, speaker diarization, UtteranceEnd detection, auto-reconnect, and building an SSE endpoint for browser clients. ## Prerequisites - `@deepgram/sdk` installed, `DEEPGRAM_API_KEY` configured - Audio source: microphone (via Sox/`rec`), file stream, or WebSocket audio from browser - For mic capture: `sox` installed (`apt install sox` / `brew install sox`) ## Instructions ### Step 1: Basic Live Transcription ```typescript import { createClient, LiveTranscriptionEvents } from '@deepgram/sdk'; const deepgram = createClient(process.env.DEEPGRAM_API_KEY!); const connection = deepgram.listen.live({ model: 'nova-3', language: 'en', smart_format: true, punctuate: true, interim_results: true, // Show in-progress results utterance_end_ms: 1000, // Silence threshold for utterance end vad_events: true, // Voice activity detection events encoding: 'linear16', // 16-bit PCM sample_rate: 16000, // 16 kHz channels: 1, // Mono }); // Connection lifecycle events connection.on(LiveTranscriptionEvents.Open, () => { console.log('WebSocket connected to Deepgram'); }); connection.on(LiveTranscriptionEvents.Close, () => { console.log('WebSocket closed'); }); connection.on(LiveTranscriptionEvents.Error, (err) => { console.error('Deepgram error:', err); }); // Transcript events connection.on(LiveTranscriptionEvents.Transcript, (data) => { const transcript = data.channel.alternatives[0]?.transcript; if (!transcript) return; if (data.is_final) { console.log(`[FINAL] ${transcript}`); } else { process.stdout.write(`\r[interim] ${transcript}`); } }); // UtteranceEnd — fires when speaker pauses connection.on(LiveTranscriptionEvents.UtteranceEnd, () => { console.log('\n--- utterance end ---'); }); ``` ### Step 2: Microphone Capture with Sox ```typescript import { spawn } from 'child_process'; function startMicrophone(connection: any) { // Sox captures from default mic: 16kHz, 16-bit signed LE, mono const mic = spawn('rec', [ '-q', // Quiet (no progress) '-r', '16000', // Sample rate '-e', 'signed', // Encoding '-b', '16', // Bit depth '-c', '1', // Mono '-t', 'raw', // Raw PCM output '-', // Output to stdout ]); mic.stdout.on('data', (chunk: Buffer) => { if (connection.getReadyState() === 1) { // WebSocket.OPEN connection.send(chunk); } }); mic.on('error', (err) => { console.error('Microphone error:', err.message); console.log('Install sox: apt install sox / brew install sox'); }); return mic; } // Usage const mic = startMicrophone(connection); // Graceful shutdown process.on('SIGINT', () => { mic.kill(); connection.finish(); // Sends CloseStream message, waits for final results setTimeout(() => process.exit(0), 2000); }); ``` ### Step 3: Live Diarization ```typescript const connection = deepgram.listen.live({ model: 'nova-3', smart_format: true, diarize: true, interim_results: false, // Only final for cleaner diarization utterance_end_ms: 1500, encoding: 'linear16', sample_rate: 16000, channels: 1, }); connection.on(LiveTranscriptionEvents.Transcript, (data) => { if (!data.is_final) return; const words = data.channel.alternatives[0]?.words ?? []; if (words.length === 0) return; // Group consecutive words by speaker let currentSpeaker = words[0].speaker; let segment = ''; for (const word of words) { if (word.speaker !== currentSpeaker) { console.log(`Speaker ${currentSpeaker}: ${segment.trim()}`); currentSpeaker = word.speaker; segment = ''; } segment += ` ${word.punctuated_word ?? word.word}`; } console.log(`Speaker ${currentSpeaker}: ${segment.trim()}`); }); ``` ### Step 4: Auto-Reconnect with Backoff ```typescript class ReconnectingLiveTranscription { private client: ReturnType<typeof createClient>; private connection: any = null; private reconnectAttempts = 0; private maxReconnectAttempts = 10; private baseDelay = 1000; constructor(apiKey: string, private options: Record<string, any>) { this.client = createClient(apiKey); } connect() { this.connection = this.client.listen.live(this.options); this.connection.on(LiveTranscriptionEvents.Open, () => { console.log('Connected'); this.reconnectAttempts = 0; // Reset on success }); this.connection.on(LiveTranscriptionEvents.Close, () => { this.scheduleReconnect(); }); this.connection.on(LiveTranscriptionEvents.Error, (err: Error) => { console.error('Connection error:', err.message); this.scheduleReconnect(); }); return this.connection; } private scheduleReconnect() { if (this.reconnectAttempts >= this.maxReconnectAttempts) { console.error('Max reconnection attempts reached'); return; } const delay = this.baseDelay * Math.pow(2, this.reconnectAttempts) + Math.random() * 1000; // Jitter this.reconnectAttempts++; console.log(`Reconnecting in ${Math.round(delay)}ms (attempt ${this.reconnectAttempts})`); setTimeout(() => this.connect(), delay); } send(chunk: Buffer) { if (this.connection?.getReadyState() === 1) { this.connection.send(chunk); } } close() { this.maxReconnectAttempts = 0; // Prevent reconnect this.connection?.finish(); } } ``` ### Step 5: SSE Endpoint for Browser Clients ```typescript import express from 'express'; import { createClient, LiveTranscriptionEvents } from '@deepgram/sdk'; const app = express(); app.get('/api/transcribe/stream', (req, res) => { res.setHeader('Content-Type', 'text/event-stream'); res.setHeader('Cache-Control', 'no-cache'); res.setHeader('Connection', 'keep-alive'); const deepgram = createClient(process.env.DEEPGRAM_API_KEY!); const connection = deepgram.listen.live({ model: 'nova-3', smart_format: true, interim_results: true, encoding: 'linear16', sample_rate: 16000, channels: 1, }); connection.on(LiveTranscriptionEvents.Transcript, (data) => { const transcript = data.channel.alternatives[0]?.transcript; if (transcript) { res.write(`data: ${JSON.stringify({ transcript, is_final: data.is_final, speech_final: data.speech_final, })}\n\n`); } }); // Client provides audio via a paired WebSocket (see browser setup) req.on('close', () => { connection.finish(); }); }); ``` ### Step 6: KeepAlive for Long Sessions ```typescript // Deepgram closes idle connections after ~10s of no audio. // Send KeepAlive messages during silence periods. connection.on(LiveTranscriptionEvents.Open, () => { const keepAliveInterval = setInterval(() => { if (connection.getReadyState() === 1) { connection.keepAlive(); } }, 8000); // Every 8 seconds connection.on(LiveTranscriptionEvents.Close, () => { clearInterval(keepAliveInterval); }); }); ``` ## Output - Live WebSocket transcription with interim/final results - Microphone capture pipeline (Sox -> Deepgram) - Speaker diarization in streaming mode - Auto-reconnect with exponential backoff and jitter - SSE endpoint for browser integration - KeepAlive handling for long sessions ## Error Handling | Issue | Cause | Solution | |-------|-------|----------| | WebSocket closes immediately | Invalid API key or bad encoding params | Check key, verify `encoding`/`sample_rate` match audio | | No transcripts received | Audio not being sent or wrong format | Verify `connection.send(chunk)` is called with raw PCM | | High latency | Network congestion | Use `interim_results: true` for perceived speed | | `rec` command not found | Sox not installed | `apt install sox` or `brew install sox` | | Connection drops after 10s | No audio + no KeepAlive | Send `connection.keepAlive()` every 8s | | Garbled output | Sample rate mismatch | Ensure audio sample rate matches `sample_rate` option | ## Resources - [Live Streaming Audio](https://developers.deepgram.com/docs/live-streaming-audio) - [Streaming API Reference](https://developers.deepgram.com/reference/speech-to-text/listen-streaming) - [End of Speech Detection](https://developers.deepgram.com/docs/understanding-end-of-speech-detection) - [SDK LiveTranscriptionEvents](https://github.com/deepgram/deepgram-js-sdk) ## Next Steps Proceed to `deepgram-data-handling` for transcript processing and storage patterns.
© Jeremy Longshore · MIT · jeremylongshore/tons-of-skills-marketplace