Resound is local-first. Set RESOUND_TRANSCRIBER. No vendor is required and
nothing is hardcoded to a single cloud provider.
| Provider | Status | Config |
|---|---|---|
mock |
✅ offline, deterministic (tests/dev) | none |
local-whisper |
✅ recommended — local, audio stays on the machine | RESOUND_WHISPER_COMMAND, RESOUND_WHISPER_MODEL, RESOUND_WHISPER_FORMAT |
openai-compatible |
✅ any OpenAI-compatible endpoint | RESOUND_OPENAI_BASE_URL, RESOUND_OPENAI_API_KEY |
openai |
✅ shorthand: compatible client → api.openai.com | OPENAI_API_KEY |
deepgram |
🧩 scaffolded (interface only) | DEEPGRAM_API_KEY |
assemblyai |
🧩 scaffolded (interface only) | ASSEMBLYAI_API_KEY |
All providers implement the Transcriber interface in packages/transcribers.
Scaffolded providers resolve by name but throw a clear "not implemented yet"
error if invoked. Adding a real provider is one new class + a case in
getTranscriber().
Shells out to a locally installed Whisper binary so audio never leaves your
machine. Defaults to whisper.cpp's whisper-cli:
# macOS example
brew install whisper-cpp
# download a model, e.g. base.en
curl -L -o models/ggml-base.en.bin \
https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.en.bin
# .env
RESOUND_TRANSCRIBER=local-whisper
RESOUND_WHISPER_COMMAND=whisper-cli
RESOUND_WHISPER_FORMAT=whisper.cpp
RESOUND_WHISPER_MODEL=./models/ggml-base.en.bin
# Use more CPU threads when no explicit RESOUND_WHISPER_ARGS are set.
RESOUND_WHISPER_THREADS=8To use the Python openai-whisper / faster-whisper CLIs instead, set
RESOUND_WHISPER_FORMAT=openai-whisper and RESOUND_WHISPER_COMMAND=whisper.
Anything else: point RESOUND_WHISPER_COMMAND at your binary and add flags via
RESOUND_WHISPER_ARGS.
For long Discord meetings, Resound transcribes speaker tracks sequentially by
default so multiple Whisper processes do not compete for the same CPU. The bot
posts elapsed time, completed tracks, analyzed audio, and an estimated remaining
time while /resound stop is finalizing. Set RESOUND_WHISPER_ARGS explicitly
to override the default thread setting.
Use a multilingual model for international meetings. The .en models are
English-only; they can handle English accents, but they are not the right choice
when participants may speak other languages. The practical progression is:
| Model | Use case | Tradeoff |
|---|---|---|
small.en |
English-only, fastest local option | More recognition errors than larger models |
medium |
Recommended first upgrade for global meetings | About 1.5 GB; slower and more CPU/RAM intensive |
large-v3-turbo |
Stronger accuracy with a large-model architecture | About 1.6 GB; benchmark locally before production use |
large-v3 |
Highest accuracy target in the official Whisper family | About 3.1 GB and usually impractical on CPU-only laptops |
The official whisper.cpp model registry includes medium, large-v3, and
large-v3-turbo conversions. Model size improves recognition, but cannot
guarantee perfect transcripts: microphone quality, crosstalk, packet loss,
language switching, and overlapping speakers still matter. On this Intel Mac,
Homebrew whisper-cli uses CPU/BLAS rather than Metal, so medium is the
recommended production starting point; benchmark large-v3-turbo against a
representative multilingual recording before moving up again.
For an optional cloud/remote path that is not locked to OpenAI:
RESOUND_TRANSCRIBER=openai-compatible
RESOUND_OPENAI_BASE_URL=https://api.groq.com/openai/v1 # or LM Studio, vLLM, OpenRouter, a local whisper server…
RESOUND_OPENAI_API_KEY=... # many local servers accept any/empty token
RESOUND_TRANSCRIBER_MODEL=whisper-1Speaker labels: REST transcription APIs do not diarize, so cloud/
local-whisperon a single mixed file labels everything as the first participant. Real per-speaker labels come from per-speaker audio (one file/stream per user) — which is exactly what the supported Discord receive adapter produces.
ReSound's supported production receiver is a pinned Pycord sidecar with DAVE receive support. It has passed dependency probes, live DAVE connection tests, real-voice capture, per-speaker WAV generation, and end-to-end transcription.
- DAVE is now mandatory. Discord's MLS-based end-to-end encryption (DAVE) was enforced across all voice channels (enforcement March 2, 2026; rollout reported complete May 19, 2026). Voice is E2EE by default.
@discordjs/voicereceive is still unreliable under DAVE. With DAVE on, bots that try to receive audio hit reconnect loops, nospeakingevents, and decryption failures such asDecryptionFailed(UnencryptedWhenPassthroughDisabled)andCannot read properties of undefined (reading 'decrypt')inVoiceReceiver.onUdpMessage. Sending works; receiving does not.@snazzah/daveyis the DAVE protocol library bundled with@discordjs/voice, but the receive decrypt path is not yet wired up.- Pycord's DAVE receive fix is pinned to an audited upstream commit because it
is not yet in the released 2.8.0 wheel. ReSound uses a Python sidecar around the
patched Pycord voice receiver by default in
RESOUND_BOT_MODE=discord/discord-native, with a real runtime preflight for the pinned Pycord build,davey,PyNaCl, andlibopus. - Legacy "subscribe to a user's Opus stream and decode it" snippets predate DAVE and will not work in DAVE-protected calls.
Sources: discord.js #11419, discord.js #10735, DAVE whitepaper, Discord blog: Bringing DAVE to all platforms.
Tradeoffs of the available approaches
| Approach | Notes |
|---|---|
@discordjs/voice receive |
Mature API surface for receive, but DAVE support for receiving is the gating question — verify the installed version's status before relying on it. Needs prism-media + an Opus decoder (@discordjs/opus or opusscript) and libsodium/sodium-native. |
| Bring-your-own DAVE stack | Implement/track an MLS + DAVE layer directly. Most control, most work; only justified if library support stalls. |
| Account/self-bot capture | ❌ Against Discord ToS. Not supported by Resound. |
| Out-of-band capture | Record the host's system/app audio outside Discord and feed the file to Resound's transcriber. Sidesteps DAVE entirely; loses per-user diarization. |
-
Alternative path: local capture + transcription. Capture the call from the operator machine (system audio, OBS, QuickTime, Audio Hijack, or the built-in
resound record/RESOUND_BOT_MODE=local-captureflow) and run it throughlocal-whisperor an OpenAI-compatible endpoint. This works now and needs no bot-side voice receive. See usage.md. -
The production live path uses Pycord.
PycordDiscordRecorder(packages/audio/src/pycord-discord-recorder.ts) launches a Python sidecar (packages/audio/python/discord_native_sidecar.py) that logs into Discord, joins the requested voice channel, records per-user PCM through Pycord's DAVE-aware receive path, writes aligned speaker WAVs plusaudio/raw/mixed.wav, and returns chunk metadata to the normal Resound pipeline.DiscordRecorderremains an unsupported compatibility backend for maintainers comparing the older@discordjs/voicereceive stack.Production deployments must use
RESOUND_DISCORD_RECEIVER_BACKEND=pycord(the default).autoanddiscordjsexist for maintainers and are not release-supported.In
autobot mode, Resound still falls back only to preflighted local-capture; if neither real recorder is ready, start fails before recording begins.
To enable the preferred live mode on a fresh machine:
python3 -m pip install -U -r packages/audio/python/requirements.txt
RESOUND_BOT_MODE=discord \
RESOUND_DISCORD_RECEIVER_BACKEND=pycord \
pnpm bot:startOn this machine, python3 packages/audio/python/discord_native_sidecar.py --probe
returns a successful readiness payload only when the installed Pycord build has
both DAVE support and the DAVE receive fix (dave_receive=true).
Each self-hosted installation must complete the acceptance check in self-hosting.md before recording real meetings. This catches server-specific permissions, receiver, storage, and transcriber configuration.
RESOUND_BOT_MODE=local-capture does not use Discord voice receive. The slash
commands run on the same operator machine that captures local audio devices
through ffmpeg/avfoundation. Use this when macOS mixed system capture is
preferred over server-side per-speaker capture:
RESOUND_BOT_MODE=local-capture
RESOUND_AUDIO_SYSTEM_DEVICE=1
RESOUND_AUDIO_MIC_DEVICE=2
pnpm bot:startThe operator still needs macOS audio routing that sends Discord/system output into a capture device such as BlackHole, plus a microphone device for their own voice.
Remote transcription uploads speaker tracks sequentially, as local Whisper does. This keeps at most one file upload resident per transcription call and stops scheduling uploads after a provider failure. Providers are not retried implicitly. A declared missing speaker track is an error rather than an omitted participant; restore the audio before retrying. Single-file upload size limits remain provider-defined.
The legacy Discord receiver flushes at most 30 seconds of decoded PCM per chunk (5,760,000 bytes at stereo 48 kHz/s16le), even without silence. Pause flushes prior samples and discards incoming paused samples; stop detaches reception and flushes immediately. Chunk filenames and metadata remain compatible, but continuous utterances may span multiple files. This does not change its unsupported DAVE status or make it a production-supported receiver.
The Pycord sidecar converts PCM to WAV in 1 MiB reads instead of loading each
complete recording into memory. python3 -m unittest discover -s packages/audio/python -p 'test_*.py' runs offline sample-equivalence and bounded-memory checks without
Discord dependencies. python3 scripts/benchmarks/wav-memory.py compares the
original conversion with the current implementation and verifies identical bytes.
Local Whisper runs have a 30-minute deadline per track. Set
RESOUND_WHISPER_TIMEOUT_MS (or LocalWhisperOptions.timeoutMs) for larger models
or long recordings. Binary preflight is limited to 30 seconds. Remote uploads
use a file-backed Blob, one track at a time, with a 10-minute deadline including
response consumption; configure RESOUND_HTTP_TIMEOUT_MS or the transcriber's
timeoutMs option. Provider resolution uses RESOUND_HTTP_TIMEOUT_MS from the
explicitly supplied env object, or the process environment when env is omitted.
Deadline values must be positive integer milliseconds.
No automatic retry occurs after a timeout: the remote server may have processed
the request already.
Strata/TotalRecall commands default to 60 seconds; webhook requests default to
30 seconds. RESOUND_SINK_TIMEOUT_MS or the sink's timeoutMs option overrides
these limits. Unused webhook response bodies are cancelled. Child commands run
without a shell; on POSIX their process group is terminated on deadline.
Subprocess output streams to private resound-command-* directories under the
OS temporary directory. Error previews contain at most 8 KiB per stream and link
to retained full logs when larger. Successful small command logs are removed.
Large provider error responses use private resound-http-* artifacts. These logs
can contain transcript text and provider details; retain/share them as private
meeting evidence and remove them when no longer needed. Failed command logs and
large diagnostic artifacts are retained until operator/OS temporary-file cleanup;
there is no automatic retention policy. ffmpeg recording diagnostics are written
beside the WAV as .stderr.log. Requesting stop has a 15-second deadline
(stopTimeoutMs for the recording APIs); forced termination is an error.
The pinned Pycord receiver delivers each SSRC's independent RTP timestamp without
cross-speaker normalization. Resound anchors each speaker's first decoded packet
to the shared receiver monotonic clock and uses unsigned 32-bit RTP deltas within
that stream. This gives receive-time alignment, not sender-clock synchronization;
network latency and decode jitter still affect the first-packet anchor. Silence,
wraparound and packet overlap are handled explicitly. SSRC changes or timestamp
jumps inconsistent with elapsed receive time are realigned with a warning.
Mixed audio includes each speaker's session offset; speaker WAVs retain relative
samples and their existing startSeconds metadata.
Finalization runs off the Discord event loop. Increasing processed-byte receipts
refresh RESOUND_SIDECAR_STOP_TIMEOUT_MS (default 15 seconds), now an idle
finalization deadline. Work that stalls still fails. Progress receipts are at
most once per second and contain no audio or transcript content.
The sidecar checks free disk space once per second, reserving estimated space for speaker WAVs, mixed PCM/WAV and a 64 MiB margin. Disk exhaustion or the WAV 4 GiB format limit fails explicitly and retains raw PCM for recovery. This is a best-effort reserve, not a filesystem quota: concurrent applications can consume the free space after a check. Production deployments should use volume quotas and monitor retention. No old recordings are automatically deleted.
The sidecar validates startup and stop deadlines before launching capture. A broken stdin command pipe rejects capture rather than crashing the bot with an unhandled stream error. Sidecar stderr writes directly to the private adjacent log file, keeping per-chunk disk writes off the bot event loop; unexpected exits return a short tail and the full diagnostic path.
A successful HTTP status or local process exit must also produce a recognized
transcription response. OpenAI-compatible and Python Whisper JSON must contain
segments (an array) or text (a string); whisper.cpp JSON must contain a
transcription array. Explicit empty arrays and text-only responses still work.
Segment text must be a string. Supplied timestamps must be finite, nonnegative
numbers; omitted optional timestamps retain their zero defaults. Remote
avg_logprob, when present and non-null, must be finite numeric data.
Malformed output fails with a field-specific diagnostic instead of completing
with an empty or coerced transcript. Audio remains available through the existing
failure/recovery flow. Resound does not automatically retry provider calls.
Speaker-track offsets and durations must be finite, nonnegative numbers. Local Whisper and OpenAI-compatible transcription validate all selected speaker tracks before launching a command or uploading any audio. Invalid timing fails explicitly instead of producing a misleading timeline; timestamp addition overflow also fails. Fractional offsets and zero duration remain supported.