feat(local-session): Redis state store and audio segmentation engine - #8
Open
Rahulkaushik01 wants to merge 1 commit into
Open
feat(local-session): Redis state store and audio segmentation engine#8Rahulkaushik01 wants to merge 1 commit into
Rahulkaushik01 wants to merge 1 commit into
Conversation
Foundation for desktop local recordings, where the desktop uploads its own mic and system audio as chunks instead of a bot joining a meeting. * local_session_store: Redis-backed queue, tail and lock. A local session has no long-lived process, so the audio "memory" a meeting bot keeps in RAM lives here -- the tail holds a sentence that straddles two uploads so the VAD only ever cuts on real silence. The queue carries a TTL and a clear helper so an abandoned session's state cannot outlive it. * local_audio_processing: 10ms VAD framing and utterance creation, plus a loudness meter that widens to float64 before squaring. The shared meter squares int16 in int16, which wraps for |sample| > 181 and under-reads loudness, cutting utterances into more fragments than there are real pauses. Both are self-contained; the ingest task in the next branch drives them. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Foundation for local recordings: the desktop records the user's own mic + system
audio and uploads it, instead of dispatching a bot into a meeting. This PR adds only
the two self-contained engine modules — no API and no task yet — so it stays small
and reviewable.
Problem
A meeting bot is a long-lived process that holds in-progress audio in RAM. A local
recording has no such process — it is a series of stateless HTTP uploads handled by
short-lived workers, so the "memory" a bot gets for free must be rebuilt explicitly.
Second, a spoken sentence rarely lines up with an upload boundary, so processing each
chunk independently chops words in half.
How it works
local_session_store— Redis-backed queue, tail and lock, per sessionand per source. The tail is the core idea: audio still mid-sentence when a batch ends
is carried over and glued to the front of the next batch, so the VAD only ever cuts on
real silence. The lock guarantees exactly one drain owns the tail. The queue carries a
TTL and a clear helper so an abandoned session's state cannot outlive it.
local_audio_processing— 10 ms VAD framing and utterance creation. 10 ms becausewebrtcvad accepts only 10/20/30 ms frames, and because the shared manager's
"too large for VAD" guard compares a byte length against a sample count, so anything
over ~15 ms silently bypasses the VAD and is reported as speech.
calculate_normalized_rms()squares an int16array in int16, which wraps for
|sample| > 181— i.e. for most real speech. Measuredon a real 94-second voice recording: 53% of samples overflow, and the meter under-reads
so badly that only 21% of frames are judged speech versus 54% with correct maths.
Widening to float64 before squaring fixes it.
Scoping that bug honestly
It does not lose audio. Once an utterance is open the buffer keeps every chunk regardless
of the silence verdict, so end-to-end capture measured 93% (buggy) vs 96% (fixed). The real
cost is over-segmentation — 14 fragments instead of 9 on that sample — and because
ElevenLabs transcribes each clip independently, a clip cut mid-sentence transcribes worse.
The fix is applied only in our subclass; the shared meeting-bot path is left byte-identical.
Changes made
bots/local_session_store.pybots/local_audio_processing.pyTotal: +293 / −0 (293 changed lines), 1 commit.
Testing performed
ruff check→ All checks passedruff format --check→ clean (CI runs this repo-wide)python manage.py check→ no issues (module imports cleanly on this branch alone)comparing both meters frame-by-frame through the actual webrtcvad
Coding-guidelines compliance
in this PR exceeds any limit.
VAD_FRAME_MS,BYTES_PER_SAMPLE,INT16_FULL_SCALE,TAIL_TTL_SECONDS,QUEUE_TTL_SECONDS,LOCK_TTL_SECONDS.is_final.Risks / notes
calculate_normalized_rms()is deliberately not modified: changing it wouldalter segmentation for every meeting bot and deserves its own PR with its own testing.