Skip to content

feat(local-session): Redis state store and audio segmentation engine - #8

Open
Rahulkaushik01 wants to merge 1 commit into
devfrom
feat/local-session-audio-engine
Open

feat(local-session): Redis state store and audio segmentation engine#8
Rahulkaushik01 wants to merge 1 commit into
devfrom
feat/local-session-audio-engine

Conversation

@Rahulkaushik01

Copy link
Copy Markdown
Collaborator

Stacked PR 1 of 4 — base dev. Order: 1 → 2 → 3 → 4.
Chain: audio-engine → ingest-and-lifecycle → api → owner-and-reaper

Summary

Foundation for local recordings: the desktop records the user's own mic + system
audio and uploads it, instead of dispatching a bot into a meeting. This PR adds only
the two self-contained engine modules — no API and no task yet — so it stays small
and reviewable.

Problem

A meeting bot is a long-lived process that holds in-progress audio in RAM. A local
recording has no such process — it is a series of stateless HTTP uploads handled by
short-lived workers, so the "memory" a bot gets for free must be rebuilt explicitly.
Second, a spoken sentence rarely lines up with an upload boundary, so processing each
chunk independently chops words in half.

How it works

  • local_session_store — Redis-backed queue, tail and lock, per session
    and per source. The tail is the core idea: audio still mid-sentence when a batch ends
    is carried over and glued to the front of the next batch, so the VAD only ever cuts on
    real silence. The lock guarantees exactly one drain owns the tail. The queue carries a
    TTL and a clear helper so an abandoned session's state cannot outlive it.
  • local_audio_processing — 10 ms VAD framing and utterance creation. 10 ms because
    webrtcvad accepts only 10/20/30 ms frames, and because the shared manager's
    "too large for VAD" guard compares a byte length against a sample count, so anything
    over ~15 ms silently bypasses the VAD and is reported as speech.
  • Corrected loudness meter. The shared calculate_normalized_rms() squares an int16
    array in int16, which wraps for |sample| > 181 — i.e. for most real speech. Measured
    on a real 94-second voice recording: 53% of samples overflow, and the meter under-reads
    so badly that only 21% of frames are judged speech versus 54% with correct maths.
    Widening to float64 before squaring fixes it.

Scoping that bug honestly

It does not lose audio. Once an utterance is open the buffer keeps every chunk regardless
of the silence verdict, so end-to-end capture measured 93% (buggy) vs 96% (fixed). The real
cost is over-segmentation — 14 fragments instead of 9 on that sample — and because
ElevenLabs transcribes each clip independently, a clip cut mid-sentence transcribes worse.
The fix is applied only in our subclass; the shared meeting-bot path is left byte-identical.

Changes made

File Lines Purpose
bots/local_session_store.py +123 Redis queue / tail / lock, TTL, clear helper
bots/local_audio_processing.py +170 VAD framing, utterance creation, corrected RMS meter

Total: +293 / −0 (293 changed lines), 1 commit.

Testing performed

  • ruff checkAll checks passed
  • ruff format --checkclean (CI runs this repo-wide)
  • python manage.py checkno issues (module imports cleanly on this branch alone)
  • RMS fix validated against a real 94-second voice recording decoded to 16 kHz PCM,
    comparing both meters frame-by-frame through the actual webrtcvad
  • Exercised end-to-end by PRs 2–3 (33/33 pipeline checks)

Coding-guidelines compliance

  • File size (§2): both new files ≤ 250 (the preferred level) — 123 and 170. No file
    in this PR exceeds any limit.
  • Functions (§6): all new functions ≤ 50 lines, single responsibility, early returns.
  • Constants (§5): no magic numbers — VAD_FRAME_MS, BYTES_PER_SAMPLE,
    INT16_FULL_SCALE, TAIL_TTL_SECONDS, QUEUE_TTL_SECONDS, LOCK_TTL_SECONDS.
  • Naming (§4): snake_case functions, descriptive names, boolean is_final.
  • Docs (§7): comments explain only non-obvious decisions (why 10 ms, why float64, why a tail).
  • No dead code / debug logs / unused imports (§15).
  • Security (§8): no secrets; nothing added to env handling in this PR.

Risks / notes

  • Nothing calls this code yet — intentional, so the PR stays small. PR 2 drives it.
  • The shared calculate_normalized_rms() is deliberately not modified: changing it would
    alter segmentation for every meeting bot and deserves its own PR with its own testing.

Foundation for desktop local recordings, where the desktop uploads its own mic
and system audio as chunks instead of a bot joining a meeting.

* local_session_store: Redis-backed queue, tail and lock. A local session has
  no long-lived process, so the audio "memory" a meeting bot keeps in RAM lives
  here -- the tail holds a sentence that straddles two uploads so the VAD only
  ever cuts on real silence. The queue carries a TTL and a clear helper so an
  abandoned session's state cannot outlive it.
* local_audio_processing: 10ms VAD framing and utterance creation, plus a
  loudness meter that widens to float64 before squaring. The shared meter
  squares int16 in int16, which wraps for |sample| > 181 and under-reads
  loudness, cutting utterances into more fragments than there are real pauses.

Both are self-contained; the ingest task in the next branch drives them.

Co-Authored-By: Claude <noreply@anthropic.com>
@Rahulkaushik01
Rahulkaushik01 requested a review from hd1801 July 20, 2026 06:58
@Rahulkaushik01 Rahulkaushik01 self-assigned this Jul 20, 2026
@Rahulkaushik01 Rahulkaushik01 added the enhancement New feature or request label Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant