Skip to content

Harden the Voice Foundation pipeline#15

Merged
logannye merged 2 commits into
mainfrom
codex/voice-foundation-hardening
Jul 24, 2026
Merged

Harden the Voice Foundation pipeline#15
logannye merged 2 commits into
mainfrom
codex/voice-foundation-hardening

Conversation

@logannye

@logannye logannye commented Jul 20, 2026

Copy link
Copy Markdown
Owner

Summary

  • replaces the prototype 10 Hz speech extractor and five prototype.speech.* values with a continuous AudioWorklet to voice-worker path and task-specific prototype.voice.* measurements
  • adds a microphone-only Voice Foundation protocol with quiet/speech preflight and five completion-gated task phases
  • centralizes audio quality, per-metric abstention, uncertainty, provenance, accepted-interval clipping, and exact trajectory compatibility
  • adds a disabled-by-default, loopback-only Python 3.11 FastAPI WavLM Large representation service with pinned model provenance and deterministic CI adapter
  • updates encounter/evidence contracts, protocol fixtures, privacy assertions, architecture, safety, validation, and operator documentation
  • keeps Facial Foundation on the existing MediaPipe pipeline and preserves its six facial measurements; face-voice fusion remains deferred

Voice architecture

  • mono audio requested at ideal 48 kHz with echo cancellation, noise suppression, and AGC disabled
  • reusable 20 ms PCM blocks transferred through a MessageChannel
  • bounded 30-second worker ring; 40 ms windows on a 10 ms hop; discontinuities are never synthesized as silence
  • 50-700 Hz F0 estimation with estimator agreement and octave checks
  • cepstral peak prominence, HNR, perturbation, LPC formants, timing, and estimated syllabic/DDK features with independent quality gates
  • two sustained /a/ trials, standardized reading, rapid /pa-ta-ka/, and spontaneous response; elapsed time never advances a task

Privacy and safety

  • no audio recording, transcript, voiceprint, native audio arrays, or WavLM embeddings enter observations, events, evidence, diagnostics payloads, reports, or trajectory artifacts
  • PCM and representation arrays are in-memory and cleared or released at lifecycle boundaries
  • WavLM accepts only finite 16 kHz mono PCM over loopback, writes no audio, logs no request body, and returns only transient layer summaries plus provenance
  • all measurements remain descriptive, deterministic engineering features with clinicalValidation: "none"

Validation

  • pnpm test - 198 unit tests, structure/privacy validation, typecheck, production build
  • pnpm test:browser - 25 Playwright scenarios
  • pnpm test:python - 4 sidecar schema/service tests

Real WavLM checkpoint extraction remains a documented manual smoke because model weights are intentionally absent from required CI.

@logannye
logannye merged commit 87273c9 into main Jul 24, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant