Skip to content

fix(speech): language ID minSpeechMs counts voiced speech, not trailing silence - #300

Merged
akirilyuk merged 2 commits into
mainfrom
fix/lid-min-speech-voiced
Oct 4, 2026
Merged

akirilyuk merged 2 commits into
mainfrom
fix/lid-min-speech-voiced

Conversation

@akirilyuk

Copy link
Copy Markdown
Owner

Summary

Symptom: a ~1 s utterance followed by silence passed a 1500 ms languageId.minSpeechMs threshold, so language ID ran on audio that was mostly silence.

Cause: both threshold checks (the early/continuous check and the identify gate) compared minSpeechMs with the length of the LID PCM buffer. That buffer runs from utterance start to close and includes the endpoint tail and STT gate hold after VAD speech end, so trailing silence counted as speech.

Change:

  • The agent tracks lid_voiced_ms: speech from the first to the last voiced VAD frame of the utterance. Gaps between words count (they are folded in once speech resumes). Silence after the last voiced frame never counts.
  • Both threshold checks and the too_short skip use lid_voiced_ms. The classifier still receives the whole buffered audio. Agents without a VAD treat every frame as voiced.
  • force semantics, the continuous re-identify cadence and the defaults are unchanged.
  • The roundtrip-language-id example threshold is retuned to speech units (2000 ms, a gate only under the default end-of-utterance timing).
  • Docs (SDK LanguageIdConfig.minSpeechMs, VOICE-API.md) and CHANGELOG updated.

Tests (crates/speech/tests/language_id_nonblocking_test.rs):

  • early_timing_short_speech_with_long_trailing_silence_is_too_short
  • end_of_utterance_timing_short_speech_with_trailing_silence_is_too_short
  • early_timing_identifies_once_voiced_reaches_min
  • pauses_between_words_count_trailing_silence_does_not
  • early_timing_does_not_identify_during_trailing_silence

Verified locally and via CI; no extra manual steps.

🤖 Generated with Claude Code

akirilyuk and others added 2 commits October 4, 2026 20:58
…ng silence

The LID gate compared minSpeechMs with the buffered audio length, which includes the
endpoint tail, so a short utterance followed by silence passed the threshold. Count VAD
speech from the first to the last voiced frame (gaps between words count, trailing silence
does not). The classifier still receives the whole buffer. The language-id example
threshold is retuned to speech units.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@akirilyuk
akirilyuk merged commit b478781 into main Oct 4, 2026
8 checks passed
@akirilyuk
akirilyuk deleted the fix/lid-min-speech-voiced branch October 4, 2026 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant