Skip to content

fix(vad): throttle speech activity events while speaking - #5805

Merged
aconchillo merged 2 commits into
pipecat-ai:mainfrom
perfloop-oss:perfloop-pr-open-c51nvt9eh7
Sep 16, 2026
Merged

aconchillo merged 2 commits into
pipecat-ai:mainfrom
perfloop-oss:perfloop-pr-open-c51nvt9eh7

Conversation

@perfloop-agent

Copy link
Copy Markdown
Contributor

Summary

VADController now routes speaking-state activity through _maybe_speech_activity(), so the configured speech_activity_period is honored while audio continues to arrive. This removes redundant UserSpeakingFrame broadcasts through the downstream pipeline while preserving raw audio forwarding and speech start/stop transitions.

The focused regression tests cover the controller's default activity period and the VADProcessor activity, start, stop, and forwarding paths. The changelog records the user-visible fix.

Testing

  • uv run --locked --group dev pytest tests/test_vad_controller.py tests/test_vad_processor.py
  • uv run --locked --group dev ruff check src/pipecat/audio/vad/vad_controller.py tests/test_vad_controller.py tests/test_vad_processor.py
  • uv run --locked --group dev ruff format --check src/pipecat/audio/vad/vad_controller.py tests/test_vad_controller.py tests/test_vad_processor.py
  • uv run --locked --group dev towncrier build --draft --version Unreleased
  • CI Unit Tests configuration: 5,653 passed and 127 skipped.
  • Repository-wide Pyright: 0 errors, with two pre-existing pyaudio warnings.

Workload: PipelineWorker VADProcessor with SileroVADAnalyzer replaying the repository's bundled 16 kHz speech trace five times as 20 ms frames with the default 200 ms speech-activity period

Metric Before After Change
VADProcessor process CPU nanoseconds for five speech-trace replays 820000000 690000000 15.9% lower
VADProcessor elapsed nanoseconds for five speech-trace replays 842079916 730178996 13.3% lower
Checks: 6 passed.

Generated by Perfloop. Human sponsor: Tomás Senart. Measurements and checks.

@tsenart
tsenart force-pushed the perfloop-pr-open-c51nvt9eh7 branch from cbc1538 to 3c994a2 Compare September 16, 2026 18:43
@tsenart

tsenart commented Sep 16, 2026 •

Copy link
Copy Markdown

Hi, I'm Tomás, I build Perfloop. I approved this contribution and will handle review feedback here.

We've had performance PRs merged in ClickHouse, Prometheus, Ruff/ty, parquet-go, RoaringBitmap, and more.

Before approving, we verified the change against main: the controller's docstring documents on_speech_activity as periodic with a 0.2 s default, but _maybe_speech_activity() has never been called since VADController landed in January, so every audio frame fires the event and the aggregator broadcasts a UserSpeakingFrame per frame. The consumers of that event, the pipeline idle timeout and the wake-phrase turn strategy, work on second-scale timers, and speech start and stop remain separate events. The tests ran exactly as your CI "Unit Tests" job runs them (5,653 passed, 127 skipped) plus repository-wide Pyright; the workflows here only need an approve-and-run.

@aconchillo

Copy link
Copy Markdown
Contributor

Good catch. LGTM!

@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
src/pipecat/audio/vad/vad_controller.py 97.56% <100.00%> (+4.87%) ⬆️

... and 13 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aconchillo
aconchillo merged commit 365552d into pipecat-ai:main Sep 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants