feat: add STT latency and RTF tracking metrics - #9894
Open
buyun14 wants to merge 1 commit into
Open
Conversation
Track speech-to-text call timing and real-time factor (RTF) in both the pipeline preprocess stage and the web chat live chat service, and surface the metrics in the Live Mode dashboard. - preprocess_stage: probe audio duration via get_media_duration, measure STT duration, log provider/model/duration/RTF, and record into event.trace for observability. - live_chat_service: emit stt_total_time / stt_audio_duration / stt_rtf over the live chat websocket. - LiveMode.vue: render the new STT latency metrics.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
Adds STT observability (latency, audio duration, and real-time factor/RTF) to AstrBot’s pipeline tracing and Live Mode dashboard so users can see STT performance alongside existing LLM/TTS metrics.
Changes:
- Record per-call STT timing, audio duration, RTF, provider/model, and status into
event.trace. - Emit STT timing/RTF metrics over the Live Mode websocket for web chat.
- Render the new STT metrics in the Live Mode UI.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| dashboard/src/components/chat/LiveMode.vue | Displays new STT metrics (total time, audio duration, RTF) in Live Mode. |
| astrbot/dashboard/services/live_chat_service.py | Measures STT timing and emits stt_total_time, stt_audio_duration, and stt_rtf metrics. |
| astrbot/core/pipeline/preprocess_stage/stage.py | Adds STT duration + audio duration probing and records detailed STT stats into event.trace. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+47
to
+51
| <span v-if="metrics.stt_total_time">STT Total Latency: {{ (metrics.stt_total_time * | ||
| 1000).toFixed(0) }}ms</span> | ||
| <span v-if="metrics.stt_audio_duration">STT Audio Duration: {{ (metrics.stt_audio_duration * | ||
| 1000).toFixed(0) }}ms</span> | ||
| <span v-if="metrics.stt_rtf">STT RTF: {{ metrics.stt_rtf.toFixed(2) }}</span> |
Comment on lines
+888
to
+890
| stt_start = time.time() | ||
| user_text = await stt_provider.get_text(audio_path) | ||
| stt_duration = time.time() - stt_start |
Comment on lines
+200
to
204
| stt_start = time.time() | ||
| status = "failed" | ||
| retry = 5 | ||
| for i in range(retry): | ||
| try: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation / 动机
Add timing and real-time factor (RTF) metrics to speech-to-text (STT) calls so users can observe STT latency directly in the Live Mode dashboard. Currently the dashboard shows TTS/chat latency but no STT latency, and there is no per-call timing/RTF observability in the pipeline.
Modifications / 改动点
astrbot/core/pipeline/preprocess_stage/stage.py: in_stt_record, probe audio duration viaget_media_duration, measure the STT call duration, log provider/model/duration/RTF, and record intoevent.tracefor observability.astrbot/dashboard/services/live_chat_service.py: measure web chat STT duration and emitstt_total_time/stt_audio_duration/stt_rtfmetrics over the live chat websocket.dashboard/src/components/chat/LiveMode.vue: render the new STT latency metrics (total latency, audio duration, RTF).This is NOT a breaking change. / 这不是一个破坏性变更。
Screenshots or Test Results / 运行截图或测试结果
Ruff check and format check pass on the modified files. (Dashboard screenshots to be provided once verified.)
Checklist / 检查清单
😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
/ 如果 PR 中有新加入的功能，已经通过 Issue / 邮件等方式和作者讨论过。
👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
/ 我的更改经过了良好的测试，并已在上方提供了"验证步骤"和"运行截图"。
🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in
requirements.txtandpyproject.toml./ 我确保没有引入新依赖库，或者引入了新依赖库的同时将其添加到
requirements.txt和pyproject.toml文件相应位置。😮 My changes do not introduce malicious code.
/ 我的更改没有引入恶意代码。
Summary by Sourcery
Add STT timing and real-time factor observability across the pipeline and Live Mode dashboard.
New Features:
Enhancements: