fix(playback): recover audio clock drift after render stalls - #131
Conversation
两个叠加的缺陷让全屏切换产生持续到 seek/暂停的音画失步(实测 733ms 停顿 + 时钟永久超前音频 0.7s): 1. CAMetalLayer.nextDrawable 默认可阻塞最长 1s:Space 动画期间 WindowServer 扣住全部 drawable,渲染线程停摆 733ms,而 pump_audio 同在该 tick 上,音频环(~290ms)被抽干 → underflow 451ms 静音。 attach 时设 allowsNextDrawableTimeout=true,拿不到 drawable 立即 返回 nil,按既有 RendererBackpressure 路径跳过本帧、下帧重试; 新增 present_backpressure_skips 计数 + power-of-two 节流诊断。 2. 时钟重锚的距离型 stale 判定(250ms)在积压回填后永久拒绝校正: 恢复后环 front 落后时钟一个积压深度(0.3-1s),35769 次重锚尝试 全部 reason=stale 被拒,失步持续整个会话(实测 trace 803s)。 改为活性判定:read/underflow 计数推进 → 设备在消费,front 可信, 一律 discipline_to(大偏差走既有 snap);计数冻结(设备真停走) 才维持跳过。 验证:cargo test -p erika 504 全绿(含 backfill 重锚/冻结环两用例)。
can_accept_audio_frame 此前用 trait 默认实现(恒 true),macOS 路径的 环深实际只由音频生产提前量(120ms)间接约束,任何渲染侧停顿超过 ~300ms 就把环抽干成静音。CoreAudioOutput 现在按 1.2s 高水位真门控, worker 提前量同步加深到 1.2s(依赖上一个 commit 的活性重锚:环深 不再让时钟重锚被拒)。1.2s 覆盖实测最坏 present 停顿(全屏 Space 切换 ~700ms;allowsNextDrawableTimeout 的兜底上限 ~1s)并留余量。 验证:cargo test -p erika 504 全绿。
Shinokawa
left a comment
Shinokawa
left a comment
There was a problem hiding this comment.
这份 PR 指出了真实缺陷：有效音频一旦落后播放时钟超过 250ms，现有距离过滤就会持续拒绝校时。我们在 Windows 使用通用核心独立复现了这个机制，真实解码、播放 worker 与 Presenter 链路也能通过注入渲染阻塞复现持续失步；不依赖 macOS 原生全屏触发器。
PR 的重新锚定方向正确，但目前有两个阻止原样合入的问题。
-
1200ms 缓冲与 #82 的倍速过渡契约冲突。 #82 将输出预算、旧速桥和 worker prefill 对齐到 250ms，避免旧速 PCM 堆积。此 PR 将 CoreAudio 目标及共享 lead/prefill 提高到 1200ms，但
AudioRingBuffer::set_playback_rate仍只保留 250ms。真实环形缓冲测试中，切倍速直接截掉 950ms PCM，且dropped_frames没有反映这次丢弃。请撤回这组 1200ms 配置变更；若保留 CoreAudio 的显式背压门控，应与统一的短队列预算一致。 -
计数活性不能替代样本有效性。 旧时间线中采集、延迟交付的快照，计数也可能比上次大。受控测试中，先采集 200ms 音频快照，再完成 3000ms seek，最后交付旧快照，PR 会将时钟拉回 200ms。这个测试不意味着标准串行宿主已自然复现了同样竞态，但证明核心缺少拒绝该反馈所需的信息。建议为内部观测补充采样时刻和媒体/输出代次，区分有效媒体消费、欠载静音及设备恢复。真实大偏差应能重锚，旧时间线或过期反馈必须被拒绝，并保留倍速过渡期间禁止混速反馈的规则。
另外请修正 Metal 的说明：Apple 文档规定 allowsNextDrawableTimeout 默认就是 true；true 仍可能等待一秒，false 会无限等待。显式设 true 可以作为防御配置，但不能描述为非阻塞、廉价跳过一帧。nil drawable 的既有 RendererBackpressure 处理来自 #115，可以保留；skip 计数也可以保留，但不覆盖“阻塞后成功返回 drawable”的耗时。
建议将本 PR 收窄为“修复有效音频大幅落后后无法重新校时”，保留和加强相应回归测试，撤回深缓冲措施，按上述有效性协议修订校时。标题与变更说明应明确修复的是失步恢复，不宣称消除了渲染阻塞导致的音频欠载。
音频供给与渲染解耦另作后续 PR，沿用同一主时钟和已有 Play/Pause/Seek/Buffering/EOF 状态机，并扩展 quiesce/ACK、generation 和倍速桥协议。当前独立供给实验在 1600ms 渲染阻塞下以短队列保持零欠载，但它只验证稳定播放，尚不是完整生产实现。
验证摘要：基线 733ms 渲染阻塞出现约 598ms 欠载并持续失步；PR 在 733ms 阻塞下恢复正常，但 1600ms 阻塞仍欠载约 395ms。确定性校时实验中基线拒绝 601/601 次反馈，PR 接受 601/601 次。现有默认库测试基线 479、PR 482 全部通过，额外启用的现有 Presenter 契约测试两边各 48 项通过。这些结果说明修复有价值，同时现有测试尚未覆盖上述交互问题。
Shinokawa
changed the title
Superseded by 2f1accb: removed the 1.2s queue, validated clock observations, corrected Metal timeout semantics, and completed audio ownership/lifecycle tests. This withdraws my obsolete objections; independent review and platform CI are still required.
There was a problem hiding this comment.
🟡 Changes recommended
The updated audio-clock discipline stores a rejected snapshot as the new baseline, which can allow later invalid observations to be accepted and incorrectly influence clock correction.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR refactors playback so decoded PCM delivery and audio-clock feedback continue even when rendering stalls, preventing long-lived underflow/clock-guard lockout (notably during macOS fullscreen Space transitions) while preserving existing lifecycle and rate-bridge behavior.
Changes:
- Introduces a dedicated presenter-side audio owner thread (
AudioService) that owns backend creation/calls/drop and commits playback-rate transitions independently of render ticks. - Adds capture-identity audio clock observations (
capture_audio_clock/update_audio_clock_observation) and updates clock discipline to tolerate render stalls while rejecting stale/out-of-order/silence-only samples. - Retains Metal drawable-timeout behavior explicitly (
allowsNextDrawableTimeout) and adds throttled skip counters/diagnostics; synchronizes docs (EN/ZH/JA) and changelog.
File summaries
| File | Description |
|---|---|
| docs/capi_reference.zh.md | Documents audio-owner behavior and host ticking expectations in Chinese C API reference. |
| docs/capi_reference.md | Updates English C API reference guidance for audio_only_tick and render-stall independence. |
| docs/capi_reference.ja.md | Mirrors C API guidance updates in Japanese. |
| docs/architecture.zh.md | Expands Chinese architecture doc for audio owner + observation-based clock feedback. |
| docs/architecture.md | Expands English architecture doc for audio owner + observation-based clock feedback. |
| docs/architecture.ja.md | Expands Japanese architecture doc for audio owner + observation-based clock feedback. |
| crates/erika/src/renderer/metal/mod.rs | Adds drawable-backpressure skip accounting + throttled reporting; exposes internal skip count. |
| crates/erika/src/renderer/metal/apple.rs | Forces allowsNextDrawableTimeout=true on host-provided CAMetalLayer to avoid indefinite waits. |
| crates/erika/src/renderer/d3d11.rs | Fixes/updates a test config struct init (adds ..default()). |
| crates/erika/src/presenter/audio.rs | New audio-owner implementation (command/ACK boundaries, pumping, rate bridge, clock observation publishing). |
| crates/erika/src/presenter/audio_integration.rs | New integration tests simulating render stalls + device callback while using the production audio owner. |
| crates/erika/src/presenter.rs | Wires presenter to the new audio owner; adjusts transitions/drains/stats and first-audio gating. |
| crates/erika/src/playback.rs | Updates audio clock discipline to use bounded delivery extrapolation + progress baselining and new output epochs. |
| crates/erika/src/core.rs | Introduces AudioClockObservation with capture identity (player/generation/command sequence/output epoch/time). |
| crates/erika/src/audio.rs | Minor doc wording updates reflecting the audio-owner change. |
| CHANGELOG.md | Records the new audio owner, Metal layer timeout behavior, and improved clock recovery. |
Review details
- Files reviewed: 16/16 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Shinokawa
changed the title
Problem and behavior
After a render stall exhausts queued audio, the old distance-based guard rejects
valid audio feedback more than 250 ms behind the playback clock. That makes the
desynchronization persist after rendering and PCM delivery recover.
This PR restores clock recovery using observation identity and actual media
progress. Audio delivery still runs in the existing presenter tick; a blocked
render call can still cause temporary audio underflow. Independent audio delivery
is being reviewed separately in #132, which depends on this clock repair.
Implementation
output epoch and monotonic capture time. The worker validates both shared intent
and executed commands; freshness expires 500 ms after capture. Buffering/prefill
uses the same identity and age checks.
actual PCM consumption and media time to advance. A rejected sample cannot
replace the trusted progress baseline; its capture-time watermark still rejects
out-of-order delivery. This addresses Copilot's review of
2f1accb.bounded by PCM queued at capture and the committed rate. Seek floors, paused
clocks, first-frame parking, and mixed-rate feedback protections remain intact.
reuses the existing audio seek-floor trimming to avoid queueing that audio a
second time. A real seek/stop/stream replacement clears this boundary. This
closes a pre-existing continuity gap exposed by stricter observation validation.
fix(audio): smooth playback-rate transitions across native outputs #82's short queue and 250 ms rate bridge, including non-1x audio-master feedback.
allowsNextDrawableTimeout, whose true/default behavior still permits a wait ofup to one second; it is not nonblocking. Existing nil-drawable backpressure
handling remains unchanged.
C ABI and host bindings are unchanged relative to
main. The Rust synchronoussnapshot API remains available; delayed feedback should use capture/update
observation APIs. The branch incorporates #129; its D3D11 renderer is unchanged
by this PR, and the earlier test initializer prerequisite is no longer needed.
Historical contracts
Preserves the stale-feedback intent of #30/#31; seek/lifecycle protections from
#35/#41/#66/#87; #42's existing producer quiesce protocol; #55's distinction between
device recovery and EOF; #69's shared clock; #71/#115's foreground recovery;
#74's buffering reanchor; and #82's rate bridge/mixed-rate protections.
Validation
wgpu: 546 passed.a still-invalid 0.3 s observation, plus independent read-counter/media-time
regressions, capture ordering, genuine recovery and output-epoch counter resets.
output tail, while an explicit seek can restart from zero.
observations after controlled underflow. Silence/frozen observations, bounded
extrapolation, seek/pause and non-1x correction tests remain passing.
upscaler-status failure no longer occurs after incorporating 修复 Windows 扫描线视频 DWM 崩溃、铺满尺寸及 D3D11VA 帧复用 #129.
The core behavior was reproduced independently on Windows. Native macOS fullscreen
animation and physical DAC/display synchronization have not been measured. The
previous combined head passed all 12 platform checks; this narrowed head starts
its own CI run and does not inherit those results. All local results above were
rerun on
d99b2dfafter resolving the D3D11 test initializer conflict with #129.The original author's commits are preserved. Scope was narrowed with ordinary
follow-up commits, without rewriting the fork's history.