Skip to content

[PR] fix(openai): route tool-less audio (TTS) requests through the side passthrough - #6638

Merged
ranxianglei merged 3 commits into
ranxianglei:masterfrom
sx4im:fix-6636-openai-wire-acp-injection
Nov 15, 2026
Merged

ranxianglei merged 3 commits into
ranxianglei:masterfrom
sx4im:fix-6636-openai-wire-acp-injection

Conversation

@sx4im

@sx4im sx4im commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

A TTS call on /chat/completions carries a top-level audio object and no tools. It ran through the full pipeline, which injected the ACP system prompt as messages[0] plus an prefix on every message. TTS models 400 on the leading system message, and the tags would be read aloud as text.

What changed: isAudioOutputCall recognizes a tool-less request with a non-empty top-level audio object and routes it through the #388 verbatim side passthrough, so the request leaves the machine exactly as the client wrote it. Same treatment as the #2500/#5073 utility shapes, including the output-budget high-water skip. A tool-carrying request with audio stays a main turn. Added WC-025 to the wire-contract ledger with a validator tripwire.

How verified: typecheck clean. 5 new tests in tests/issue6636-tts-side-request.test.ts (predicate positives/negatives, side-lane label, and an e2e test using the issue’s exact TTS body: forwarded byte-identical, no system, no tags, kernel untouched; plus a control proving a real tool-carrying turn still gets the ACP system prompt and tags). 153 tests green across the touched and neighboring side-lane suites.

Fixes #6636

…ssthrough

A TTS call on /chat/completions carries a top-level audio object and no
tools. It ran through the full pipeline, which injected the ACP system
prompt as messages[0] plus an <acp> prefix on every message, and TTS
models 400 on the leading system message. isAudioOutputCall now
recognizes the shape and routes it through the ranxianglei#388 verbatim side
passthrough, with a WC-025 ledger entry and tripwire in the fakes.
@ranxianglei ranxianglei changed the title fix(openai): route tool-less audio (TTS) requests through the side passthrough [PR] fix(openai): route tool-less audio (TTS) requests through the side passthrough Nov 14, 2026
ci-check added 2 commits November 14, 2026 12:18
…eto (ranxianglei#6636 review)

The tripwire flagged ANY audio-shaped body with a leading system message,
but isAudioOutputCall vetoes tool-carrying requests (a main turn whose model
has an action surface and accepts a leading system). A legitimate
tool-carrying voice-chat main turn, decorated by bili, would have tripped the
fake upstream. Add the same no-tools veto so the validator matches the
predicate, and pin the boundary with a legal test row.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Review of PR #6638 — duplicate screening: not a duplicate (no other open issue/PR covers the tool-less audio side-passthrough; this PR is the fix for #6636).

Verdict: code is good to merge, pending the fresh CI run (details below). I verified the mechanism end-to-end, ran the touched suites locally, and made two branch updates (both non-destructive, additive):

  1. Merged current master into the branch (merge commit a92c1c0a). The branch was based on 36a04a79, and master moved to e5e1647b ([PR] fix: watermark fastpath (plugin full-resend watermark fastpath) silently disabled by input release mutation | fix: 水印快路径(plugin full-resend watermark fastpath)被输入释放改动静默禁用 #6640, the watermark-fastpath fix) after the PR's CI dispatch. Both PRs touch src/server/handle.ts but in disjoint regions — the merge was clean. On the merged head: typecheck clean, and the touched + neighboring suites (the 5 new TTS tests, side-lane-invariants, wire-contract, watermark-fastpath) are 47/47 green locally.
  2. One consistency fix, own commit 6321ba05: the WC-025 validator in tests/wire-contract-fakes.ts flagged any audio-shaped body with a leading system message, but isAudioOutputCall carries a no-tools veto (a tool-carrying request is a main turn whose model has an action surface and accepts a leading system). A legitimate tool-carrying voice-chat main turn, decorated by bili, would have tripped the fake upstream. I added the same no-tools veto to the validator plus a legal test row pinning the boundary. 30/30 green in the wire-contract + TTS suites after the change.

CI state (live-verified): the original head cc2d09eb (dispatched 02:16Z) had all 9 e2e lanes green (native pi/opencode/dsh, real+fake codex, pi-codex-ws, local-registry, theorem lane); the main CI matrix + build artifact were cancelled, not failed — no code fault on the PR. Master was red at the time on 3 e2e lanes (watermark-fastpath regression, now fixed by #6640 and tracked in #4599). The new head 6321ba05 is pushed and its CI run is pending — that run is the one to watch.

Behavior change (flagged per policy): a tool-less request with a non-empty top-level audio object changes from full pipeline (ACP system injection + per-message tags + kernel tracking/compression) to verbatim side passthrough (no kernel touch, #388 lane). For TTS that is the fix. Accepted residual, documented in the predicate's doc comment: a tool-less voice-chat turn that asks for audio output (e.g. a no-tools audio-preview voice assistant) now also rides the side lane for that turn — the reply still works (the model gets exactly what the client wrote), but the kernel does not track/compress those turns; the next tool-carrying turn rejoins the full pipeline. The shape signal cannot distinguish one-shot TTS from a voice-chat turn (model names are unreliable — WC-025 deliberately keys on shape, not spelling), and availability of the TTS path outranks the compression coverage of that edge case.

(Apology note: an earlier copy of this review was misposted to the wrong mirror thread and has been withdrawn.)


八问(中文结构化结论)

  1. 是什么问题:OpenAI 兼容 /chat/completions 上的 TTS 调用(无 tools + 顶层 audio 对象)被 bili 当主轮处理,注入了 ACP system prompt(messages[0])并给每条消息打 标签;TTS 模型对前置 system 直接 400,标签还会被当文本念出来。
  2. 什么原因导致的:side-request 分类器(isSideRequest)不认识 TTS 形状——既有信号(agent 头、server-tool 形状、structured-output 形状、小 max_tokens 预算)全部漏掉「顶层 audio 对象 + 无 tools」,请求因此进入完整管线被装饰。根因是分类缺失,不是注入逻辑本身有 bug。
  3. 影响范围与严重程度:任何经 bili 的 OpenAI 兼容 TTS 模型(OpenAI wire: ACP ref tags + leading system are injected into every /chat/completions, breaking non-conversational side calls (TTS 400 "messages[0] system role is not allowed for TTS model") #6636 中 dsh-fish-soul 经小米 MiMo mimo-v2.5-tts 复现,bili 0.1.213)——TTS 完全不可用(400),属 p1 可用性缺陷。
  4. 怎么修复的:新增 isAudioOutputCall 形状谓词(无 tools 否决 + 非空顶层 audio 对象),挂进 isSideRequest 子句(位于 [bug] bili dsh 长会话中 max_output_tokens 被压到 ≤200,主请求被误判为 side request 并绕过压缩 #546 tools 否决之后、预算启发式之前),让 TTS 走 标题生成等侧请求污染主会话状态:prepareAnthropic 缺 isTitleGen 守卫、快照无条件覆盖、survivedCount 每请求空转 #388 逐字 side passthrough(不注入、不打标签、kernel 不触碰);同步加入 restoreOutputBudget 高水位跳过(与 OMP extension injects tools and compression philosophy in helper prompts #1897/[bug] Claude Code 的 WebSearch 子请求被当成主对话:nudge 基准被搜索的 usage 重锚,导致提前折叠;每次搜索还被注入 bili 工具 #2500/Hermes lane: proxy adds ACP tools to toolless structured-output requests from child processes, which strict backends reject (400) #5073 同类,防止 TTS 的 max_tokens 毒化首次 starved restore);wire-contract 台账加 WC-025 + 校验器绊线。
  5. 是否有回归问题:审过的既有行为——isSideRequest 五个既有子句(agent 头、main 显式否决、server-tool、[bug] bili dsh 长会话中 max_output_tokens 被压到 ≤200,主请求被误判为 side request 并绕过压缩 #546 tools 否决、structured-output、小预算)逐一未动,side-lane-invariants 套件通过;restoreOutputBudget 跳过只作用于新形状;sideLabel 只新增一个分支;server.ts 导出表 +1 符号。有意行为变更(已披露):无 tools 且带非空顶层 audio 的请求从「完整管线」变为「逐字 side passthrough」——TTS 是修复;残留:无 tools 的语音聊天轮(audio 输出)同轮也走 side lane,回复仍可用但 kernel 不跟踪/不压缩该轮,下一个带 tools 的轮回归完整管线(形状信号无法区分一次性 TTS 与语音聊天轮,按可用性优先原则接受并已在谓词注释中记录)。高危模式逐项过:无阈值/时序/输出格式漂移、无 fail-fast 变吞错、无缓存/重试策略变化。另发现 WC-025 校验器缺谓词同款的无 tools 否决(带 tools 的语音聊天主轮被 bili 装饰后会被 fake 上游误判)——已在本 PR 分支独立 commit 修复。
  6. 是否破坏缓存:不涉及。side passthrough 是既有机制(标题生成等侧请求污染主会话状态:prepareAnthropic 缺 isTitleGen 守卫、快照无条件覆盖、survivedCount 每请求空转 #388);TTS 请求此前因 400 根本走不到前缀缓存路径;主轮缓存路径未动(e2e 对照测试证明带 tools 的主轮仍获得完整 ACP 装饰)。
  7. 是否改了设计方向:不涉及。完全沿用既有 side-lane / utility-shape 纪律([bug] Claude Code 的 WebSearch 子请求被当成主对话:nudge 基准被搜索的 usage 重锚,导致提前折叠;每次搜索还被注入 bili 工具 #2500/Hermes lane: proxy adds ACP tools to toolless structured-output requests from child processes, which strict backends reject (400) #5073),无新架构面。
  8. 是否引入了新的配置:不涉及。无新 env/config/CLI。

收工判定:代码可合(等新 head 6321ba05 的 CI 跑绿);行为变更已按上述披露。

@ranxianglei

Copy link
Copy Markdown
Owner

Closes pair #6636 was auto-closed by the batch merge.

Superseded by #6765 (merged to master as 78d128d).

Your commit 6321ba0 is in master verbatim — same content, same authorship, only the object ID changed (the batch used a real git merge, no rebase or cherry-pick rewrite). Merged under a different branch name, so GitHub does not flip this PR by itself.

Thanks for the fix — it is live in master from 78d128d onward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants