Skip to content

[PR] Make a held head dropped at drain auditable (#5192) - #6537

Merged
ranxianglei merged 2 commits into
ranxianglei:masterfrom
tinysunspot:2026-11-13_audit-drain-dropped-head
Nov 15, 2026
Merged

ranxianglei merged 2 commits into
ranxianglei:masterfrom
tinysunspot:2026-11-13_audit-drain-dropped-head

Conversation

@tinysunspot

Copy link
Copy Markdown
Contributor

What this changes

The streaming tag filter reports drops through a one-shot notice: recordDrop fires onDrop for the first drop of a response and then never again. A tag head still held when the stream ends is dropped inside flush(), so whenever a completed tag came first — the common case — the drain drop produced a dropCount bump and no log line at all.

This gives drain drops their own budget and an explicit flag on the callback:

-export function createTagEchoFilter(onDrop?: (snippet: string) => void, ...
+export function createTagEchoFilter(onDrop?: (snippet: string, duringDrain?: boolean) => void, ...
 const recordDrop = (snippet: string, duringDrain = false) => {
     droppedAny = true;
+    if (duringDrain) {
+        if (onDrop && !drainNoticed) { drainNoticed = true; onDrop(snippet, true); }
+        return;
+    }
     if (onDrop && !notified) { notified = true; onDrop(snippet); }
 };

The two drop sites that resolve a held tail at EOF (TRUNC_OPEN and TRUNC_CLOSE in the flush() settle branch, src/loop/tag-echo-filter.ts:1608-1617) pass the flag; all eight callers name the event in their own wording. Ordinary strips keep the one-shot semantics untouched — still exactly one line per response.

Why

Field evidence from #5192. In a 40-truncation session, 36 stripped model-emitted render tag lines were logged and 35 of them carried a full opening tag. The one exception was five characters long:

17:50:59.273 [tag-echo] stripped model-emitted render tag (plugin passthrough): <acp 

That line is the drained head — the same fragment the persisted assistant text then carried as prose (1-13 17:51:02">m00582). But because it rendered with the identical wording and prefix as a successful strip, it was indistinguishable from one, and the second and later drain drops of that session left no line at all. An operator reading bili.log had no way to tell "tag stripped cleanly" from "stream died mid-tag".

Naming the event is the whole point: a drain drop means the stream ended while the filter still held bytes, which is a truncation symptom, not a strip.

Reproduction (red before the fix)

tests/issue5192-drain-head-audit.test.ts — six unit tests against the filter plus one end-to-end test through pipePluginChatWithStrip. Against the unpatched filter, 4 of the 7 fail:

✖ #5192: a tag head discarded by flush() is reported as a drain drop
✖ #5192: a drain drop is reported even after an earlier strip claimed the one-shot notice
✖ #5192: a truncated close tag discarded by flush() is also a drain drop
✖ #5192: the passthrough logs a held head dropped at stream end as a drain drop

With the fix: 7/7, and the emitted line is

[testsess] [tag-echo] held head dropped during drain: <acp tokens="12" type="text" time="2026-11-13 17:51:02"

The end-to-end case asserts both directions: the completed tag still produces exactly one stripped model-emitted render tag line, the drained head produces exactly one held head dropped during drain line, and the prose around both survives byte-for-byte.

Verification

  • npx tsc --noEmit --project tsconfig.json → exit 0
  • 200/200 pass across the eleven in-family suites (issue5192-drain-head-audit, tag-echo, verif-tag-echo-filter, issue2190-degen-close, issue4164-dead-ref-chunk-boundary, issue1881-reasoning-tag-leak, plugin-passthrough-tag-strip-chat, plugin-passthrough-progress-frame, plugin-passthrough-progress-frame-google, plugin-passthrough-finish-tail, plugin-passthrough-tag-strip)
  • Red/green verified by reverting only the two duringDrain arguments in the filter: 4 fail, then 7/7 after restore
  • tests/advisory.test.ts has one pre-existing failure (checkForUpdate: a forced manual update bypasses the first-check deferral, 4 !== 3 fetch calls); it reproduces on a clean master stash, so it is unrelated to this change

Scope

One module (src/loop/tag-echo-filter.ts, +18/−6) plus the callback wording at its eight call sites (src/plugin.ts, five adapters) and one new test file. The new parameter is optional and additive — no TagEchoFilterStats field, so no consumer (composeStreamFilters stats merge, maybeWarnDegenerate, degenerateTurnWarning) needs to change, and no config surface or wire shape moves. No version bump. Fixes #5192 is deliberately not claimed — the owner's ruling keeps all facets under that issue; this PR only makes one facet observable.

Related

The streaming tag filter reports its drops through a one-shot notice: the
first drop of a response claims it and every later one is silent. A tag head
still held when the stream ends is dropped inside flush(), so when a
completed tag came first (the common case) the drain drop produced a counter
bump and no line at all — the only trace was the strip snippet, which is why
a 5-char "<acp " in bili.log was indistinguishable from a successful strip.

Give drain drops their own budget and an explicit second argument on the
onDrop callback so every caller can name the event: the filter now reports a
drop made during flush() as "held head dropped during drain", while the
one-shot notice for ordinary strips keeps its current semantics (still one
line per response).

Refs ranxianglei#5192.
@ranxianglei ranxianglei changed the title Make a held head dropped at drain auditable (#5192) [PR] Make a held head dropped at drain auditable (#5192) Nov 13, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

PR #6537 review in progress — interim status:

Verified so far:

  • Mechanism is correct for the case it covers: a truncated tag head held at EOF (the [Pi/OpenAI] 0.1.209 leaks render-tag suffix type="text">mNNNNN into assistant text #5192 repro) now gets its own one-shot drain-drop budget, so a drain drop no longer stays silent when a strip already consumed the one-shot. Probe-confirmed: drop({duringDrain:true}) fires for the truncated-head case.
  • 7/7 tests pass locally (node --import tsx --test tests/issue5192-drain-head-audit.test.ts).
  • No regression risk: additive change (new optional param + new one-shot budget), no wire/persistence/config changes.

Gap found (extending the fix now):
The PR flags only 2 of the EOF drop sites (TRUNC_OPEN/TRUNC_CLOSE settle branch). The swallow-span EOF drops (stream dies mid-tag after a complete opening tag — the more common "died mid-tag" shape) still go through dropSwallowSpan with duringDrain:false and produce no drain line. Probe-confirmed: a stream that ends after <acp tokens="12" type="text">m00582 (no close) logs a normal "stripped model-emitted render tag" for the head but no drain line for the swallowed tail.

Per 连贯修复不拆片 (one mechanism = one PR), I'm extending the fix to cover the swallow-span EOF drop sites. Remaining: flag the 7 swallow-span EOF call sites, add tests, run, push to the PR branch, then post the full 八问 review.

The original PR flagged only the TRUNC_OPEN/TRUNC_CLOSE settle-branch
sites (a truncated tag head held at EOF). The swallow-span EOF drops
(stream dies mid-tag AFTER a complete opening tag — the more common
died-mid-tag shape) still went through dropSwallowSpan with
duringDrain=false and produced no drain line.

Extend the duringDrain flag to dropSwallowSpan and pass true at all 7
swallow-span EOF call sites (lines 1537, 1546, 1551, 1554, 1579, 1585,
1594). Add 2 regression tests for the swallow-span drain case.

Per 连贯修复不拆片: one mechanism (drain-drop auditability) = one PR.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

PR #6537 Review — "Make a held head dropped at drain auditable (#5192)"

查重: 不重复。PR 是 #5192 的一个可审计切面(owner 裁定所有切面留在 #5192 下),PR 本身不 claim Fixes #5192。

Review 结论: 机制正确,但原始 PR 只覆盖了 EOF drop 站点的一部分。我直接扩展了修复(见下方说明),已 push 到 PR 分支。

我做了什么(扩展说明)

原始 PR 只 flag 了 flush() settle 分支的 2 个站点(TRUNC_OPEN/TRUNC_CLOSE,即"截断的 tag head 在 EOF 被丢弃"——#5192 的复现场景)。但 flush() 里还有 7 个 swallow-span EOF drop 站点(stream 在 complete opening tag 之后死掉——更常见的"died mid-tag"形态),这些站点走 dropSwallowSpan 且 duringDrain=false,不产生 drain 日志行。

Probe 确认:stream 在 <acp tokens="12" type="text">m00582(无 close)后死掉时,operator 只看到 "stripped model-emitted render tag"(tag head),没有 drain 行——无法区分"tag 被干净剥掉"和"stream 在 tag 中间死掉"。

按 连贯修复不拆片(一个机制 = 一个 PR),我扩展了修复:

  • dropSwallowSpan 新增 duringDrain 参数,透传给 recordDrop。
  • 7 个 swallow-span EOF 调用站点(lines 1537, 1546, 1551, 1554, 1579, 1585, 1594)传 true。
  • 新增 2 个回归测试(swallow-span drain case)。
  • 9/9 测试通过(原 7 + 新 2),83/83 tag-echo 测试通过,typecheck 干净。

行为变更披露:swallow-span EOF drop 从"静默丢弃"变为"产生一条 held head dropped during drain 日志行"。这是有意的(修复目标),不影响 wire/persistence/config。

八问

  1. 是什么问题:model 响应在 render tag 中间被截断(stream died mid-tag)时,filter 在 drain 阶段静默丢弃 held bytes,operator 没有日志行可以区分"tag 被干净剥掉"和"stream 在 tag 中间死掉"。
  2. 什么原因导致的:tag-echo filter 的 drop 日志共用一个 one-shot notified 预算。当同一响应里先有一个 complete tag strip 消耗了预算,后续的 drain drop(EOF 处的截断 tag)就被静默丢弃——没有日志行。drain drop 和 strip 共享同一个 one-shot 预算,drain drop 被 strip 挤掉。
  3. 影响范围与严重程度:受影响场景——同一响应里先有 complete tag strip,后续 stream 在 tag 中间死掉。严重程度:可观测性缺口(不是数据丢失——bytes 被正确丢弃,但审计线索不完整)。operator 无法从日志判断 stream 是否在 tag 中间死掉。
  4. 怎么修复的:给 drain drop 独立的 one-shot 预算(drainNoticed),与 strip 预算(notified)分离。flag 所有 EOF drop 站点(2 个 settle 分支 + 7 个 swallow-span)传 duringDrain=true。调用方根据 flag 选择措辞:"held head dropped during drain" vs "stripped model-emitted render tag"。
  5. 是否有回归问题:无。改动是增量(新可选参数 + 新 one-shot 预算)。现有 strip 行为不变(同一 one-shot 预算、同一措辞)。之前静默的 drain drop 现在产生日志行——这是修复目标。无 wire/persistence/config 变更。83/83 tag-echo 测试通过。
  6. 是否破坏缓存:不涉及。改动仅在日志路径,无 cache/retry/增量游标变更。
  7. 是否改了设计方向:不涉及。扩展现有 drain-drop 机制覆盖更多 EOF drop 站点,无架构变更。
  8. 是否引入了新的配置:不涉及。无新 env var/config key/默认值。

判定

可合。机制正确,测试充分(9/9 通过 + 83/83 回归),diff 干净(2 files, +35/−9 扩展部分),无回归。扩展部分已 push 到 PR 分支(commit 296e2c0f),PR 现有 2 commits / 8 files。

@ranxianglei

Copy link
Copy Markdown
Owner

Superseded by #6765 (merged to master as fcee1c0).

Your commit 296e2c0 is in master verbatim — same content, same authorship, only the object ID changed (the batch used a real git merge, no rebase or cherry-pick rewrite). Merged under a different branch name, so GitHub does not flip this PR by itself.

Thanks for the fix — it is live in master from fcee1c0 onward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants