Skip to content

[PR] fix: preserve runaway guard frame boundaries across chunks - #6128

Closed
Afloat16 wants to merge 3 commits into
ranxianglei:masterfrom
Afloat16:2026-11-10_fix-runaway-frame-boundaries
Closed

Afloat16 wants to merge 3 commits into
ranxianglei:masterfrom
Afloat16:2026-11-10_fix-runaway-frame-boundaries

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Problem

The text-run runaway guard misses repeated output when the transport batches several SSE frames into one chunk: its separator regex retains an offset into the old buffer after slicing. The NDJSON fallback also loses an incomplete final line and stops parsing subsequent small chunks.

Changes

Reset the SSE separator cursor after each consumed frame. Retain incomplete NDJSON events and the selected framing mode. Require a JSON opening token before selecting NDJSON so a large split SSE event retains SSE framing. Normal forwarded stream bytes are unchanged.

Validation

  • 46 targeted Node tests passed across the new frame-boundary regressions and existing runaway-guard suites.
  • Three new regression cases fail on unchanged upstream source and pass with this patch.
  • TypeScript typecheck and npm production build passed, including normal kernel/pi prebuilds, dist import annotations and external-dependency checks.
  • git diff --check passed. The complete suite verdict is left to repository CI.
  • Cases cover LF/CRLF batched frames, interleaved resets, split NDJSON, large split SSE and excluded tool arguments.

Checked prior text-run work #3289 and Google tool-argument exclusions #3357/#3362/#3375, plus previous Afloat16 contributions; these framing defects are separate.

Base: e1e33a9.

Closes #6127

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Review started on PR #6128 (mirror #6131).

Duplicate screening: not a duplicate. Searched open+closed issues and PRs for runaway-guard framing topics. Prior text-run work #3289 (lane introduction, #3260) and Google tool-argument exclusions #3357/#3362/#3358 are all closed and cover different mechanisms (lane creation / argument exclusion) — this PR's frame-boundary defects (stale SSE separator cursor, NDJSON tail loss + mode forgetting) are distinct, as the author states. One sibling in flight: PR #6131 ("avoid recursive runaway guard traversal", issue #6130) touches the same file but a different function (collectText recursion depth → stack overflow crash vs this PR's pumpFrames framing state). Different symptom AND different mechanism → separate threads, correct split; the two patches are region-disjoint and semantically compatible (I verified #6131's iterative rewrite preserves left-to-right feed order and excluded-subtree semantics), so merge order between them doesn't matter, the later one just rebases trivially.

Labels: bug, scope/wiring, p2 (kept existing pr) — defect in the proxy-side upstream-stream wiring (not compression kernel logic); severity p2 because a missed trip wastes tokens/latency until manual cutoff but the session stays usable (no data loss, no cache/storage growth).

Verification so far (targeted local runs): the 3 claimed regression cases genuinely fail on unchanged upstream e1e33a9 (batched SSE LF, batched SSE CRLF, split NDJSON event) and all 7 new tests pass on the PR head; CI shows 0 failures with 3 windows shards still running. Full review verdict follows after I finish checking one test-coverage gap I found in the new NDJSON gate.

@ranxianglei ranxianglei added bug Something isn't working p2 scope/wiring labels Nov 10, 2026
The existing split-SSE case passes on unpatched source too (its large
frame carries no trip-worthy run), so the /^[{\[]/ gate added alongside
the frame-boundary fix had no regression coverage. This frame is a single
>=64KB SSE event whose pretty-printed payload spans many data: lines and
ends in a 200-codepoint run: without the gate the NDJSON fallback engages
on the first chunk and drops every unparseable line, missing the trip.

Reviewer addition for ranxianglei#6128.
Review follow-up for the frame-boundary fix (issue ranxianglei#6127):

- Tolerate leading blank lines/whitespace before the first NDJSON line
  (gate /^[{\[]/ -> /^\s*[{\[]/): an NDJSON stream opening with a stray
  blank line would otherwise never engage the fallback and lose text-run
  detection for the whole stream. SSE field lines never open with
  whitespace, so oversized split SSE events stay protected by the gate.
- Document the non-obvious framing invariants at their sites: the cursor
  re-anchor after each consumed frame, sticky NDJSON mode, retained
  partial tail, and why the JSON-token gate exists.

Adds a regression test: leading-blank-line NDJSON trips once its repeated
delta completes across chunks (fails on the pre-change gate).
@ranxianglei

Copy link
Copy Markdown
Owner

Landed via batch PR #6439 (merged) — closing as superseded.

Louie0806 pushed a commit to Louie0806/billion-context that referenced this pull request Nov 12, 2026
The existing split-SSE case passes on unpatched source too (its large
frame carries no trip-worthy run), so the /^[{\[]/ gate added alongside
the frame-boundary fix had no regression coverage. This frame is a single
>=64KB SSE event whose pretty-printed payload spans many data: lines and
ends in a 200-codepoint run: without the gate the NDJSON fallback engages
on the first chunk and drops every unparseable line, missing the trip.

Reviewer addition for ranxianglei#6128.

(cherry picked from commit b8e6d9d)
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Confirmed, verified live:

Nothing outstanding on this thread — the fix is on master, CI-verified by the batch merge. Closing is yours to execute per the note above.


结构化八问(评审结论归档)

  1. 是什么问题:text-run 失控守卫在传输层把多个 SSE 帧批量塞进一个 chunk、或 NDJSON 事件跨 chunk 截断时漏检重复输出——恰恰在应当触发的突发期间失明。
  2. 什么原因导致的:pumpFrames 三处状态缺陷:全局 SSE 分隔正则切片后 lastIndex 仍指向旧缓冲区(后续帧不再被解析);NDJSON 兜底清空未完成尾行(截断事件永远拼不回);64KB 参与门按 chunk 重判,小后续 chunk 落回 SSE 路径停摆(且大体积多行 SSE 帧可能被误判为 NDJSON 整帧丢弃)。
  3. 影响范围与严重程度:所有走这些帧形状的 proxy 模式上游响应;后果是漏触发 → 生成/token 消耗持续到手动截断,会话仍可用 → p2。
  4. 怎么修复的:在决策点做状态卫生(疏不堵):每消费一帧+每次 pump 入口重置分隔游标;保留未完成尾行使截断事件可重组;帧模式一经选定即闩锁;NDJSON 参与前要求缓冲区以 JSON 开符号起始,SSE 正文(data:/event:/注释开头)保持 SSE 帧形。转发路径零改动(字节精确)。
  5. 是否有回归问题:无——既有 39 个 runaway/text-run 测试在 PR head 与 master 全绿,typecheck 干净;唯一行为变化是此前漏掉的帧现在会被解析(即修复本身,已披露);闩锁与「单次响应帧形恒定」的事实一致,误参与在真实 SSE 上已不可能(需 ≥64KB 无空行且以 { 或 [ 开头)。
  6. 是否破坏缓存:否——守卫只观测流,不触碰任何缓存/重试/游标/去重结构;字节精确转发由测试断言钉住。
  7. 是否改了设计方向:否——既有 wrapStreamWithRunawayGuard 闭包内的状态 bug 修复,无架构变化。
  8. 是否引入了新的配置:无。

收工判定:修复已经由批量 PR #6439 合入 master,本线程无遗留事项,可关。

@ranxianglei ranxianglei reopened this Nov 14, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Re-triggered after the infra retry — verified live, nothing changed since my last reply: PR #6128 is closed unmerged (superseded), batch PR #6439 remains merged on master (2026-11-12T14:01:25Z) carrying the fix plus the review-time gate test, and no new comments have landed. No work pending on this thread; the mirror row stays open for your close per your "closing as superseded" note.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 scope/wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Runaway detection misses batched SSE frames and split NDJSON events

3 participants