Skip to content

[Bug] Runaway detection misses batched SSE frames and split NDJSON events #6127

Closed

Description

@Afloat16

Reproduction

At upstream e1e33a9, pass two complete SSE events in one ReadableStream chunk to wrapStreamWithRunawayGuard. Each event has choices[0].delta.content containing 100 consecutive = characters. With the normal 200-character text-run threshold, the trip callback is never called; sending the same events separately does trip. Both LF and CRLF separators reproduce this.

An NDJSON case uses a complete JSON line with a 66 KB non-repeating prefix followed by the beginning of a 200-character repeated delta. Deliver its remainder and newline in the next chunk. The fallback drops the unfinished line and misses the repeated output.

Expected behavior

Inspect the same model text independently of network chunk boundaries, retain partial events, and forward ordinary bytes unchanged. Missing a trip allows unnecessary generation and token consumption.

Cause and verified fix

The global SSE separator regex retains lastIndex after the consumed prefix is sliced away. NDJSON fallback clears the unfinished tail and forgets its framing mode on subsequent small chunks. Resetting the cursor and preserving mode plus tail fixes both cases.

A prepared patch passes 46 targeted tests, typecheck and production build; three new regressions fail on the original source. Existing #3289 and tool-argument exclusions #3357/#3362/#3375 do not cover these framing failures. No kernel or configuration changes are required.

Activity

  1. ranxianglei commented on Nov 11, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: opencode/qwen3.8-27b)

    Duplicate screening: searched open+closed issues and PRs for this symptom/mechanism — the fix is already being delivered as PR #6128 ("fix: preserve runaway guard frame boundaries across chunks", closes #6127). Per the one-fix-PR-per-thread policy I did not open a parallel PR; instead I independently reproduced both reported failures, audited that patch, and pushed one hardening commit onto its branch.

    Independent verification (at e1e33a9):

    • Batched SSE: two complete events × 100 = chars in one ReadableStream chunk never trip; the same events in separate chunks do. LF and CRLF both affected. Root cause confirmed in source: the module-global /\r?\n\r?\n/g separator regex keeps its lastIndex after frameBuf is sliced — the next exec() resumes past the end of the shrunk string, so frames 2..N of a chunk are silently never scanned.
    • Split NDJSON: a ≥64 KB line split across chunks loses its incomplete tail (frameBuf = "") and the fallback mode is not sticky, so later small chunks are re-read as SSE where a lone \n never forms a separator — the repeated run is never counted. Matches the report exactly.

    Review outcome on PR #6128:

    • Patch direction is correct: cursor reset after each consumed frame; sticky NDJSON mode; retained partial tail; JSON opening-token gate so oversized split SSE events keep SSE framing.
    • One real gap, fixed directly (additive commit 3b150bff92 on the PR branch): the opening-token gate /^[{\[]/ rejects NDJSON streams whose first non-blank content follows leading blank lines — the fallback then never engages and the whole stream loses text-run detection. Widened to /^\s*[{\[]/; SSE field lines never start with whitespace, so the oversized-split-SSE protection stays intact. Added a regression test for that case (it fails on the pre-change gate).
    • Also documented the non-obvious invariants at their sites (cursor re-anchor, sticky mode, retained tail) so they don't read as removable dead code.
    • A companion review session had meanwhile pinned the token gate itself with a discriminating large multi-line data:-frame test (b8e6d9d873); the rebase kept both tests.

    Local evidence: 23/23 targeted tests pass (new framing suite + existing runaway-guard suite); reverting src/runaway-guard.ts to the pre-fix source fails exactly the three positive-direction regressions (batched-SSE LF/CRLF, NDJSON partial tail) — the tests have teeth. Typecheck clean. Full-suite verdict comes from CI on the new head; the cached run for the previous head showed 0 failed checks (1 still pending at snapshot time).

    Status: PR #6128 head is now 3b150bff92, mergeable, awaiting CI on the new head. Labels verified in place (bug / p2 / scope/wiring). No kernel or config changes required, as the report states.

    八问

    1. 是什么问题:失控输出守卫在「一个网络块里塞了多个 SSE 事件」或「NDJSON 事件被拆到两个块」时漏检重复输出,本该中止的退化生成继续烧 token。
    2. 什么原因导致的:两个帧边界记账 bug——(a) 全局分隔符正则 /\r?\n\r?\n/g 在切片缩短缓冲后保留旧 lastIndex,下一次 exec 从已越界位置恢复,同块内第 2..N 帧全被跳过;(b) NDJSON 兜底清掉未完成的尾行且模式不粘滞,后续小块按 SSE 重读(单个 \n 永远不构成分隔符),重复段从未被计数。
    3. 影响范围与严重程度:所有流经守卫的流式响应(两种压缩模式都接线,relay.ts:1918/:2219);触发条件 = 传输层把多事件批进一块(代理/缓冲常见)或 NDJSON 服务端跨块拆行。p2:守卫是纯观察性的(字节逐字转发不变、会话不坏),漏检只意味着退化生成不被中止、多耗 token;正常流量无感,故不是 p1。
    4. 怎么修复的:PR [PR] fix: preserve runaway guard frame boundaries across chunks #6128 贡献者补丁:每消费一帧后重置正则游标、NDJSON 模式粘滞化、保留未完成尾行、加 JSON 开头 token 门防止超大拆分 SSE 事件误入兜底。我补的评审硬化(3b150bff92):token 门放宽为允许前导空白(/^\s*[{\[]/)——否则以空行开头的 NDJSON 流整个丢失检测(SSE 字段行绝不以空白开头,保护不受损)+ 注释钉住非显然不变量 + 该场景回归测试。
    5. 是否有回归问题:逐项过既有行为——字节级转发不变量([bug] 插件透传会剥掉工具调用参数里的 render tag(arguments 流挂了 tag-echo 过滤器) #1039)未动(所有测试断言输出与输入逐字节相等)、工具参数排除(fix: runaway text guard no longer reads Google tool-call arguments as model output #3357 系)在批处理下仍生效(有测试钉住)、ACP tag / tool-XML 计数器未触碰、阈值常量未变、SSE 流不受 NDJSON 改动影响(门要求无空行分隔 + JSON 开头)。本地定向 23/23 绿 + typecheck 干净;全量结论等新 head 的 CI。
    6. 是否破坏缓存:不涉及——守卫不碰前缀缓存/重试/增量游标,只读不改字节。
    7. 是否改了设计方向:不涉及——仍是观察性守卫架构,只修帧边界记账。
    8. 是否引入了新的配置:不涉及——无新 env/config,阈值不变。

    结论:bug 属实,修复方向正确并已加固,交付收敛在 PR #6128 —— 等新 head 的 CI 结果后即可合并。

  2. ranxianglei commented on Nov 12, 2026

    @ranxianglei
    Owner

    Fixed in batch PR #6439 (merged).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions