Skip to content

Add single-codepoint repetition lane to the output-side runaway guard (Closes #3260) - #3289

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-15_output-text-run-guard
Oct 15, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-15_output-text-run-guard

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #3260

What

Adds a third detection lane to the existing output-side runaway guard (src/runaway-guard.ts, #2346/#2347): single-codepoint repetition in decoded model text. A model stuck in a symbol loop (#3946: one symbol string for ~22 minutes — DSH form ②) now trips the stream mid-generation instead of running to max-output or manual abort. This is the bili-side landing for layer (A) of #3260 per owner decision (选项①, default on; dsh source untouched); layer (B) stays tracked at Tyan66666/billion-context-dsh#212.

Semantics ported from pi's findDegenerateRuns (minRun=200, production-tuned in billion-context-pi): ≥200 consecutive identical codepoints in assistant text/thinking → trip. Astral characters count as one unit (emoji ×200 trips).

How it works

  • The wrapper parses each SSE frame (JSON), walks the object, and feeds only text-family leaves (content / text / thinking / reasoning / reasoning_content / delta) into a stateful codepoint run counter. Covers every wire bili speaks: openai chat deltas, anthropic content_block_delta, responses output_text/reasoning_summary deltas, google parts[].text.
  • Tool-call argument subtrees are excluded — user intent, never counted ([bug] 插件透传会剥掉工具调用参数里的 render tag(arguments 流挂了 tag-echo 过滤器) #1039 invariant: a write-tool payload of 300 '=' must not abort the turn). Same for Responses function_call_arguments.* events.
  • Responses cumulative snapshot events (.done / .completed / .incomplete) are skipped: they re-send finished payloads and would double-count runs already seen as deltas.
  • Every frame is parsed — deliberately NO raw pre-gate: token-granular streaming interleaves framing bytes between identical characters, so a raw run-length arm would stay blind to exactly this incident shape. Structure-only frames cost a parse plus a short walk (~µs each); nothing they carry enters the counter.
  • NDJSON bodies (bare-newline JSON events, some OpenAI-compatible servers) are handled by a fallback line splitter that engages at 64KB with no blank-line separator present; unparseable tails are capped at 8MB and dropped.
  • The trip path reuses the existing contract verbatim: onTrip(verdict) → caller aborts upstream, downstream closes cleanly (EOF, not an error chunk), bytes forwarded up to the trip point byte-exact. New verdict reason "text-run"; detail carries maxTextRun + textRunChar.
  • Always on, zero new config (detector thresholds stay internal constants per 彻底整治环境变了滥用,除非必要都走配置文件 #2030 discipline).

Behavior change (old → new)

  • Old: single-codepoint repetition streams ran to max-output or manual abort; only ACP-tag enumeration and tool-call XML floods were cut intrinsically.
  • New: any decoded assistant text/thinking run of ≥200 identical codepoints aborts the upstream stream mid-flight and the downstream sees clean EOF. Affects every client routed through bili (including bili dsh, whose model traffic rides the proxy envs).
  • Why safe: threshold parity with pi's production value; legit prose/code tops out far below 200 consecutive identical codepoints (longest realistic separator lines ≈150); error direction is fail-safe (a miss, never a rewrite); forwarded bytes remain byte-exact.

Known limitations

  • Runs inside non-JSON bodies (neither SSE nor NDJSON shaped) are not detected — the guard covers the JSON wires bili speaks.
  • Cumulative snapshot payloads are intentionally not counted (double-count hazard): a provider delivering an entire turn ONLY as a typed snapshot event would not trip it.

Tests

16 new tests in tests/issue3260-text-run-guard.test.ts: core threshold / statefulness / reset / astral-codepoint / near-miss observability; wrap-level openai cross-frame trip with byte-exact prefix assertion; threshold−1 boundary; tool-argument exclusion (#1039); anthropic + responses lanes; snapshot skip; function_call_arguments exclusion; 3-byte-chunk boundary robustness; NDJSON fallback; 200KB legit-prose passthrough; 150-char separator margin; lane independence (acp flood still trips as acp-enumeration). All 15 pre-existing tests/runaway-guard.test.ts cases green locally (targeted run); full-suite verdict comes from CI on this head.

No config surface changes. No kernel/ or pi-subagents/ changes. npm run typecheck clean locally.

…guard (#3260)

Port pi's findDegenerateRuns minRun semantics (200) onto the existing intrinsic
stream terminator: a model stuck in a symbol loop (#3946: one symbol string for
~22 minutes) now trips the stream mid-generation instead of running to
max-output or manual abort.

Detection runs on DECODED model text only — per-frame JSON walk over text-family
keys (openai delta.content/reasoning_content, anthropic text_delta/thinking_delta,
responses top-level delta, google parts[].text). Tool-call arguments are excluded
(#1039 user intent), and Responses cumulative snapshot events (.done/.completed/
.incomplete) are skipped so runs are never double-counted. Every frame is parsed:
token-granular streaming interleaves framing bytes between identical characters, so
a raw run-length pre-gate would stay blind to the incident shape. NDJSON bodies
(bare-newline events) are handled by a fallback splitter; byte-exact forwarding and
the clean-EOF trip contract are unchanged. Always on, zero new config.

Also neutralizes the trip log wording in relay.ts (two sites): it said 'runaway
enumeration detected' even when the reason is now text-run.
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-15_output-text-run-guard (880d1cc)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3811-pr.3289.ca80c4d

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3289.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 这是 #3260 选项①(bili 加单 codepoint 连击 lane、默认打开)的唯一实现 PR;按主题/失败签名检索 open+closed issue 与近期 PR,无并列或重复条目。

审核独立性:以下结论由本 PR head 的 diff 独立重读产出(拉了 pr-3289,通读三处改动 + 本地定向验证),不是复述 #3260 侧的分析。

⓪ 方向关(审原 issue):#3260 病灶是「正在生成中的模型单符号复读」(形态②/#3946:一个符号串刷约 22 分钟)在经 bili 路由的客户端上没有输出侧熔断——现有守卫只认 ACP 标签枚举与 tool XML 洪水两种形态,纯符号流里没有 < 直接穿过。owner 已拍板选项①,本 PR 正是该拍板的落地:给现有 #2346 熔断机制族加第三条 lane。修的是机制缺失本身(第三个检测器不存在),不是掩盖症状。方向正确。

② 回归关(逐项清点既有行为,给证据):

  • 触发前逐字节原样转发 —— 保持(byte-exact 前缀断言钉住 deepEqual(out, Buffer.concat(parts.slice(0,5))));
  • 触发即干净 EOF(非错误块)—— 不变(controller.close()+reader.cancel() 路径逐字复用);
  • 两条既有 lane(acp-enumeration / tool-xml-flood)—— 逻辑逐字未变(仅重新缩进进 if(text.length>0)),lane 独立性测试确认 acp 洪水仍以 acp-enumeration 触发;
  • release v0.1.189 #2451 的 tool-xml gap 记账 —— 未动;空块行为等价(旧版对空串调 feedGapOnly 是 no-op,新版直接跳过,可观察结果一致);
  • 请求重建 / 会话状态 / 前缀缓存 —— 未触碰;
  • 日志措辞 runaway enumeration detected → output-side runaway guard tripped(relay.ts 两处)—— 仅日志层,grep 确认无代码/测试依赖旧串,且现在有三条 lane 后更准确;
  • 两种压缩模式(plugin/proxy)—— 守卫对模式无感(作用在原始上游响应流上),两个挂载点本就都包了它,新 lane 在守卫内部,两模式同等受益,无差异语义引入。

结论:无回归。 唯一实质变化=新增一条始终开启的中止路径 + 每帧 JSON 解析开销(见 Q3)。

③ 代码关(主关):

  • feedText 用 for..of 按 codepoint 迭代(astral emoji 算一个单位),状态化跨帧续接,阈值 200 对齐 pi 生产值——正确;
  • collectText 递归只喂 TEXT_LEAF_KEYS 叶子且在 EXCLUDED_BRANCHES 之外,排除位沿父键向下传播——我手工 trace 了 openai chat delta.content / anthropic text_delta.thinking / responses 顶层 delta(string) / google parts[].text 四种形状,均正确;tool_calls/function_call_arguments 子树整体排除([bug] 插件透传会剥掉工具调用参数里的 render tag(arguments 流挂了 tag-echo 过滤器) #1039);
  • Responses 累计快照(.done/.completed/.incomplete)与 function_call/custom_tool_call 事件跳过 → 防双重计数 + [bug] 插件透传会剥掉工具调用参数里的 render tag(arguments 流挂了 tag-echo 过滤器) #1039;
  • SSE 解析 CRLF 容忍、多行 data 拼接、[DONE]/非 JSON 容错;NDJSON 回退 ≥64KB 才启用(fail-safe:漏报不假报)、8MB 封顶防内存膨胀(正常流 frameBuf 只留最后一个未完成帧,有界);
  • 无 as any/@ts-ignore,常量全部具名,注释讲「为什么」不讲「是什么」,风格与原文件一致。typecheck(tsc --noEmit) 干净。

④ 测试关:17 例全是真测试(有牙)——byte-exact 前缀、threshold−1 边界(单元+跨帧)、#1039 工具参数排除(openai+responses 两处)、快照双重计数防护(.completed 单独不触发)、误报护栏(200KB 合法散文 passthrough 逐字节 + 150 字符分隔线余量)、lane 独立性、3 字节分片跨 decoder/frame 边界、NDJSON 回退、astral codepoint。这个 bug 回归时抓得住。本地定向跑新文件 + 既有 tests/runaway-guard.test.ts 共 31 例全绿;全量以 CI 为准。

⑤/⑤b diff 整洁度 + 代码整洁硬标准:3 个文件(主体 relay 守卫 + relay.ts 两行日志 + 新增测试),每一行都与本 PR 目的相关,无跑题文件、无空行/重排噪音、无 lockfile 连带。命名达意、无死代码、无魔法数字、边界处理完整、风格一致——外部提交从严清单全过。


结构化八问:

  1. 是什么问题:长会话模型陷入单符号复读(同一字符连击,形态②/[PR] storage: MySQL backend — step 2 of the pluggable-storage plan (#3931) #3946)时,经 bili 路由的客户端没有输出侧熔断——现有守卫只认 ACP 标签枚举与 tool XML 洪水,纯符号流没有 < 直接穿过,只能跑到 max-output 或人工中止。
  2. 什么原因导致的:生成侧缺「单 codepoint 连击」的内在规模终止器;pi 的 guard 挂在出站视图折叠上、DSH 挂载点拿不到该接缝,而 bili 中继正好压在原始响应流上是本仓库内天然拦截点。根因是第三条检测 lane 不存在,PR 补的是机制本身。
  3. 影响范围与严重程度:波及所有经 bili 路由的客户端(含 bili dsh)。严重度=特定条件下有问题——仅当解码后模型文本出现连续 ≥200 相同 codepoint 才中止,正常会话零影响;现状是这类会话白烧 token 到上限/人工介入。新增成本:每条流式响应每帧多一次 JSON 解析+短遍历(微秒级,<1% 回合延迟),对所有经 bili 客户端生效(不只 DSH)——这是避免对 token 级流式变盲的必要代价。误报面:某合法文本若真含 ≥200 连击(现实最长分隔线≈150)会中止该回合,阈值取 pi 生产值,风险低但非零。
  4. 怎么修复的:复用现有 【bug1】失控枚举无内在终止条件:单条消息 265 个标签,撞输出上限或需手动掐断 #2346 机制族加第三 lane——逐帧解析 JSON,只把文本族叶子喂进状态化 codepoint 计数器,≥200 触发即复用「中止上游+干净 EOF」;工具参数子树排除([bug] 插件透传会剥掉工具调用参数里的 render tag(arguments 流挂了 tag-echo 过滤器) #1039)、Responses 快照事件跳过防双重计数;因 token 级流式在相同字符间夹框架字节故不做原始字节预门、每帧都解析;NDJSON 载体走 64KB 回退。
  5. 是否有回归问题:无(② 已逐项清点并给证据)。唯一实质变化=新增一条始终开启的中止路径 + 每帧解析开销;日志措辞变更仅日志层。
  6. 是否破坏缓存:不涉及——作用在响应流出站流,不碰请求重建、前缀缓存、会话状态。
  7. 是否改了设计方向:不涉及——沿用现有输出侧熔断机制族,只增一条 lane,触发契约逐字复用,无架构转向。
  8. 是否引入新配置:不涉及——零配置,TEXT_RUN_THRESHOLD=200 为内部常量(彻底整治环境变了滥用,除非必要都走配置文件 #2030 纪律:检测器阈值不入配置面)。

判定:可合(待你批准合并)。 auto-merge 不适用——这是 feat(新增检测能力面)+ 共享 relay 热路径 + 行为变更(新增始终开启的中止路径),按 §7.4 需人审。本地定向证据:31/31 守卫测试绿 + typecheck 干净;全量以 CI 为准,红了我会定向跟进。(B) 插件层工作仍在 Tyan66666/billion-context-dsh#212 跟踪,dsh 源码按你的决定未动。

@ranxianglei
ranxianglei merged commit 6ee4a2a into master Oct 15, 2026
33 of 41 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.469-master.6ee4a2a (rolling master tag).

npm install -g billion-context@0.0.469-master.6ee4a2a  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

ranxianglei pushed a commit that referenced this pull request Oct 15, 2026
… guard

The fake-model growth filler "x".repeat(360) emits a run of 360 identical
codepoints per reply. Since #3289 merged the output-side runaway guard (#3260,
src/runaway-guard.ts: textRun >= TEXT_RUN_THRESHOLD = 200), every synthetic
reply is aborted mid-stream ("output-side runaway guard tripped (text-run;
maxTextRun:200)") and the client sees an empty response, so T5's "every
client turn must get a visible reply" fails deterministically on every
shard-2 CI lane (run 38777578495: ubu22-s2 / ubu24-s2 / win22-s2 / win24-s2).

Fix at the producer: "x ".repeat(180) — same 360 bytes per reply, max run
of identical codepoints is 1, so the fixture stops emitting the degenerate
symbol-loop shape the guard exists to catch (#3946). Guard untouched; no
runtime code changed.

Verified locally: tests/issue1086-chain-self-exemption.test.ts 21/21.
ranxianglei added a commit that referenced this pull request Oct 16, 2026
…ler-under-runaway-guard

修 CI 无故变红:#1086 T5 用例的 'x'×360 填充触发 #3289 输出护栏,含 #3289 的每棵树必挂
LoveDoLove pushed a commit to LoveDoLove-Forked-Projects/billion-context that referenced this pull request Oct 16, 2026
…-run lane (ranxianglei#3315 sub-defect 2)

The T5 liveness fake upstream echoed "x".repeat(360) per turn. ranxianglei#3289's
output-side runaway guard trips at TEXT_RUN_THRESHOLD=200 identical
codepoints (src/runaway-guard.ts), aborting every such stream — so each
turn lost its visible reply and the suite failed deterministically
(25 !== 60) on every tree containing ranxianglei#3289: master push run 38767006158
and PR ranxianglei#3275 run 38776693851, plus local repro (35 guard trips).

Mixed-char filler "xyz".repeat(120) keeps the identical 360-byte payload
size (all assertions depend on length/byte count only) while maxTextRun
drops to 1. Guard behavior itself is untouched — ranxianglei#3289 is intentional
production behavior; the fixture simply must not emit pathological runs.

Verified: node --import tsx --test tests/issue1086-chain-self-exemption.test.ts
-> 21/21 pass, 0 guard trips (was 35).
@github-actions github-actions Bot mentioned this pull request Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

degenerationGuard 移植到 DSH 版(billion-context-dsh):插件层方案已定待实现,另需上游输出侧熔断

1 participant