Skip to content

[PR] fix(util): parse the window out of CJK context-overflow rejections - #3950

Merged
ranxianglei merged 3 commits into
ranxianglei:masterfrom
DadayaSuwen:2026-10-19_cjk-context-overflow
Oct 20, 2026
Merged

ranxianglei merged 3 commits into
ranxianglei:masterfrom
DadayaSuwen:2026-10-19_cjk-context-overflow

Conversation

@DadayaSuwen

Copy link
Copy Markdown

What this changes

parseOverflowWindow (src/util.ts:301) was English-only. #3787 / #3792 taught CONTEXT_OVERFLOW_PATTERNS to classify the 上下文 family of Chinese rejections, but the window parser was left behind, so a recognised Chinese body still yields window === undefined.

That matters because of what armOverflowShrink does with an unparsed window (src/server/relay.ts):

const arm = Math.max(0, Math.min(declared, Number.isFinite(est) ? est : declared));
// log: "upstream context overflow (window not parseable, model=…) — armed
//       emergency shrink at ~N tokens (min of declared N and payload estimate),
//       nothing learned (#987)"

With no window, the arm is sized by the rejected payload's own estimate — a number larger than the cap it is trying to get under. When the relay states the cap in plain text (超过单次上限 1500000 tokens), that statement is strictly better evidence than the estimate, and today it is thrown away.

Two additions, both narrow:

  1. parseOverflowWindow: a CJK branch that reads the number attached to the limit word (上限 / 最大 / 限制), never the payload size quoted elsewhere in the same sentence.
  2. CONTEXT_OVERFLOW_PATTERNS: four markers for the phrasings #3787 still misses — 上下文 described as 已满/过长/超长/太大, the ceiling named 模型上限 / 上下文窗口, input-side subjects (输入过长 / 提示词太长 / 请求过长 / 对话太长 / 消息超长), and the token counter as subject (输入 token 数量超出允许范围).

The field repro this comes from

请求上下文超限:本次请求约 1748713 tokens,超过单次上限 1500000 tokens。请缩短对话历史或开启上下文压缩后重试(请求未发送到上游,未计费)。
  • The payload number (1748713) precedes the limit number (1500000), so a naive (\d+) tokens would read the wrong one. The branch reads 1500000.
  • With #3787 merged this body is now classified correctly, so the arm fires — but at min(declared, defaultCountTokens(rawBody)) ≈ the payload size, above the cap. The three-400 loop it came from is in the linked issue.

Ordering is load-bearing

The trailing form is tried first:

m =
    text.match(/(\d[\d,]*)\s*(?:个\s*)?tokens?\s*(?:的)?(?:上限|最大|限制)/i) ??
    text.match(/(?:上限|最大(?:值|限制)?|限制)[^。;;!!\n]{0,20}?(\d[\d,]*)\s*(?:个\s*)?tokens?/i);

A relay that writes both numbers in the trailing shape (已突破 200000 tokens 的上限,本次请求约 240000 tokens) would otherwise have the leading form reach past the limit word and grab the payload quoted after it. Both directions are pinned by tests (tests/cjk-context-overflow.test.ts).

A trailing unit (tokens / token / 个token) is required either way, so a bare rate-limit line (本分钟调用上限 3 次) cannot masquerade as a window.

Both directions

A false positive arms a shrink against a payload that fits, so every gate ships its negative twin (#3617). tests/cjk-context-overflow.test.ts has 18 cases: 9 positive (field repro, 模型上限, trailing 的上限, JSON envelope, 502 relay, 413, input-side phrasing, token-side phrasing, 请求/对话/消息) and 9 negative (频率超限, 触发限流, 请求过于频繁, 余额不足, 参数错误, 服务暂时不可用 5xx, 输出内容过长, status 200, 上限 3 次).

Relationship to #3246

#3246 covers the same area and is currently mergeable: false against master — #3787 has since landed the same 上下文 direction, so the two overlap. This PR deliberately does not touch tests/context-overflow.test.ts (the file #3246 edits) and keeps its cases in a separate file, so either can land first. The CJK window parse is the piece #3246's own body calls a leftover: "错误正文不带数字时仍学不到精确窗口(遗留项…)".

Verification

node --import tsx --test tests/cjk-context-overflow.test.ts
# tests 18 / pass 18 / fail 0

node --import tsx --test tests/context-overflow.test.ts tests/cjk-context-overflow.test.ts \
  tests/issue2484-overflow-and-summary.test.ts tests/issue1195-overflow-refold.test.ts \
  tests/fix-arm-headroom-clamp.test.ts tests/side-request-isolation.test.ts
# tests 90 / pass 89 / fail 1

The single failure is pre-existing and unrelated: side-request-isolation.test.ts → EBUSY: resource busy or locked, unlink '…\Temp\bili-side-arm-<rand>\index.db' on Windows. It reproduces on master without this branch (verified by re-running the suite alone, twice, with different random temp dirs) — a SQLite handle released after the test's cleanup.

npx tsc --noEmit is clean after npm --prefix kernel run build and npm --prefix pi-subagents run build (both required before typecheck in a fresh checkout).

Closes #3947

@ranxianglei ranxianglei changed the title fix(util): parse the window out of CJK context-overflow rejections [PR] fix(util): parse the window out of CJK context-overflow rejections Oct 20, 2026
DadayaSuwen and others added 3 commits October 20, 2026 23:31
…m can fire

Chinese-fronted relays (DeepSeek / Qwen / GLM / Moonshot and the
[OI]-compatible resellers between them) translate the provider's
context-overflow rejection into Chinese. CONTEXT_OVERFLOW_PATTERNS held 18
English markers only, so those 400s fell through as generic bad requests:
armOverflowShrink never ran, no shrink was armed, and the identical
oversized payload was re-forwarded on every retry while bili's meter stayed
frozen at its last (stale) reading — the client saw the same error forever.

Add five CJK markers that pair a CONTEXT subject with an exceed verb inside
one clause, so throttle phrasing (频率超限 / 请求过于频繁 / 触发限流) can
never match. parseOverflowWindow gains the CJK twin: the window is the
number attached to the limit word (上限 / 最大 / 限制), never the payload
size quoted earlier in the same sentence, and a trailing token unit is
required so a bare 上限 3 次 quota line cannot masquerade as a window.

Field repro (v0.1.202, DeepSeek-compatible relay, 5.0MiB outbound):
  {"message":"请求上下文超限:本次请求约 1748713 tokens,超过单次上限
   1500000 tokens。请缩短对话历史或开启上下文压缩后重试"}
  before: isOverflow=false, window=undefined -> 400 loop, no arm
  after:  isOverflow=true,  window=1500000  -> arm fires, fold happens
Review supplement for ranxianglei#3947 cause-5 (agreed in the ranxianglei#3947 thread): the CJK
branch of parseOverflowWindow landed in this PR, but parseOverflowCounts
stayed English-only — '本次请求约 1748713 tokens,超过单次上限 1500000
tokens' classifies as overflow yet yields no requestedTotal, while the
limit number must never be read as the payload size. The branch reads the
number adjacent to the overflow predicate (subject-anchored on 请求/输入,
limit vocabulary between subject and number vetoes the guess) and books it
as total+messages (a rejection bills input only, ranxianglei#3450). Both directions
pinned: field repro → requestedTotal 1748713 with window still 1500000;
ceiling phrasing ('请求上限为 N tokens,超出部分将被截断') → no
requestedTotal, window N.
… this PR

ranxianglei#3246 (Closes ranxianglei#3240) predates this PR and is now mergeable:false against
master because ranxianglei#3787 landed the 上下文 direction it carried. After ranxianglei#3787 +
this PR's (a)-(d), two of its patterns keep unique coverage: the input side
named with an EXCEED predicate ('输入长度超出限制' — (c)'s length predicates
do not reach it, and extending (c)'s subjects to 超出/超过/超限 would light
up throttle phrasings like '请求超出频率限制') and the mixed-language
'prompt 过长'. Both are folded in here so the CJK detector surface lands in
one PR; ranxianglei#3246's field-phrasing regression cases (b-c-pi#639) move into
tests/cjk-context-overflow.test.ts alongside them.
@ranxianglei
ranxianglei force-pushed the 2026-10-19_cjk-context-overflow branch from 33bbebd to 85a8166 Compare October 20, 2026 15:35
@ranxianglei ranxianglei added bug Something isn't working dsh p2 scope/wiring labels Oct 20, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复 —— 但与 #3246 同机制族(都改 src/util.ts 的 CJK 溢出检测表 + 窗口/计数解析)。#3246 先到但基线已旧(#3787 落了上下文正逆对 → mergeable:false);本 PR 是 #3947 线程里批准开出的载体。处置(已执行):把 #3246 的残值唯一覆盖移植进本 PR、关闭 #3246 并在其帖内写明对照——内容零丢失,按「一个机制一个 PR」收口到本条目。

标签:bug · dsh · scope/wiring · p1(上游错误体分类属外部 API 边界;中文措辞下自愈完全失效 = 会话被 400 锁死至升级,按 owner 修订属 p1 会话不可恢复类;图像计量那半的完整严重度在 #3947 同标 p1)。

独立审核结论(非复读 issue 分析)

方向关:#3947 cause-5 的代码层判断成立(捕获时 dsh profile 运行副本旧于 #3787 故分类没中;即便分类修好,窗口解析缺失让 arm 落 no-window 分支按 min(declared, payload estimate) 定尺)。本 PR 修在检测/提取决策点(根因层),不是症状遮蔽——方向正确。

代码关(逐行读过两个 hunk + 消费端):

我做的动作(已推送到本 PR 分支,独立标注 commit,作者署名不变):

  1. 8b811c5 — parseOverflowCounts 补 CJK 分支(Image-heavy sessions under-count the outbound payload by ~19×, never arm a shrink, and re-forward the same 5 MiB body on every retry #3947 线程约定项:「本次请求约 N tokens」→ requestedTotal/requestedMessages;主语锚定 请求/输入 + 区间含限制词汇即否决,「请求上限为 N tokens,超出部分将被截断」不会误读成载荷尺寸)。该字段被 PR A 的拒绝教学直接消费,放这里避免两个 PR 在 util.ts 上互相踩。
  2. 85a8166 — [PR] fix: recognize Chinese relay overflow wording in the context-overflow detector (#3240) #3246 残值两条 pattern(输入侧 EXCEED 谓词「输入长度超出限制」;混合语言「prompt 过长」)连同 b-c-pi#639 回归用例移植。
  3. rebase 到最新 master(原落后 196 提交,无文本冲突)。

验证证据:rebase 后本地定向 59/59(cjk-context-overflow 22 + context-overflow 34 + issue2484 3)、tsc --noEmit clean(kernel/pi-subagents 重建后);原 head 09c1ae5 的远程 CI 执行 lane 全绿(40 runs,skipped 均为门控 lane);新 head 85a8166c6 的 CI 运行中,绿了即可合。

结构化八问

  1. 是什么问题:中文中转站的溢出报错要么不被判为溢出、要么判了却读不出窗口/载荷尺寸 → 紧急收缩与同请求 refold 不触发或按错尺寸武装,会话每轮重发同一个超窗 body 反复吃 400。
  2. 什么原因导致的:标记表只有英文措辞(Context Overflow | 上下文溢出 #3787 后仅上下文正逆对)+ 两个数字解析器无 CJK 分支——标记表覆盖不全 + 提取覆盖不全;检测点本身没错。
  3. 影响范围与严重程度:走国内中转站、payload 超窗的用户面;触发前完全卡死(同一 400 循环、需升级或人工裁剪历史才恢复),英文 provider 零影响。本 PR 覆盖检测器半;图像主导载荷的完整回路另需 PR A/B(Image-heavy sessions under-count the outbound payload by ~19×, never arm a shrink, and re-forward the same 5 MiB body on every retry #3947 跟踪)。
  4. 怎么修复的:标记表扩充(单句内主语-谓词配对、方向词强制)+ 两个解析器各加 CJK 分支——窗口读绑 limit 词的数、计数读邻接谓词的数,两者互斥且都要求 token 单位。
  5. 是否有回归问题:逐项审过既有行为——① 英文 pattern 全部原样保留、追加表尾、.some() 语义不变;② status 门禁(400/413/5xx)未动,裸 5xx 无标记仍非溢出;③ 超窗拒答以 5xx 返回时出现探测死循环;外部摘要链路三处静默/上限问题(含文件:行号与本地验证) #2484「5xx+标记才算溢出」由新用例再钉一层;④ Bedrock "too many tokens" 限流排除立场未被破坏(反例含频率超限/触发限流/过于频繁);⑤ 新增两条 pattern 不改变任何既有正例的行为(旧用例全绿)。无意回归 0 处。
  6. 是否破坏缓存:不涉及——只改错误分类与数字提取,不碰缓存键/重试策略/增量游标/去重键。
  7. 是否改了设计方向:不涉及——同一检测点、同一武装机制,标记账本扩充(wire-contract ledger only grows 同精神);context window problem continues #987「窗口归声明、永不从流量学习」维持——解析出的数只用于一次性 arm/refold,不持久化。
  8. 是否引入新配置:不涉及。

行为变更披露(旧 → 新)

  • 旧:带中文溢出措辞的 400/413/5xx → 判非溢出(Context Overflow | 上下文溢出 #3787 后部分措辞可判但 window=undefined → arm 走 no-window 分支,按 min(declared, payload estimate) 定尺)。
  • 新:同类措辞 → 判溢出 → 窗口解析出绑 limit 词的数(字段复现 → 1500000)、计数解析出自报载荷尺寸(→ 1748713)→ arm 于 min(stated, judgingLimit) + 尝试同请求 refold(使用pi 压缩后就提示prompt is too long #1195)。英文行为零变化。

判定

可合并:方向正确、修在根因层、回归审计 0 处、定向 59/59 + tsc clean、已 rebase 到最新 master、行为变更已披露。等新 head CI 绿后请你合入(合并按规矩留给人)。

@ranxianglei ranxianglei added the p1 label Oct 20, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

状态补记(完整审核结论见上一条,不重复):新 head 85a8166c6 的 CI 仍在排队等 runner(test 分片未开跑),已完成的 lane(static-gates / kernel-guard / config-surface-guard / docs 系等)无失败。本地定向验证已全绿:59/59 + tsc clean。CI 绿后即可合入;若红了我会在本帖跟进排查。

@ranxianglei
ranxianglei merged commit 3450c55 into ranxianglei:master Oct 20, 2026
72 of 150 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.692-master.3450c55 (rolling master tag).

npm install -g billion-context@0.0.692-master.3450c55  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Image-heavy sessions under-count the outbound payload by ~19×, never arm a shrink, and re-forward the same 5 MiB body on every retry

2 participants