Repository navigation
[PR] fix(util): parse the window out of CJK context-overflow rejections - #3950
Conversation
…m can fire
Chinese-fronted relays (DeepSeek / Qwen / GLM / Moonshot and the
[OI]-compatible resellers between them) translate the provider's
context-overflow rejection into Chinese. CONTEXT_OVERFLOW_PATTERNS held 18
English markers only, so those 400s fell through as generic bad requests:
armOverflowShrink never ran, no shrink was armed, and the identical
oversized payload was re-forwarded on every retry while bili's meter stayed
frozen at its last (stale) reading — the client saw the same error forever.
Add five CJK markers that pair a CONTEXT subject with an exceed verb inside
one clause, so throttle phrasing (频率超限 / 请求过于频繁 / 触发限流) can
never match. parseOverflowWindow gains the CJK twin: the window is the
number attached to the limit word (上限 / 最大 / 限制), never the payload
size quoted earlier in the same sentence, and a trailing token unit is
required so a bare 上限 3 次 quota line cannot masquerade as a window.
Field repro (v0.1.202, DeepSeek-compatible relay, 5.0MiB outbound):
{"message":"请求上下文超限:本次请求约 1748713 tokens,超过单次上限
1500000 tokens。请缩短对话历史或开启上下文压缩后重试"}
before: isOverflow=false, window=undefined -> 400 loop, no arm
after: isOverflow=true, window=1500000 -> arm fires, fold happens
Review supplement for ranxianglei#3947 cause-5 (agreed in the ranxianglei#3947 thread): the CJK branch of parseOverflowWindow landed in this PR, but parseOverflowCounts stayed English-only — '本次请求约 1748713 tokens,超过单次上限 1500000 tokens' classifies as overflow yet yields no requestedTotal, while the limit number must never be read as the payload size. The branch reads the number adjacent to the overflow predicate (subject-anchored on 请求/输入, limit vocabulary between subject and number vetoes the guess) and books it as total+messages (a rejection bills input only, ranxianglei#3450). Both directions pinned: field repro → requestedTotal 1748713 with window still 1500000; ceiling phrasing ('请求上限为 N tokens,超出部分将被截断') → no requestedTotal, window N.
… this PR ranxianglei#3246 (Closes ranxianglei#3240) predates this PR and is now mergeable:false against master because ranxianglei#3787 landed the 上下文 direction it carried. After ranxianglei#3787 + this PR's (a)-(d), two of its patterns keep unique coverage: the input side named with an EXCEED predicate ('输入长度超出限制' — (c)'s length predicates do not reach it, and extending (c)'s subjects to 超出/超过/超限 would light up throttle phrasings like '请求超出频率限制') and the mixed-language 'prompt 过长'. Both are folded in here so the CJK detector surface lands in one PR; ranxianglei#3246's field-phrasing regression cases (b-c-pi#639) move into tests/cjk-context-overflow.test.ts alongside them.
33bbebd to
85a8166
Compare
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复 —— 但与 #3246 同机制族(都改 src/util.ts 的 CJK 溢出检测表 + 窗口/计数解析)。#3246 先到但基线已旧(#3787 落了上下文正逆对 → mergeable:false);本 PR 是 #3947 线程里批准开出的载体。处置(已执行):把 #3246 的残值唯一覆盖移植进本 PR、关闭 #3246 并在其帖内写明对照——内容零丢失,按「一个机制一个 PR」收口到本条目。 标签:bug · dsh · scope/wiring · p1(上游错误体分类属外部 API 边界;中文措辞下自愈完全失效 = 会话被 400 锁死至升级,按 owner 修订属 p1 会话不可恢复类;图像计量那半的完整严重度在 #3947 同标 p1)。 独立审核结论(非复读 issue 分析)方向关:#3947 cause-5 的代码层判断成立(捕获时 dsh profile 运行副本旧于 #3787 故分类没中;即便分类修好,窗口解析缺失让 arm 落 no-window 分支按 min(declared, payload estimate) 定尺)。本 PR 修在检测/提取决策点(根因层),不是症状遮蔽——方向正确。 代码关(逐行读过两个 hunk + 消费端):
我做的动作(已推送到本 PR 分支,独立标注 commit,作者署名不变):
验证证据:rebase 后本地定向 59/59( 结构化八问
行为变更披露(旧 → 新)
判定可合并:方向正确、修在根因层、回归审计 0 处、定向 59/59 + tsc clean、已 rebase 到最新 master、行为变更已披露。等新 head CI 绿后请你合入(合并按规矩留给人)。 |
(ai: vllm-qwen/qwen3.8-27b) 状态补记(完整审核结论见上一条,不重复):新 head |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.692-master.3450c55 # this exact build
npm install -g billion-context@master # always the newest merged build
|
What this changes
parseOverflowWindow(src/util.ts:301) was English-only.#3787/#3792taughtCONTEXT_OVERFLOW_PATTERNSto classify the 上下文 family of Chinese rejections, but the window parser was left behind, so a recognised Chinese body still yieldswindow === undefined.That matters because of what
armOverflowShrinkdoes with an unparsed window (src/server/relay.ts):With no window, the arm is sized by the rejected payload's own estimate — a number larger than the cap it is trying to get under. When the relay states the cap in plain text (
超过单次上限 1500000 tokens), that statement is strictly better evidence than the estimate, and today it is thrown away.Two additions, both narrow:
parseOverflowWindow: a CJK branch that reads the number attached to the limit word (上限 / 最大 / 限制), never the payload size quoted elsewhere in the same sentence.CONTEXT_OVERFLOW_PATTERNS: four markers for the phrasings#3787still misses — 上下文 described as 已满/过长/超长/太大, the ceiling named 模型上限 / 上下文窗口, input-side subjects (输入过长/提示词太长/请求过长/对话太长/消息超长), and the token counter as subject (输入 token 数量超出允许范围).The field repro this comes from
1748713) precedes the limit number (1500000), so a naive(\d+) tokenswould read the wrong one. The branch reads1500000.#3787merged this body is now classified correctly, so the arm fires — but atmin(declared, defaultCountTokens(rawBody))≈ the payload size, above the cap. The three-400loop it came from is in the linked issue.Ordering is load-bearing
The trailing form is tried first:
A relay that writes both numbers in the trailing shape (
已突破 200000 tokens 的上限,本次请求约 240000 tokens) would otherwise have the leading form reach past the limit word and grab the payload quoted after it. Both directions are pinned by tests (tests/cjk-context-overflow.test.ts).A trailing unit (
tokens/token/个token) is required either way, so a bare rate-limit line (本分钟调用上限 3 次) cannot masquerade as a window.Both directions
A false positive arms a shrink against a payload that fits, so every gate ships its negative twin (#3617).
tests/cjk-context-overflow.test.tshas 18 cases: 9 positive (field repro, 模型上限, trailing 的上限, JSON envelope, 502 relay, 413, input-side phrasing, token-side phrasing, 请求/对话/消息) and 9 negative (频率超限, 触发限流, 请求过于频繁, 余额不足, 参数错误, 服务暂时不可用 5xx, 输出内容过长, status 200, 上限 3 次).Relationship to #3246
#3246covers the same area and is currentlymergeable: falseagainstmaster—#3787has since landed the same 上下文 direction, so the two overlap. This PR deliberately does not touchtests/context-overflow.test.ts(the file#3246edits) and keeps its cases in a separate file, so either can land first. The CJK window parse is the piece#3246's own body calls a leftover: "错误正文不带数字时仍学不到精确窗口(遗留项…)".Verification
The single failure is pre-existing and unrelated:
side-request-isolation.test.ts→EBUSY: resource busy or locked, unlink '…\Temp\bili-side-arm-<rand>\index.db'on Windows. It reproduces onmasterwithout this branch (verified by re-running the suite alone, twice, with different random temp dirs) — a SQLite handle released after the test's cleanup.npx tsc --noEmitis clean afternpm --prefix kernel run buildandnpm --prefix pi-subagents run build(both required before typecheck in a fresh checkout).Closes #3947