Skip to content

[PR] fix: refuse oversized degraded direct-sends using the lane known window (#4934) - #4940

Closed
ranxianglei wants to merge 4 commits into
masterfrom
2026-10-27_degraded-direct-oversize-guard
Closed

ranxianglei wants to merge 4 commits into
masterfrom
2026-10-27_degraded-direct-oversize-guard

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

What & why (root cause)

Field shape from #4941: {"code":"context_length_exceeded","message":"WorkBuddy: prompt is too long: 1866803 tokens > 1048576 maximum"}.

The attached log shows the compression engine working correctly for the whole session (all 79 logged /v1/chat/completions compressed, usage ~8–10%, max input well under the window). The failure happened at one instant: the proxy self-killed on the anti-orphan parent-gone watchdog (stale-heartbeat pids: 508) at 07:09:35, and at 07:09:56 a model request resolved no live proxy and went direct (uncompressed) — sending the full raw history (~1.87M context) to a 1M-window model. A replacement proxy only came up at 07:09:58 (~2s after the give-up).

Root cause (a client-side guard blind spot, not an engine bug): when the proxy is down, the request rides the degraded direct-send path in src/agent/native-intercept.ts (directSendOrRefuse). That path already carries the #4516 local oversize refusal, but it arms only when the request carries x-bili-plugin-context-window. On this path bili measures the host's original request, which never carries that header — bili stamps it only onto the proxy-rewritten request (headersFor, applied after the degraded decision). So the window was unknown → the doom check disarmed → the oversized history went straight out. The server-side egress fuse cannot help here because the proxy was down.

Fix

Expose the lane's own already-resolved window through a new optional state.resolveContextWindow hook on NativeInterceptState, consulted as a secondary source (after the request header) inside directSendOrRefuse. dsh wires it to its per-model runtime-info cache (effectiveContextWindow), which is warm for any session that ran — exactly the #4941 case.

Behavior change (disclosed)

Tests

New tests/issue4934-degraded-direct-oversize.test.ts (4 cases): lane-known window + oversized headerless body → local 422 (upstream never hit); body that fits the lane's window → direct (no false refusal); no resolver + no header → fail-open (preserves #4516); declared header wins over the resolver. Backward-compat verified locally green: issue4516-pi-direct-fallback, native-intercept, dsh-native. Full-suite verdict = remote CI.

Follow-ups (out of scope here)

Closes #4941

…n window (#4934)

When the proxy dies mid-session (anti-orphan parent-gone), the next model
request rides the degraded direct-send path in native-intercept.ts. That path
measures the host's ORIGINAL request, which never carries
x-bili-plugin-context-window (bili stamps that header only onto the
proxy-rewritten request), so the #4516 oversize guard disarmed and an oversized
raw history went straight to the provider — the #4934 field shape
("WorkBuddy: prompt is too long: 1866803 tokens > 1048576 maximum").

Expose the lane's own resolved window through a new optional
state.resolveContextWindow hook, consulted as a SECONDARY source after the
request header inside directSendOrRefuse. dsh wires it to its per-model
runtime-info cache (effectiveContextWindow). Unknown window keeps #4516's
fail-open (never a false refusal from missing data); the request's own declared
header still wins over the resolver.

Regression test: tests/issue4934-degraded-direct-oversize.test.ts.
@ranxianglei ranxianglei changed the title fix: refuse oversized degraded direct-sends using the lane known window (#4934) [PR] fix: refuse oversized degraded direct-sends using the lane known window (#4934) Oct 27, 2026
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-27_degraded-direct-oversize-guard (7c3c360)

Latest build — install this exact version

npm install -g billion-context@0.0.6468-pr.4940.7c2351f

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
7c3c360 billion-context@0.0.6468-pr.4940.7c2351f build
d0287b7 billion-context@0.0.6447-pr.4940.1cc41a7 build
74f7db1 billion-context@0.0.6431-pr.4940.5abe471 build
7be4a78 billion-context@0.0.6430-pr.4940.4e4fec0 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4940.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

补上首条回复里挂起的三项,全部落地:

1. CI(当前 head 74f7db109):全绿。 49 个 check-run = 36 success + 13 skipped,零 failure/cancelled。加上本地定向 178 例全绿 + typecheck/build 干净,测试关通过。

2. Closes 引用已修正。 已直接 PATCH PR 描述:全部 3 处 #4934(含 Closes 行与两处正文引用)改为 GitHub 真实编号 #4941(WorkBuddy bug report)。现在合并会正确关闭 bug issue,不会再误关 streamMarkers 那个无关 PR。

3. 与 GH #4894 交叉核对完毕(已读正文)。 同家族、同收口点:#4894 的触发路径「归因正常但代理未就绪」在 src/agent/native-intercept.ts:1466,同样走 directSendOrRefuse——本 PR 的二级窗口源让该路径的 doom 检查一并武装,#4894 的窗口守卫面已被本 PR 覆盖。但 #4894 还有两个本 PR 不碰的面,维持 open:(a) declareDshCompactionActive() 在就绪检查之前执行——bili 不可用时宿主自己的 compaction 已经关了,叠加直发 = 最坏组合;(b) 慢启动(schema-v8 backfill 等)超过 SPAWN_WAIT_MS 子进程被杀后「direct forever」。这两面建议留在 #4894 后续处理,不与本 PR 混。

最终判定:可合。 代码审核通过(八问见上一条)、CI 绿、Closes 引用已修。剩下的动作只有 owner 合并(agent 不代合);分支无冲突(mergeable=true)。watchdog stale-heartbeat 误杀(B1/A′)与 pi/opencode/omp 一行接线按原计划走后续。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: vllm-qwen/qwen3.8-27b)

Overlap notice from the #4894 lane: this PR and PR #4904 (opened ~5.5h earlier, Closes #4894) implement the same mechanism — an optional lane-window hook on NativeInterceptState consulted in directSendOrRefuse AFTER the request-header lookup, dsh wired to its effectiveContextWindow cache, fail-open preserved on miss, header precedence kept. Both rewrite the same comment block and the same line in src/agent/native-intercept.ts, so both cannot merge (textual conflict + a redundant second hook).

Convergence recommendation (first-come-first-served + test superset): merge #4904 and close this one as superseded. #4904's shared-intercept cases strictly cover this PR's four (lane-known + oversized headerless → 422; fits → direct; no resolver → fail-open; header wins), plus two more: media stage-2 exclusion (#4111 media-excluded floor math — avoids false refusals when the body carries large images) and two dsh hook-resolution unit tests (session model beats global default; unknown window → undefined fail-open). After the surviving PR lands, #4941/#4934 are resolved by the same code and should be linked/closed there.

CI state today: this head is fully green (36 success / 13 skipped / 0 failure); #4904's head shows zero failures but 39 concurrency-cancelled runs — a clean re-run is possible if needed. If you'd rather keep THIS PR as the survivor, say so in the thread and I will converge #4904 onto it (adopt your hook name where it is clearer) instead of the other way around.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

结论:同意收敛到 #4904——它存活,本 PR 按 superseded/duplicate 处理。 没有选「保留本 PR」,因为独立复核后确认 #4904 不弱于本 PR,而且有一处严格更强。

独立复核(拉取 #4904 head 973237f7f,对 master e4cc685 逐段 diff 并与本 PR 对照,未复用 overlap notice 的现成结论):

  1. 机制相同、同线冲突坐实:两者都是给 NativeInterceptState 加可选同步钩子,在 directSendOrRefuse 里头查找之后引用(头优先、undefined→fail-open、其他车道零变化);两者都重写 src/agent/native-intercept.ts 同一行(declaredWindowFromHeaders(...) ?? state.<hook>?.())和同一注释块——只能活一个。
  2. 测试超集成立:其 5 个 shared-intercept 用例与本 PR 4 例一一对应(not-ready+钩子+无头→本地 422 且上游零命中 / body 放得下→直发不误拒 / miss→fail-open / 声明头优先于钩子),外加:媒体 stage-2 排除用例(4MB base64 图 + ~2.75K 核心文本 → 200,把 DSH's unattributed over-limit gate prices base64 images as text (bytes/4) — media-heavy requests are falsely rejected wi | dsh 无归属超限门把 base64 图片当文本计价(bytes/4)——媒体重的请求被假拒 422,上游本会正常接受 #4111 两阶段数学经钩子路径钉住,防大图片会话被误拒)+ 2 个 dsh 钩子解析单测(会话模型压过全局默认;无服务→undefined fail-open)。
  3. 我额外找到的一处差异(利于 [PR] fix(#4894): arm the direct-fallback oversize guard on dsh not-ready sends #4904):其 dsh 接线是先 refreshModelInfo(register.base, target) 再读缓存(target = 会话 live 模型 ?? 全局默认,cooldown+dedup);本 PR 只读 effectiveContextWindow(ctx)。我核了 refreshModelInfo 全部调用点:激活期 warm-up 只焐热全局默认模型的 key,而 headersFor(唯一的每请求 warm 点)在 raw-URL not-ready/give-up 分支根本不执行——那正是本修复针对的场景(directSendOrRefuse 的 not-ready 分支不经过 headersFor)。后果:会话模型 ≠ 宿主全局默认、且代理自进程启动起从未就绪时,本 PR 的钩子对该模型永远 fail-open(缺陷原样保留),[PR] fix(#4894): arm the direct-fallback oversize guard on dsh not-ready sends #4904 会在首个降级请求触发缓存预热、第二个请求起守卫生效。[PR] fix(#4894): arm the direct-fallback oversize guard on dsh not-ready sends #4904 不是等价,是严格更强。
  4. 先来后到:[PR] fix(#4894): arm the direct-fallback oversize guard on dsh not-ready sends #4904 早开约 5.5 小时。其 head 上 39 个并发取消的 CI run 属表面问题(可重跑),不推翻以上。

已做的动作:

八问(精简版):

  1. 是什么问题:两个 PR 修同一个缺陷(降级直发路径的超窗护栏失能),同线冲突不能共存。
  2. 什么原因导致:[Bug][dsh] 0.1.208: attributed-but-not-ready direct send has no window guard, and takeover is declared before readiness — long sessions 400 with prompt_too_long | 归因正常但代理未就绪时直发无窗口护栏 + 就绪前就宣告接管 → 长会话必 400 #4894/[PR] feat(compress): add streamMarkers setting to suppress streaming receipts (#3906) #4934 两条线各自发现同一根因(护栏的窗口来源耦合在只有代理分支才会打的请求头上),两个 agent 独立实现到了同一行。
  3. 影响范围/严重程度:仅仓库调度面(一个冗余 PR 待关),用户零影响。
  4. 怎么处置的:收敛到 [PR] fix(#4894): arm the direct-fallback oversize guard on dsh not-ready sends #4904(更早 + 测试超集 + 覆盖冷缓存角);本 PR 标 duplicate、停推。
  5. 是否有回归问题:本次未推送任何新代码;[PR] fix(#4894): arm the direct-fallback oversize guard on dsh not-ready sends #4904 自身的回归审计见其线程 floor 5 的 review。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不变——沿用 NativeInterceptState 既有钩子模式。
  8. 是否引入新配置:无。

判定:本 PR 使命已完成(根因钉死、方向验证正确、CI 绿);收敛到 #4904 无损且有增益。建议 #4904 CI 重跑转绿后合并,随后关闭本 PR。注意 #4934 的看门狗根因问题(B1 误杀 vs A′ 真换)仍是独立残项、等你证据,不因本次收敛而消失。

@ranxianglei ranxianglei added the duplicate This issue or pull request already exists label Oct 27, 2026
ranxianglei pushed a commit that referenced this pull request Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

duplicate This issue or pull request already exists

Projects

None yet

1 participant