Repository navigation
[PR] fix: refuse oversized degraded direct-sends using the lane known window (#4934) - #4940
ranxianglei wants to merge 4 commits into
Conversation
…n window (#4934) When the proxy dies mid-session (anti-orphan parent-gone), the next model request rides the degraded direct-send path in native-intercept.ts. That path measures the host's ORIGINAL request, which never carries x-bili-plugin-context-window (bili stamps that header only onto the proxy-rewritten request), so the #4516 oversize guard disarmed and an oversized raw history went straight to the provider — the #4934 field shape ("WorkBuddy: prompt is too long: 1866803 tokens > 1048576 maximum"). Expose the lane's own resolved window through a new optional state.resolveContextWindow hook, consulted as a SECONDARY source after the request header inside directSendOrRefuse. dsh wires it to its per-model runtime-info cache (effectiveContextWindow). Unknown window keeps #4516's fail-open (never a false refusal from missing data); the request's own declared header still wins over the resolver. Regression test: tests/issue4934-degraded-direct-oversize.test.ts.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.6468-pr.4940.7c2351fThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4940.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 补上首条回复里挂起的三项,全部落地: 1. CI(当前 head 2. 3. 与 GH #4894 交叉核对完毕(已读正文)。 同家族、同收口点:#4894 的触发路径「归因正常但代理未就绪」在 最终判定:可合。 代码审核通过(八问见上一条)、CI 绿、Closes 引用已修。剩下的动作只有 owner 合并(agent 不代合);分支无冲突( |
|
(ai: vllm-qwen/qwen3.8-27b) Overlap notice from the #4894 lane: this PR and PR #4904 (opened ~5.5h earlier, Closes #4894) implement the same mechanism — an optional lane-window hook on NativeInterceptState consulted in directSendOrRefuse AFTER the request-header lookup, dsh wired to its effectiveContextWindow cache, fail-open preserved on miss, header precedence kept. Both rewrite the same comment block and the same line in src/agent/native-intercept.ts, so both cannot merge (textual conflict + a redundant second hook). Convergence recommendation (first-come-first-served + test superset): merge #4904 and close this one as superseded. #4904's shared-intercept cases strictly cover this PR's four (lane-known + oversized headerless → 422; fits → direct; no resolver → fail-open; header wins), plus two more: media stage-2 exclusion (#4111 media-excluded floor math — avoids false refusals when the body carries large images) and two dsh hook-resolution unit tests (session model beats global default; unknown window → undefined fail-open). After the surviving PR lands, #4941/#4934 are resolved by the same code and should be linked/closed there. CI state today: this head is fully green (36 success / 13 skipped / 0 failure); #4904's head shows zero failures but 39 concurrency-cancelled runs — a clean re-run is possible if needed. If you'd rather keep THIS PR as the survivor, say so in the thread and I will converge #4904 onto it (adopt your hook name where it is clearer) instead of the other way around. |
(ai: vllm-qwen/qwen3.8-27b) 结论:同意收敛到 #4904——它存活,本 PR 按 superseded/duplicate 处理。 没有选「保留本 PR」,因为独立复核后确认 #4904 不弱于本 PR,而且有一处严格更强。 独立复核(拉取 #4904 head
已做的动作:
八问(精简版):
判定:本 PR 使命已完成(根因钉死、方向验证正确、CI 绿);收敛到 #4904 无损且有增益。建议 #4904 CI 重跑转绿后合并,随后关闭本 PR。注意 #4934 的看门狗根因问题(B1 误杀 vs A′ 真换)仍是独立残项、等你证据,不因本次收敛而消失。 |
…h a sparse-padded store (#4940)
What & why (root cause)
Field shape from #4941:
{"code":"context_length_exceeded","message":"WorkBuddy: prompt is too long: 1866803 tokens > 1048576 maximum"}.The attached log shows the compression engine working correctly for the whole session (all 79 logged
/v1/chat/completionscompressed, usage ~8–10%, max input well under the window). The failure happened at one instant: the proxy self-killed on the anti-orphanparent-gonewatchdog (stale-heartbeat pids: 508) at07:09:35, and at07:09:56a model request resolved no live proxy and went direct (uncompressed) — sending the full raw history (~1.87M context) to a 1M-window model. A replacement proxy only came up at07:09:58(~2s after the give-up).Root cause (a client-side guard blind spot, not an engine bug): when the proxy is down, the request rides the degraded direct-send path in
src/agent/native-intercept.ts(directSendOrRefuse). That path already carries the #4516 local oversize refusal, but it arms only when the request carriesx-bili-plugin-context-window. On this path bili measures the host's original request, which never carries that header — bili stamps it only onto the proxy-rewritten request (headersFor, applied after the degraded decision). So the window was unknown → the doom check disarmed → the oversized history went straight out. The server-side egress fuse cannot help here because the proxy was down.Fix
Expose the lane's own already-resolved window through a new optional
state.resolveContextWindowhook onNativeInterceptState, consulted as a secondary source (after the request header) insidedirectSendOrRefuse. dsh wires it to its per-model runtime-info cache (effectiveContextWindow), which is warm for any session that ran — exactly the #4941 case.Behavior change (disclosed)
context_length_exceeded.kernel/change.Tests
New
tests/issue4934-degraded-direct-oversize.test.ts(4 cases): lane-known window + oversized headerless body → local 422 (upstream never hit); body that fits the lane's window → direct (no false refusal); no resolver + no header → fail-open (preserves #4516); declared header wins over the resolver. Backward-compat verified locally green:issue4516-pi-direct-fallback,native-intercept,dsh-native. Full-suite verdict = remote CI.Follow-ups (out of scope here)
windowsmap keyed byprovider/model, stamped only while a proxy is live), whereasresolveContextWindowhere is a single-value session-window accessor — sufficient for dsh, whose current session model resolves synchronously. Generalizing to those lanes needs a request-aware resolver (passing the target model): a deliberate interface change tracked separately, not bundled into this stop-bleed.Closes #4941