Repository navigation
[PR] fix(#4894): arm the direct-fallback oversize guard on dsh not-ready sends - #4904
Conversation
…ends The shared guard in directSendOrRefuse (#4516) reads its window from the stamped context-window header — but that header is only stamped by the proxy branch, so a request that finds no live proxy (slow boot past the spawn budget, #4609) carried none and the guard stayed disarmed in exactly the state where an uncompressed egress most likely blows the window: long dsh sessions 400 prompt_too_long for the whole process lifetime. Add an optional declaredWindowForDirect hook to NativeInterceptState, consulted after the header lookup at every fall-back site; dsh arms it from its own cached model info (session model over global default, the #2381/#2924 resolution headersFor stamps), failing open on any miss. Other lanes leave it unset — byte-identical behavior. Behavior change (dsh lane only): an oversized model request sent while the proxy is not ready now fails locally with a 422 (oversize-refusal-fallback) instead of egressing uncompressed and taking an upstream 400 prompt_too_long.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.6367-pr.4904.9aec8a8This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4904.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 查重: 不重复——本 PR 是 #4894 Part 2(直发护栏侧)的实现交付物( 标签: bug / dsh / scope/wiring / p1(dsh 车道长会话慢启动后每轮必上游 400、进程生命周期内不可用——按「用户不能用」定 p1;请求管道/协议接线面)。已打好。 收到,开始独立 review:拉分支核代码、对原 issue 方向、逐项回归清点、看 CI 状态,结论按八问回。 |
|
(ai: vllm-qwen/qwen3.8-27b) Merge-guard sync done: merged origin/master (e4cc685) into |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Merge guard 同步完成:把 origin/master(e4cc68575)合入分支 2026-10-27_direct-window-guard-4894,零冲突(无需设计决策),已推送 973237f。PR 状态 behind → blocked(剩余门禁只有 CI/review);推送时点 guard/static 各 lane 全绿、test shards 运行中。本地已做定向验证:tests/dsh-native.test.ts + tests/issue4894-direct-window-guard.test.ts 共 75/75 通过,typecheck 干净——master 自分叉后动过 dsh-native.ts(#4825/#4755/#4811/#4848),故做了语义级核对。结果评论已留:#4904 (comment) |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 这条是我刚才用 API 发的结果评论被同步层镜像回来的(作者显示为账号本身),不是新指令,无需处理。当前状态:分支已同步 master 并推送(973237f7f),behind 已清除,等 CI test shards 跑完即可合并;CI 结果出来前我这边没有待办。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.887-master.9b6ec23 # this exact build
npm install -g billion-context@master # always the newest merged build
|
…t — merge-guard sync Master advanced 7f2e119 -> 9b6ec23 (+5 commits: version 0.1.208 #4980; dsh direct-fallback oversize guard #4904/#4894 in dsh-native.ts/native-intercept.ts + tests). Clean auto-merge, zero conflicts (delta touches no PR-area or prior-conflict file). PR change intact: src/server.ts zero-fold calibrated-fit predicate (#4930) + tests/issue4930-zero-fold-calibrated-fit.test.ts. Typecheck clean; targeted PR/preflight/yield suites 48/48 green.
Closes #4894
What breaks. In the dsh native lane, when attribution is ok but the proxy is not ready (slow boot past the spawn budget — e.g. #4609's O(index.db) open), model requests fall back to a direct send. The shared oversize guard added by #4516/#4546 reads its window from the
x-bili-plugin-context-windowheader — but that header is stamped only by the proxy branch (headersFor), so on the not-ready path it is absent and the guard skips measuring entirely. A long session then egresses uncompressed and takes an upstream400 prompt_too_longon every turn for the whole process lifetime.Fix. One optional hook on
NativeInterceptState:declaredWindowForDirect?: () => number | undefined— consulted bydirectSendOrRefuseAFTER the header lookup, at every fall-back site (not-ready, respawn give-up, unattributed direct). Header wins when present (request-specific); the hook answers from lane state synchronously;undefined→ skip measuring, byte-identical to today (fail-open preserved, no body buffering for windowless lanes).headersForstamps (DSH插件按「全局默认模型」上报上下文窗口,导致会话真实模型匹配不到窗口、回落默认200K #2381 session live model first, host global default fallback, [dsh-native] takeover-gate refusal (threw state) fails open with the FULL uncompressed transcript; a >window local history dies upstream with 400 (maximum context length) #2924 cached-only window), minus the tools/attribution gates those serve plugin-mode stamping. A miss anywhere fails open.Behavior change (dsh lane only, intended): an oversized model request sent while the proxy is not ready now fails locally with a 422 (
x-bili-gate: oversize-refusal-fallback, wording distinguishes bootstrap-failed vs dead vs busy-exhausted vialastUnavailableReason) instead of egressing uncompressed and receiving an upstream400 prompt_too_long. Old: egress + upstream 400 + retry loop re-sending the same history. New: instant local refusal with an actionable message; the turn can succeed as soon as the proxy recovers.Scope note. Part 1 of #4894 (takeover declared before readiness — one-way verdict global) is a design decision (preset config-resolution timing interacts with declaration timing, double-compression risk #1729) and will be tracked as a separate p1 decision-request issue after this PR. This PR fixes the guard side only.
Tests. New
tests/issue4894-direct-window-guard.test.ts(5 cases: main repro not-ready+hook+no-header→422; fail-open on miss; fits-window direct; media stage-2 exclusion via hook; header-beats-hook precedence). Two dsh-native cases: session-model-beats-global and unknown-window-fails-open via_nativeStateForTest().declaredWindowForDirect. Targeted suites green locally: issue4894 5/5, native-intercept 66/66, native-intercept-selfheal 22/22, issue4516-pi-direct-fallback 11/11, dsh-native #4894 pattern 2/2; typecheck clean. Full suite = CI.Note: open PR #4878 also edits
directSendOrRefuse(hop hygiene, sits after the local-refusal paths) — if it merges first this branch needs a quick rebase.