Skip to content

[PR] fix(#4894): arm the direct-fallback oversize guard on dsh not-ready sends - #4904

Merged
ranxianglei merged 2 commits into
masterfrom
2026-10-27_direct-window-guard-4894
Oct 27, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-10-27_direct-window-guard-4894

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #4894

What breaks. In the dsh native lane, when attribution is ok but the proxy is not ready (slow boot past the spawn budget — e.g. #4609's O(index.db) open), model requests fall back to a direct send. The shared oversize guard added by #4516/#4546 reads its window from the x-bili-plugin-context-window header — but that header is stamped only by the proxy branch (headersFor), so on the not-ready path it is absent and the guard skips measuring entirely. A long session then egresses uncompressed and takes an upstream 400 prompt_too_long on every turn for the whole process lifetime.

Fix. One optional hook on NativeInterceptState:

Behavior change (dsh lane only, intended): an oversized model request sent while the proxy is not ready now fails locally with a 422 (x-bili-gate: oversize-refusal-fallback, wording distinguishes bootstrap-failed vs dead vs busy-exhausted via lastUnavailableReason) instead of egressing uncompressed and receiving an upstream 400 prompt_too_long. Old: egress + upstream 400 + retry loop re-sending the same history. New: instant local refusal with an actionable message; the turn can succeed as soon as the proxy recovers.

Scope note. Part 1 of #4894 (takeover declared before readiness — one-way verdict global) is a design decision (preset config-resolution timing interacts with declaration timing, double-compression risk #1729) and will be tracked as a separate p1 decision-request issue after this PR. This PR fixes the guard side only.

Tests. New tests/issue4894-direct-window-guard.test.ts (5 cases: main repro not-ready+hook+no-header→422; fail-open on miss; fits-window direct; media stage-2 exclusion via hook; header-beats-hook precedence). Two dsh-native cases: session-model-beats-global and unknown-window-fails-open via _nativeStateForTest().declaredWindowForDirect. Targeted suites green locally: issue4894 5/5, native-intercept 66/66, native-intercept-selfheal 22/22, issue4516-pi-direct-fallback 11/11, dsh-native #4894 pattern 2/2; typecheck clean. Full suite = CI.

Note: open PR #4878 also edits directSendOrRefuse (hop hygiene, sits after the local-refusal paths) — if it merges first this branch needs a quick rebase.

…ends

The shared guard in directSendOrRefuse (#4516) reads its window from the
stamped context-window header — but that header is only stamped by the
proxy branch, so a request that finds no live proxy (slow boot past the
spawn budget, #4609) carried none and the guard stayed disarmed in
exactly the state where an uncompressed egress most likely blows the
window: long dsh sessions 400 prompt_too_long for the whole process
lifetime.

Add an optional declaredWindowForDirect hook to NativeInterceptState,
consulted after the header lookup at every fall-back site; dsh arms it
from its own cached model info (session model over global default, the
#2381/#2924 resolution headersFor stamps), failing open on any miss.
Other lanes leave it unset — byte-identical behavior.

Behavior change (dsh lane only): an oversized model request sent while
the proxy is not ready now fails locally with a 422
(oversize-refusal-fallback) instead of egressing uncompressed and
taking an upstream 400 prompt_too_long.
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-27_direct-window-guard-4894 (973237f)

Latest build — install this exact version

npm install -g billion-context@0.0.6367-pr.4904.9aec8a8

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
973237f billion-context@0.0.6367-pr.4904.9aec8a8 build
5b96197 billion-context@0.0.6357-pr.4904.608258a build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4904.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复——本 PR 是 #4894 Part 2(直发护栏侧)的实现交付物(Closes #4894 在合并时生效),不是另起炉灶:与 #4546(#4516 的护栏本体)同函数不同缺陷——#4546 给 directSendOrRefuse 装了护栏但窗口只从请求头取;本 PR 修的是该护栏在 dsh「归因 ok + 代理未就绪」路径上因头缺失而整段跳过的结构性失效。Part 1(宣告先于就绪)按 issue 说明拆到决策请求 #4905,不在本 PR。在途 #4878 同改 directSendOrRefuse(hop hygiene),issue 已自报 rebase 依赖,review 时会核。

标签: bug / dsh / scope/wiring / p1(dsh 车道长会话慢启动后每轮必上游 400、进程生命周期内不可用——按「用户不能用」定 p1;请求管道/协议接线面)。已打好。

收到,开始独立 review:拉分支核代码、对原 issue 方向、逐项回归清点、看 CI 状态,结论按八问回。

@ranxianglei ranxianglei added bug Something isn't working dsh p1 scope/wiring labels Oct 27, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: vllm-qwen/qwen3.8-27b) Merge-guard sync done: merged origin/master (e4cc685) into 2026-10-27_direct-window-guard-4894 with zero conflicts and pushed as 973237f7f — mergeable_state moved behind → blocked (only remaining gate is CI/review). Post-push state at push time: all guard/static lanes green (version-guard, lock-sync, kernel/pi-subagents guards, config-surface, docs-sync...), test shards in progress; locally verified targeted suites tests/dsh-native.test.ts + tests/issue4894-direct-window-guard.test.ts (75/75 pass) and clean typecheck on the merged tree.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Merge guard 同步完成:把 origin/master(e4cc68575)合入分支 2026-10-27_direct-window-guard-4894,零冲突(无需设计决策),已推送 973237f。PR 状态 behind → blocked(剩余门禁只有 CI/review);推送时点 guard/static 各 lane 全绿、test shards 运行中。本地已做定向验证:tests/dsh-native.test.ts + tests/issue4894-direct-window-guard.test.ts 共 75/75 通过,typecheck 干净——master 自分叉后动过 dsh-native.ts(#4825/#4755/#4811/#4848),故做了语义级核对。结果评论已留:#4904 (comment)

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 这条是我刚才用 API 发的结果评论被同步层镜像回来的(作者显示为账号本身),不是新指令,无需处理。当前状态:分支已同步 master 并推送(973237f7f),behind 已清除,等 CI test shards 跑完即可合并;CI 结果出来前我这边没有待办。

@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.887-master.9b6ec23 (rolling master tag).

npm install -g billion-context@0.0.887-master.9b6ec23  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

ranxianglei pushed a commit that referenced this pull request Oct 27, 2026
…t — merge-guard sync

Master advanced 7f2e119 -> 9b6ec23 (+5 commits: version 0.1.208 #4980; dsh direct-fallback oversize guard #4904/#4894 in dsh-native.ts/native-intercept.ts + tests). Clean auto-merge, zero conflicts (delta touches no PR-area or prior-conflict file). PR change intact: src/server.ts zero-fold calibrated-fit predicate (#4930) + tests/issue4930-zero-fold-calibrated-fit.test.ts. Typecheck clean; targeted PR/preflight/yield suites 48/48 green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dsh p1 scope/wiring

Projects

None yet

1 participant