Skip to content

[PR] fix: empty-slot direct send now refuses over-window bodies locally (#5996) - #5999

Merged
ranxianglei merged 4 commits into
masterfrom
2026-11-09_empty-slot-window-doom-5996
Nov 9, 2026
Merged

ranxianglei merged 4 commits into
masterfrom
2026-11-09_empty-slot-window-doom-5996

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Summary

Fixes #5996 — bili-native (dsh) empty intercept slot sends an over-window body direct and the provider answers 400 CONTEXT_WINDOW_EXCEEDED.

Root cause

The native interceptor's empty-slot branch (the live-state slot is cleared between dsh activations — #2082/#2101) guarded oversized model POSTs only with the 8 MiB silent-direct byte cap. A CJK-heavy session body under that ceiling yet over the model's token window (the reporter's ~6.7 MiB / ~2.1M tokens is < 8 MiB yet far past a 1M window) sailed through raw and was rejected upstream with an opaque 400 CONTEXT_WINDOW_EXCEEDED, while bili itself accounted the prompt as in-window.

The degraded path already prices such bodies against the declared window and refuses them locally (bodyOverWindowDetail). The empty-slot branch had no window source at all — headersFor only stamps x-bili-plugin-context-window on the proxy branch (#4894), so when the slot is empty there is neither state nor header to read. It could only see bytes.

Fix

Remember the last context window any lane resolved while routing normally — captured from headersFor output on every successful proxy route (main + routed branches) and from the degraded path's resolved window. That steady state is precisely what precedes a slot-empties event. The empty-slot branch now runs the same window-aware doom the degraded path uses against that last-known value, after the existing byte-cap check.

  • Cold start (no window ever seen in this process) → the check is skipped → today's byte-cap-only semantics are preserved (fail open, exactly as before).
  • Buffering for the media walk is bounded by the byte cap (only reached when size <= cap).

⚠️ Behavior change (flagged per policy)

Old New
Empty-slot body under the 8 MiB cap but over the last-known window sent direct → provider 400 CONTEXT_WINDOW_EXCEEDED (opaque) refused locally → retryable 422 naming the token estimate vs the resolved window

Same gate (oversize-refusal-no-live-state), same retryable: true, same remedy (slot rebinds on the next activation / host restart). Only the failure surface moves from an opaque upstream 400 to a clear local refusal. This is a degradation/fallback-semantics change, so it is not an auto-merge candidate — human review requested.

Tests

New regression tests in tests/native-intercept.test.ts:

  1. over-window refuses — a lane stamps a 1000-token window on one live request; the slot then empties; a 4 MiB body (under the 8 MiB cap, ~1.19M est tokens > the 1M floor) is refused locally with the oversize-refusal-no-live-state gate, naming tokens + window.
  2. under-window passes — same setup, a 1 MiB body (~299K est tokens < 1M) still rides direct while the slot is empty.
  3. cold-start fails open — no headersFor (no window ever stamped); the same 4 MiB body goes direct (200) — the guard does not invent a refusal it has no basis for.

Verified locally: npm run typecheck clean, npm run build clean, tests/native-intercept.test.ts 81/81 green, plus the targeted suites on the touched paths (issue4894-direct-window-guard, issue5686-compaction-direct-refusal, native-intercept-selfheal, issue4516-pi-direct-fallback, pf-channel) all green. Full-suite verdict deferred to CI.

Fixes #5996

ework-agent and others added 2 commits November 9, 2026 10:49
…n LIVE_FILES

Converged master-red fix (sentinel ci-fixes lane): #5989 merged the
spawning suite without registering it, so the #1248 SPAWN_RE tripwire
kills all 11 core-matrix test shards before any test runs on every PR
head. Hunk is byte-identical to open PR #5990 (which carries the same
registration plus a test-tolerance change to master).
…5996)

The native interceptor's empty-slot branch (live-state slot cleared between dsh
activations, #2082/#2101) guarded oversized model POSTs only with the 8 MiB
silent-direct byte cap. A CJK-heavy body under that ceiling yet over the model's
token window (~6.7 MiB / ~2.1M tokens < 8 MiB, far past a 1M window) sailed
through raw and was rejected upstream with an opaque 400 CONTEXT_WINDOW_EXCEEDED,
while bili itself accounted the prompt as in-window.

The degraded path already prices such bodies against the declared window and
refuses them locally; the empty-slot branch had no window source at all
(headersFor only stamps x-bili-plugin-context-window on the proxy branch, #4894),
so it could only see bytes.

Remember the last context window any lane resolved while routing normally
(captured from headersFor output on every successful proxy route — the steady
state that precedes a slot-empties event) and run the same window-aware doom the
degraded path uses against it in the empty-slot branch. Cold start (no window
ever seen) fails open to today's byte-cap-only semantics.

Behavior change: an empty-slot body under the byte cap but over the last-known
window used to go direct and 400 upstream; it is now refused locally with the
same 422 policy-refusal class and retryable remedy (the slot rebinds on the next
activation). Same gate (oversize-refusal-no-live-state); only the stated basis
changes from the byte ceiling to the token estimate vs the resolved window.

Fixes #5996
@ranxianglei ranxianglei added bug Something isn't working p2 labels Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-09_empty-slot-window-doom-5996 (15c1983)

Latest build — install this exact version

npm install -g billion-context@0.0.8743-pr.5999.05a03e4

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
15c1983 billion-context@0.0.8743-pr.5999.05a03e4 build
12b18e9 billion-context@0.0.8742-pr.5999.124e6f3 build
4ec9778 billion-context@0.0.8741-pr.5999.c77b1c6 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5999.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: empty-slot direct send now refuses over-window bodies locally (#5996) [PR] fix: empty-slot direct send now refuses over-window bodies locally (#5996) Nov 9, 2026
… resolved window in the empty-slot refusal (#5996 review)

Review finding on #5996: the new doom basis said 'against the N token window the active lane resolved', but N is max(resolved, MIN_ASSUMED_WINDOW_TOKENS) - when the resolved window is below the 1M floor that misstates the model's real window. Mirror the sibling fallbackOversizeRefusal declared-vs-bar distinction (#4079); adds a regression row pinning the >=1M verbatim naming.
… in LIVE_FILES) per CI-coordination protocol (#4599)
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复(open [PR] 行全量核对,无其他行指向 #5996)。
标签: bug + dsh + scope/wiring + p2 —— dsh 宿主侧拦截层的缺陷修复;特定条件下明显异常但有绕行(重试),故 p2。

Review verdict: direction correct, implementation sound; one wording flaw found in review, fixed directly on the branch.

Mechanism (verified against the PR head tree, not just the diff):

  • lastKnownContextWindow is captured at all three places a window is resolved while routing normally: the degraded path (noteResolvedContextWindow, runs even on refused sends), and the routed + rewrite/attach branches (captureStampedWindow on headersFor output). headersFor fires only on those two proxy routes, so capture is complete; the self-branch (already-proxy-addressed URLs) does not capture → fail-open, acceptable.
  • The empty-slot branch now runs the same bodyOverWindowDetail doom the degraded path uses, against that last-known value, after the existing byte-cap check (unchanged, still first). Cold start (no window ever seen this process) skips → today's byte-cap-only semantics preserved exactly (fail open).
  • The new refusal is shape-identical to the pre-existing empty-slot refusal: same x-bili-gate: oversize-refusal-no-live-state, same retryable: true, same remedy text.

Regression inventory (pre-change behavior walked, not just the diff): byte-cap arm unchanged; all three capture sites are observation-only (no control-flow change); _resetForTest clears the new module state (no cross-test bleed); proxy mode untouched; no config/kernel/wire-shape/persistence changes. Reasoned in both compression modes (§6): plugin-mode-only surface.

Tests: the three new ones have teeth and cover both directions (#3617 discipline) — over-window refuses / under-window rides direct / cold-start fails open. Token math independently recomputed (4 MiB vs the 1M floor bar → est 1,195,377 > 1M refuse; 1 MiB → 298,845 ≤ 1M pass). Minor gap: the degraded/routed capture helpers aren't unit-tested directly (tiny, low risk).

Local verification at PR head (isolated worktree, kernel + pi-subagents built first): tests/native-intercept.test.ts 81/81 green, typecheck clean — matches the author's claims.

Finding, fixed directly (commit 12b18e917d): the new doom basis said "against the N token window the active lane resolved", but N is max(resolved, MIN_ASSUMED_WINDOW_TOKENS) — for a sub-1M resolved window that misstates the model's real window as the floored bar. The sibling fallbackOversizeRefusal already distinguishes declared-vs-bar (#4079); I mirrored that distinction (short form when declared == window; long form citing the resolved value + the floor rule otherwise) and added a regression row pinning the ≥1M verbatim naming. Post-fix: 82/82 green, typecheck clean.

CI (live-verified, timeline):

  1. First head 4ec97781a9 — last cached snapshot showed all 30 checks "skipped" (stale intermediate state, taken minutes after creation).
  2. Second head 12b18e917d — all 11 test shards failed uniformly within ~90s across ubuntu+windows × node 22/24, while the identical tree is green locally → inherited master born-red, not this PR's diff. Diagnosis: tests/issue5955-wedged-same-lane-takeover.test.ts landed via [PR] test(launcher): cover the #5955 wedged-takeover escalation ladder + win32 ESRCH silence #5989 without registration in scripts/live-files.mjs (known problem; the ci-fixes hunk is byte-identical to open PR [PR] test(launcher): tolerate the documented one-observation loss in the #5955 takeover ledger #5990).
  3. Merged origin/ci-fixes (831b645804) into the PR branch per the CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 protocol (one-line comment posted there). Current head = 15c198362a; at last check: 24 success / 0 failed / 13 still running. Final green verdict pending the running shards.

Behavior change (PR's disclosure assessed accurate): empty-slot bodies under the 8 MiB cap but over the last-known window now get a local retryable 422 naming est-tokens vs window, instead of an opaque upstream 400 CONTEXT_WINDOW_EXCEEDED. Bounded trade-off, documented in the PR: the remembered window is process-global most-recent-wins; a false refusal needs a multi-model host with differing >1M windows + an empty slot + a body in the band between them — and the outcome is a retryable local 422, not an opaque upstream 400. My review fix additionally changes the refusal wording for the floored case: old "…against the 1,000,000 token window the active lane resolved" → new "…against the 1,000,000 token local bar (1,000 resolved by the active lane; the bar never assumes below 1,000,000, #4079)".

Auto-merge judgment: NOT eligible — it adds a new local-refusal surface on a degradation/fallback path (fallback/default-value semantics are human-gated per §7.4). Everything else qualifies (single module, regression tests, Fixes #5996, clean diff). Human review requested, as the PR states.


结构化八问(中文)

  1. 是什么问题:dsh 空拦截槽位(插件两次激活之间)会把小于 8MiB 字节上限的模型请求原样直发上游;若该 body 超过模型的 token 窗口(CJK 长会话典型,如 ~6.7MiB/~2.1M tokens 对 1M 窗口),provider 返回不透明的 400 CONTEXT_WINDOW_EXCEEDED,而 bili 自己把 prompt 记为在窗内。
  2. 什么原因导致的:空槽分支只有字节上限守卫、完全没有窗口来源——headersFor 只在代理分支盖 x-bili-plugin-context-window,槽空时既无状态也无头可读,于是「低于字节上限但超 token 窗口」的 body 裸奔直通。(降级路径本就有对声明窗口的计价与本地拒绝,空槽分支缺的是同一套机制。)
  3. 影响范围与严重程度:dsh 桌面用户;触发条件 = 槽位为空 且 body<8MiB 且 body 超模型 token 窗口。严重度:个别请求以不透明上游 400 失败 + 账目误记为在窗内;有绕行(重试即可,槽位下次激活/重启重绑)→ p2。
  4. 怎么修复的:记住任何车道正常路由时解析出的最后一个上下文窗口(两个代理分支从 headersFor 输出捕获 + 降级路径的解析值——正是槽位清空事件前的稳态);空槽分支在字节上限检查之后对同一 bodyOverWindowDetail doom 用该值计价。冷启动跳过 → 原有 fail-open 语义逐字保留。
  5. 是否有回归问题:逐项审过既有行为——字节上限臂未动且仍最先执行;三处捕获点纯观测、不改控制流;新 422 与既有拒绝同形同 gate 同 retryable;冷启动语义保留;proxy 模式不受影响;_resetForTest 清理新状态防串测。评审中发现一处措辞缺陷(floored 1M 下限被说成「车道解析出的窗口」),已按小问题直接修策略在分支上修掉并补回归行(12b18e917d)。另 CI 继承性红([PR] test(launcher): cover the #5955 wedged-takeover escalation ladder + win32 ESRCH silence #5989 带入的未注册 live 文件)已按协议 merge ci-fixes 收敛,非本 PR 缺陷。除此之外无回归。
  6. 是否破坏缓存:不涉及——未触碰缓存/重试/游标/去重键逻辑。
  7. 是否改了设计方向:不涉及——把既有 doom 机制扩展到一个分支,无架构/边界/数据流转向。有界取舍已在 PR 中披露:进程级最近一次生效的窗口记忆;跨模型误拒需 >1M 窗口互异 + 空槽 + body 落在区间带,结果是可重试的本地 422 而非不透明上游 400。
  8. 是否引入了新的配置:不涉及——无新 env/config/默认值。

判定:15c198362a 剩余分片转绿后即可合入;人工合并(降级语义变更,§7.4 人审档)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b) CI update (live-verified): head 15c198362a is now fully green — all 50 check runs completed: 37 success / 13 skipped / 0 failure. The inherited-red convergence via origin/ci-fixes held on the full matrix. Ready for human merge whenever you are.

@ranxianglei
ranxianglei merged commit 39fc2ee into master Nov 9, 2026
50 checks passed
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.1110-master.39fc2ee (rolling master tag).

npm install -g billion-context@0.0.1110-master.39fc2ee  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dsh p2 scope/wiring

Projects

None yet

1 participant