Repository navigation
[PR] fix: empty-slot direct send now refuses over-window bodies locally (#5996) - #5999
Conversation
…n LIVE_FILES Converged master-red fix (sentinel ci-fixes lane): #5989 merged the spawning suite without registering it, so the #1248 SPAWN_RE tripwire kills all 11 core-matrix test shards before any test runs on every PR head. Hunk is byte-identical to open PR #5990 (which carries the same registration plus a test-tolerance change to master).
…5996) The native interceptor's empty-slot branch (live-state slot cleared between dsh activations, #2082/#2101) guarded oversized model POSTs only with the 8 MiB silent-direct byte cap. A CJK-heavy body under that ceiling yet over the model's token window (~6.7 MiB / ~2.1M tokens < 8 MiB, far past a 1M window) sailed through raw and was rejected upstream with an opaque 400 CONTEXT_WINDOW_EXCEEDED, while bili itself accounted the prompt as in-window. The degraded path already prices such bodies against the declared window and refuses them locally; the empty-slot branch had no window source at all (headersFor only stamps x-bili-plugin-context-window on the proxy branch, #4894), so it could only see bytes. Remember the last context window any lane resolved while routing normally (captured from headersFor output on every successful proxy route — the steady state that precedes a slot-empties event) and run the same window-aware doom the degraded path uses against it in the empty-slot branch. Cold start (no window ever seen) fails open to today's byte-cap-only semantics. Behavior change: an empty-slot body under the byte cap but over the last-known window used to go direct and 400 upstream; it is now refused locally with the same 422 policy-refusal class and retryable remedy (the slot rebinds on the next activation). Same gate (oversize-refusal-no-live-state); only the stated basis changes from the byte ceiling to the token estimate vs the resolved window. Fixes #5996
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.8743-pr.5999.05a03e4This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5999.tgz
npm install -g packageInstall instructions are refreshed on each push. |
… resolved window in the empty-slot refusal (#5996 review) Review finding on #5996: the new doom basis said 'against the N token window the active lane resolved', but N is max(resolved, MIN_ASSUMED_WINDOW_TOKENS) - when the resolved window is below the 1M floor that misstates the model's real window. Mirror the sibling fallbackOversizeRefusal declared-vs-bar distinction (#4079); adds a regression row pinning the >=1M verbatim naming.
… in LIVE_FILES) per CI-coordination protocol (#4599)
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复(open [PR] 行全量核对,无其他行指向 #5996)。 Review verdict: direction correct, implementation sound; one wording flaw found in review, fixed directly on the branch. Mechanism (verified against the PR head tree, not just the diff):
Regression inventory (pre-change behavior walked, not just the diff): byte-cap arm unchanged; all three capture sites are observation-only (no control-flow change); Tests: the three new ones have teeth and cover both directions (#3617 discipline) — over-window refuses / under-window rides direct / cold-start fails open. Token math independently recomputed (4 MiB vs the 1M floor bar → est 1,195,377 > 1M refuse; 1 MiB → 298,845 ≤ 1M pass). Minor gap: the degraded/routed capture helpers aren't unit-tested directly (tiny, low risk). Local verification at PR head (isolated worktree, kernel + pi-subagents built first): Finding, fixed directly (commit CI (live-verified, timeline):
Behavior change (PR's disclosure assessed accurate): empty-slot bodies under the 8 MiB cap but over the last-known window now get a local retryable 422 naming est-tokens vs window, instead of an opaque upstream Auto-merge judgment: NOT eligible — it adds a new local-refusal surface on a degradation/fallback path (fallback/default-value semantics are human-gated per §7.4). Everything else qualifies (single module, regression tests, 结构化八问(中文)
判定: |
(ai: vllm-qwen/qwen3.8-27b) CI update (live-verified): head |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.1110-master.39fc2ee # this exact build
npm install -g billion-context@master # always the newest merged build
|
Summary
Fixes #5996 — bili-native (dsh) empty intercept slot sends an over-window body direct and the provider answers
400 CONTEXT_WINDOW_EXCEEDED.Root cause
The native interceptor's empty-slot branch (the live-state slot is cleared between dsh activations — #2082/#2101) guarded oversized model POSTs only with the 8 MiB silent-direct byte cap. A CJK-heavy session body under that ceiling yet over the model's token window (the reporter's ~6.7 MiB / ~2.1M tokens is
< 8 MiByet far past a 1M window) sailed through raw and was rejected upstream with an opaque400 CONTEXT_WINDOW_EXCEEDED, while bili itself accounted the prompt as in-window.The degraded path already prices such bodies against the declared window and refuses them locally (
bodyOverWindowDetail). The empty-slot branch had no window source at all —headersForonly stampsx-bili-plugin-context-windowon the proxy branch (#4894), so when the slot is empty there is neither state nor header to read. It could only see bytes.Fix
Remember the last context window any lane resolved while routing normally — captured from
headersForoutput on every successful proxy route (main + routed branches) and from the degraded path's resolved window. That steady state is precisely what precedes a slot-empties event. The empty-slot branch now runs the same window-aware doom the degraded path uses against that last-known value, after the existing byte-cap check.size <= cap).400 CONTEXT_WINDOW_EXCEEDED(opaque)422naming the token estimate vs the resolved windowSame gate (
oversize-refusal-no-live-state), sameretryable: true, same remedy (slot rebinds on the next activation / host restart). Only the failure surface moves from an opaque upstream 400 to a clear local refusal. This is a degradation/fallback-semantics change, so it is not an auto-merge candidate — human review requested.Tests
New regression tests in
tests/native-intercept.test.ts:oversize-refusal-no-live-stategate, naming tokens + window.headersFor(no window ever stamped); the same 4 MiB body goes direct (200) — the guard does not invent a refusal it has no basis for.Verified locally:
npm run typecheckclean,npm run buildclean,tests/native-intercept.test.ts81/81 green, plus the targeted suites on the touched paths (issue4894-direct-window-guard,issue5686-compaction-direct-refusal,native-intercept-selfheal,issue4516-pi-direct-fallback,pf-channel) all green. Full-suite verdict deferred to CI.Fixes #5996