Repository navigation
[PR] fix: preflight — refuse doomed bodies before folding when tools[]/system/other alone exceed the window - #7175
ranxianglei wants to merge 4 commits into
Conversation
…tem/other alone exceed the window (#7167) The preflight trigger's text channel includes tools[] (estimateWireOverhead counts them), so a body whose tools[] alone clear the window triggers the fold — irreversibly rewriting session history and billing real summary tokens — only for the post-fold measured dooms (#1839/#3146) to answer with the 413 anyway. Measure the UNMODIFIED prepared body with the #554 guard meter right after the trigger guard: if tools+system+other (dimensions folding cannot touch) >= the doom bar (native window when bili knows it, headroom limit as fallback — the SAME bar the post-fold dooms use), fail fast with the 413 before probeForEvidence / preflightCompress. The gate can only refuse a body the post-fold flow would have 413'd with the same meter — no new failure mode. Growth-armed payloads keep the autoFold fail-open rule; unparseable bodies fall through as before. Tests: tests/issue7167-preflight-irreducible-gate.test.ts (e2e through the proxy: doomed body -> 413 with zero summary calls and history intact on the next request; foldable control body still folds).
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.10840-pr.7175.b83e951This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr7175.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 查重: 不重复 — this PR is the only open PR for the pre-fold irreducible-dimension gate. Related-but-distinct open PRs touch other mechanisms: #4928 (convergence walk past the doom bar), #5006 (doom byte-floor pricing), #7123 (egress-fuse 413 wording). None gates the irreducible dimensions before the fold. Review status: BLOCKED — the PR head is red on CI, and I've pinned the root cause. CI
Root cause (confirmed): both are Responses-API tests (they post to Plan (fixing now): the meter's bucketing must be wire-aware — |
…7167) The #7175 irreducible gate reads tools+system+other from estimateRawBodyBreakdown, which bucketed the outbound body by top-level key and only recognized 'messages'/'system'. On the Responses wire the foldable message array lives under 'input' (Google: 'contents'), so it landed in the 'other' bucket and the gate 413'd an entirely-foldable body before any fold ran — the #3686 / #488-B Responses e2e regressions on the PR head. Make the bucketing wire-aware: 'input'/'contents' -> messages, 'instructions'/'systemInstruction' -> system. 'total' is the sum of the buckets either way, so the doom verdicts that price 'total' (post-fold #1839/#3146 arms, the egress fuse) are untouched; only the gate's bucket split and the measuredBreakdownClause display shift.
…post-fold measured doom The new pre-fold irreducible-dimension gate (src/server.ts) now answers a tools-heavy body before the fold runs, so the e2e #5000 case (120 tools × 40KB, core text clearing the byte floor) 413s with the irreducible message instead of the post-fold 'outbound body measures ~N TEXT tokens' doom. The behavior is the PR's intended change (no history rewrite, no summary billing); the test's assertion and comment were stale. Update them to pin the new verdict: code preflight_compress_failed, retryable false, the irreducible-dimension message, 'Compaction was NOT attempted', zero upstream calls.
…ges bucket) The #7167 irreducible-dimension gate prices total - messages, so the foldable message channel must be bucketed as messages for every wire shape (chat messages[], Responses input, Google contents). Pin that with a unit test so the gate cannot doom an over-window Responses/Google body before a fold could run (the CI break where input/contents mass landed in other and the gate 413'd perfectly-foldable bodies).
(ai: vllm-qwen/qwen3.8-27b) The red CI reported by this wake (head 290384d, 11 failing test shards) is already fixed. The current head of the PR branch is 八问摘要
结论: CI 已绿,PR 可合并。 |
(ai: vllm-qwen/qwen3.8-27b) Confirmed green. The current head of the PR branch is |
Closes #7167
查重: 不重复 — related-but-distinct: #6459 (probe-for-evidence doom-bar 413 before any fold — the #2490 byte-floor gate), #7168 (egress fuse counts non-contextual upload fields into the model window), #7092/#7016 (egress fuse and doom gates price prompt mass, not the whole body). None measures the irreducible dimensions (tools[]/system/other) of the unmodified outbound body before
preflightCompressruns — that is this issue's distinct mechanism.What
The preflight trigger's text channel includes
tools[](estimateWireOverheadcounts them), so a body whosetools[]alone clear the window triggers the fold — irreversibly rewriting session history and billing real summary tokens — only for the post-fold measured dooms (#1839/#3146 arms) to discovertools[] ≥ windowand answer with the 413 anyway. The fold was guaranteed-doomed before it started.Fix
Right after the trigger guard (before
probeForEvidence/preflightCompress), measure the unmodified prepared body with the #554 guard meter (measuredOutbound+estimateRawBodyBreakdown). Iftools + system + other— dimensions folding cannot touch — measure ≥ the doom bar (the native window when bili knows it, the headroom limit as fallback — the same bar the post-fold dooms use), fail fast with 413 /retryable:falseand a message that states compaction was NOT attempted, the body was NOT forwarded, and names the remedies (trim the tool surface / lazy tool discovery, shorten the system prompt, raise the effective window when the native window is unknown).Behavior change (intended — per the issue's request)
tools+system+other ≥ doomBar) ran the full preflight fold — irreversible history rewrite + summary billing — then 413'd via the post-fold measured dooms ("context ~N tokens exceeds the model window M … preflight compression could not bring it under").autoFoldTargetTokenscauses a fold storm, andexternalSummary.*file edits do not take effect until aPUT /__bili/config#3444 floor mechanism owns that regime); unparseable bodies fall through as before.Tests
tests/issue7167-preflight-irreducible-gate.test.ts— e2e through the proxy (window 20K):preflight_compress_failed,retryable:false, message asserts compaction was NOT attempted; zero upstream calls, zero summary calls; then the same session with a small tool surface forwards with the ORIGINAL history intact (MARKER-0…MARKER-49 present, no fold block).Targeted suites green: this file +
preflight-fail-fast+issue5000-count-gate+issue3846-local-refusal-evidence+issue2313-probe-forward+external-summary-autofold(35 pass) +preflight-compress+issue1839-ghost-denominator+issue1800-image-arbitration-compress+preflight-window-clamp+preflight-round-budget(12 pass).npm run typecheckclean.