Skip to content

fix: rescue context-overflow turns in-request instead of arming a rescue that never comes (#1195) - #1328

Merged
ranxianglei merged 1 commit into
masterfrom
2026-09-25_overflow-refold-retry
Sep 25, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-09-25_overflow-refold-retry

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Problem

#1195: after a successful ACP compression, the next request still dies with an upstream Prompt is too long 400 and the session locks. Root cause (verified against pi/omp sources): wire clients (pi / omp on the plain proxy path) treat an upstream context-overflow 400 as fatal — they end the session on the spot. The existing #987 recovery arms a one-shot emergency shrink and waits for the next turn to fold. That next turn never comes: the client has already given up. Session locked.

This is the mirror image of #1310: there the ledger lied while the wire was fine; here the ledger is honest, the wire really is over, and the rescue itself is unreachable.

Fix

When forward() sees an upstream context overflow (400/413 matching the overflow patterns) and the error states a window:

  1. arm the one-shot emergency shrink exactly as before (extracted into armOverflowShrink, once per request);
  2. re-run prepare + preflight under the session lock with the stated window as a per-call modelContextLimit override — the fold the declared window could never trigger now runs against the REAL window;
  3. re-send the folded body once within the same request;
  4. success → the client gets the retry's 200 and never sees the 400; retry rejected → the client gets the retry's verdict (not the stale first 400); unfoldable payload / null refold / transport failure → today's verbatim passthrough of the original 400.

Nothing is learned (#987 keeps governing: no confirmedContextLimits, declared window unchanged) — the override is per-call, scoped to the rescue only. The #496 image-forward-once hatch stays closed on the retry (the arm sets lastInputTokensSource="usage").

Tests

Fixes #1195

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-25_overflow-refold-retry (ac6a538)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-1328

Each push to this PR publishes a new version under the pr-1328 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1328.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review started. PR #1328 (2026-09-25_overflow-refold-retry, 1 commit, 3 files, +445/-77, base master). CI: version-guard / build-artifact / e2e / hermetic-registry-e2e / pi-lane / codex-overlay all green; the 4 test matrix jobs were still in_progress at time of writing. Pulling the branch now for local verification (typecheck + full suite + build) and a full code/regression review per the PR review spec; final verdict to follow in this thread.

…cue that never comes (#1195)

Wire clients (pi/omp on the plain proxy path) treat an upstream
context-overflow 400 as fatal and end the session, so the #987
arm-and-fold-on-the-next-turn recovery never gets its next turn and the
session locks. When forward() sees a context overflow with a stated
window, re-run prepare+preflight under the session lock with the stated
window as a per-call limit override, fold the payload below the REAL
window, and re-send it once within the same request. Nothing is learned;
the declared window keeps governing (#987). An unfoldable payload keeps
today's verbatim passthrough; a rejected retry passes the RETRY's
verdict through, not the stale first 400.

Fixes #1195
@ranxianglei
ranxianglei force-pushed the 2026-09-25_overflow-refold-retry branch from 8b5fa77 to ac6a538 Compare September 25, 2026 12:05
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review complete — mergeable. Full walkthrough below (direction → regression → code → tests → diff), plus one action I took directly on the PR branch.

① Original issue direction (#1195)

Verified against the #1195 thread: this is the genuine-upstream-400 case (owner cross-determination floor 35), not the #1310 fake local reject — the ledger is honest, the wire really is over, and pi/omp treat an overflow 400 as fatal, so #987's armed rescue never gets its "next turn".

Layer judgment: the fix targets the correct layer. It does not mask (no filtering/rewriting of the error or payload), it does not treat the symptom at the wrong boundary (the fold happens in preflight under the session lock, then the real folded body is re-sent — the client-visible contract only improves). It also correctly refuses to over-reach: nothing is learned (confirmedContextLimits untouched, declared window unchanged, per-call override scoped to the rescue), so root cause A (window over-declaration) stays open for its own fix, exactly as the issue's per-mechanism plan intends. Complementary work already exists for mechanism C (#1247 drift detection, open).

② Regression review — existing behaviors on touched paths

Existing behavior Verdict
First-pass preflight fail-fast response (503/500 + JSON/SSE) Preserved — suppression only applies to the rescue pass (respondFailFast=false); first pass still responds exactly as before
Arm-once semantics (#987/#1110 overflowArmTokens) Preserved — extracted into armOverflowShrink, once-per-request flag; legacy !ok passthrough re-inspection no-ops; tests B/C pin overflowArmTokens === 128000
Client-visible overflow contract Intended change (flagged): stated-window overflow now returns the retry's 200 instead of the first 400; if the retry is also rejected, the client sees the retry's verdict instead of the stale first 400 (old: first 400). Unfoldable payload keeps the original 400 byte-verbatim (test B pins byte identity)
#496 image-forward-once hatch on retry Stays closed — verified in code: arm sets lastInputTokens = limit + source="usage" before the second pass, so noOverflowEvidence is false
Nothing learned / declared window unchanged Preserved — no confirmedContextLimits/effectiveContextLimit writes anywhere in the new path; pinned undefined in all four tests
Stale prepared after successful retry Safe — every downstream consumer post-fetch uses either the shared live session object (mutated by the refold pass under lock) or immutable per-request fields (protocol/stream/compressInjected/originalMessages, identical across both passes); nobody consumes prepared.body downstream
#568 early-200 hold interaction No new hazard — rescue can fire after the hold committed 200; all outcomes degrade through the existing in-band-error machinery (src/server.ts:4749)
Retry fetch framing Safe — init carries no explicit content-length (undici recomputes); identical pattern to the pre-existing role-ladder compat retry
Both compression modes (§6) OK — preflight force-fold already fires in both modes (#470 backstop); the change only makes it fold against the window the upstream actually stated. Plugin mode's tool-call carrier is untouched

Minor nits found (non-blocking): (a) logRequestCost runs once per pipeline pass → two cost lines logged on a rescued request (cosmetic); (b) the hook's try/catch {} swallows a hypothetical refold failure silently — a single log("warn", ...) there would help debuggability. Happy to take either as a follow-up commit if you want them.

③ Test validity

  • Red-on-master empirically confirmed: I ran the new file at the merge-base (a7a51474) — A fails (gets 400), C fails (stale first 400 served), B passes (regression guard on the existing contract). Exactly the claimed shape.
  • The assertions have teeth: exact upstream call count (A: precisely original+summary+retry), marker-based fold-range checks (tail kept, early range dropped), byte-identical passthrough (B), no-learning pins ×4, test: pin the overflowArmTokens one-shot RELEASE direction (arm → real usage report → cleared → side request forwards again) + restart round-trip #1129 arm retirement by the retry's usage, sticky fold on the next turn (T1 r2). Each of A/B/C would catch a distinct regression mode (rescue regresses to arm-and-wait / passthrough re-serializes the body / retry verdict regresses to the stale 400).
  • T1 update in tests/e2e-overflow-selfheal.test.ts is the intended behavior change documented above, not test-weakening; T2/T3 untouched.

④ Verification performed (rebased head)

Diff cleanliness

3 files, every line serves the fix; no whitespace churn, no generated artifacts, no version bump. Clean.


中文摘要:修复了 #1195「上游报窗口溢出 400 后救援永远等不到下一轮、会话锁死」的问题——在同一个请求内用上游声明的真实窗口重跑 prepare+preflight 折叠并重发一次(不学习、不改声明窗口、不可折叠时原样透传原 400),已本地验证 rebase 后全绿并直接把分支 rebase 推到了当前 master,可以合并。

@ranxianglei
ranxianglei merged commit 60df397 into master Sep 25, 2026
11 checks passed
ranxianglei pushed a commit that referenced this pull request Sep 26, 2026
…on; reframe as observability complement to #1328 (#1247 review, #1417)
@ranxianglei
ranxianglei deleted the 2026-09-25_overflow-refold-retry branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

使用pi 压缩后就提示prompt is too long

1 participant