Repository navigation
Description
Symptom
A session whose payload exceeds the model window and whose entire foldable content sits inside the soft-protected recent zone gets stuck in a permanent 502 loop. Every request fails fast before forwarding, and no recovery path exists — the request never reaches the model, so the EMERGENCY nudge (the only other recovery mechanism, #299) can never fire.
Found while building the e2e suite (#329): the suite's load-growth phase rides large user-messages on the wire verbatim, and under a forced small window (BILI_LAUNCHER_MODEL_WINDOWS) the recent zone ends up covering all foldable content.
Mechanism
- Preflight trigger:
tokenCount >= config.modelContextLimit(src/server.ts:1788). - The kernel's nudge computes
compressibleRangesexcluding the soft-protected recent zone (preserveRecentMessages, default 5, plus the last user message) — acp-kernel: "messages there are excluded from the range but do not fail it (an entirely-protected range fails with a clear error)". - When nothing viable remains (>= 200 tokens per
viableRanges):src/server.ts:1802-1807→failFast(502, "no part of the conversation is compressible (nothing left to fold)", false), orsrc/preflight.ts:264-266→failure = { kind: "exhausted", detail: "no compressible ranges remain in the conversation" }→src/server.ts:1853→failFast(502, ..., false).
- The payload is never forwarded ("The over-window payload was NOT forwarded." —
src/server.ts:1798). The model never sees it, so the EMERGENCY nudge / model-driven compress can never fire. - Nothing changes between attempts (same session state, same payload, same window) → every retry is an identical 502.
retryable: falsetells the client it is a hard failure, but it is actually a permanent one — the session is bricked until the window is raised or history is trimmed externally.
This is the same stuck-session class that #247 fixed for model switches (preflight was that fix); #301's fail-fast introduced a new variant: when preflight itself cannot help, the proxy 502s instead of forwarding (where the upstream 400 would at least surface to the user and the reactive path could still try).
Impact
- A session can be permanently stuck with no recovery path — worse than a transient error.
- Likely in real operation: any workload where one large recent message (big tool output, big user payload) pushes context over a small forced window. The e2e suite hits this deterministically under a forced window.
- Compounded by relay usage inflation (~2×, noted in Model: glm-5.3 — test: e2e suite — real codex through bili (warmup/load/compress/purity/forge) #329): the trigger fires earlier than true usage, making the state easier to reach.
Suggested fix
When the payload is over-window and no viable range exists outside the protected zone, fall back in order:
- Relax the soft protection (preferred): re-run preflight with the recent zone relaxed (oldest-first within the zone), still honoring the hard
protectedToolsexclusion. Soft protection is a preference, not a constraint — under overflow, folding the oldest soft-protected content beats bricking the session. - Emergency truncation of large tool outputs inside the protected zone (the kernel already ships
truncateLargeToolOutputsas a last-resort safety valve; it never touches summaries). - If both are exhausted, keep the fail-fast 502 but make the error actionable: name the protected zone, the sizes involved, and what an operator can do (raise the window / restart the session).
Alternative (weaker): forward as-is when the payload's own estimate is close to the limit — the window estimate is approximate and relay usage inflation means true usage may fit — letting the reactive nudge handle it.
Repro (from the #329 e2e suite)
- Force a small window via
BILI_LAUNCHER_MODEL_WINDOWS. - Send a payload where a recent user-message alone exceeds the window (the suite's load-growth pattern: deterministic payload user-messages riding the wire verbatim).
- Observe on every request, indefinitely:
preflight fail-fast 502 (retryable=false): context ~N tokens exceeds the model window W (model=M) and preflight compression could not bring it under: no part of the conversation is compressible (nothing left to fold). The over-window payload was NOT forwarded.
Reported by @ranxianglei while building #329 (report-only there; filed separately per repo convention).
Activity
ranxianglei commentedon Aug 29, 2026
[bot] Picked up. Reproducing the mechanism first: preflight's
viableRanges(turn.nudge?.compressibleRanges)is empty because the kernel'scomputeProtectedRefs(soft recent zone:preserveRecentMessages+preserveRecentTokens+ last user message) swallows all foldable content, andsrc/preflight.ts:264then gives up withkind: "exhausted"→src/server.ts:1853→ permanent 502.Plan (per the issue's suggested fix):
preserveRecentMessages: 0/preserveRecentTokens: 0), still honoring hardprotectedTools, and fold oldest-first within the zone.truncateLargeToolOutputsas a last resort.Working on it now — will report back with a PR.
ranxianglei commentedon Aug 29, 2026
[bot] Fixed — PR: #337
Root cause (two independent brickers):
preflightCompressIfNeededfailed fast on the normal-configcompressibleRanges— which exclude the soft-protected recent zone — beforepreflightCompressever ran. So a payload whose only foldable content is inside the zone never got a chance.The fix:
src/preflight.ts): when no range is foldable outside the soft zone and the payload's own estimate still overflows, preflight relaxes the soft zone to zero (preserveRecentMessages/preserveRecentTokens→ 0) and retries, folding oldest-first within it. The hardprotectedToolsexclusion is computed independently and still applies. Only when truly nothing is foldable does it fail fast — now with an actionable error ("Raise the model context window or restart the session to recover") instead of the opaque "no compressible ranges remain".result.payloadEstimate, notcurrentTokens— the latter is floored by a possibly-stalelastInputTokensfrom a prior model, which would fold protected content even when the real payload already fits. (This regression was caught by the existing bili 模式切换模型会报错上下文满 #247 model-switch test during development.)usage < truncate.threshold; compressible ranges derive from the protected zone, not usage). Without this, the node trims the very tool output overflowing the window, making the fit check undershoot and break before the soft zone is relaxed.src/server.ts): re-measure after the rebuild instead of trustingresult.payloadEstimate, which the relaxed-zoneprocessTurntrims more than the normal-config prepare does (could turn a guaranteed-400 forward into a false "fits").Tests: new e2e for the #330 scenario (over-window payload, only foldable content in the protected zone → relaxed, folded, forwarded 200); the "truly incompressible" e2e is now a hard-protected tool result (502 + actionable remedy, no summarization call, no forward). Full suite 746/746, typecheck + build clean.
On the issue's second suggested lever (kernel
truncateLargeToolOutputs): it's stateless/non-persistent and only touches tool-results (this repro is a large user message), so it's largely redundant with the soft-zone relaxation — noted as a possible follow-up in the PR rather than bundled here.Merge pr-337 into master: #330 soft-zone relax + fix #496 image-cost …
Merge pr-337 into master: #330 soft-zone relax + fix #496 image-cost …
Merge pull request #337 from ranxianglei/2026-08-29_preflight-soft-zo…
fix: try all viable ranges before preflight exhaustion (#569)
fix: try all viable ranges before preflight exhaustion (#569)
fix: hold the client through long preflight compression (#568)
fix: judge anonymous zero-baseline forks by the char-count upper bound (
fix: count system+tools wire overhead in the preflight trigger (ranxi…
fix: hold the client through long preflight compression (#568)
fix: hold the client through long preflight compression (#568)
fix: hold the client through long preflight compression (#568)
fix(ranxianglei#1001): pin acp-kernel 0.0.79 (line-form split-entry c…
fix: dispatch preflight folds by payoff, keep the recent zone last-re…
test: restore margin in preflight fixtures knife-edged by the attribu…
test: #3726 re-tune knife-edge e2e margins for the +~300 est-tok syst…