Repository navigation
Description
Symptom
An over-window payload where every compressible range is inside the kernel's soft-protected zone causes a permanent 502 loop: no further request in the session can be sent until a new conversation is started. Found while exercising the codex compact intercept path (PR #325 / PR-E2) on a real-device e2e.
Mechanism (traced in code)
preflightCompressIfNeeded(src/server.ts:1863) fires whentokenCount = max(session.stats.lastInputTokens, payloadEstimate) >= limit.- Preflight (src/preflight.ts:224) only folds ranges from
turn.nudge.compressibleRanges— the kernel's soft-protected recent ranges are excluded (they live innudge.protectedRanges, src/loop/core.ts:143). - If a round applies zero ranges (all candidates protected, chunks below
MIN_CHUNK_TOKENS=2000, or summarization responses unusable), preflight stops withfailure = {kind:"exhausted", detail:"no range could be compressed (chunks below minCompressRange or the summarization responses were unusable)"}. - Payload still over window →
failFast(502, …, retryable=false)(src/server.ts:1928-1929; the empty-ranges variant is src/server.ts:1882:"no part of the conversation is compressible (nothing left to fold)"). Response body:context ~N tokens exceeds the model window L (model=M) and preflight compression could not bring it under: <detail>. The over-window payload was NOT forwarded. - Livelock: the 502 is deterministic — same payload, same state, same result on every retry. The only in-band recovery for this overflow shape is the EMERGENCY nudge (fix: detect chained bili instances (hop marker) — double processing corrupts session state #299), which instructs the model to force-compress — but that requires the request to reach the model. Fail-fast returns before forwarding, so the nudge can never fire and the client's retry loop spins on 502 forever.
Impact
- Session death: over-window state with the overflow concentrated in the protected zone is unrecoverable in-band; the user must start a new conversation (losing context).
- This is the "bili rejects" variant of the exact stuck-session class preflight was added to fix (bili 模式切换模型会报错上下文满 #247: "the request itself is rejected by the upstream, so the model never sees the nudge and every subsequent request overflows identically (stuck session)"). Fail-fast (fix: honor anthropic-beta context-1m to size the context window #301) trades the upstream 400 for a local 502 — but when nothing outside the protected zone is foldable, both outcomes are equally unrecoverable in-band.
Suggested fix (options)
- A: When preflight exhausts with the payload still over window, don't fail-fast — forward the request with the EMERGENCY nudge injected (or with soft protection lifted for this one request) so the model gets one chance to force-fold in-band. Upstream 400 remains the backstop, but the recovery path is restored.
- B: Allow preflight itself to override soft protection for the oldest protected range when
failure.kind === "exhausted"and the payload is still over window (preflight is already the emergency path). - C (minimal): Keep the 502 but make the message name the protected-zone cause explicitly; does not unstick the session, only improves diagnosability.
Related observation (minor, report-only)
Preflight runs before compact-trigger detection on the pass-through path (preflight call at src/server.ts:1169; trigger detection/forge decision after): a compaction_trigger request that fails the forge gate (→ native compaction pass-through) still gets preflight-folded first, rebuilding its payload. Wasted summarization calls + ACP-state divergence that PR-C's semantic gate then rebases. Fix: detect compact requests (trigger form + POST /responses/compact) before preflight and skip preflight for them — the forge path never forwards anyway, and the pass-through path is precisely the case where upstream native compaction is the intended over-window escape.
Activity
ranxianglei commentedon Aug 29, 2026
[bot] Superseded by #332 (same livelock, with real-device log evidence from the e2e run). The related observation from this issue's last section (preflight/clamp running before compact-trigger detection) is tracked separately in #333. Closing to avoid duplicate tracking.
ranxianglei commentedon Aug 29, 2026
[bot] Acknowledged — closing #331 as superseded.
For the record, my trace here reached the same mechanism you describe:
preflightCompressIfNeeded(src/server.ts:1767) only foldsturn.nudge.compressibleRanges(src/preflight.ts:263), so when every candidate sits in the kernel's soft-protected recent zone (src/loop/core.ts:143protectedRanges), the loop exits withfailure.kind === "exhausted"and the payload still overflows →failFast(502, …, retryable=false)(src/server.ts:1854). The EMERGENCY nudge (#299) can only fire if the request reaches the model, but fail-fast returns before forwarding, so the recovery path is unreachable — a deterministic 502 on every retry.I'll leave the fix to #332 (which carries the real-device log evidence) and the preflight-before-compact-trigger ordering note to #333, so we don't double-track. Happy to pick up either of those next — say the word and I'll start on #332.