Skip to content

preflight fail-fast × soft-protected zone → permanent 502 livelock (EMERGENCY nudge unreachable) #331

Closed

Description

@ranxianglei

Symptom

An over-window payload where every compressible range is inside the kernel's soft-protected zone causes a permanent 502 loop: no further request in the session can be sent until a new conversation is started. Found while exercising the codex compact intercept path (PR #325 / PR-E2) on a real-device e2e.

Mechanism (traced in code)

  1. preflightCompressIfNeeded (src/server.ts:1863) fires when tokenCount = max(session.stats.lastInputTokens, payloadEstimate) >= limit.
  2. Preflight (src/preflight.ts:224) only folds ranges from turn.nudge.compressibleRanges — the kernel's soft-protected recent ranges are excluded (they live in nudge.protectedRanges, src/loop/core.ts:143).
  3. If a round applies zero ranges (all candidates protected, chunks below MIN_CHUNK_TOKENS=2000, or summarization responses unusable), preflight stops with failure = {kind:"exhausted", detail:"no range could be compressed (chunks below minCompressRange or the summarization responses were unusable)"}.
  4. Payload still over window → failFast(502, …, retryable=false) (src/server.ts:1928-1929; the empty-ranges variant is src/server.ts:1882: "no part of the conversation is compressible (nothing left to fold)"). Response body: context ~N tokens exceeds the model window L (model=M) and preflight compression could not bring it under: <detail>. The over-window payload was NOT forwarded.
  5. Livelock: the 502 is deterministic — same payload, same state, same result on every retry. The only in-band recovery for this overflow shape is the EMERGENCY nudge (fix: detect chained bili instances (hop marker) — double processing corrupts session state #299), which instructs the model to force-compress — but that requires the request to reach the model. Fail-fast returns before forwarding, so the nudge can never fire and the client's retry loop spins on 502 forever.

Impact

  • Session death: over-window state with the overflow concentrated in the protected zone is unrecoverable in-band; the user must start a new conversation (losing context).
  • This is the "bili rejects" variant of the exact stuck-session class preflight was added to fix (bili 模式切换模型会报错上下文满 #247: "the request itself is rejected by the upstream, so the model never sees the nudge and every subsequent request overflows identically (stuck session)"). Fail-fast (fix: honor anthropic-beta context-1m to size the context window #301) trades the upstream 400 for a local 502 — but when nothing outside the protected zone is foldable, both outcomes are equally unrecoverable in-band.

Suggested fix (options)

  • A: When preflight exhausts with the payload still over window, don't fail-fast — forward the request with the EMERGENCY nudge injected (or with soft protection lifted for this one request) so the model gets one chance to force-fold in-band. Upstream 400 remains the backstop, but the recovery path is restored.
  • B: Allow preflight itself to override soft protection for the oldest protected range when failure.kind === "exhausted" and the payload is still over window (preflight is already the emergency path).
  • C (minimal): Keep the 502 but make the message name the protected-zone cause explicitly; does not unstick the session, only improves diagnosability.

Related observation (minor, report-only)

Preflight runs before compact-trigger detection on the pass-through path (preflight call at src/server.ts:1169; trigger detection/forge decision after): a compaction_trigger request that fails the forge gate (→ native compaction pass-through) still gets preflight-folded first, rebuilding its payload. Wasted summarization calls + ACP-state divergence that PR-C's semantic gate then rebases. Fix: detect compact requests (trigger form + POST /responses/compact) before preflight and skip preflight for them — the forge path never forwards anyway, and the pass-through path is precisely the case where upstream native compaction is the intended over-window escape.

Activity

  1. ranxianglei commented on Aug 29, 2026

    @ranxianglei
    OwnerAuthor

    [bot] Superseded by #332 (same livelock, with real-device log evidence from the e2e run). The related observation from this issue's last section (preflight/clamp running before compact-trigger detection) is tracked separately in #333. Closing to avoid duplicate tracking.

  2. ranxianglei commented on Aug 29, 2026

    @ranxianglei
    OwnerAuthor

    🤖 ework agent · vllm-qwen/qwen3.8-27b

    [bot] Acknowledged — closing #331 as superseded.

    For the record, my trace here reached the same mechanism you describe: preflightCompressIfNeeded (src/server.ts:1767) only folds turn.nudge.compressibleRanges (src/preflight.ts:263), so when every candidate sits in the kernel's soft-protected recent zone (src/loop/core.ts:143 protectedRanges), the loop exits with failure.kind === "exhausted" and the payload still overflows → failFast(502, …, retryable=false) (src/server.ts:1854). The EMERGENCY nudge (#299) can only fire if the request reaches the model, but fail-fast returns before forwarding, so the recovery path is unreachable — a deterministic 502 on every retry.

    I'll leave the fix to #332 (which carries the real-device log evidence) and the preflight-before-compact-trigger ordering note to #333, so we don't double-track. Happy to pick up either of those next — say the word and I'll start on #332.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions