Skip to content

Preflight fail-fast 502 "no range could be compressed" misjudges a huge over-window session 鈥?minutes later the same session compresses 88% in one shot #569

Closed

Description

@shengbuding

Environment

  • billion-context v0.1.86 (npm), proxy mode; client = dsh CLI; upstream = OpenAI-compatible remote (deepseek-v4-flash), 1,000,000-token nominal window with effective window 653,896 after output reservation.

Symptom

With context 鈮?54K tokens (> effective window 653,896), preflight compression immediately fails closed and returns a structured 502, retryable=false, claiming nothing can be compressed:

2026-09-06T02:40:30.336Z [error] [<session>] preflight fail-fast 502 (retryable=false):
context ~954192 tokens exceeds the model window 653896 (model=deepseek-v4-flash) and preflight
compression could not bring it under: no range could be compressed (chunks below minCompressRange
or the summarization responses were unusable). The over-window payload was NOT forwarded.

Repeated identically at 02:40:33.753Z and 02:42:32.628Z (client retries).

However, the same session, minutes later and while still massively over-window, successfully executed a single in-band compression that shrank the history 88%:

03:22:51 [acp-obs] compress m00797鈥搈01781 shrink 88% (~563795 tokens): 641566 鈫?65637

So compressible ranges clearly existed; the preflight decision "no range could be compressed" was wrong for this payload. The difference: the in-band compress was driven by the session owner (the model itself calling the compress tool over chosen ranges), while preflight's automatic scan found nothing usable.

Observations (facts only, no proposed fix)

  1. The fail-fast path returns 502 with retryable=false and does NOT forward 鈥?the client cannot make progress through its own retries; the session is only recoverable if the model independently issues an in-band compress (i.e. the failure is bypassed from the client side of the protocol).
  2. "chunks below minCompressRange or the summarization responses were unusable" 鈥?in this case the history consisted of many small blocks (earlier small compressions during the same crisis each saved only 3鈥?%, e.g. preflight compressed 2 range(s), ~7435 tokens saved (702031 鈫?694596) in 265882ms), so automatic range selection may indeed have found only sub-minimum chunks 鈥?yet a large contiguous range (the 88% shrink) was later compressible. The automatic strategy and the manual strategy disagree on what is compressible.
  3. Fail-fast was retryable=false, so the client's own retry policy restarts the whole crisis (including repeated 300s preflight hangs, see companion issue) rather than degrading gracefully.
  4. The 502 body is emitted with retryable=false, which for an OpenAI-compatible client means "give up"; the client (dsh) then retried anyway and got the same 502 three times before the session owner intervened.

Repro sketch

Drive a session to context 鈮?.4脳 effective window with many already-compressed small blocks (so no single auto-selected range exceeds minCompressRange), then issue any request whose payload is over the window. Expect the fail-fast 502 above, even though a large manual/forced compression of the same history succeeds.

(Log excerpts redacted to remove session identifiers.)

Activity

  1. ranxianglei commented on Sep 6, 2026

    @ranxianglei
    Owner

    🤖 ework agent · qwen3.8-27b

    [bot] 🏷 Verified, root-caused, and fixed. PR: #574

    What I verified (repro). I reproduced the exact failure end-to-end through the real proxy against a mock upstream: an over-window session whose oldest viable range yields an unusable summary (sub-minCompressRange chunk, empty content, or a rejected/short summarization response) gets the 502 retryable=false "no range could be compressed" verdict — while a later span of the same history compresses fine. Deterministic, so client retries hit the identical wall every time, exactly matching your three repeated 502s at 02:40–02:42. Your observation that the in-band compress then shrank 88% is consistent: model-driven compression chooses its own ranges; preflight never got past the first one.

    Layer judgment — where the bug actually is. It is not the minCompressRange threshold, and not that "only small chunks existed." Root cause: preflightCompress (src/preflight.ts) selects exactly one range per round (oldest viable startRef) and, when that single range yields no applied chunk, declares global exhaustion and breaks out of the round loop. Remaining viable ranges are never tried — one bad leading edge produces a false terminal verdict on a recoverable session. That's your observation #2 (auto vs manual disagreeing) in one line.

    A secondary finding that makes block-dense histories especially prone to it: the kernel's min-gate sums only non-summary message text per spec, while preflight's own char precheck counts all view text including compressed-block summaries. So after several small folds (your 3–4% savings rounds), a leading range can pass preflight's check yet be rejected at apply time — or simply sit below minCompressRange once its blocks are folded — while the large uncompressed tail remains fully compressible (as your 03:22 88% shrink proves).

    The fix (src/preflight.ts, core-only, no wire-shape change):

    • Iterate all viable ranges (oldest-first, keeping the head byte-stable for prefix caching) within a round until one applies; declare exhausted only after every range has been genuinely tried.
    • Ranges that deterministically fail (unusable summary, apply rejection, unresolvable refs) go into a skip-set — refs are stable across folds, so they're never retried within the invocation. No wasted upstream calls, no churn.
    • New cap MAX_SUMMARY_CALLS_PER_PREFLIGHT = 8 bounds summarization spend per invocation (legacy worst case was also 8×1); hitting it reports a distinct budget-exhausted detail.
    • The exhaustion message now names how many ranges were tried, so a future 502 is truthful and diagnosable instead of asserting "nothing could be compressed."
    • Upstream-error mapping (429→503 retryable), abort handling, and token credit math are unchanged. Mode-agnostic: plugin and proxy modes are affected identically.

    Verification: typecheck ✅ · new suite tests/preflight-multi-range.test.ts 3/3 ✅ (regression case confirmed RED on legacy code: expected 200, actual 502) · full suite 1050/1051 (the single failure is a pre-existing, environment-specific launcher test that fails identically on pristine master — unrelated) · build ✅.

    Out of scope here (companion territory, per your obs #1/#3/#4): whether a truly exhausted fail-fast should degrade to forwarding rather than stalling the client (retryable=false = "give up" for OpenAI-compatible clients), the repeated 300s preflight hangs (companion issue), and the terminal-fragmentation case where every remaining raw fragment is below minCompressRange (would need multi-spec batched applies in preflight — different design question). Happy to take those up separately if you want them tracked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions