Skip to content

fix: recover preflight from empty SSE summaries + dead-end cooldown (#726) - #727

Merged
ranxianglei merged 2 commits into
masterfrom
2026-09-12_preflight-empty-summary-recovery
Sep 12, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-09-12_preflight-empty-summary-recovery

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #726

Problem

After the #626 (stream) and #663 (max_output_tokens) compatibility retries, a ChatGPT-backend summarization call can still return HTTP 200 SSE with zero extractable summary text — an in-stream error event or truncated stream that extractSummaryFromSse silently discarded. The only trace was summary too short (0 chars), one viable range got skipped whole, and every codex auto-retry re-ran the identical doomed walk (~8 large summary calls per retry), producing the stable 502 retryable=false loop from #726.

Changes

src/preflight.ts

  1. Diagnose empty summaries — new diagnoseEmptySummary(text, json?) scans a no-text body for terminal error signals: SSE error / response.failed / response.error events (response.error.code + message), response.incomplete (status + optional message), bare JSON {error: {...}}, plus fallbacks for empty bodies, event-carrying streams without text, and non-SSE bodies. The diagnosis goes into the warn log and is threaded into the fail-fast 502 message (Last unusable summary: …), so the operator/client finally sees WHY the summary was empty.
  2. Shrink-and-retry — the chunk pass became a worklist: a chunk whose summary comes back unusable is halved (oldest half first) and retried down to a floor (2×MIN_CHUNK_TOKENS tokens / 2×minCompressRange chars) before the range is skipped. This recovers size-driven rejections — the prime suspect in [Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726, where the first chunk was CHUNK_FRACTION × window ≈ 630K tokens of summarization input — while staying bounded by the existing MAX_SUMMARY_CALLS_PER_PREFLIGHT budget. Skip-on-exhaustion semantics for genuinely unsalvageable ranges are preserved.

src/server.ts

  1. Dead-end cooldown — when preflight fails with compressedRanges === 0 (zero progress under unchanged state = deterministic dead-end), the failure fingerprint (model\u0000limit) + status + full message is cached in session.metadata.preflightDeadEnd for a cooldown (default 5 min, BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS). Matching requests fail fast with the cached diagnosis and zero upstream calls; any non-failure outcome clears the marker; aborted failures never arm it. The 502 message gains a recovery hint (restart the session). This stops the quota-burning retry loop from the issue's logs (8 identical walks in 48s).

tests/preflight-empty-summary.test.ts (new)

  • Size-driven recovery: oversized chunk → 200 SSE response.failed; halved chunk succeeds → request 200, payload forwarded, no dead-end marker left behind.
  • Systemic failure: diagnosis present in the 502 message, bounded summary calls, nothing forwarded, cooldown blocks the immediate retry with zero upstream calls, expiry re-runs the walk.
  • Unit coverage for diagnoseEmptySummary across all signal shapes.

Pre-flight

  • npm run typecheck ✅
  • npm test — 1422 pass / 0 fail / 2 skipped (gated e2e) ✅
  • npm run build ✅
  • E2E_CHECK=1 e2e preflight ✅. Full ACP_TEST_E2E=1 run not possible in this environment: the default local upstream (http://127.0.0.1:8199/v1) is down here (proxy log shows only ECONNREFUSED to it); please dispatch .github/workflows/ci-e2e.yml before merging since this touches the request pipeline.

Out of scope (separate issue filed)

The deeper blind spot — nudge never fires when the upstream never reports usage (usage=0% in #726's logs), letting context grow to 1.32M without any incremental compression — is kernel-layered and needs its own design discussion. Filed separately with this issue as source.

…726)

- Diagnose HTTP-200 summarization bodies that carry no summary text: parse terminal error signals (SSE error / response.failed / response.incomplete events, bare JSON error) into the log line and the fail-fast message, so a 0-char summary no longer hides the upstream's actual rejection reason.

- Retry unusable chunks at half size (oldest half first) down to a floor before skipping the range, bounded by the existing per-regime call budget — recovers size-driven rejections (a CHUNK_FRACTION-sized chunk exceeding the upstream's real input cap), which was #726's stable 502 loop on ChatGPT-backend codex.

- Zero-progress preflight failures arm a per-session dead-end cooldown (default 5m, BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS): identical over-window requests fail fast with the cached diagnosis and zero upstream calls instead of re-burning quota on the same doomed walk; any non-failure outcome clears it.
@github-actions

github-actions Bot commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-12_preflight-empty-summary-recovery (40ab16b)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-727

Each push to this PR publishes a new version under the pr-727 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr727.tgz
npm install -g package

This comment is automatically updated on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR review (branch 2026-09-12_preflight-empty-summary-recovery)

Duplicate screening: no duplicates. This is the only open PR touching preflight recovery; it directly fixes #726 (whose triage already established the three-layer root cause this PR addresses), and the out-of-scope growth-source issue #728 carries proper source attribution (来源: #726 …). No overlap with other open PRs (#715/#711/#706/#697/#688 are unrelated).

Diff cleanliness: clean — 3 files, all on-purpose (src/preflight.ts, src/server.ts, one new test file), no stray/generated changes.

Verified locally (on 5f51a47, then re-verified after my fix below): npm run typecheck ✅ · npm test ✅ · npm run build ✅. Code-level review of the three changes:

  • diagnoseEmptySummary / extractStreamError: covers all claimed signal shapes (SSE error/response.failed/response.error/response.incomplete, bare {error}, empty body, events-without-text, non-SSE); unit tests match every branch. ✓
  • Shrink-and-retry worklist: ordering verified (.slice().reverse() + pop() preserves oldest-first; halved halves push newer-first so the older half retries next); termination guaranteed (span size strictly decreases; each attempt costs a budgeted call); skip-on-exhaustion semantics for unsalvageable ranges preserved; success path still breaks to the next round exactly like the old flat pass. ✓
  • Dead-end cooldown: armed only on zero-progress non-aborted failures, keyed model\u0000limit, cleared on any non-failure outcome, persisted via markDirty. Both compression modes considered: preflightCompressIfNeeded runs in plugin mode too (server.ts call site is not plugin-gated), and both new mechanisms are mode-agnostic (session metadata keyed by model+limit; folds applied to session state, carrier rendering untouched) — no plugin/proxy divergence introduced. ✓

Two issues found, fixed directly on the PR branch (commit 40ab16b):

  1. Dead-end check sat above the safe-forward paths. It ran right after the trigger, before the PR #489 后续: 默认图片计费 b64/4 在像素 tile 计费上游(官方 Anthropic/OpenAI)会硬 502 误杀多模态会话 #496 image-arbitration forward and the fix: stitched-stream usage double-count after compress re-request (adapter-anthropic >0 guard) #300 stale-baseline fit check. Consequence: a payload whose own estimate actually fits the window — but which trips the trigger only via a stale-high lastInputTokens (e.g. right after a client-side/native compaction, before the first successful usage report lands) — would get a false cached 502 instead of a safe forward while the marker from an earlier larger request was still warm. The cooldown exists to suppress the doomed walk; paths that return without running the walk must not be blocked by it. Moved the check below all safe-forward returns (just before the walk starts). Added a regression test (seed high usage baseline → systemic failure arms marker → smaller same-session payload must still forward): verified it fails on the original placement and passes with the fix.
  2. Cached marker dropped retryable. A 429-armed dead-end was originally 503/retryable:true; the cooldown replay hard-coded retryable: false, losing the retry signal. Now stored and replayed verbatim.

Post-fix verification: typecheck ✅ · full suite 1423 pass / 0 fail / 2 skipped (was 1422; +1 new regression test) ✅ · build ✅.

Minor observations (report only, no action taken): under a systemic rejection the halving ladder now concentrates up to the full 8-call budget on the oldest range instead of spreading ~1 call across ranges — bounded and acceptable given the cooldown caps cross-request burning; partial-progress failures (folded ≥1 range but still over) neither arm nor clear the marker — correct, since each such retry makes durable progress so no livelock forms; stale expired markers persist in metadata until the next arm/clear — single slot per session, harmless.

E2E dispatched as requested: workflow run https://github.com/ranxianglei/billion-context/actions/runs/34690025787 (in progress on 40ab16b). Merging stays human-only — once that run is green, this is ready.

中文摘要:修复了死锁冷却检查的放置位置(原来在 #300/#496 安全转发路径之前,会把"实际能装下窗口、仅因陈旧高基线触发"的请求误判成缓存 502)并补上丢失的 retryable 标志(429 触发的冷却重放不应丢掉可重试信号),已在 PR 分支直接补了修复+回归测试(验证过旧代码必挂、新代码通过),全量测试 1423 通过,E2E 已 dispatch,等其变绿即可合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 E2E dispatch result (run https://github.com/ranxianglei/billion-context/actions/runs/34690025787, on 40ab16b): the full suite failed for an infra reason, not a code regression — both E2E_UPSTREAM_URL and E2E_UPSTREAM_KEY arrived empty in the runner env (repo secrets not set), so the log shows upstream /models probe: unreachable and the codex warmup exited 1 before any proxy logic ran (warmup exit=1 last= at tests/e2e/e2e-codex.test.ts:212). The zero-token preflight stage (E2E_CHECK=1: codex CLI install + dist build + reachable-check) passed. Nothing in this PR touched the warmup path; local verification stands (typecheck ✅, 1423 pass / 0 fail ✅, build ✅).

Action needed from you: set/refresh the E2E_UPSTREAM_URL + E2E_UPSTREAM_KEY repo secrets, then re-dispatch ci-e2e.yml on this branch (I'll re-verify if you'd like me to). Everything else in the review above holds; merge stays with you once E2E is green.

中文摘要:E2E 已按要求 dispatch 但失败原因是仓库缺少 E2E_UPSTREAM_URL/KEY 两个 secret(上游为空、warmup 直接 exit 1,与代码无关;零 token preflight 阶段通过),请补上 secret 后重新 dispatch,其余审查结论不变,等 E2E 变绿后可合并。

@ranxianglei
ranxianglei merged commit 0d04002 into master Sep 12, 2026
6 of 7 checks passed
antclt pushed a commit to antclt/billion-context that referenced this pull request Sep 15, 2026
…anxianglei#780)

extractSummaryFromSse conflated 'the stream delivered a complete summary'
with 'the stream died mid-delivery': a gateway truncation (ranxianglei#764 shape: half-line
data, no final event) left a partial accumulator that was persisted as a
complete tier-1 summary - silently worse than an empty one, because ranxianglei#727's
diagnosis chain only fires on empty results.

- a data line that fails to parse is corruption (badFrame), not skipped noise
- event: frame headers type frames whose payload lacks a type field, reset at
  every frame boundary (no cross-frame leakage)
- failure terminals (response.incomplete/.failed/.error, generic error, bare
  error object) invalidate any text accumulated before them
- responses requires the spec-mandatory response.completed terminal; its full
  response object stays authoritative over delta accumulation
- delta-less terminals (response.output_text.done / output_item.done) now carry
  their text instead of extracting an empty string
- anthropic/openai bodies must end on a frame boundary (CRLF-tolerant);
  finish_reason/[DONE]/message_stop are NOT required (ranxianglei#764: real gateways
  omit these occasionally)
- diagnoseEmptySummary names truncation explicitly (incomplete data lines /
  missing final frame terminator) so logs say what actually happened

Rejected streams return '' and flow into the existing diagnose + halving +
cooldown chain (ranxianglei#726/ranxianglei#727), closing that recovery loop for truncated streams.

Preflight: typecheck clean, 1535 pass / 0 fail / 2 skipped, build ok.
New coverage: 23 unit tests (tests/preflight-sse-extraction.test.ts) + 1
full-pipeline integration test through the real proxy HTTP server.
@ranxianglei
ranxianglei deleted the 2026-09-12_preflight-empty-summary-recovery branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502

1 participant