Repository navigation
fix: recover preflight from empty SSE summaries + dead-end cooldown (#726) - #727
Conversation
…726) - Diagnose HTTP-200 summarization bodies that carry no summary text: parse terminal error signals (SSE error / response.failed / response.incomplete events, bare JSON error) into the log line and the fail-fast message, so a 0-char summary no longer hides the upstream's actual rejection reason. - Retry unusable chunks at half size (oldest half first) down to a floor before skipping the range, bounded by the existing per-regime call budget — recovers size-driven rejections (a CHUNK_FRACTION-sized chunk exceeding the upstream's real input cap), which was #726's stable 502 loop on ChatGPT-backend codex. - Zero-progress preflight failures arm a per-session dead-end cooldown (default 5m, BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS): identical over-window requests fail fast with the cached diagnosis and zero upstream calls instead of re-burning quota on the same doomed walk; any non-failure outcome clears it.
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-727Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr727.tgz
npm install -g packageThis comment is automatically updated on each push. |
…rve its retryable flag
[bot] 🏷 PR review (branch Duplicate screening: no duplicates. This is the only open PR touching preflight recovery; it directly fixes #726 (whose triage already established the three-layer root cause this PR addresses), and the out-of-scope growth-source issue #728 carries proper source attribution ( Diff cleanliness: clean — 3 files, all on-purpose ( Verified locally (on
Two issues found, fixed directly on the PR branch (commit
Post-fix verification: typecheck ✅ · full suite 1423 pass / 0 fail / 2 skipped (was 1422; +1 new regression test) ✅ · build ✅. Minor observations (report only, no action taken): under a systemic rejection the halving ladder now concentrates up to the full 8-call budget on the oldest range instead of spreading ~1 call across ranges — bounded and acceptable given the cooldown caps cross-request burning; partial-progress failures (folded ≥1 range but still over) neither arm nor clear the marker — correct, since each such retry makes durable progress so no livelock forms; stale expired markers persist in metadata until the next arm/clear — single slot per session, harmless. E2E dispatched as requested: workflow run https://github.com/ranxianglei/billion-context/actions/runs/34690025787 (in progress on 中文摘要:修复了死锁冷却检查的放置位置(原来在 #300/#496 安全转发路径之前,会把"实际能装下窗口、仅因陈旧高基线触发"的请求误判成缓存 502)并补上丢失的 retryable 标志(429 触发的冷却重放不应丢掉可重试信号),已在 PR 分支直接补了修复+回归测试(验证过旧代码必挂、新代码通过),全量测试 1423 通过,E2E 已 dispatch,等其变绿即可合并。 |
[bot] 🏷 E2E dispatch result (run https://github.com/ranxianglei/billion-context/actions/runs/34690025787, on Action needed from you: set/refresh the 中文摘要:E2E 已按要求 dispatch 但失败原因是仓库缺少 E2E_UPSTREAM_URL/KEY 两个 secret(上游为空、warmup 直接 exit 1,与代码无关;零 token preflight 阶段通过),请补上 secret 后重新 dispatch,其余审查结论不变,等 E2E 变绿后可合并。 |
…anxianglei#780) extractSummaryFromSse conflated 'the stream delivered a complete summary' with 'the stream died mid-delivery': a gateway truncation (ranxianglei#764 shape: half-line data, no final event) left a partial accumulator that was persisted as a complete tier-1 summary - silently worse than an empty one, because ranxianglei#727's diagnosis chain only fires on empty results. - a data line that fails to parse is corruption (badFrame), not skipped noise - event: frame headers type frames whose payload lacks a type field, reset at every frame boundary (no cross-frame leakage) - failure terminals (response.incomplete/.failed/.error, generic error, bare error object) invalidate any text accumulated before them - responses requires the spec-mandatory response.completed terminal; its full response object stays authoritative over delta accumulation - delta-less terminals (response.output_text.done / output_item.done) now carry their text instead of extracting an empty string - anthropic/openai bodies must end on a frame boundary (CRLF-tolerant); finish_reason/[DONE]/message_stop are NOT required (ranxianglei#764: real gateways omit these occasionally) - diagnoseEmptySummary names truncation explicitly (incomplete data lines / missing final frame terminator) so logs say what actually happened Rejected streams return '' and flow into the existing diagnose + halving + cooldown chain (ranxianglei#726/ranxianglei#727), closing that recovery loop for truncated streams. Preflight: typecheck clean, 1535 pass / 0 fail / 2 skipped, build ok. New coverage: 23 unit tests (tests/preflight-sse-extraction.test.ts) + 1 full-pipeline integration test through the real proxy HTTP server.
Fixes #726
Problem
After the #626 (stream) and #663 (max_output_tokens) compatibility retries, a ChatGPT-backend summarization call can still return HTTP 200 SSE with zero extractable summary text — an in-stream error event or truncated stream that
extractSummaryFromSsesilently discarded. The only trace wassummary too short (0 chars), one viable range got skipped whole, and every codex auto-retry re-ran the identical doomed walk (~8 large summary calls per retry), producing the stable502 retryable=falseloop from #726.Changes
src/preflight.tsdiagnoseEmptySummary(text, json?)scans a no-text body for terminal error signals: SSEerror/response.failed/response.errorevents (response.error.code+message),response.incomplete(status + optional message), bare JSON{error: {...}}, plus fallbacks for empty bodies, event-carrying streams without text, and non-SSE bodies. The diagnosis goes into the warn log and is threaded into the fail-fast 502 message (Last unusable summary: …), so the operator/client finally sees WHY the summary was empty.2×MIN_CHUNK_TOKENStokens /2×minCompressRangechars) before the range is skipped. This recovers size-driven rejections — the prime suspect in [Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726, where the first chunk wasCHUNK_FRACTION × window≈ 630K tokens of summarization input — while staying bounded by the existingMAX_SUMMARY_CALLS_PER_PREFLIGHTbudget. Skip-on-exhaustion semantics for genuinely unsalvageable ranges are preserved.src/server.tscompressedRanges === 0(zero progress under unchanged state = deterministic dead-end), the failure fingerprint (model\u0000limit) + status + full message is cached insession.metadata.preflightDeadEndfor a cooldown (default 5 min,BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS). Matching requests fail fast with the cached diagnosis and zero upstream calls; any non-failure outcome clears the marker; aborted failures never arm it. The 502 message gains a recovery hint (restart the session). This stops the quota-burning retry loop from the issue's logs (8 identical walks in 48s).tests/preflight-empty-summary.test.ts(new)response.failed; halved chunk succeeds → request 200, payload forwarded, no dead-end marker left behind.diagnoseEmptySummaryacross all signal shapes.Pre-flight
npm run typecheck✅npm test— 1422 pass / 0 fail / 2 skipped (gated e2e) ✅npm run build✅E2E_CHECK=1e2e preflight ✅. FullACP_TEST_E2E=1run not possible in this environment: the default local upstream (http://127.0.0.1:8199/v1) is down here (proxy log shows only ECONNREFUSED to it); please dispatch.github/workflows/ci-e2e.ymlbefore merging since this touches the request pipeline.Out of scope (separate issue filed)
The deeper blind spot — nudge never fires when the upstream never reports usage (
usage=0%in #726's logs), letting context grow to 1.32M without any incremental compression — is kernel-layered and needs its own design discussion. Filed separately with this issue as source.