Repository navigation
fix: cross-spec compress-failure circuit breaker + stale-generation diagnosis (#2146) - #2147
Conversation
[bot] 🏷 收到,开始 review PR #2147(closes #2146)。已拉取分支 |
7e7aa6f to
984c8d4
Compare
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.2381-pr.2147.fb3dff0This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2147.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 Review 完成。分支原本与 master 冲突(mergeable_state=dirty),已 rebase 到最新 master 并解决冲突后推送:984c8d4(原 7e7aa6f)。CI 20/20 全绿,PR 现 本轮 review 做的事
结构化七问
测试有效性:6 个新测试走真实 kernel 执行(applyRanges × 真实 session state),断言钉死 receipt 精确措辞与 metadata 状态(arm@3、continue@4、warn 日志、成功清除后重启、11min 衰减 vs 9min 延续、beyond-note 文本+混合端点抑制、bN 端点/空态单元形、loopTracking:false 五项隔离)——有牙,能抓住该机制的回归;issue847 更新断言钉住 3× 时的取代关系。已知缺口:没有端到端复现完整事故场景(stale-generation 重启)的 hermetic harness,单测钉桩 + CI 真实客户端 lane 覆盖下可接受。 判定:可合并(mergeable clean,CI 20/20 绿)。合并动作留给人:#2147 |
3773b9f to
40097a2
Compare
40097a2 to
62bb0c7
Compare
|
已 rebase 到当前 master(818d24e+)——零冲突,全绿。 本地验证:tsc 干净、 与 #2110 家族的关系:本 PR 修的是"压缩一直失败无降级"(161 连败烧 106K×N input 的陈旧 generation 场景),与 #2110 的"无人压缩"互补——失败侧的止损。熔断逻辑不掩盖任何东西:每次失败照常入账,跨 spec 累计才升级为 hard stop + stale-generation 诊断,与 #2110 留言里"失效要显性失败"的处置一致。 可独立 review 合并;与 #1818/#1851 同在 preflight/compress 区域但不冲突(本 PR 主要在 src/stream.ts)。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.88-master.9f82bbe # this exact build
npm install -g billion-context@master # always the newest merged build
|
Closes #2146
Problem
After a dsh client restart (plugin mode), the model (glm-5.3-flash) kept calling
compresswith ranges that could never resolve against the live session — stale-generation refs left in its persisted history. The incident log (bili (17).log, v0.1.179) shows 161 consecutive compress failures over 2h42m, each on a distinct spec (walking spec space: sequential 50-ref windows above the frontier, then tracking slightly ahead of it, finally huge tail ranges), while the model never calledacp_statusonce despite every receipt asking it to. Each round trip burned ~106K input tokens.Root cause
The existing anti-loop guard is keyed on the EXACT spec (#847: fires only when the same normalized range repeats ≥2×). A model that issues a fresh failing range each call never trips it. The #1026 span hint and #1029 ref snapshot were already inside every failure receipt — present but unread by this model. Nothing tracked "still failing" across DISTINCT specs, so no escalation path existed from diagnostic hints to a hard stop.
Secondary gap: when ALL requested refs sit strictly above the session's highest mapped ref, the kernel's generic "typo or wrong session" wording misleads the model into hunting a typo that cannot exist — the deterministic cause is stale-generation artifacts in the client's persisted history.
Fix (host-side, no kernel change)
src/stream.ts:noteCompressLoopFailure): a session-level streak of consecutive TOTAL compress failures (any spec), persisted insession.metadata["compressFailStreak"]alongside 出现持续压缩失败,无法自动恢复 #847's map. At 3 consecutive failures the failure receipt swaps the diagnostic hint cluster for a hard-stop paragraph ("STOP calling compress now … Continue your actual task"). Disarms on any successful compress or after 10 min quiet (refs keep advancing DURING the loop, so frontier-based decay would disarm mid-loop). The breaker NEVER refuses execution — it only rewrites what a FAILED call tells the model, so a legitimate call can always still succeed and reset the streak. Warn log on arm and on each armed failure ([warn: compress-loop] …) for operator diagnosis.beyondFrontierNote): when every requested endpoint is above the session's highest m-ref, the receipt states those refs cannot exist here, names stale artifacts from an earlier generation as the cause, and notes that NO superset of them can ever compress.applyRanges(unparseable call / zero-blocks gate+kernel rejection / throw).src/preflight.ts: preflight overflow-compress passes{ loopTracking: false }— bili's own internal lane must not arm the model-facing breaker on attempts the model never made. Its existingstartsWith("[Compression FAILED")check is untouched (failure prefix unchanged).Behavior changes (disclosed)
[…errors][Current context…][Repeat-failure guard…][span hint][no-viable…], now[…errors][beyond-note?] [COMPRESS CIRCUIT BREAKER: N …]— the hint cluster is REPLACED by the hard-stop paragraph (stacking more ignored advice is exactly what failed in the incident). Applies to BOTH modes: wire and plugin shareapplyRanges.tests/issue847-compress-failure-guards.test.tsupdated accordingly).No other output-format, threshold, timing, or default changes.
Config surface
None. Threshold (3) and decay window (10 min) are hardcoded constants by design, per env/config-surface discipline (cf.
DEGENERATE_FOLD_COVERAGE).Both compression modes
Shared
applyRanges→ plugin mode (dsh, the incident lane viasrc/loop/core.ts) and wire mode (OpenAI/Responses/Anthropic streams) both get breaker + diagnosis; preflight lane explicitly excluded; success path clears the streak in all lanes.Tests
tests/issue2146-compress-loop-breaker.test.ts(6): distinct-spec arm at 3 + count continuation + warn logs; success clear; 10-min decay vs 9-min continuation; beyond-frontier receipt text + mixed-endpoint suppression; unit shapes (bN endpoints, empty state);loopTracking:falseisolation.tests/issue847-compress-failure-guards.test.ts(the intended behavior change fix: keep system prompt byte-stable for prefix cache #1 above).Addendum (commit 3773b9f): receipts no longer echo phantom refs back
Owner asked whether the fix was stop-the-bleeding without locating WHY the loop runs. Deeper log analysis found a self-sustaining channel the first commit left open: every failed compress receipt quotes the kernel's per-range error text verbatim (
startId="m29971" does not exist…), which lands in the client-persisted history as a tool result — where the looping model reads its own past failures as unfinished work and slides further up the same phantom ladder (incident log: m29971–m30020 → m30021–m30070 → m30071–m30120 within 20s, retries widening to m30121–m30260 hours later).Fix: when
beyondFrontierNotefires (every requested ref provably above the session's highest mapped ref — the stale-generation case), the requested endpoint refs are scrubbed from the model-visible receipt ([stale-ref]placeholder); the server log keeps the real refs for operators. In-frontier unknown endpoints stay verbatim (a genuine typo the model can correct). Safety invariant: when the note fires, no live snapshot number can collide with a scrubbed token. Composes with the #1875ProxyToolResultenvelope (compressResult(receipt, "refused", 0)).