Repository navigation
fix: preflight classifies normal-stop zero-text summaries as deterministic (#2832) - #2840
Conversation
…istic (#2832) - diagnoseEmptySummary gains per-wire block fingerprints (text_chars / text_blocks / thinking_blocks / reasoning_chars / text_parts) so a thinking-only answer is distinguishable from relay-stripped text in logs and fail-fast details - new isDeterministicEmptyCompletion: a normal stop (end_turn / stop / completed) carrying zero answer text is deterministic for that (model, host) pair — it earns neither #1767's same-span retries nor #726's halving recovery, because re-sending the identical payload returns the identical empty answer and a normal stop is not a size signal - three consecutive deterministic empties on the same (host, model) arm a session-persisted dead-probe verdict (session.metadata deterministicEmptyProbe); while dead, at most ONE canary probe per preflight invocation bills the wall, and a successful canary clears the verdict so folding resumes on a self-healed relay - fail-fast detail names the actionable exits: configure compress.externalSummary, switch the session's model, or fix the relay's thinking/text handling - tests: classifier matrix + fingerprint pins (unit), systemic e2e (bounded strikes, sticky verdict, exactly-one-canary contract, nothing forwarded), recovery e2e (canary clears, fold proceeds, 200); #574 multi-range fixture moves to max_tokens so that file keeps pinning the walk/budget mechanics on the size-driven path independent of this reclassification
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.3143-pr.2840.e2f862dThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2840.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 PR #2840 复核完成。本 PR 是我上一轮针对 #2832 的实现;本轮按 [PR] 唤醒规则拉取分支核对代码、确认 CI 并给出审查结论(未重做)。 核查范围:单 commit ① 是什么问题:preflight 把「正常结束(end_turn/stop/completed)+零正文」的确定性空摘要误判为瞬时故障——每个 span 烧 2 次同-span 重试 + 共享预算 + #726 减半级联,跨 span/range/turn 无上限计费,且诊断串无法区分「全答在 thinking」与「中继吞掉 text」。 ② 什么原因导致的: ③ 影响范围与严重程度:走 preflight LLM fold 且摘要模型经中继回 thinking-only 正常结束(或中继剥 text)的用户;触发 = 超窗 + 该形态稳定复现。特定条件下完全无法折叠(每 turn 烧光探测预算、零 fold),最终 fail-fast 502 或落入 #2313 evidence-forward 全额计费空 turn(即事故)。非该形态用户不受影响。 ④ 怎么修复的(机制层三件事,已逐一读码验证):
⑤ 是否有回归问题(逐项清点既有行为):尺寸驱动(#726/#853/#874 max_tokens/length 仍减半)✓保持;瞬时光(#1767 content_filter、#780/#784 truncated/incomplete 仍重试)✓保持;external-summary / batch-fold / futility(#1841) / round-budget(#574/#1933) / dead-end cooldown ✓均未触碰。一处测试夹具配套改: ⑥ 是否破坏缓存:不涉及。不碰消息重建/system 锚点/prefix-cache 字节;diff 只落 ⑦ 是否改了设计方向:不涉及架构转向。在既有 transient/size-driven 二分上补第三类 deterministic,判决持久化沿用 session.metadata 纪律。两种压缩模式行为一致(preflight 本就是双模式 last-resort backstop #470;改动只涉分类/retry/halving/walk-stop,不碰 carrier/rendering——diff 未触及 server.ts/adapter)。 ⑧ 是否引入新配置:无。阈值硬常量 测试有效性:真测试有牙。classifier 矩阵 14 例钉住每个边界(content_filter/max_tokens/length/incomplete/google/SSE 均判 not-deterministic);指纹串精确匹配区分 thinking-only vs stripped-text;两个 e2e 断言 bounded billing(3–6 vs 旧 unbounded)+ exactly-one-canary + self-heal 清 verdict。3 个既有 pin 更新都加了牙(text_chars/text_blocks 精确串)+ content_filter 仍 transient 断言。 CI:Linux 双 node(22/24)全绿;guard 全绿(lane/docs/lock/version/config-surface/kernel-guard/build/image-billing);Windows 双 node 仍在跑(同一套套件,Linux 已过)。 收工判定:代码核对成立、方向正确(修根因非症状)、无意回归 0、测试有牙、Linux CI 全绿。等 Windows CI 收尾后即可由人合并 PR #2840,我这边无需再改动。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.297-master.b7be571 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Closes #2832
What was wrong
A relay model that answers every preflight summary probe with a NORMAL stop (
end_turnon the anthropic wire,stopon openai,completedon responses) and zero answer text — the whole answer riding in thinking blocks, or the relay stripping the text blocks — was classified transient by preflight (no size signal matched). Every span therefore burned two same-span retries plus the per-regime retry budget, then the #726 halving cascade re-billed doomed sub-spans, for EVERY span of EVERY range EVERY turn. Incident (2026-10-11, bili 0.1.188): six failed probes over six minutes, ZERO folds landed, no sticky verdict, no actionable advice — the turn fell through to an estimate-only evidence forward that billed the full payload. The diagnosis string also could not tell thinking-only from relay-stripped-text, so operators had nothing to act on.The fix (mechanism-level)
isDeterministicEmptyCompletion(protocol, json): a plain-JSON completion that ended normally yet carries zero extracted answer text is deterministic for that (model, host) pair — re-sending the identical payload returns the identical empty answer. Such outcomes settransient: false(no bili会突然暂停发继续又可用 #1767 same-span retries) and a newdeterministicflag. Non-zero short answers, SSE bodies, size-driven stops, content_filter, truncated/incomplete streams all stay exactly where they were.(host, model)key arm a verdict persisted in session metadata (deterministicEmptyProbe, same discipline as the existing learned flags). While dead, at most ONE canary probe per preflight invocation bills the wall; the walk stops with an actionable fail-fast detail. A successful canary clears the verdict and folding resumes on a self-healed relay. Keyed by (host, model), so a stale entry is inert after a relay/model switch; the external-summary lane bypasses the verdict entirely (different executor).emptyCompletionDetail()now reports where the model's output went:text_chars=N/text_blocks=N/thinking_blocks=N/reasoning_chars=N(openai + anthropic) andtext_parts=N(+reasoning_chars) on responses — logs and fail-fast details finally distinguish thinking-only from stripped-text from truly-nothing.Behavior changes (old → new)
(finish_reason=stop, answered as model=X)(stop_reason=end_turn, text_blocks=0, thinking_blocks=2, reasoning_chars=1873, answered as model=k3-256k)— three pinned test strings updated accordinglySize-driven (#726/#853/#874), transient (#1767/#780/#784), external-summary, batch-fold, futility (#1841), round-budget (#574/#1933), and dead-end-cooldown behavior are unchanged.
Both compression modes
Preflight fires in BOTH modes as the last-resort backstop when the input alone exceeds the window (TECHNICAL-NOTES.md "Two compression modes", #470). This change touches only probe classification, retry/halving gating, and walk stopping — no summary carrier, no system-message handling, no tool/nudge injection — so plugin mode (agent-owned compression) and proxy mode get identical treatment. Verified: sole entry is
preflightCompressIfNeeded(src/server.ts:2354) called from src/server/handle.ts:2011; no carrier/rendering paths touched.Config surface
None added. The strike limit is a hard constant (
DETERMINISTIC_EMPTY_STICKY_LIMIT = 3, matching the auto-update 3-strikes convention); the verdict lives in internal session metadata, not config/env/CLI. The operator exits already exist and are named in the advice:compress.externalSummary, switching the session model, or fixing the relay.Tests
tests/issue2832-preflight-deterministic-empty.test.ts: classifier matrix (14 cases incl. thinking-only→deterministic, any-text→not, max_tokens/length/content_filter/incomplete→not, google/SSE/null→not), exact fingerprint strings per wire, systemic e2e (verdict arms within 3 turns, total strikes bounded 3–6 vs unbounded before, nothing forwarded, EXACTLY one canary call on the carried-verdict turn, verdict survives), recovery e2e (canary clears → fold proceeds → 200, metadata deleted).preflight-empty-summary,preflight-transient-empty-retry).tests/preflight-multi-range.test.tsfixture moved fromend_turn+empty tomax_tokens+empty ON PURPOSE: its failure shape was silently inside the class this PR reclassifies, which would let the dead-probe verdict shadow the budget-cap mechanism that file exists to pin. It now stays on the size-driven halving path its "2 calls per failed range" arithmetic describes (provenance comment added).Kernel untouched; pi-subagents untouched; no version bump.