Skip to content

[PR] fix(#4331): report uncovered mid-history spans instead of claiming "at its minimum" - #4340

Closed
ranxianglei wants to merge 3 commits into
masterfrom
2026-10-23_uncovered-history-audit
Closed

ranxianglei wants to merge 3 commits into
masterfrom
2026-10-23_uncovered-history-audit

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #4331

查重: 无 — closest relative #4227 was a reporting-layer baseline mismatch (fixed by #4229); related-but-distinct mechanisms: #4307 (stuck compress calls), #4289/#4293 (failed exchanges exempt from folding), #4240 (batch rejection). This is a distinct failure domain: genuinely-uncovered mid-history spans plus an ungrounded global-minimum verdict.

What this does

The stop signal ("No compressible ranges remain — the context is already at its minimum") fired on PRE-FOLD OFFER exhaustion alone (#2404 pinning), while nothing anywhere compared the union of active-block ranges against the session's ref set. The issue's live ZCode session proved the gap at scale: 138 mid-history refs (~65K tok ≈ 16% of context) in no active block, largest run m00312–m00411, with the receipt declaring the minimum three times in a row. Repro on master confirmed the kernel DOES find such spans when they are offered — what was missing was a GLOBAL coverage audit plus a verdict that only claims "minimum" when one holds.

  • src/uncovered.ts (new) — uncoveredHistoryRuns(): contiguous runs outside every active block's coverage (the collectVisible predicate), classified by: skip if any current offer covers part of it; empirical protected-zone diff via two core.processTurn passes over a cloned state (normal config vs preserve-0/0 relaxed) — reusing the kernel's own zone rules instead of copying them, so it self-maintains as those rules evolve; runs below VIABLE_RANGE_MIN_TOKENS (200) stay silent. Advisory only: any throw degrades to "no runs found" and never breaks receipts or status.
  • acp_status — additive UNCOVERED HISTORY section (top-5 runs by tokens, rest counted), filtered against the live Compressible-ranges list above it; names isolated sub-gate residue between covered blocks that no merge will ever fold (the permanent-hole class of 出现持续压缩失败,无法自动恢复 #847/[Bug] compress reports 'no compressible ranges remain' while mid-history spans sit outside every block #4331). Explicitly diagnostic, not a fold suggestion.
  • postCompressTail — on clean whole-offer exhaustion with no tier distillation, the audit runs BEFORE the verdict: uncovered mass → honest "not at its minimum" text naming the runs; none → the verbatim compress 成功输出缺剩余范围快照与停止提示(pi 侧 #420/#521 对齐) #1387 contract string fires byte-stable, as before.

Behavior changes (disclosed)

Surface Old New Why
compress success tail, whole-offer exhaustion unconditional "already at its minimum" claim only when the global audit verifies it; otherwise name the uncovered runs the old claim was structurally unverifiable — exactly the #4331 false stop
acp_status default scope no coverage view beyond the ranges list additive UNCOVERED HISTORY section for material (≥200 tok/run) uncovered runs visibility for permanent holes; rows drill down via scope:"uncompressed" unchanged
sub-gate residue (runs below minCompressRange) invisible on every surface named in the acp_status diagnostic (still never offered/folded) diagnostic ≠ offer; #847 gate discipline preserved on the offer side

Nudge prompt text untouched (kernel-owned). Fold eligibility / apply behavior untouched.

Config surface

None — no new env/config/CLI fields; the floor reuses VIABLE_RANGE_MIN_TOKENS.

kernel-change: none

Regression review

Existing behavior audited on touched paths: #2404 offer-pinning (subset folds still advertise the rest; protected-at-nudge ranges still never leak AS OFFERS — updated pinned tests assert that form), #1387 verbatim stop contract (byte-stable for genuinely-drained sessions — negative test), tier-distillation precedence (no stop AND no uncovered verdict while a tier is active), #2029 status-baseline provenance (every status-reader kernel call feeds the provenance-aware baseline — audit included), #847 failure-guard offer gating, viable-range floors on recommending surfaces. Local: typecheck clean + targeted suites green (9 new gates + all four updated pinned files, 43 pass); full-suite verdict = CI on this head.

Known limitation (separate observation, not fixed here)

The receipt's remaining-calc removes a WHOLE offered batch when any submitted span overlaps it, so a partially-folded merged batch can under-report rather than over-report (safe direction). Worth a follow-up issue if it proves user-visible.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-23_uncovered-history-audit (94819f4)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.5230-pr.4340.f861986

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4340.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix(#4331): report uncovered mid-history spans instead of claiming "at its minimum" [PR] fix(#4331): report uncovered mid-history spans instead of claiming "at its minimum" Oct 22, 2026
@ranxianglei ranxianglei added bug Something isn't working p3 scope/wiring labels Oct 22, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复——按失败签名(#4331 / uncovered / "at its minimum")检索 open+closed issues 与 PRs,唯一相关项是 PR #4339:同一工作线的前身(同分支名、同 issue、旧 head 776b636 正是本分支 force-push 前的提交)。本 PR(squash 成单提交 b502443、rebase 到当前 master)取代 #4339,建议合并时顺手关掉它(我不自行关 PR)。master CI 红已有 #4326 跟踪(#4174+#4330 在途修复),不另立项。
标签: bug / scope/wiring / p3 —— 压缩回执与状态面的诊断诚实性缺陷,不涉及可用性,打这三个。

审核结论:代码本身可合

独立核过代码与测试(不是复读 PR 描述):

审核中直接修掉的一个小问题(已推独立标注 commit)

unused-exports 门红了:UncoveredRun / UncoveredAuditInput 两个接口只用于导出函数的签名、无任何跨文件导入,被门判为 INTERNAL_ONLY。按门自己的指引去掉前导 export(commit 7981fee05,2 行,行为零变化;本地复跑门+typecheck+定向测试全绿)。这个红原先让 fail-fast 层把 test/static-gates/red-signal 三条车道整体 SKIP 掉——修掉后车道才会真正跑起来。

CI 现状说明(PR 上还会看到的红与归属)

八问

  1. 是什么问题:compress 回执「上下文已到最小」只看当前折叠候选列表是否耗尽,不检查历史里是否还有未被任何活跃块覆盖的 span——模型和用户被告知「压无可压」,而实测(ZCode 现场)还有 ~16% 上下文没被覆盖。
  2. 什么原因导致的:停止信号由 pre-fold offer 单独驱动(post-compress tail immediately re-advertises newly-released ranges -> chained folds at zero growth -> upstream prefix-cache misses #2404 钉死),管线里没有任何一处比较「活跃块覆盖并集 vs 会话 ref 全集」;受保护窗口让部分 span 永久不可 offer,旧文案把「不可 offer」误当成「已压到底」。
  3. 影响范围与严重程度:所有走到 offer 耗尽且存在覆盖空洞的会话;严重度 = 误导性诊断(回执文案 + acp_status 漏报),不破可用性、不破数据 → p3。
  4. 怎么修复的:判词前加一个 advisory-only 审计(src/uncovered.ts)——用两次 core.processTurn(正常配置 vs 清空保留窗口)对 state 克隆做经验性保护区差分,得到 zone-hidden ref;剔除已被活跃块覆盖与当前可操作 span 覆盖的区间后,按数组连续性报 ≥200 tok 的连续 run。有残余 → 回执点名区间说「未到最小」,acp_status 加 UNCOVERED HISTORY 段(top-5);无残余 → 原契约字符串逐字不动。
  5. 是否有回归问题:逐项过既有行为——post-compress tail immediately re-advertises newly-released ranges -> chained folds at zero growth -> upstream prefix-cache misses #2404 offer 钉死不变;compress 成功输出缺剩余范围快照与停止提示(pi 侧 #420/#521 对齐) #1387 逐字停止契约在无残余时保持(负向测试钉死);tier 蒸馏优先级高于两个新信号;Status readers still trust estimate-grade failure baselines after #1846 #2029 状态基线溯源保持(每次 kernel 调用喂 provenance 基线);出现持续压缩失败,无法自动恢复 #847 sub-gate 残余只进诊断、永不进 offer/fold;viable-range 下限不变。未发现无意回归。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:否——纯增量诊断 + 判决点文案更诚实,折叠决策机制本体未动。
  8. 是否引入新配置:无(Config surface: None)。

已知局限(PR 里已披露、不在本 PR 修):回执 remaining 计算在任一已提交 span 重叠时整批扣减——部分折叠的合并批次可能少报剩余(安全方向);用户侧若可见再立后续。

合并判定:代码就绪(review 通过 + 本地定向验证绿 + 审核小修已推)。等待两件事:① 7981fee05 的新 CI run 确认本 PR 自身车道绿;② 人工合并 #4174+#4330 清掉继承的 master 红(否则 shard 车道无论怎样都红)。顺带建议关闭被取代的 #4339。

@ranxianglei ranxianglei added the ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR label Oct 22, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 状态跟进(接上一条):

  1. 我这边造成的一次 CI 干扰,已纠正:给本镜像 issue 打标签时,平台同步把标签推到了 GitHub PR 上,触发 labeled 事件——ci.yml 的 plan 逻辑对非 ci:full 标签事件按 scope=skip 处理,把我 push 触发的 synchronize run 顶掉了(只剩 guards 跑、重车道全 skip)。已给 PR 补上 ci:full 标签重新拉起完整矩阵(run 39829266389)。教训记下:平台标签同步会打 GitHub 的 CI 事件。
  2. 新 run 当前状态:tier-1 全绿——unused-exports ✓(确认小修生效)、kernel-guard / lock-sync / test-import-graph / docs 等全部 success。test shards 与 mysql-contract 的红逐条核对过日志:全部是 CI red on master since #4177: unregistered spawn test kills all test shards; mysql-contract lane fails #4326 继承红的原签名——shard 死在 process-spawning test files missing from LIVE_FILES: tests/storage-durability.test.ts(tripwire,一个测试都没跑),mysql-contract 死在 ERR_MODULE_NOT_FOUND ... acp-kernel/dist/index.js。没有任何一条失败能归因到本 PR。
  3. 结论不变:代码就绪;等两件事——① 剩余 in-progress 车道收尾(预期结局同上:PR 自身零新增红);② 人工合并 [PR] web: Rotate the "Compressed Structure" tree view of large sessions to vertical full size, no longer shrunk to sub-pixel | web: 大会话「压缩结构」树图旋转为竖向全尺寸,不再缩成亚像素 (#4173) #4174 + [PR] fix(ci): build acp-kernel in the mysql-contract lane (sub-defect B of #4326) #4330 清掉继承的 master 红,shard 车道才会真正跑到测试体。顺带再提一次:[PR] feat: audit uncovered history behind the "no ranges remain" verdict (#4331) #4339 是本 PR 的前身,建议合并时关闭。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 收尾确认(run 39829266389,full matrix,head 7981fee05):

最终判定不变:代码可合。放行条件只剩两件事,都在人这边:① 合并 #4174 + #4330 让 shard 车道真正跑到测试体(届时若 shard 绿,本 PR 即全绿);② 关闭前身 PR #4339。

awork added 3 commits October 23, 2026 16:58
…nest minimum verdict

"no compressible ranges remain — the context is already at its minimum" fired on
offer exhaustion alone, with no global coverage audit anywhere: live sessions
carried ~16% of context as mid-history refs outside every active block while the
receipt declared the global minimum (#4331).

- src/uncovered.ts: uncoveredHistoryRuns() — contiguous runs outside every
  active block's coverage, classified by an empirical protected-zone diff (two
  processTurn passes: normal config vs preserve-0/0 relaxed, over a cloned
  state), current-offer exclusion, VIABLE_RANGE_MIN_TOKENS floor. Advisory
  only: any throw degrades to "no runs found".
- acp_status: additive UNCOVERED HISTORY section (top-5 runs by tokens, rest
  counted), filtered against the live Compressible-ranges list above it; names
  isolated sub-gate residue between covered blocks that no merge will ever fold.
- postCompressTail: on clean whole-offer exhaustion with no tier distillation,
  run the audit first — uncovered mass -> honest "not at its minimum" text
  naming the runs; none -> the verbatim #1387 contract string, byte-stable for
  genuinely-drained sessions.

Behavior change (disclosed): old = unconditional minimum claim on offer
exhaustion; new = claim only when the global audit verifies it, honest
uncovered-listing otherwise. Sub-gate residue became visible in acp_status
(was invisible on every surface). No config/env surface added; kernel untouched;
nudge prompt text untouched.

Known limitation (separate observation, not fixed here): the receipt's
remaining-calc removes a whole offered batch when any submitted span overlaps it,
so a partially-folded merged batch can under-report.

Tests: tests/issue4331-uncovered-history.test.ts (9 gates incl. negative
verbatim-contract + tier precedence); updated pinned contracts in
tests/compress-success-tail.test.ts (#2404/#1387 now pin "never leaks AS AN
OFFER"), tests/issue2029-status-baseline-provenance.test.ts (every status-reader
kernel call feeds the provenance baseline), tests/issue847-compress-failure-guards.test.ts
(sub-gate residue visible in the diagnostic, still never offered). TECHNICAL-NOTES.md
entry appended.
…d-exports gate)

The two audit types are only used in the signatures of the exported functions; no cross-file import exists, so the unused-exports gate flags them as INTERNAL_ONLY. Drop the leading export per the gate's own guidance; behavior unchanged.
… pin, status-pass reuse

Response to reviewer feedback on the #4331 PR:
- postCompressTail: the honest uncovered verdict now fires only when total
  uncovered mass >= UNCOVERED_RECEIPT_FLOOR_TOKENS (10K); below the floor the
  verbatim #1387 wording fires byte-identical to master (pi #521 anti-nag:
  small leftovers must not re-nag on every fold). tests/compress-success-tail.
  test.ts is back at master bytes — its sub-floor residue fixtures now pin the
  floor directly (zero diff vs master).
- uncoveredHistoryRuns: optional normalRanges input — a caller that already
  ran the normal-config offer pass may reuse it (acp_status now pays ONE extra
  processTurn instead of two); undefined = audit runs its own pass.
- UncoveredRun/UncoveredAuditInput unexported (unused-exports gate).
- New P4 regression test: three adjacent T1 blocks distilled into one T2
  parent through the real host path must not leak child spans into the audit
  (pins coveredMessageIds' active-block union + unit-projection child-id
  propagation + child deactivation).
- TECHNICAL-NOTES: floor, tier-2 invariant, cost profile.
@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

已并入合并列车 #4525(master 6d7c587,合并方式 merge commit,提交保留原作者署名)。本 PR 关单收尾;若列车中对应提交出问题,回滚点为 #4525 单提交 revert。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR p3 scope/wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] compress reports 'no compressible ranges remain' while mid-history spans sit outside every block

1 participant