Skip to content

fix(#2202): condition ledger accrual on measured fold coverage - #2231

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-05_cache-ledger-coverage-accrual
Oct 7, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-05_cache-ledger-coverage-accrual

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #2202

What this does

The cache ledger books every usage sample against every elapsed fold (requestsAfter += 1), and the panel turns that counter into netSaved = Σ (S − σ) × requestsAfter. The booking is a claim that the request benefited from the fold — but when a host removes the covered originals outside bili's knowledge (dsh native compaction, #2193), the claim stops being true while the counter keeps climbing. That is exactly what the #2193 incident showed: netSaved at 355M with 1510 covered ids permanently off the wire.

This conditions the claim on the evidence bili already collects per pass — a writer-side fix at the accrual site, not a display patch:

  1. Per-block coverage records (src/fold-reconcile.ts) — reconcileFoldCoverage already classifies every covered id on every inbound pass (present / re-anchored / unmatched). It now also stores {p, r, t, e} per blockId under session.metadata["foldCoverageByBlock"]: present verbatim / reclaimed via claim / total covered / ever-present latch.
  2. Fold→block link (src/cache-ledger.ts) — sparse LedgerFold.bid, set once in recordCacheFoldsFromBlocks, the single choke point for both the eager proxy path and the lazy plugin-mode path. Pre-[cache-ledger] netSaved 面板在 fold 覆盖坍缩后继续累计虚增——ledger 与实际重发行为无交叉校验 #2202 ledger entries lack it and keep status-quo booking.
  3. Fractional accrual (recordCacheSample) — the increment becomes the measured fraction (p + r) / t: 1 full coverage (byte-identical to today), f partial, 0 verified coverage-lost (frozen until coverage returns). One warn/info log per flip (foldCoverageLostLogged dedup key).
  4. Panel (sessions-data.ts/client.ts/i18n.ts) — coverage-lost folds render as separate labeled segments (row tooltip, overview sub-line, detail stat), never inside the headline number. Frozen pre-loss savings stay visible; post-loss growth stops.

Behavior change (disclosed)

old new
Accrual per sample per elapsed fold unconditional +1 measured fraction (p+r)/t
Healthy session (coverage intact) +1 +1 — byte-identical
Verified coverage loss (#2193 class) keeps climbing forever frozen at loss; resumes on recovery
Folds whose ids never appeared (view-folding hosts) +1 +1 — unverifiable stays status quo
Side-request-shaped passes (<10 msgs) fed drift streak; would feed coverage records take no evidence at all

The last row is an adjacent fix found while working this issue: the reconcile path had no side-request guard (the sibling #1195 warn gates on !isTitleGen && len >= 10). A short pass rolls the order backbone onto side-request ids, which breaks the next real pass's alignment and sends missing covered ids straight to unmatched — phantom total-loss feeding both #2193's drift escalation and these coverage records. Such passes now return early without touching anchors, backbone, streak, or records (and without resetting the drift episode — skip ≠ recovery).

Tests

  • tests/fold-reconcile.test.ts — existing fixtures expanded to conversation-sized passes (real hosts resend full history; single-message passes no longer exist in production shape); new describe pins per-block records, the ever-latch, the never-present class, side-pass innocence, and the alignment-skip case.
  • tests/issue2202-coverage-accrual.test.ts (new) — hermetic seam test: fold materialization → healthy accrual → shadowed freeze (exactly one transition warn) → recovery resume (exactly one info); legacy parity and never-present status quo pinned.
  • Targeted suites green locally (fold-reconcile, fold-drift, cache-ledger, issue2202: 73/73); typecheck green. Full regression left to CI.

Stacking note

Built on the tip of PR #2201 (66a2cf0b9) because both work the same evidence path (fold-reconcile per-pass classification): #2201 is the alarm, this is the accounting fix. Will rebase onto master once #2201 merges.

Config surface

None added. No new env vars, config fields, or CLI flags. Internal metadata keys only (foldCoverageByBlock, foldCoverageLostLogged), sparse LedgerFold.bid, optional web summary fields.

Docs: TECHNICAL-NOTES.md "Cache-ledger accrual is coverage-conditioned (#2202)".

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-05_cache-ledger-coverage-accrual (3dc26e6)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2388-pr.2231.89306aa

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2231.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到,开始 review PR #2231。已拉取分支(pr-2231 @ 90bb5d5),stacked 在 #2201 tip(66a2cf0)上。按七问体例出结论,先审原 issue #2202 方向与回归面,再走代码关。稍后给完整结论。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

Review 完成。七问结论如下。

1. 是什么问题:#2202 —— 宿主在 bili 之外移除已被覆盖的原文(dsh native compaction,#2193)时,cache ledger 仍对"覆盖字节已不在线上"的 fold 持续记账 netSaved,导致虚增(#2193 事故:1510 个被覆盖 id 已离线,账面仍记 355M)。

2. 什么原因导致的:记账与实测覆盖脱钩——recordCacheSample 对每个 elapsed fold 无条件 requestsAfter += 1,没有任何环节验证该 fold 的 covered messages 是否仍真实存在于 live history(或经 claim 被回收)。症状是面板数字虚高,根因是 accrual 生产者不测量。

3. 怎么修复的:把记账条件化到实测覆盖上。每轮 inbound pass 按 blockId 记录 {p 原样在场, r 经 claim 回收, t 总数, e ever-present latch}(reconcileFoldCoverage);accrual 从恒 1 变为实测分数 (p+r)/t;verified-lost 冻结至恢复为止;<10 条消息的 side-request 形状 pass 完全不取证据(skip ≠ recovery,不重置 drift episode);面板把 coverage-lost fold 单独标注呈现。与 #2202 floor 2 中 owner 批准的设计包逐条一致。

4. 是否有回归问题:逐项审过既有行为,未发现无意回归:健康会话记账字节级不变(+1/sample,legacy-parity 测试钉住);#2202 之前持久化的无 bid 条目保持现状记账;never-present 类(view-folding 宿主,id 从未出现)自校准为现状、不会误冻;requestsAfter 浮点化后所有消费点(kernel cache-report 的 breakeven 比较/savingPerTurn 算术、web 面板算术)逐一验证过,无整数假设;drift episode 状态在 skip pass 下刻意不动(有注释)。一个已披露边界:compress.reconcile=warn(opt-in)模式下 r 分类用的是未实际应用的 planned claims,理论上可能比现状的无条件 +1 略多记——仅影响该可选模式且不比现状差,已记录。面板新增字段全部可选、仅在 >0 时输出。

5. 是否破坏缓存:不涉及——未触碰任何缓存键、重试策略、去重键或增量游标。

6. 是否改了设计方向:不涉及——按 owner 批准的方案原样实现。一处解释性偏差提请知悉:floor 2 措辞是冻结份额"不混入 netSaved",实现保留了 pre-loss 的真实节省在 headline 内(那些字节确实没被计费)并单独标注——判断物理上正确,若 owner 倾向完全剔除请指出。

7. 是否引入新配置:不涉及——只有内部 metadata key(foldCoverageByBlock、foldCoverageLostLogged、稀疏 LedgerFold.bid、web 汇总可选字段),无 config/env/CLI 面变化(PR 已按规范声明)。

行为变更披露:ledger 每 sample 记账 无条件 +1 → 实测分数 (p+r)/t;健康会话字节级不变;coverage-lost fold 由"继续虚增"改为"冻结至恢复"。这正是本 PR 的目的。

验证与 CI 状态:

判定:PR 本身代码/测试/文档扎实、与批准设计一致、diff 干净;剩余 CI 红是 #2239 的既有问题,非本 PR 引入。#2239 修复后(或 owner 接受已知无关红)即可合并;合并动作按规矩留给人工。

@ranxianglei

Copy link
Copy Markdown
Owner Author

冲突 rebase master

@ranxianglei
ranxianglei force-pushed the 2026-10-05_cache-ledger-coverage-accrual branch from 6fd6c25 to 121dd7f Compare October 6, 2026 12:21
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 冲突已解,rebase 完成并推送:6fd6c252c → 121dd7f67(force-with-lease,PR 头分支 2026-10-05_cache-ledger-coverage-accrual)。七问摘要:

  1. 是什么问题:上游冲突唤醒——master 前进了 5 个合并(fix(#2241): continuity-aware dsh persona anchor — model switches migrate the anchor instead of forking the main lane #2247/session self-heal: zombie plugin lanes degrade to proxy tools, idle nudges suppress, 524 summaries learn streaming (#2155) #2246/fix: stop labeling first-party siblings as third-party compressors in conflict alarms #2262/fix(#2260): v0.1.186 review follow-ups — dsh spawn leak, inert resign keys named at config load, codex arg/TOML fixes, web pristine root, misc #2263/fix: wait for both disconnect-fault log lines before asserting (#2212) #2213,新 tip c4ae70796),本 PR 落后且与之一处文本冲突。
  2. 什么原因导致的:双方在同一位置向 TECHNICAL-NOTES.md 插入新章节——master 侧的「Stuck-session self-heal(会话卡死自愈机制:卡死模式检测与代理侧主动补救 #2155)」与本 PR 的「Cache-ledger accrual is coverage-conditioned([cache-ledger] netSaved 面板在 fold 覆盖坍缩后继续累计虚增——ledger 与实际重发行为无交叉校验 #2202)」落在同一锚点后,产生同位插入冲突。其余 7 个文件(cache-ledger/fold-reconcile/web 面板/测试)git 自动合入,无冲突。
  3. 怎么修复的:rebase 到 c4ae70796;唯一冲突按并集解决——两节都保留(master 的在前、本 PR 的在后),未删任何一方内容。diff 完整性核对过:仍是 8 文件 +514/−40,与 rebase 前一致。
  4. 是否有回归问题:无。新 master 提交虽也碰了 src/web/{client,i18n,sessions-data}.ts,但 hunk 不重叠;在新 head 上重跑定向验证全绿——fold-reconcile / fold-drift / cache-ledger / issue2202-coverage-accrual 共 73/73,typecheck 干净,build 通过。
  5. 是否破坏缓存:不涉及(纯机械 rebase + 文档并集合并,零代码语义变化)。
  6. 是否改了设计方向:不涉及。
  7. 是否引入了新的配置:不涉及。

CI 会在 121dd7f67 上重跑。提醒一点:master 目前自带一个与本 PR 无关的红(tests/pi-subagent-registry.test.ts:275,#2186 接线改动破坏了 #2185 的单 handler fake,已在 #2239 立项跟踪)——CI 里若只剩这一条红,属预期,与本 PR 无关。分支已转干净,等 CI 后即可由人合并。

@ranxianglei
ranxianglei force-pushed the 2026-10-05_cache-ledger-coverage-accrual branch from 121dd7f to a7197c3 Compare October 6, 2026 15:11
@ranxianglei

Copy link
Copy Markdown
Owner Author

解决冲突

@ranxianglei
ranxianglei force-pushed the 2026-10-05_cache-ledger-coverage-accrual branch from a7197c3 to 3dc26e6 Compare October 6, 2026 15:47
@ranxianglei

Copy link
Copy Markdown
Owner Author

Rebased onto master (post-#1818/#1851/#2147) — head 3dc26e6.

Single textual conflict in src/cache-ledger.ts (import block): union of master's clearPostRebuildAnchor addition to the session.js import with this PR's new METADATA_FOLD_COVERAGE import from fold-reconcile.js. No semantic intersection (master's change is the post-rebuild anchor clear; this PR's is the coverage metadata key).

Verified locally on the rebased head:

  • tsc clean, build clean
  • issue2202-coverage-accrual + cache-ledger ×2 + fold-reconcile + fold-drift + issue1547 + issue2131 ×2 + host-usage-postfold: 122/122
  • fold-reconcile-perf + fold-round2-shape + compress-fingerprint-refold + issue1195 + issue1908 + refold-wiring + responses-fold-cache-seam + tight-fold-summaries: 34/34

Also confirming the earlier red CI lane (codex accepts CODEX_HOME overlay..., run 37485302214) was a Windows runner flake (spawnSync node.exe ETIMEDOUT in win-cmd-argv #456) — unrelated to this PR's ledger changes; the rerun is green.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到冲突唤醒,核查后无需再动——冲突已被解掉,本条告警是过渡态快照。七问摘要:

  1. 是什么问题:系统转发「PR 与 base 冲突(dirty)」。但核查时 GitHub 已报 mergeable=True / mergeable_state=clean,head 为 3dc26e60e。
  2. 什么原因导致的:时间线——我上一轮把分支 rebase 到 c4ae70796 并推送(121dd7f67)后,master 又前进了(fix: cross-spec compress-failure circuit breaker + stale-generation diagnosis (#2146) #2147/fix(#1841): preflight futility gate — skip doomed rounds, bail mid-walk #1851/fix(#1812): fold integrity — net-shrink gate, post-rebuild meter anchor, summary budget, image-arb compression (A/B/D/E, review pending) #1818 等合并 → 9f82bbed8),期间产生过 dirty 窗口;随后一个兄弟 agent 会话已完成新一轮 rebase 并 force-push 到 3dc26e60e(坐在当前 master tip 上)。告警就发在这个窗口里。
  3. 怎么修复的:未重复 rebase(政策:冲突解完不要重复 rebase)。做的是等价性核验:把 3dc26e60e 的 PR patch 与我上一轮的 patch 逐行比对——内容一致,仅两处无害差异:①TECHNICAL-NOTES.md 新章节位置不同(对方放在文件末尾 tag-echo 节后,我放在 install-lane 段后,两节都在、无丢失);②src/cache-ledger.ts import 行采纳了 master post-rebuild usage estimate inflates ~3.4x after a big fold, firing an EMERGENCY T1 nudge into an over-budget context #1820 新增的 clearPostRebuildAnchor(正确吸收 master 变更,我的版本基于更老 master 没有这行)。
  4. 是否有回归问题:无。在 3dc26e60e 上本地验证:build OK、typecheck 干净、定向套件(fold-reconcile / fold-drift / cache-ledger / issue2202-coverage-accrual)fail 0。
  5. 是否破坏缓存:不涉及(零代码改动,纯核验)。
  6. 是否改了设计方向:不涉及。
  7. 是否引入了新的配置:不涉及。

当前状态:head 3dc26e60e 包含完整修复 + 最新 master,merge state clean,等 CI 跑完即可由人合并。本地工作区分支指针也已同步到 3dc26e60e,后续不会再有旧 head 干扰。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 刚核查过:冲突已不存在,无需再解。当前实况(刚刚拉取):

按政策不重复 rebase(冲突解完状态自动转干净)。现在万事俱备:等您合并即可——#2231 。若您 UI 里仍看到 conflict 标记,刷新一下即可,API 侧已是 clean。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 交叉核对一致,无新增动作。两个会话的独立验证互相印证:

  • 同一 head 3dc26e60e:我这边做的是 patch 级等价比对(与原始修复逐行一致)+ build/typecheck/定向套件全绿 + CI 22/22 绿;您这条补充了更宽的本地套件(156 用例)和 codex lane 红是 Windows runner flake(spawnSync node.exe ETIMEDOUT)的判断——与我观察到的「CI 已转绿」吻合。
  • 冲突描述也对得上:我 rebase 到 c4ae70796 时撞的是 TECHNICAL-NOTES.md 同位插入;这轮 rebase 到 9f82bbed8(含 post-rebuild usage estimate inflates ~3.4x after a big fold, firing an EMERGENCY T1 nudge into an over-budget context #1820)时撞的是 cache-ledger.ts import 块——都是同位并集解决,无语义交集。
  • 刚再次确认:master tip 未动(9f82bbed8),head 包含它,mergeable_state=clean。

状态维持不变:万事俱备,等合并。

@ranxianglei
ranxianglei merged commit 8f62fd4 into master Oct 7, 2026
22 checks passed
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.93-master.8f62fd4 (rolling master tag).

npm install -g billion-context@0.0.93-master.8f62fd4  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[cache-ledger] netSaved 面板在 fold 覆盖坍缩后继续累计虚增——ledger 与实际重发行为无交叉校验

1 participant