Skip to content

[PR] fix: recover fold coverage after short-request deactivation under id churn (#4050) - #4085

Merged
ranxianglei merged 4 commits into
masterfrom
2026-10-21_issue4050-short-replay-coverage
Oct 24, 2026
Merged

ranxianglei merged 4 commits into
masterfrom
2026-10-21_issue4050-short-replay-coverage

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

查重

不重复:无既有 issue/PR 覆盖本失败签名(短请求使 fold 失活 + 规范化等价重发 → 永久失去压缩覆盖)。相关但不同:#3844/#3845(reap 防护——本复现位于其合入后的 master,块并未被 reap);#4036/#4040(compaction-trigger 活动探测——不同失败域,本例无 trigger);#3301(迁移材料缺失——本例材料充分,no-short 对照可恢复)。同根因无在途修复。

What this fixes (#4050)

A short request deactivates a retained fold (kernel syncBlocks — by design, #3844/#3845). When the main history then returns with canonical-equivalent bytes under churned ids (client re-serialization), reconcileFoldCoverage never re-keyed the block: coveredRealHistoryIds skips inactive blocks, so covered.size === 0 short-circuited the whole pass, the stale ids could never be found again, and syncBlocks could never reactivate them. Result: summary dropped from the wire and the folded originals re-sent every turn (field evidence in the issue: input 230K→591K tokens, cache hit 99%→7%).

The fix feeds the retained view-deactivated lineage back into the claim ladder as recovery candidates (recoveryCandidateIds): active coverage is inserted first (the anchor cap's survivor order, #2334, is unchanged), then inactive non-expanded blocks' covered ids. The claim ladder decides per cell with multi-channel corroboration (positional fingerprint / normalized identity / simhash ≤8 / order bracket) — nothing is restored blindly. Corroborated cells get re-keyed by rewriteBlocks; the kernel's syncBlocks then re-activates the block naturally on the next pipeline run. Non-corroborated cells stay honest mismatches and complete the EXISTING lifecycle: z-streak reap for verified substrates (#2695), truncation suppression (wireContinuity/tailTruncated), epoch verdict (#3641). Excluded: expanded blocks (host decompressed on purpose — fresh compress creates a NEW block; including them would feed #2293-style missing inflation). Consumed blocks need no exclusion (ids ⊆ surviving parent's, Set dedupes).

Behavior changes (disclosed)

# change old → new why
a all-inactive sessions now run full reconcile passes noop → same cost class as any session with ≥1 active fold the pass is exactly what recovers coverage; side-request gate still skips short passes
b short pass on an all-inactive session no longer resets drift state reset → preserved resets are reserved for true episode boundaries ("no folds at all"); a retained-but-inactive fold is not an episode boundary — aligns with the documented intent at that gate
c wholesale history replacement on one session id with all folds inactive now escalates to epoch confirmation after 3 passes (+ error line) silent noop → escalation consistent with the active-fold variant, which escalates today; snapshot retained, auto-clears on realignment
d truly-gone spans of view-deactivated blocks with verified substrate now drain via z-streak reap (~3 passes, repair mode, proxy lane only) zombie-retained forever → reaped completes the #2695 lifecycle for a class that previously could never accrue z once fully deactivated; plugin mode unaffected (reap gated on pluginAgent === undefined)
e #2297 unit fixture refined dead blocks implicit → explicitly expanded: true same exclusion semantics, sharper classification now that "inactive" splits into recoverable vs dead-lineage

No user-visible behavior change when everything is healthy: active folds reconcile exactly as before (their ids were already in covered; insertion order unchanged), and the reported shape now recovers instead of leaking.

Config surface

None — no new config field, env var, or CLI flag.

kernel-change

N/A — host-side only (src/fold-reconcile.ts). Kernel untouched.

Tests

  • NEW tests/issue4050-short-replay.test.ts — the issue's repro as an end-to-end HTTP regression (real proxy + loopback mock upstream, storage disabled, snapshotRender off): 4 scenarios. Pre-fix the short-normalized scenario asserted the buggy behavior (finalActive=0, summary off wire, folded originals on wire, +17,017B); it now pins recovery byte-identical to the no-short control (65,884B vs 65,884B). The genuinely-edited negative control pins HONEST MISMATCH (no blind restore).
  • tests/fold-reconcile.test.ts — 4 new unit tests: churn re-keys both blocks (reconcile re-keys, kernel re-activates); sanctioned head-truncation of a verified span is never reaped (structural absence, no z accrual over 3 passes); mid-history edit accrues honest drift (z climbs, same [Bug][proxy] unframed host-side history deletion leaves zombie fold blocks rendering carriers forever — one carrier flaps in/out every turn (persistent prefix-cache churn) #2695 lifecycle as active folds); consumed child ids dedupe against the surviving parent.
  • Local targeted suites green: fold-reconcile 72/72 · fold-drift+issue2202+issue2627+issue2915 39/39 · responses-fold-cache-seam+refold-wiring+compaction-archive+issue3477 23/23 · typecheck clean. Full-suite verdict = CI on the pushed head.

Closes #4050

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-21_issue4050-short-replay-coverage (b1cf2d9)

Latest build — install this exact version

npm install -g billion-context@0.0.5446-pr.4085.0bf4274

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
b1cf2d9 billion-context@0.0.5446-pr.4085.0bf4274 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4085.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: recover fold coverage after short-request deactivation under id churn (#4050) [PR] fix: recover fold coverage after short-request deactivation under id churn (#4050) Oct 21, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复——本 issue 是 PR #4085(修复 #4050)的镜像条目,与既有 issue/PR 无重复;按 [PR] 流程处理:已拉取分支核对代码、审 diff、确认 CI,不重做。

初步状态(head 354240b98,base=master 56a0f5a46):

  • 静态门禁 + e2e 车道全绿;上一轮红的 ubuntu verif 两处(开发机绝对路径 + 缺 no-solver skip)已由独立 commit 修掉,ubuntu 转绿。
  • 当前红:test (windows-latest, 22) 两个 shard——正在查失败签名,判断是本 PR 引入还是存量。

完整八问审查结论随后补上。

awork added 3 commits October 21, 2026 12:23
…churn (#4050)

A short request deactivates a retained fold (kernel syncBlocks, by design). When the main history then returns under canonical-equivalent id churn (client re-serialization), reconcileFoldCoverage never re-keyed the block: coveredRealHistoryIds skips inactive blocks, so covered.size===0 short-circuited the whole pass and the stale ids could never be found again - permanent coverage loss, summary off the wire, folded originals re-sent every turn.

Feed retained view-deactivated lineage back into the claim ladder as recovery candidates (active coverage inserted first, so the anchor cap's survivor order is unchanged). The ladder decides per cell with multi-channel corroboration - nothing is restored blindly; non-corroborated cells stay honest mismatches and complete the existing lifecycle (z-streak reap #2695, truncation suppression, epoch verdict #3641). Host-side only; kernel untouched.
…wire artifacts (#4050)

process.env.TMPDIR is unset on Windows (it uses TEMP/TMP), so the
artifact write fell back to "/tmp" and resolved to a nonexistent
D:\tmp\ on the windows-latest runners, failing all four scenarios with
ENOENT (CI run 39538358942, windows-22 shard 2). Use node:os tmpdir(),
which is valid on every platform. Verified locally: 4/4 pass, result
lines unchanged.
@ranxianglei
ranxianglei force-pushed the 2026-10-21_issue4050-short-replay-coverage branch from b4b678a to ccc586f Compare October 21, 2026 04:24
@ranxianglei ranxianglei added bug Something isn't working p1 scope/kernel labels Oct 21, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

标签: bug / scope/kernel / p1(缓存命中崩坏致成本爆炸类,按 2026-10-19 口径 p1;客户端无法从证据确定,不打)。

独立审核完成(不照抄 #4050 侧结论,diff 与行为均重新过了一遍)。

① 方向关

修的是根本原因,不是症状。缺陷生产者 = reconcileFoldCoverage 入口用 active-only 的 coveredIdsOf 建覆盖集:短请求把保留块置 inactive(kernel syncBlocks 既有设计)后,该块 id 永远进不了 claim ladder,canonical 等价字节回来也认不回 → 永久失覆盖。修在决策点(覆盖集定义),不是在中途过滤/改写 wire——符合同一机制不同形态仍成立的检验。recoveryCandidateIds 的排除项也对:expanded=true(宿主故意解压,#2293)排除;consumed 子块无需排除(id ⊆ 存活父块,Set 去重,有测试钉住)。

② 回归审查(触碰路径的既有行为逐项)

③④ 代码与测试有效性

核心 diff 仅 src/fold-reconcile.ts +41,单模块。测试是真测试:e2e 走真实 HTTP 代理+真实 kernel applyCompression,四场景含负控,断言直接打症状(summaryOnWire/foldedAssistantOriginalOnWire/finalActiveBlocks)+ 字节预算(replay−baseline < 5KB,防 +17KB 回退);四个单测分别钉 re-key 契约、truncation 抑制 reap、honest z 累积、consumed 去重。断言有牙,bug 回归时抓得住。

⑤ diff 整洁度 + master 已移动的影响

master 已并入 #4084/#4087(PR base 之后)。对照当前 master:verif-wire-anthropic-roundtrip 与 verif-wire-google-rebuild-stability 两个文件与 master 逐字节相同(冗余);verif-view-latch-discipline 仅 haveSolver 常量块位置不同(rebase 冲突点,取 master 版即可);issue3769 EBUSY 修复只在 master(#4087),PR 未碰该文件,无冲突。rebase 到 9a97169 后 PR 应收敛为 fold-reconcile.ts + 两个测试文件,三个 verif 文件的 hunk 可整段丢弃。

CI 状态(截至 04:22Z)

结构化八问

  1. 是什么问题:短请求把保留折叠块置 inactive 后,主历史带着等价内容(churned id)回来时,折叠覆盖永远认不回来,摘要掉线、被折原文每轮重发。
  2. 什么原因导致的:reconcile 入口的覆盖集只认 active 块(fold-reconcile drift 告警在客户端侧删除历史场景下持续误报(计数含死块、#2193 升级后不抑制、措辞误导) #2297 为治 missing 虚高而收窄),把「视图暂时看不见」和「真正死掉的谱系」混为一谈;inactive 块的 id 进不了 claim ladder,失去唯一的重锚机会。
  3. 影响范围与严重程度:Responses legacy-render 车道(非快照渲染)上「短请求 + 后续主历史 churned 回归」的会话;现场 input 230K→591K(+157%)、缓存命中 99%→7%——压缩这一核心承诺在该形态下整体失效,成本爆炸,p1。
  4. 怎么修复的:覆盖集补入「inactive 且非 expanded」的保留块 id(视图失活 ≠ 宿主解压),复用既有 claim ladder 逐 cell 认证后 re-key;不加任何新的认领通道或盲恢复。
  5. 是否有回归问题:② 所列各项均保持或为披露内的有意变更(a-e);高危模式逐项过——无语义漂移、无 fail-fast 变吞错、无阈值/时序漂移;输出变化仅限恢复后的 wire 收敛回基线(测试钉 <5KB)。
  6. 是否破坏缓存:反向修复——正是它恢复了前缀稳定(摘要载体回到 wire);re-key 只改 session state 元数据,wire 字节收敛回基线。
  7. 是否改了设计方向:不涉及——完全在既有 reconcile/re-key 架构内,恢复走既有 ladder。
  8. 是否引入了新的配置:不涉及——无新 env/config/默认值。

判定:代码与测试审核通过,行为变更 a-e 披露完整且我逐项核实成立。合并前置只剩机械步骤:conflict-worker rebase 到 9a97169(顺手丢弃两个与 master 相同的 verif 文件、view-latch 取 master 版)→ rebased head CI 全绿 → 可合。本 PR 触碰 id 邻接逻辑,按 §7.4 保持人工合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

状态更新(接上条审查结论):rebase 已落地且干净。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

收工判定:全绿,可合。

  • 全量 test matrix(run 39543846750,head ccc586f29)已于 04:45Z completed / success——含 windows node-22 两个 shard(此前两处红的根因均已修掉)。
  • 该 head 上所有 guard jobs(kernel-guard / config-surface-guard / lane-decisions-guard / docs-sync 等)+ 8 条 e2e 车道全部绿。
  • PR 当前 mergeable: True(无冲突;master 仍在前进,合并前 GitHub 会自动 rebase 校验)。

至此本 issue 的审核闭环:方向(根因在决策点修,非症状遮罩)、回归清单、测试有效性、diff 整洁度四关均已独立审过并逐条写在上条八问回复里。按 §7.4(触碰 id 邻接逻辑)留人工合并:#4085

@ranxianglei
ranxianglei merged commit 1c38b7d into master Oct 24, 2026
20 of 21 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.809-master.1c38b7d (rolling master tag).

npm install -g billion-context@0.0.809-master.1c38b7d  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p1 scope/kernel

Projects

None yet

1 participant