Skip to content

[PR] fix: cross-version restart no longer destroys fold state on explained id churn (#4131) - #4141

Closed
ranxianglei wants to merge 2 commits into
masterfrom
2026-10-21_4131-cross-version-churn-gate
Closed

ranxianglei wants to merge 2 commits into
masterfrom
2026-10-21_4131-cross-version-churn-gate

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #4131

What this changes

A bili upgrade across v0.1.201 re-mints non-text message ids by design (#3642 canonical identity projection; D1 accepted this one-time mass churn and promised the reconcile ladder would absorb it). On the first turn after a cross-version restart the #1001 unannounced-rewrite detector — which sits OUTSIDE that ladder and judges on raw-id hash identity alone — saw the collapse (incident: 95/392 pre-turn refs), marked a compaction boundary anyway, and applyCompactionArchive pruned the ref map to the incoming ids and tombstoned ~284 still-cited refs: fold coverage destroyed for a pure transform. User-visible: whole session re-sent unfolded, replay loop, prefix-cache misses.

Fix: this pass's fold-reconcile verdict is now passed into detectUnannouncedHistoryRewrite (openai/anthropic/google lanes; the responses lane carries no detector). When kind is reanchored/resend with unmatched === 0, the collapse is explained churn — the destructive boundary is suppressed and an id-churn-adoption conflict event + info log are recorded instead. Off/noop verdicts (reconcile disabled, side-request skip) carry no evidence → legacy behavior unchanged; partial claims (unmatched > 0) keep the legacy escalation path intact.

Second facet (issue expectation #4): the applyCompactionArchive log line now names how many refs the prune drops and which block coverages they sit under, so any remaining destructive prune is auditable instead of silent.

Behavior change disclosure (old → new)

Regression check

  • Both directions pinned in tests/issue4131-cross-version-churn-gate.test.ts: POSITIVE (real reconcileFoldCoverage over the re-minted fixture reports unmatched=0 → gate suppresses, no boundary, no deadRefs), NEGATIVE 1 (模型切换后压缩块跨模型失配,新模型收到 61.3 万 token 全量上下文 #1001 shape — covered substrate truly gone → still escalates), NEGATIVE 2 (append-only), NEGATIVE 3 (off/noop/null verdicts do not suppress; legacy callers bit-identical), plus the log facet.
  • Adjacent suites green locally: fix-1001-history-rewrite (incl. both e2e wire tests pinning genuine-rewrite escalation end-to-end), compaction-archive, fold-anchor-doctrine, fix-1075-side-request, conflict-remediation, prefix-affinity-orphan. Full-suite verdict = remote CI on the pushed head.

Boundaries

@ranxianglei ranxianglei changed the title fix: cross-version restart no longer destroys fold state on explained id churn (#4131) [PR] fix: cross-version restart no longer destroys fold state on explained id churn (#4131) Oct 21, 2026
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-21_4131-cross-version-churn-gate (283a2ef)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4932-pr.4141.73a83b5

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4141.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 本条目是 #4131 的修复 PR(Closes #4131)。检索 open PR:没有其他 PR 处理同一机制(#1001 rewrite 检测器把跨版本 id 重铸误判为客户端重写)。#4085 同属 id churn 家族但机制不同(短请求 deactivation 后折叠覆盖丢失,#4050 线),不构成重复。

标签:bug, opencode, scope/kernel, p2(缺陷类;opencode 车道;src 侧 ref/账本与折叠判定面;能用但有明显问题——折叠状态随后续 re-fold 自愈、无永久数据丢失,不达 p1 的「会话不可用/缓存泄露」档)。

审核已开始:分支 2026-10-21_4131-cross-version-churn-gate(单 commit d34ae7ab9,+222/−15,7 文件)已拉取,正在逐段独立审读 diff 与既有行为面;本地已在该 head 上装依赖并 build 通过。CI 现状:test 分片仍在跑,已完成的 gate(static-gates/mysql-contract/test-import-graph/unused-exports/lifecycle-ledger-guard/docs-sync/lock-sync)全绿,e2e 车道按门控 skip。审完给结构化八问结论。

awork added 2 commits October 21, 2026 17:30
… fold-reconcile fully re-anchors (#4131)

The #1001 unannounced-rewrite detector judged history rewrites on raw-id
hash identity alone, outside the order-centric reconcile ladder. A bili
upgrade across v0.1.201 re-mints non-text message ids by design (#3642 D1
accepted this one-time churn and promised the reconcile ladder would absorb
it), so on the first turn after a cross-version restart the detector saw
the collapse, marked a compaction boundary anyway, and applyCompactionArchive
pruned the ref map + tombstoned hundreds of still-cited refs — destroying
fold coverage for a pure transform (incident: 212 covered refs re-anchored
with 0 unmatched, then pruned to 392 live ids).

detectUnannouncedHistoryRewrite now takes this pass's fold-reconcile verdict:
when kind is reanchored/resend with unmatched === 0, the collapse is
explained churn — the destructive boundary is suppressed and an
id-churn-adoption conflict event + info log are recorded instead. Off/noop
verdicts carry no evidence and keep legacy behavior; partial claims
(unmatched > 0) keep the legacy escalation path unchanged.

Second facet (issue expectation #4): applyCompactionArchive now names in
its log line how many refs the prune drops and which block coverages they
sit under, so any remaining destructive prune is auditable instead of silent.

Tests: tests/issue4131-cross-version-churn-gate.test.ts pins both directions
(explained churn suppressed; genuine rewrite still escalates; off/noop
verdicts do not suppress; legacy callers unchanged) plus the log facet.
…n events (#4141 review)

Review of #4141 found that a session whose conflict ledger contains ONLY the new id-churn-adoption events (the exact shape this PR produces on a clean cross-build restart) fell through formatConflictSection into the [suspected]-plugin tier: header claimed 'something outside bili touched this conversation' and the advice told readers to 'confirm each named plugin really compresses' — both false, since the event is bili's own fully-handled re-mint adoption (#3642 D1). Same misframing class as #2802 (orphan-only) and #2912 (handled-native-only).

Fix: dedicated adoptionOnly tier in the branch ladder (gated to pure ledgers so mixed ledgers keep their existing wording verbatim) with accurate header + active/historical advice; regression test pins the framing and the absence of plugin-hunt/remediation output. No behavior change beyond the panel/advice text.
@ranxianglei
ranxianglei force-pushed the 2026-10-21_4131-cross-version-churn-gate branch from d34ae7a to 283a2ef Compare October 21, 2026 09:31
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

审核结论(独立审读,非复读 issue 分析)

① 原 issue 方向:方向正确。#4131 的根因是检测层用单一证据维度(原始 id 哈希同一性)去否决 fold-reconcile 的多证据判决——直接违反 KDD#11「顺序为中心、hash 为辅助」;#3642 D1 承诺的「一次性 mass churn 由 reconcile 阶梯吸收」在检测层没兑现。修法打在机制上(把本 pass 的 reconcile 判决喂给检测器),不是堵症状。

② 回归审查(逐一点名审过的既有行为面):

③ 代码审查:接线正确——reconcileOutcome 来自同一 pass 的 reconcileFoldCoverage(session, msgs, …),与检测器看到的 msgs 是同一数组,中间无变更,判决与被判对象严格同 pass。effectiveMessageIds 为原始 sha256 id(核过 kernel/src/compress.ts:206、decompress.ts:116、sync.ts:119),审计日志的交集计算成立。

④ 测试有效性:新测试有牙——POSITIVE 跑真实 reconcileFoldCoverage 于重铸 fixture 断言无边界+无 deadRefs+adoption 事件;三个 NEGATIVE 各自钉住一条不得误伤的路径;日志 facet 断言精确行内容。非恒真断言。

⑤/⑤b diff 整洁度/代码整洁:单提交、+222/−15、每行都与目的相关;命名达意(churnExplained/adoptionOnly),注释讲为什么且带 issue 溯源,风格与文件既有惯例一致。

Review 发现并已当场修复(小问题直接修)

adoption-only 账本的错框:新增的 id-churn-adoption 事件落进 formatConflictSection 时会掉进 [suspected] 插件档——纯 adoption 账本(恰是本 PR 在干净跨版本重启后产生的形态)会显示「something outside bili touched this conversation」头 + 「confirm each named plugin really compresses」建议,两句都是假的(事件是 bili 自己已处置完的重铸采纳)。同类错框先例:#2802(orphan-only)、#2912(handled-native-only)。已在 PR 分支补一个标注 commit 修掉:adoptionOnly 专用档(仅限纯账本,混合账本措辞逐字保持不变)+ 精确的 active/historical 文案 + 回归测试钉死(断言无插件狩猎建议、无 Fix 页脚、无外部干扰头)。

同步做了 rebase:原 head 落后 master(v0.1.203 已发)。master 前进 ~50 个提交与本 PR 触碰的 7 个文件零重叠,rebase 干净;rebased head 上重跑了 typecheck(干净)+ 全部定向套件(8 文件 58/58 绿),已 force-push。

CI 现状:旧 head d34ae7ab9 上所有 guard/gate 车道全绿、门控 e2e 按设计 skip、ubuntu-22 三分片绿(ubuntu-24/windows-22 分片当时在跑)。新 head 283a2ef1a 的 CI 刚被 push 触发,全量以新 head 的 CI 为准;本地定向验证已过。

结构化八问

  1. 是什么问题:跨 v0.1.201 升级重启后 bili 按设计重铸非文本消息 id,模型切换后压缩块跨模型失配,新模型收到 61.3 万 token 全量上下文 #1001 重写检测器看到 id 塌缩就标了破坏性压缩边界,ref map 被剪到仅剩入站 id、~284 个仍被引用的 ref 被墓碑化——折叠覆盖被一次纯变换毁掉。
  2. 什么原因导致的:检测器在 reconcile 阶梯之外、只用哈希同一性一个维度做全局否决;The intermediate message (core) is not protocol-agnostic: switching protocols/models causes identity churn; cross-protoc | 中间消息(core)没有做到协议无关:切协议/切模型制造身份 churn,补齐跨协议映射 #3642 D1 的一次性 mass churn 承诺没有传导到检测层(单一证据维度否决多证据判决,KDD#11 违规)。
  3. 影响范围与严重程度:跨 ≥v0.1.201 版本重启宿主且会话带折叠状态的 plugin 车道用户(opencode/dsh 等),每会话每升级窗口一次性;后果为整会话折叠态丢失重发 + 前缀缓存击穿 + replay 循环,但随后续 re-fold 自愈、无永久数据丢失 → p2。
  4. 怎么修复的:把本 pass 的 fold-reconcile 判决传入检测器——全量恢复(reanchored/resend 且 unmatched=0)判定为「可解释 churn」,抑制破坏性边界、改记 id-churn-adoption 冲突事件 + info 日志;配套让 applyCompactionArchive 的 prune 日志点名丢弃的 ref 数与其所在块,残余破坏性剪枝从此可审计。
  5. 是否有回归问题:未发现(既有行为清单逐项见上);唯一由本 PR 引入的面板错框已在 review commit 中修复并钉测试。
  6. 是否破坏缓存:不——缓存/重试/游标/去重策略零改动;ref cursor 与 REF_FLOOR 高水位逻辑未触碰;修复本身正是恢复该场景的前缀缓存稳定性。
  7. 是否改了设计方向:否——纯宿主侧记账决策,无线条/配置/kernel 变化,转发载荷在所有场景保持字节一致。
  8. 是否引入了新的配置:无。

PR 正文声明「不在本 PR 内」的两项(#1724 drain-barrier 双实例竞态、重铸后 stale startRef/endRef 使 decompress(bN) 显示退化)均已在 #4131 清单内单独跟踪,范围切割合理。

判定:无拦点,可合(§7.4 身份/会话绑定类 → 人工合并;等新 head 283a2ef1a 的 CI 全绿即可)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

已并入合并列车 #4551(master 合并,merge commit 保留原作者署名)。本 PR 关单收尾;回滚点为 #4551 单提交 revert。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

1 participant