Skip to content

[PR] fix: preserve position fingerprints when reaping damaged folds - #5954

Closed
mariopasquali wants to merge 1 commit into
ranxianglei:masterfrom
mariopasquali:2026-11-08_fold-position-fingerprints
Closed

mariopasquali wants to merge 1 commit into
ranxianglei:masterfrom
mariopasquali:2026-11-08_fold-position-fingerprints

Conversation

@mariopasquali

Copy link
Copy Markdown
Contributor

Removing a damaged fold shortened the saved position IDs without removing
their paired fingerprints. The following replay could then cache a fingerprint
under the wrong message ID.

This keeps both arrays aligned during removal. The regression checks actual
fold removal, each remaining fingerprint association, and the following replay.
It fails before the fix. After rebasing onto current master, all 73 reconciliation
tests, build and typecheck pass with an independent dependency install.

This prevents new corruption; it does not repair existing historical metadata.

Related architecture: #2480.

Fixes #5953.

@ranxianglei ranxianglei changed the title fix: preserve position fingerprints when reaping damaged folds [PR] fix: preserve position fingerprints when reaping damaged folds Nov 8, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重 (duplicate screen): 不重复。 Searched all open PRs — #5954 is the only one targeting this mechanism (reap desyncing the foldPositions {ids, canon} pair). Nearby fold-reconcile PRs (#5737 direct-ID majority, #5691 dsh drift remediation, #5651 dead-fold token freeze) are different symptoms/mechanisms (drift classification, long-session remediation, token accrual) — none fix the removal-alignment bug.

Verdict: MERGEABLE. Clean root-cause fix, a regression test with real teeth (verified fails pre-fix / passes post-fix), no regressions in the affected area, CI green. Recommend approval + human merge (external contributor PR touching an identity-adjacent store ⇒ human-review tier regardless).

What I verified

The bug is real and self-perpetuating. Traced the mechanism end-to-end:

  • Invariant (src/fold-reconcile.ts:45, :2017): foldPositions.canon[i] must be the fingerprint of foldPositions.ids[i], kept 1:1 ("the pair can never desync from itself").
  • The [Bug][proxy] unframed host-side history deletion leaves zombie fold blocks rendering carriers forever — one carrier flaps in/out every turn (persistent prefix-cache churn) #2695 zombie-reap site (src/fold-reconcile.ts:2267-2276) splices dead ids out of the order backbone (nextOrder, which is foldPositions.ids) but never out of canon. After a reap, ids.length < canon.length and every entry past the splice point is shifted.
  • On the next replay, planReconciliation zips ids[i]↔canon[i] into canonById unconditionally (:899-904) — shifted entries get the wrong fingerprint — and the rebuild loop prefers priorCanon over fresh computation (:926-927), persisting the wrong fingerprints back into the store (:2019). Corruption is permanent for that session.
  • The existing desync guard (:910-917) does not save you: it only disables Pass-0's old-side stored coordinates. The corrupted canonById still feeds the rebuild, so that guard is insufficient against this path.

The fix is correct. It splices canon at the same index in both branches — steady state (positionIds === nextOrder) and the legacy distinct-array shape — preserving alignment without changing what/when gets reaped. Confirmed there is no sibling site: the only other writer of foldPositions is the normal write-back (:2019), which rebuilds {ids, canon} in lockstep.

The regression test has teeth (both directions verified). In an isolated checkout I ran the new tests/fold-position-reap-integrity.test.ts:

  • WITH the fix: PASS.
  • WITHOUT the fix (only src/fold-reconcile.ts reverted): FAIL — reaping must remove each position ID together with its fingerprint / 12 !== 13. Exactly the reported desync.
  • Beyond the length check it asserts (a) the fixture actually reaps (blocks.length === 0), (b) ids.length === canon.length, (c) each remaining canon[i] is the true fingerprint of ids[i], (d) the following replay doesn't cache a shifted fingerprint. Assertions (c)/(d) would also reject an incorrect "truncate canon to ids.length" fix (lengths match but middle-shifted associations survive) — so it pins the real invariant, not just array length.

No regressions in the affected area. Ran the targeted fold-reconcile / positional / drift / order-centric / reap suites (120 tests) with the fix: all pass. Typecheck: clean. (Full-suite verdict left to CI per local-testing discipline.)

CI on pushed head 55c853ae: 48 check-runs — 36 success, 12 skipped, 0 failures. Green. (mergeable_state: blocked is the review gate, not CI.)

Scope/hygiene: 2 files only (src/fold-reconcile.ts +11/−4, new test +32). No version bump, no lockfile churn, no kernel/ touch, no config/wire/persistence-shape change. Commit message clean (no secrets), fix: convention, Fixes #5953.

Known limitation (agreed with author, non-blocking)

"This prevents new corruption; it does not repair existing historical metadata." Correct and honest. For sessions already desynced pre-fix: the structural misalignment self-heals on the first post-fix pass (the store is rewritten in lockstep at :2019), but a wrong fingerprint value already cached under an id can persist because priorCanon wins over recomputation (:926-927). Impact is bounded — fingerprints are a per-cell confidence comparator, never identity (:51-52), and order-centric resync (#2915) + the content passes are the fallbacks. Repairing historical data is a separate migration concern; I'd keep it out of this fix (scope discipline). Not a blocker.


结构化八问(PR 审核结论)

  1. 是什么问题:僵尸折叠块被 reap(移除)时,只从保存的位置 ID 数组里删了死 ID,没同步删掉配对的指纹数组,两数组错位;后续回放会把指纹缓存到错误的消息 ID 上并自我延续。
  2. 什么原因导致的:根因在 [Bug][proxy] unframed host-side history deletion leaves zombie fold blocks rendering carriers forever — one carrier flaps in/out every turn (persistent prefix-cache churn) #2695 的 reap 站点(src/fold-reconcile.ts:2267-2276)——它原地 splice 了 order backbone(即 foldPositions.ids),却漏掉并行的 canon;下游 planReconciliation 无条件按索引 zip 两数组建 canonById(:899-904),重建时又优先取旧指纹而非重算(:926-927),把错误指纹固化回存储。既有 desync 守卫(:910-917)只关 Pass 0 旧侧坐标,拦不住这条污染路径。
  3. 影响范围与严重程度:所有走 proxy 模式 repair 且触发僵尸折叠 reap 的会话;某会话一旦被污染,其位置指纹层(身份层级里的置信比较器)对该会话持续失真。属「特定条件下有问题」的数据一致性缺陷,非整体不可用;指纹只作置信比较、不作身份,且有 order-centric 兜底,故非数据丢失级 p1,但属会自我延续的静默腐蚀,应尽快合入止血。
  4. 怎么修复的:在 reap 的两个分支(稳态 positionIds===nextOrder 与 legacy 异数组)里,删 ID 的同时按同一索引 splice canon,恢复「两数组永不错位」的既定不变量;不改 reap 的判定与时机,纯补齐缺失的对称删除。
  5. 是否有回归问题:无。逐项核过触碰路径的既有行为——reap 对 backbone 的删除、对异数组 positionIds 的删除、coverage/anchor 记录删除、orphan-reap 日志全部保持;唯一新增行为就是补删 canon(即修复本身)。高危模式逐项:无语义漂移、无 fail-fast→吞错、阈值(FOLD_DRIFT_ESCALATE_PASSES=3)/时序/输出格式均未动、wire 不变。定向跑了 fold-reconcile/positional/drift/order-centric/reap 共 120 个测试全绿,typecheck 干净。
  6. 是否破坏缓存:不破坏,反而修复。foldPositions 是宿主侧位置指纹缓存,本修复让它更一致;未改任何失效/去重/增量游标策略。
  7. 是否改了设计方向:没有。仍是 折叠身份改为「位置为主 + 自存规范副本」:彻底终结跨协议 fold 漂移补丁链 (#2454 根因范式) #2480 位置身份层的同一机制,只补齐一处漏掉的对称删除,未转向架构/边界/数据流。
  8. 是否引入了新的配置:无。无新 env/config/默认值。

判定:可合入。 建议 approve + 人工合并(外部贡献者 PR 且触及身份相邻存储,本就落在人工评审档)。已知限制(历史已污染元数据不回填)作者已诚实披露、影响有界,不阻塞;如需回填另立后续,不塞进本 PR。

@ranxianglei

Copy link
Copy Markdown
Owner

Landed via batch PR #6482 (merged) — closing as superseded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fold reaping can detach saved fingerprints from message IDs

2 participants