diff --git a/AGENTS.md b/AGENTS.md index a2fc3109b4..0ccab986f5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -49,6 +49,8 @@ The full file-by-file map lives in [reference/architecture.md](reference/archite 10. **Thinking-family blocks are signature-verified — NEVER rewrite them (#1960)** — Anthropic `thinking`/`redacted_thinking` blocks and Gemini `thoughtSignature` parts are verified byte-for-byte when the latest assistant message is replayed ("thinking blocks ... cannot be modified"). The bytes the proxy forwards become the client's persisted history, and the bytes the loop accumulates become the re-request rebuild — BOTH must stay verbatim. Any transformation (tag/marker stripping, dedup, trimming, re-laying-out block order, dropping empty segments) desynchronizes text from signature and bricks the session: first on the in-turn re-request, then on every subsequent request via the client's echo. #1882 put a prose filter on the thinking channel and it was exactly this bug. Treat ANY future change touching thinking-family payloads as high-risk: it needs loop-level byte-equality tests (re-request body AND client-visible stream) on both signed wires, not just adapter-level ones. The prose filters (#206/#1881) are for the TEXT channel only. +11. **顺序为中心,hash 为辅助 (order-centric reconciliation — owner rule, 2026-10-12, #2821/#2915)** — fold-state reconciliation must treat ORDER as the primary carrier of message identity across resent histories: position claims verified spans (with per-cell fingerprint gates), content hashes and tool-call ids are auxiliary evidence, and NO single dimension's miss may cascade into total unfold. The identity dimensions survive different churn classes — order survives re-encoding AND id re-minting, content survives id churn, ids survive re-encoding — so the ladder is the union of their survival domains, and each rung fails soft: an id/fingerprint mismatch voids that CELL's claim only; the positional scans resync at the next run of same-index agreements (MIN_RESYNC_RUN consecutive matches, bounded gaps) and keep claiming verified spans behind the break. Genuinely changed cells (real edits, regenerations) stay honestly unmatched and re-enter the wire — "never fuzzy similarity" (mis-attribution is the most expensive failure, MESSAGE-IDENTITY.md) is unchanged: the hash stays a HARD per-cell gate, it just no longer holds a global veto over its neighbors. Escalation to "substrate destroyed" may only count gaps that survived resync AND the content passes (byTool/byTwin/byNorm/byTurn). #2821 was the canonical violation: a mid-session model switch re-minted every tool-call id on re-projection (content byte-identical), both positional flank scans died at their FIRST cell (changed preamble at the head, appended turn at the tail), 3289 covered ids went missing, three passes → destroyed verdict → total unfold → doomed-probe 502. #2396's detection-only twins are promoted to a claiming pass (Pass 1.5, k-th discipline + length guard) so id re-mints recover even without a stored positional copy — and after one churn pass the self-saved {ids, canon} copy upgrades recovery to the positional path. A Pass-0 gap cell (order break) is resolved by tiers, owner rule 2026-10-12: any exact channel that matches claims it; failing that, simhash corroboration (≤8 bits on the 64-bit word-shingle sketch — measured margins: re-wording ≈6-8 bits inside, negation flips ≈11 / word drops ≈13 / key reorders ≈21 outside) claims it; failing that, the ORDER BRACKET — same index strictly between VERIFIED claims on both flanks — asserts the SLOT (the owner-sanctioned delete+insert tradeoff: coverage rides the slot, the stale bytes do not); with zero corroboration and no fence the cell stays honest unmatched drift. Deletion is honest loss: the flanks absorb the shift, the deleted id never gets a bracket claim (its slot cell is consumed by the verified tail). Tests: `tests/issue2915-order-centric-reconcile.test.ts`. + ### Install-Lane & Update-Ownership Contract (#1196) Every bili presence on a machine follows ONE contract — decide changes against it, not ad hoc: diff --git a/src/fold-reconcile.ts b/src/fold-reconcile.ts index bea8186f0b..8dfa40b348 100644 --- a/src/fold-reconcile.ts +++ b/src/fold-reconcile.ts @@ -54,6 +54,42 @@ // while the vanished head ids fall into the honest unmatched/archive path — // no death spiral, and the copy self-heals on the next full pass. // +// #2915 — ORDER-CENTRIC upgrade (owner rule 2026-10-12: 顺序为中心, +// hash 为辅助). Identity hierarchy: order → content → id. Pass 0's scans +// used to STOP at the first fingerprint mismatch, so one changed cell (a +// re-serialized preamble, a single edited message) stranded every later cell +// between the break and the far flank — and with every id re-minted by a +// provider re-projection (#2821's dsh model switch) BOTH scans died at their +// first cells while the order was fully intact (byPos=0, 3289 missing, +// "substrate destroyed", all folds dropped). An order break is now a GAP, +// not a wall: each scan resyncs at the next run of MIN_RESYNC_RUN consecutive +// same-index agreements (bounded by MAX_RESYNC_GAPS) and keeps claiming +// verified spans; gap cells fall through to the content passes and the honest +// unmatched path. Tool-id twins (#2396) graduate from detection-only to a +// claim pass (1.5): exact normalized content equality under the rewritten +// toolCallId, k-th discipline, length-guarded — the id is a fast path, never +// a death sentence. The fingerprint stays a HARD per-cell gate (owner +// phrasing "hash 只是参考" means no GLOBAL kill-switch — a mismatch voids +// that cell only, never the whole history); escalation to "substrate +// destroyed" still exists but now counts only gaps that survived positional +// resync AND the twin/norm content passes. +// +// #2915 gap resolution — tiered fallback for the cells that fall OUT of +// Pass 0 (owner rule, same day: any channel matches → claim; none match but +// the flanks do → claim; order holds within the session → claim). After the +// exact content passes have had their bite, a same-index gap cell is +// resolved by (a) simhash corroboration — a re-rendered cell keeps its +// word-shingle mass (measured: JSON key reorder 8 bits, one-word edit of a +// long line 7, vs re-run "3 failed"→"all green" 11, JSON-vs-prose 30; +// threshold 8 keeps the churn inside and the re-runs outside), then (b) +// order bracket — the cell sits BETWEEN verified positional claims inside +// one session (reconcile is session-scoped by construction): same role, +// bounded length drift, and the fold asserts the SLOT, not the stale bytes. +// The one stale cell bracketing can carry (delete+insert at the same index) +// is the owner-sanctioned tradeoff for never losing a session to a +// re-rendered cell. Zero corroboration anywhere stays honest drift — "order +// is the same" is only a claim when something verified it. +// // Pass 3 (#2636) — ASSISTANT-TURN key. A host replaying one conversation // across models re-serializes every thinking-bearing assistant turn: the // same-model replay carries reasoning in a structured field (kernel: adjacent @@ -96,6 +132,7 @@ // "repair" — rewrite block ids (default). import { createHash } from "node:crypto"; import { defaultCountTokens, type CoreMessage } from "acp-kernel"; +import { simhashHex, hammingHex } from "./affinity-simhash.js"; import { reapDestroyedSubstrate, type Session } from "./session.js"; import { coveredRealHistoryIds, isResponsesTurnSeparatorId } from "./session.js"; import { dshNativeCompactionWitness, recordConflict } from "./conflict-watch.js"; @@ -149,6 +186,26 @@ const FOLD_DRIFT_ESCALATE_PASSES = 3; /** Below this many permanently-missing ids the loss is small enough (a few * edited/deleted messages) to stay at warn level. */ const FOLD_DRIFT_ESCALATE_MIN_UNMATCHED = 10; +/** #2915: order-centric resync — an order break is a GAP, not a wall. A + * positional scan that stops at the first fingerprint mismatch strands every + * verified cell behind the break; with ids re-minted across a provider + * re-projection (#2821) BOTH flank scans died at their first cells and + * byPos read 0 while the order was fully intact. Each scan now resyncs at + * the next run of MIN_RESYNC_RUN consecutive same-index agreements and + * keeps claiming verified spans, bounded by MAX_RESYNC_GAPS breaks per scan + * so pathological inputs stay linear. The gap cells fall through to the + * content passes and the honest unmatched path. */ +const MIN_RESYNC_RUN = 3; +const MAX_RESYNC_GAPS = 8; +/** #2915 gap resolution — simhash corroboration tier. Measured on the repo's + * own shingle simhash (src/affinity-simhash.ts, word-bigram features): + * JSON key reorder = 8, one-word edit of a long line = 7, while a re-run + * tool call ("3 failed" → "all green") = 11 and JSON-vs-prose = 30. The + * 8-bit line keeps every observed formatting churn inside and every + * genuinely-different content outside; anything looser starts claiming + * re-runs. Bracket-only cells (tier below) catch what formatting churn + * cannot explain. */ +const GAP_SIMHASH_MAX_HAMMING = 8; /** #2202: per-block coverage evidence, consumed by the cache ledger's * conditional avoided-token accrual (src/cache-ledger.ts). */ export const METADATA_FOLD_COVERAGE = "foldCoverageByBlock"; @@ -216,6 +273,14 @@ export interface FoldAnchor { * core and vice versa) — lets a pair claim collapse onto one merged * witness without a second lookup. */ x?: string; + /** 64-bit simhash (16 hex, affinity-simhash) of the canonical text at + * anchor time — #2915 gap-resolution corroboration: near-duplicate + * detection for re-rendered cells whose normalized bytes changed + * (formatting churn that NFC/whitespace/canonical-Json normalization + * still misses). Absent on anchors minted before the field existed + * (tier degrades gracefully) and on reasoning cores (their core text + * is the provider item id, not content). */ + sh?: string; } interface ReconciliationPlan { @@ -225,9 +290,16 @@ interface ReconciliationPlan { claims: Map; byPos: number; byTool: number; + byTwin: number; + bySim: number; + byOrder: number; byNorm: number; byTurn: number; byImage: number; + /** #2915: order breaks the positional scans resynced over (gaps claimed + * as order-carried spans past a fingerprint break) — observability for + * the order-centric doctrine, surfaced in the drift log line. */ + resyncs: number; /** Covered ids missing from the resent history with no match — either * mutation (originals re-enter the wire unfolded) or benign client-side * deletion/truncation (originals no longer on the wire) (#2297/#1195). */ @@ -257,6 +329,9 @@ interface FoldReconcileResult { claims: number; byPos: number; byTool: number; + byTwin: number; + bySim: number; + byOrder: number; byNorm: number; byTurn: number; byImage: number; @@ -501,6 +576,9 @@ function anchorFrom(message: CoreMessage, ctx?: TurnCtx): FoldAnchor { anchor.t = t; anchor.m = normalizedIdentityNoToolCallId(message); } + if (message.text !== undefined && message.text !== "" && message.contentType !== "reasoning") { + anchor.sh = simhashHex(canonicalTextOf(message.text)); + } if (ctx?.p !== undefined) anchor.p = ctx.p; if (ctx?.p2 !== undefined) anchor.p2 = ctx.p2; if (ctx?.x !== undefined) anchor.x = ctx.x; @@ -537,7 +615,7 @@ export function planReconciliation( covered: Set, positions?: { ids: string[]; canon: string[] }, ): ReconciliationPlan { - const plan: ReconciliationPlan = { claims: new Map(), byPos: 0, byTool: 0, byNorm: 0, byTurn: 0, byImage: 0, unmatched: [], idRewriteSuspects: 0, newOrder: [], nextCanon: undefined }; + const plan: ReconciliationPlan = { claims: new Map(), byPos: 0, byTool: 0, byTwin: 0, bySim: 0, byOrder: 0, byNorm: 0, byTurn: 0, byImage: 0, unmatched: [], idRewriteSuspects: 0, newOrder: [], nextCanon: undefined, resyncs: 0 }; // #2480: the positional copy is built on EVERY pass, so the first pass // after an upgrade establishes it and every later drift is // position-claimable. The fast path is reuse-by-ID (#2487): same id = @@ -614,40 +692,91 @@ export function planReconciliation( if (missingSet.has(id)) missingMiddle.push(id); } - // Pass 0 — positional identity (#2480). A codec switch mints new ids for - // unchanged content, but the array COUNT and ORDER survive; bili's stored - // canonical copy is the coordinate system. Scan head-aligned from the - // exact-prefix run and tail-aligned from the exact-suffix run; claim a - // missing covered id onto the inbound id at the SAME index only while the - // canonical fingerprints agree. First mismatch STOPS the scan — past a - // positional break (insertion/deletion) position carries no identity. The - // full-content fingerprint gates every claim, so consecutive duplicate - // messages can only pair onto identical content — harmless either way. - // The tail scan is also the graceful landing for client-native - // compaction (truncated head, surviving tail): tail blocks keep coverage - // while the vanished head ids fall into the honest unmatched path. + // Pass 0 — positional identity (#2480), order-centric since #2915. A + // codec switch mints new ids for unchanged content, but the array COUNT + // and ORDER survive; bili's stored canonical copy is the coordinate + // system. Scan head-aligned from the exact-prefix run and tail-aligned + // from the exact-suffix run; claim a missing covered id onto the inbound + // id at the SAME index only while the canonical fingerprints agree. An + // order break is a GAP, not a wall (#2915): on mismatch each scan + // resyncs at the next run of MIN_RESYNC_RUN consecutive same-index + // agreements (bounded by MAX_RESYNC_GAPS) and keeps claiming verified + // spans — the dual flanks absorb any NET offset (insertion, deletion, + // compaction), so same-index agreement inside a flank's region is + // ownership evidence, and the full-content fingerprint gates every + // claim, so consecutive duplicate messages can only pair onto identical + // content — harmless either way. Gap cells stay honestly unmatched and + // fall through to the content passes. The tail scan is also the graceful + // landing for client-native compaction (truncated head, surviving tail): + // tail blocks keep coverage while the vanished head ids fall into the + // honest unmatched path. + // #2915 gap-resolution evidence: same-index (oldId, newId) pairs recorded + // at Pass-0 fingerprint breaks + the old indices that WERE positionally + // claimed. A gap cell is bracketed when verified claims exist on both + // sides — order ownership that the tiers below may corroborate. + const gapPairs: { oldIdx: number; oldId: string; newId: string }[] = []; + const claimedIdx: number[] = []; + const recordGap = (oldIdx: number, oldId: string, newId: string): void => { + if (oldId !== newId && missingSet.has(oldId) && !covered.has(newId) && !claimedCandidates.has(newId)) { + gapPairs.push({ oldIdx, oldId, newId }); + } + }; if (stored !== undefined) { const head = Math.min(oldOrder.length, newOrder.length) - suffix; - for (let i = prefix; i < head; i++) { - if (canon[i] !== stored[i]) break; + let gaps = 0; + let i = prefix; + // #2915 gap resolution evidence lives above the stored block so the + // tier (below Pass 4) can see it. + while (i < head) { + if (canon[i] !== stored[i]) { + recordGap(i, oldOrder[i]!, newOrder[i]!); + let r = -1; + for (let k = i + 1; k + MIN_RESYNC_RUN <= head; k++) { + if (canon[k] === stored[k] && canon[k + 1] === stored[k + 1] && canon[k + 2] === stored[k + 2]) { r = k; break; } + } + if (r === -1 || ++gaps > MAX_RESYNC_GAPS) break; + plan.resyncs++; + i = r; + continue; + } const oldId = oldOrder[i]; const newId = newOrder[i]; - if (oldId === newId || !missingSet.has(oldId) || covered.has(newId) || claimedCandidates.has(newId)) continue; + if (oldId === newId || !missingSet.has(oldId) || covered.has(newId) || claimedCandidates.has(newId)) { i++; continue; } plan.claims.set(oldId, [newId]); claimedCandidates.add(newId); plan.byPos++; + claimedIdx.push(i); + i++; } const tail = Math.min(oldOrder.length, newOrder.length) - prefix; - for (let j = 1; j <= tail; j++) { + gaps = 0; + let j = 1; + while (j <= tail) { const iOld = oldOrder.length - j; const iNew = newOrder.length - j; - if (stored[iOld] !== canon[iNew]) break; + if (stored[iOld] !== canon[iNew]) { + recordGap(iOld, oldOrder[iOld]!, newOrder[iNew]!); + let r = -1; + for (let k = j + 1; k + MIN_RESYNC_RUN - 1 <= tail; k++) { + let ok = true; + for (let u = 0; u < MIN_RESYNC_RUN; u++) { + if (stored[oldOrder.length - (k + u)] !== canon[newOrder.length - (k + u)]) { ok = false; break; } + } + if (ok) { r = k; break; } + } + if (r === -1 || ++gaps > MAX_RESYNC_GAPS) break; + plan.resyncs++; + j = r; + continue; + } const oldId = oldOrder[iOld]; const newId = newOrder[iNew]; - if (oldId === newId || !missingSet.has(oldId) || covered.has(newId) || claimedCandidates.has(newId) || plan.claims.has(oldId)) continue; + if (oldId === newId || !missingSet.has(oldId) || covered.has(newId) || claimedCandidates.has(newId) || plan.claims.has(oldId)) { j++; continue; } plan.claims.set(oldId, [newId]); claimedCandidates.add(newId); plan.byPos++; + claimedIdx.push(iOld); + j++; } } @@ -678,6 +807,55 @@ export function planReconciliation( } } + // Pass 1.5 — tool-id twins (#2915, promoting #2396's detection-only + // buckets to a claim pass — owner rule 2026-10-12: order/content first, + // the id is a fast path, never a death sentence). A provider + // re-projection re-mints the authoritative toolCallId while the content + // stays byte-stable (dsh model switch: call_x|fc_0 → call_x_fc_0 on every + // pair, #2821). Pass 1 keys on exactly the field that was rewritten and + // misses by construction; the anchor's no-toolCallId norm (m) pairs it + // with an unclaimed inbound twin whose ONLY difference is the id. Same + // misattribution discipline as Pass 2: exact normalized content equality + // (role+contentType+toolName+text are all hash factors), k-th occurrence + // to k-th occurrence in conversation order, and the 4x/256 length + // collision guard — never fuzzy. A genuinely re-run tool call whose + // content differs never matches; an identical-content duplicate claims + // onto semantically identical bytes (harmless by content). Anchors + // persisted before #2396 lack `m` and skip this pass (backfilled while + // present verbatim — see the seed loop; pre-#2396 residue otherwise, an + // empty set in practice). + if (missingMiddle.length > 0 && candidates.length > 0) { + const twinBuckets = new Map(); + for (const cand of candidates) { + if (claimedCandidates.has(cand.id)) continue; + const t = cand.message.toolCallId; + if (t === undefined || t === "") continue; + const nt = normalizedIdentityNoToolCallId(cand.message); + const b = twinBuckets.get(nt); + if (b === undefined) twinBuckets.set(nt, { k: 0, ids: [cand.id], messages: [cand.message] }); + else { b.ids.push(cand.id); b.messages.push(cand.message); } + } + if (twinBuckets.size > 0) { + for (const oldId of missingMiddle) { + if (plan.claims.has(oldId)) continue; + const anchor = anchors[oldId]; + if (anchor === undefined || anchor.t === undefined || anchor.m === undefined) continue; + const b = twinBuckets.get(anchor.m); + if (b === undefined || b.k >= b.ids.length) continue; // absent or exhausted → skip + const message = b.messages[b.k]; + // Collision guard (Pass 2's): normalized equality at a hash + // slice plus a sane length ratio — reformatting cannot move + // length by 4x. + const len = message.text?.length ?? 0; + if (Math.abs(len - anchor.b) > Math.max(256, anchor.b >> 2)) continue; + plan.claims.set(oldId, [b.ids[b.k]]); + claimedCandidates.add(b.ids[b.k]); + b.k++; + plan.byTwin++; + } + } + } + // Pass 2 — normalized identity, k-th occurrence to k-th occurrence inside // the churn region (duplicate-cluster discipline at the normalized level: // deleting one of N identical messages shifts the survivors together, and @@ -878,16 +1056,82 @@ export function planReconciliation( } } + // #2915 gap resolution — owner rule 2026-10-12 (tiered fallback for + // Pass-0 gap cells: “任何通道匹配就认领;都不匹配但前后匹配也认领; + // 顺序在且同 session 就兜底”). The exact content passes (byTool / + // byTwin / byNorm / byTurn / byImage) run BELOW Pass 0, so by the time + // this tier sees a gap cell every exact channel has already had its + // bite — what remains is tiered corroboration, in order of evidence + // strength: + // (a) simhash: a re-rendered cell (formatting churn normalization + // still misses — key reordering beyond canonical-Json's reach, + // template re-stamps) keeps its word-shingle mass, so a 64-bit + // sketch within GAP_SIMHASH_MAX_HAMMING bits corroborates + // “same content, different bytes”; a re-run or edited message + // does not (measured: reorder 8, edit 7 vs re-run 11, JSON-vs- + // prose 30). Role/contentType cross-check + the byNorm length + // guard. Fence-independent: near-identical shingles at the SAME + // INDEX are ownership evidence on their own. + // (b) order bracket: the gap sits at the same index between + // VERIFIED positional claims (order was proven by fingerprint + // agreement on both flanks, within this session — reconcile is + // session-scoped by construction, which is the owner's + // “sessionid 也一样” premise). A replaced-in-place cell keeps + // its slot: same role, same toolName family, bounded length + // drift → the fold asserts the slot, not the stale bytes. The + // one stale cell this can carry (delete+insert at the same + // index) is the owner-sanctioned tradeoff for never losing the + // session to a single re-rendered cell (#2821). + // Zero corroboration anywhere (no fences, no sketch match, no exact + // channel) stays honest drift — “order is the same” is only a claim + // when something verified it. + if (gapPairs.length > 0) { + const fenced = plan.byPos > 0 || plan.resyncs > 0; + const minClaimed = claimedIdx.length > 0 ? Math.min(...claimedIdx) : -1; + const maxClaimed = claimedIdx.length > 0 ? Math.max(...claimedIdx) : -1; + for (const gap of gapPairs) { + if (plan.claims.has(gap.oldId) || claimedCandidates.has(gap.newId)) continue; + const anchor = anchors[gap.oldId]; + const message = byId.get(gap.newId); + if (anchor === undefined || message === undefined) continue; + if (anchor.r !== undefined && anchor.r !== message.role) continue; + const len = message.text?.length ?? 0; + if (Math.abs(len - anchor.b) > Math.max(256, anchor.b >> 2)) continue; + // (a) simhash corroboration + if (anchor.sh !== undefined) { + const sketch = simhashHex(canonicalTextOf(message.text ?? "")); + if (hammingHex(anchor.sh, sketch) >= 0 && hammingHex(anchor.sh, sketch) <= GAP_SIMHASH_MAX_HAMMING) { + plan.claims.set(gap.oldId, [gap.newId]); + claimedCandidates.add(gap.newId); + plan.bySim++; + continue; + } + } + // (b) order bracket — same index, claims verified on both sides + if (fenced && gap.oldIdx > minClaimed && gap.oldIdx < maxClaimed) { + const loose = Math.abs(len - anchor.b) > Math.max(1024, anchor.b); + if (loose) continue; + if (anchor.t !== undefined && (message.toolCallId === undefined || message.toolCallId === "")) continue; + plan.claims.set(gap.oldId, [gap.newId]); + claimedCandidates.add(gap.newId); + plan.byOrder++; + } + } + } + for (const id of missing) { if (!plan.claims.has(id)) plan.unmatched.push(id); } - // #2396 — detection-only rewrite suspects. Both claim passes key on the - // toolCallId itself, so a host that rewrites authoritative tool-call ids - // defeats them by construction; pairing old→new without host knowledge - // would be guessing (misattribution is the expensive failure mode). Cost: - // one no-toolCallId norm per unclaimed candidate carrying a toolCallId, - // drift passes only — steady state early-returns above (#2334 discipline). + // #2396 — residual rewrite suspects. Pass 1 keys on the toolCallId itself + // and misses rewritten ids by construction; Pass 1.5 (#2915) claims the + // content-identical twins, so what reaches here survived EVERY claim pass + // (length-guard rejects, exhausted duplicate families, real edits) — + // suspect evidence, never a repair target: pairing old→new beyond verified + // content equality would be guessing (misattribution is the expensive + // failure mode). Cost: one no-toolCallId norm per unclaimed candidate + // carrying a toolCallId, drift passes only — steady state early-returns + // above (#2334 discipline). if (plan.unmatched.length > 0) { const noToolWitnesses = new Map(); for (const cand of candidates) { @@ -976,13 +1220,13 @@ export function reconcileFoldCoverage(session: Session, msgs: CoreMessage[], opt const mode = opts.mode ?? resolveFoldReconcileMode(process.env); if (mode === "off") { resetFoldDriftState(session); - return { kind: "off", missing: 0, claims: 0, byPos: 0, byTool: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: 0, unmatched: 0 }; + return { kind: "off", missing: 0, claims: 0, byPos: 0, byTool: 0, byTwin: 0, bySim: 0, byOrder: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: 0, unmatched: 0 }; } const blocks = (session.state?.blocks ?? []) as BlockLike[]; const covered = coveredIdsOf(blocks); if (covered.size === 0) { resetFoldDriftState(session); - return { kind: "noop", missing: 0, claims: 0, byPos: 0, byTool: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: 0, unmatched: 0 }; + return { kind: "noop", missing: 0, claims: 0, byPos: 0, byTool: 0, byTwin: 0, bySim: 0, byOrder: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: 0, unmatched: 0 }; } // #2202: auxiliary side-requests (title-gen, WebSearch refinement — #1075) // share the conversation id but do not carry the conversation. Reconciling @@ -995,9 +1239,9 @@ export function reconcileFoldCoverage(session: Session, msgs: CoreMessage[], opt // the episode state is left exactly as found (the resets above stay // reserved for true episode boundaries: reconcile off / no folds at all). if (msgs.length < SIDE_REQUEST_MAX_MSGS) { - return { kind: "noop", missing: 0, claims: 0, byPos: 0, byTool: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: 0, unmatched: 0 }; + return { kind: "noop", missing: 0, claims: 0, byPos: 0, byTool: 0, byTwin: 0, bySim: 0, byOrder: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: 0, unmatched: 0 }; } - if (!session.metadata) return { kind: "noop", missing: 0, claims: 0, byPos: 0, byTool: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: 0, unmatched: 0 }; + if (!session.metadata) return { kind: "noop", missing: 0, claims: 0, byPos: 0, byTool: 0, byTwin: 0, bySim: 0, byOrder: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: 0, unmatched: 0 }; const anchors: Record = (session.metadata[METADATA_ANCHORS] as Record | undefined) ?? {}; @@ -1019,7 +1263,7 @@ export function reconcileFoldCoverage(session: Session, msgs: CoreMessage[], opt if (process.env.FOLD_RECONCILE_DEBUG === "1") { const active = blocks.filter((b) => b.active).length; const sample = msgs.slice(0, 6).map((m) => `${m.role}/${m.contentType ?? "-"}/${m.id ?? "?"}`); - process.stderr.write(`[fold-reconcile-dbg] blocks=${blocks.length} active=${active} covered=${covered.size} oldOrder=${oldOrder.length} positions=${positions?.canon.length ?? -1} missing=${plan.unmatched.length + plan.claims.size} claims=${plan.claims.size} byPos=${plan.byPos} byTool=${plan.byTool} byNorm=${plan.byNorm} byTurn=${plan.byTurn} byImage=${plan.byImage} unmatched=${plan.unmatched.length} | msgs[0..5]=${sample.join(" | ")}\n`); + process.stderr.write(`[fold-reconcile-dbg] blocks=${blocks.length} active=${active} covered=${covered.size} oldOrder=${oldOrder.length} positions=${positions?.canon.length ?? -1} missing=${plan.unmatched.length + plan.claims.size} claims=${plan.claims.size} byPos=${plan.byPos} resyncs=${plan.resyncs} byTool=${plan.byTool} byTwin=${plan.byTwin} bySim=${plan.bySim} byOrder=${plan.byOrder} byNorm=${plan.byNorm} byTurn=${plan.byTurn} byImage=${plan.byImage} unmatched=${plan.unmatched.length} | msgs[0..5]=${sample.join(" | ")}\n`); } // Seed/refresh anchors for covered ids present in this pass (including @@ -1321,15 +1565,15 @@ export function reconcileFoldCoverage(session: Session, msgs: CoreMessage[], opt } if (plan.unmatched.length === 0 && plan.claims.size === 0) { - return { kind: "resend", missing: 0, claims: 0, byPos: 0, byTool: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: reapedCount, unmatched: 0 }; + return { kind: "resend", missing: 0, claims: 0, byPos: 0, byTool: 0, byTwin: 0, bySim: 0, byOrder: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: reapedCount, unmatched: 0 }; } if (opts.log !== undefined) { if (plan.claims.size > 0 && mode === "repair") { opts.log(plan.unmatched.length > 0 ? "warn" : "info", - `${tag}[fold-reconcile] resent history drifted: ${plan.claims.size + plan.unmatched.length} covered id(s) missing — reanchored ${plan.claims.size} (${plan.byPos} positional, ${plan.byTool} by toolCallId, ${plan.byNorm} by normalized identity${plan.byTurn > 0 ? `, ${plan.byTurn} by assistant-turn key` : ""}${plan.byImage > 0 ? `, ${plan.byImage} by image-capability variant` : ""}) onto churned bytes, ${plan.unmatched.length} unmatched re-enter the wire unfolded${plan.idRewriteSuspects > 0 ? `; ${plan.idRewriteSuspects} suspected host-rewritten tool-call ids (identical content under a different toolCallId — #2396)` : ""} (#1921)${driftTok}`); + `${tag}[fold-reconcile] resent history drifted: ${plan.claims.size + plan.unmatched.length} covered id(s) missing — reanchored ${plan.claims.size} (${plan.byPos} positional${plan.resyncs > 0 ? `, ${plan.resyncs} order break(s) resynced` : ""}, ${plan.byTool} by toolCallId${plan.byTwin > 0 ? `, ${plan.byTwin} by toolCallId twin` : ""}${plan.bySim > 0 ? `, ${plan.bySim} by simhash corroboration` : ""}${plan.byOrder > 0 ? `, ${plan.byOrder} by order bracket` : ""}, ${plan.byNorm} by normalized identity${plan.byTurn > 0 ? `, ${plan.byTurn} by assistant-turn key` : ""}${plan.byImage > 0 ? `, ${plan.byImage} by image-capability variant` : ""}) onto churned bytes, ${plan.unmatched.length} unmatched re-enter the wire unfolded${plan.idRewriteSuspects > 0 ? `; ${plan.idRewriteSuspects} suspected host-rewritten tool-call ids (identical content under a different toolCallId — #2396)` : ""} (#1921)${driftTok}`); } else if (plan.claims.size > 0) { opts.log("warn", - `${tag}[fold-reconcile] resent history drifted: ${plan.claims.size + plan.unmatched.length} covered id(s) missing, ${plan.claims.size} matchable by anchor (${plan.byPos} positional, ${plan.byTool} toolCallId, ${plan.byNorm} normalized${plan.byTurn > 0 ? `, ${plan.byTurn} turn key` : ""}${plan.byImage > 0 ? `, ${plan.byImage} image` : ""}) but reconcile=warn made no repair (#1921)${driftTok}`); + `${tag}[fold-reconcile] resent history drifted: ${plan.claims.size + plan.unmatched.length} covered id(s) missing, ${plan.claims.size} matchable by anchor (${plan.byPos} positional${plan.resyncs > 0 ? `, ${plan.resyncs} resynced` : ""}, ${plan.byTool} toolCallId${plan.byTwin > 0 ? `, ${plan.byTwin} twin` : ""}${plan.bySim > 0 ? `, ${plan.bySim} simhash` : ""}${plan.byOrder > 0 ? `, ${plan.byOrder} order-bracket` : ""}, ${plan.byNorm} normalized${plan.byTurn > 0 ? `, ${plan.byTurn} turn key` : ""}${plan.byImage > 0 ? `, ${plan.byImage} image` : ""}) but reconcile=warn made no repair (#1921)${driftTok}`); } else if (reasoningOnlyDrop) { opts.log("info", `${tag}[fold-reconcile] ${plan.unmatched.length} covered reasoning core(s) dropped by the host (cross-model replay discards structured thinking; every text/tool core intact) — fold substrate healthy, tolerated (#2193 follow-up)`); @@ -1361,6 +1605,9 @@ export function reconcileFoldCoverage(session: Session, msgs: CoreMessage[], opt claims: plan.claims.size, byPos: plan.byPos, byTool: plan.byTool, + byTwin: plan.byTwin, + bySim: plan.bySim, + byOrder: plan.byOrder, byNorm: plan.byNorm, byTurn: plan.byTurn, byImage: plan.byImage, diff --git a/tests/fold-reconcile-positional.test.ts b/tests/fold-reconcile-positional.test.ts index 91f7ceb59c..27b3b17d07 100644 --- a/tests/fold-reconcile-positional.test.ts +++ b/tests/fold-reconcile-positional.test.ts @@ -165,16 +165,34 @@ describe("planReconciliation Pass 0 (#2480)", () => { assert.equal(positionalFingerprintWorkCount(), 2, "only the appended messages hash — the resent prefix is free"); }); - test("mid-history edit: scans stop at the break, no misattribution", () => { + test("mid-history edit inside verified flanks: the order bracket claims the slot (#2915 owner rule)", () => { const old = oldHistory(); const pos = storedOf(old); const { plan } = planFor(old, codecSwitch(old, 5), pos); - // head claims 0-4, tail claims 6-9, edited 5 is honestly unmatched + // head claims 0-4, tail claims 6-9; the edited 5 sits at the SAME + // index between two VERIFIED runs — owner rule 2026-10-12: none of + // the content channels match, but the flanks do → the fold asserts + // the SLOT, not the stale bytes. assert.equal(plan.byPos, 9); - assert.equal(plan.unmatched.length, 1); - assert.equal(plan.unmatched[0], "old_5"); + assert.equal(plan.byOrder, 1); + assert.equal(plan.unmatched.length, 0); assert.deepEqual(plan.claims.get("old_0"), ["new_0"]); assert.deepEqual(plan.claims.get("old_9"), ["new_9"]); + assert.deepEqual(plan.claims.get("old_5"), ["new_5"], "bracketed edit keeps coverage — no island loss"); + }); + + test("mid-history edit WITHOUT a verified far flank stays honestly unmatched (no guessing)", () => { + const old = oldHistory(); + const pos = storedOf(old); + // cells 5..9 ALL rewritten: head claims 0-4, the tail scan finds no + // 3-run agreement anywhere — old_5 is NOT bracketed (no verified + // claim beyond it), so no tier may claim it. + const switched = codecSwitch(old); + for (let i = 5; i <= 9; i++) switched[i] = msg(`new_${i}`, "user", `completely different body ${i}`); + const { plan } = planFor(old, switched, pos); + assert.equal(plan.byPos, 5, "only the verified head span claims"); + assert.equal(plan.byOrder, 0, "unbracketed — the bracket tier must stay silent"); + assert.equal(plan.unmatched.length, 5); assert.equal(plan.claims.get("old_5"), undefined); }); diff --git a/tests/fold-reconcile.test.ts b/tests/fold-reconcile.test.ts index 463d890e78..77da2656cb 100644 --- a/tests/fold-reconcile.test.ts +++ b/tests/fold-reconcile.test.ts @@ -15,7 +15,9 @@ import { type FoldAnchor, type FoldBlockCoverage, type ReconcileOptions, + canonicalTextOf, } from "../src/fold-reconcile.ts"; +import { simhashHex } from "../src/affinity-simhash.ts"; import { defaultCountTokens } from "acp-kernel"; import type { CoreMessage } from "acp-kernel"; import type { Session } from "../src/session.ts"; @@ -812,12 +814,13 @@ describe("reconcileFoldCoverage anchor cap boundary (#2334)", () => { r: "user", c: "text", b: msgs[5].text!.length, + sh: simhashHex(canonicalTextOf(msgs[5].text!)), }, "retained anchor values keep the fresh anchorFrom shape"); const snapshot = JSON.stringify(session.metadata); resetNormalizedIdentityWork(); const second = reconcileFoldCoverage(session, msgs, opts); - assert.deepEqual(second, { kind: "resend", missing: 0, claims: 0, byPos: 0, byTool: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: 0, unmatched: 0 }); + assert.deepEqual(second, { kind: "resend", missing: 0, claims: 0, byPos: 0, byTool: 0, byTwin: 0, bySim: 0, byOrder: 0, byNorm: 0, byTurn: 0, byImage: 0, reaped: 0, unmatched: 0 }); assert.equal(normalizedIdentityWorkCount(), 0, "the overflow tail must not be normalized+hashed and dropped AGAIN every pass (#2334)"); assert.equal(JSON.stringify(session.metadata), snapshot, "steady-state resend leaves metadata byte-stable"); @@ -891,20 +894,23 @@ describe("reconcileFoldCoverage anchor cap boundary (#2334)", () => { }); }); -describe("rewrite-suspect detection (#2396)", () => { +describe("rewrite-suspect detection (#2396) → twin claiming (#2915)", () => { // A host may rewrite authoritative tool-call ids across provider // projections (Prime: call_x|fc_0 → call_x_fc_0 on both sides of every - // pair). Both claim passes key on the toolCallId, so such pairs defeat - // reconciliation by construction; the module must COUNT them (detection - // only) and never claim or repair onto them. + // pair). Pass 1 keys on exactly that field and misses by construction. + // #2396 made this detection-only; #2915 (owner rule 顺序为中心/hash 为 + // 辅助) promotes the no-toolCallId norm to a claim pass: exact normalized + // content equality + k-th discipline + the length guard. The detection + // counter now reports only twins that STILL went unclaimed. const oldCall = msg("tc1", "tool_result", "payload body words", { toolCallId: "call_seed|fc_0", toolName: "probe" }); const rewrittenTwin = msg("tc1-rw", "tool_result", "payload body words", { toolCallId: "call_seed_fc_0", toolName: "probe" }); - test("identical-content twin under a different toolCallId is counted, never claimed", () => { + test("identical-content twin under a different toolCallId is CLAIMED by the twin pass (#2915)", () => { const plan = planReconciliation(["tc1"], { tc1: anchorOf(oldCall) }, [rewrittenTwin], new Set(["tc1"])); - assert.equal(plan.claims.size, 0, "no host knowledge → no pairing"); - assert.deepEqual(plan.unmatched, ["tc1"]); - assert.equal(plan.idRewriteSuspects, 1); + assert.equal(plan.byTwin, 1); + assert.deepEqual([...plan.claims.entries()], [["tc1", ["tc1-rw"]]]); + assert.deepEqual(plan.unmatched, []); + assert.equal(plan.idRewriteSuspects, 0, "claimed twins are no longer residual suspects"); }); test("stable toolCallId with churned bytes still claims (unchanged path)", () => { diff --git a/tests/issue2915-order-centric-reconcile.test.ts b/tests/issue2915-order-centric-reconcile.test.ts new file mode 100644 index 0000000000..8c796866b0 --- /dev/null +++ b/tests/issue2915-order-centric-reconcile.test.ts @@ -0,0 +1,356 @@ +// [#2915] Order-centric reconciliation — owner rule 2026-10-12: +// 顺序为中心,hash 为辅助 (order is the carrier, content the per-cell gate, +// the id a fast path — never a death sentence). +// +// Replays the #2821 incident (beicause, dsh 0.2.0-rc.2 + bili 0.1.193): a +// mid-session model switch re-minted EVERY tool-call id on re-projection +// (content byte-identical, ids different). The old Pass 0 stopped at the +// first fingerprint mismatch and both flank scans died at their first cells +// (changed preamble at the head, appended turn at the tail): byPos=0, +// 3289 covered ids missing, three passes → "compression substrate appears +// destroyed" → ALL folds dropped → ~4MB raw out → doomed-probe 502. +// +// The order-centric upgrade (#2915): +// - Pass 0 RESYNCS at the next run of consecutive same-index agreements: +// an order break is a gap, not a wall — verified spans past the break +// keep their coverage, gap cells fall through honestly. +// - Pass 1.5 claims tool-id twins (#2396 detection promoted): exact +// normalized content equality under a rewritten toolCallId, k-th +// discipline, length-guarded — the fallback when no stored copy exists. +// - Escalation to "substrate destroyed" only counts gaps that survived +// resync AND the twin/norm content passes. +// - Gap resolution (owner rule, same day): a Pass-0 gap cell is claimed by +// (a) simhash corroboration (≤8 bits on the 64-bit word-shingle sketch — +// re-rendered content, measured margins) or (b) the order bracket (same +// index BETWEEN verified claims — the fold asserts the SLOT). Zero +// corroboration stays honest drift. +// - Latch recovery: an escalated session heals the moment a pass verifies +// again (streak reset clears the latch) — upgrade-in-place recovery. +import { test, describe } from "node:test"; +import assert from "node:assert/strict"; +import { simhashHex, hammingHex } from "../src/affinity-simhash.ts"; +import { + normalizedIdentity, + normalizedIdentityNoToolCallId, + positionalFingerprint, + canonicalTextOf, + planReconciliation, + reconcileFoldCoverage, + type FoldAnchor, + type ReconcileOptions, +} from "../src/fold-reconcile.ts"; +import type { CoreMessage } from "acp-kernel"; +import { SessionStore, _setStoreForTest } from "../src/persist.ts"; +import type { Session } from "../src/session.ts"; + +function msg(id: string, role: string, text: string, extra?: Partial): CoreMessage { + return { id, role, contentType: "text", text, ...extra } as CoreMessage; +} +/** Real-identity anchors (n + t + m + sh) — the shape reconcileFoldCoverage seeds. */ +function anchorOf(m: CoreMessage): FoldAnchor { + const a: FoldAnchor = { n: normalizedIdentity(m), r: m.role, c: m.contentType, b: m.text?.length ?? 0 }; + if (m.toolCallId) { + a.t = m.toolCallId; + a.m = normalizedIdentityNoToolCallId(m); + } + if (m.text && m.text !== "" && m.contentType !== "reasoning") { + a.sh = simhashHex(canonicalTextOf(m.text)); + } + return a; +} +function fakeSession(ids: string[]): Session { + return { + state: { + blocks: [{ active: true, blockId: "blk-2915", effectiveMessageIds: [...ids] }], + messageRefs: { byRaw: {}, byRef: {} }, + }, + metadata: {}, + } as unknown as Session; +} +const opts = (log?: (level: string, m: string) => void): ReconcileOptions => ({ mode: "repair", sessionId: "s2915", log: log ?? (() => {}) }); + +/** A #2821-shaped history: preamble + prose turns + tool calls with composite + * ids (`call_x|fc_N`) — exactly the field dsh re-projection rewrites. */ +function history(): CoreMessage[] { + return [ + msg("pre", "user", "session preamble: project setup instructions"), + msg("m1", "user", "first user turn body"), + msg("t1", "assistant", '{"query":"step one","limit":1}', { contentType: "tool-call", toolName: "search", toolCallId: "call_x|fc_1" }), + msg("m2", "assistant", "assistant reply after step one"), + msg("t2", "assistant", '{"query":"step two","limit":2}', { contentType: "tool-call", toolName: "search", toolCallId: "call_x|fc_2" }), + msg("m3", "user", "third user turn body"), + msg("t3", "assistant", '{"query":"step three","limit":3}', { contentType: "tool-call", toolName: "search", toolCallId: "call_x|fc_3" }), + msg("m4", "assistant", "fourth message body"), + msg("m5", "user", "fifth message body"), + msg("m6", "assistant", "sixth message body"), + msg("m7", "user", "seventh message body"), + msg("end", "assistant", "final turn body"), + ]; +} + +/** dsh-style re-projection: EVERY id re-minted, every composite toolCallId + * rewritten (`|` → `_`), content byte-identical. `changePreamble` also + * replaces the first message's text (a model switch rewrites its wrapper). */ +function remint(old: CoreMessage[], gen: string, changePreamble = false, append?: string): CoreMessage[] { + const out = old.map((m, i) => { + const text = i === 0 && changePreamble ? `${m.text} [rewritten by model switch]` : m.text; + return msg(`${gen}_${i}`, m.role, text ?? "", { + contentType: m.contentType, + toolName: m.toolName, + toolCallId: m.toolCallId !== undefined ? m.toolCallId.replace("|", "_") : undefined, + }); + }); + if (append !== undefined) out.push(msg(`${gen}_app`, "user", append)); + return out; +} + +function planFor(old: CoreMessage[], next: CoreMessage[], positions?: { ids: string[]; canon: string[] }) { + const order = old.map((m) => m.id as string); + const anchors: Record = {}; + for (const m of old) anchors[m.id as string] = anchorOf(m); + const covered = new Set(order); + return planReconciliation(order, anchors, next, covered, positions); +} +const storedOf = (msgs: CoreMessage[]): { ids: string[]; canon: string[] } => + ({ ids: msgs.map((m) => m.id as string), canon: msgs.map((m) => positionalFingerprint(m)) }); + +describe("Pass 0 order-centric resync (#2915)", () => { + test("#2821 replay: full id re-mint + changed preamble + appended turn — order carries, one resync, no mass loss", () => { + const old = history(); + const next = remint(old, "g1", true, "brand new request turn"); + const plan = planFor(old, next, storedOf(old)); + // the head scan crosses the changed-preamble island and claims every + // verified cell behind it; the tail scan crosses the appended turn + assert.ok(plan.resyncs >= 1, `expected resyncs, got ${plan.resyncs}`); + assert.equal(plan.byPos, 11, "all non-preamble ids claimed positionally"); + assert.deepEqual(plan.unmatched, ["pre"], "only the honestly-changed preamble is unmatched"); + for (let i = 1; i < 12; i++) assert.deepEqual(plan.claims.get(old[i]!.id), [`g1_${i}`]); + assert.equal(plan.byTwin, 0, "positional coverage leaves nothing for the twin pass"); + }); + + test("replacement island mid-history: resync claims both flanks, the bracket tier claims the slot (owner rule)", () => { + const old = history(); + const next = remint(old, "g1"); + next[5] = msg("g1_5", "user", "EDITED: content genuinely changed"); // index 5 island (m3, prose) + const plan = planFor(old, next, storedOf(old)); + // the island content is neither normalized-equal nor simhash-close + // (JSON vs prose ≈ 30 bits), but it sits at the SAME INDEX between + // verified claims — owner rule: flanks match → the fold asserts the + // slot, not the stale bytes. + assert.ok(plan.resyncs >= 1, `expected resyncs, got ${plan.resyncs}`); + assert.equal(plan.byPos, 11); + assert.equal(plan.byOrder, 1, "the bracket tier claims the island slot"); + assert.deepEqual(plan.unmatched, [], "no island loss"); + assert.deepEqual(plan.claims.get("end"), ["g1_11"], "the tail span past the island keeps coverage"); + assert.deepEqual(plan.claims.get("m3"), ["g1_5"]); + }); + + test("insertion (net offset) still lands on the flank scans — resync never mispairs shifted cells", () => { + const old = history(); + const switched = remint(old, "g1"); + const next = [...switched.slice(0, 5), msg("ins_0", "user", "brand new inserted turn"), ...switched.slice(5)]; + const plan = planFor(old, next, storedOf(old)); + assert.equal(plan.byPos, 12, "head + tail flanks absorb the offset; every old id keeps coverage"); + assert.equal(plan.unmatched.length, 0); + }); + + test("mid-list deletion (net-negative offset): flanks absorb, deleted cell honestly unmatched", () => { + const old = history(); + const switched = remint(old, "g1"); + const next = [...switched.slice(0, 5), ...switched.slice(6)]; // drop index 5 (m3) + const plan = planFor(old, next, storedOf(old)); + assert.equal(plan.byPos, 11, "head + tail flanks absorb the negative offset"); + assert.deepEqual(plan.unmatched, ["m3"], "the deleted cell is the only honest loss"); + assert.deepEqual(plan.claims.get("t3"), ["g1_6"], "the cell after the gap re-anchors past it"); + }); +}); + +describe("Pass 1.5 tool-id twin claiming (#2396 → #2915)", () => { + test("no stored copy: re-minted toolCallIds with identical content are claimed by the twin pass", () => { + const old = history(); + const next = remint(old, "g1"); // byte-identical content, every id (incl. tool ids) rewritten + const plan = planFor(old, next, undefined); // pre-#2480 fallback: no positional copy + assert.equal(plan.byTwin, 3, "the three tool anchors pair onto content-identical twins"); + assert.equal(plan.byTool, 0, "rewritten ids defeat the toolCallId fast path"); + assert.equal(plan.unmatched.length, 0, "prose falls to byNorm — nothing is lost"); + assert.deepEqual(plan.claims.get("t1"), ["g1_2"]); + assert.deepEqual(plan.claims.get("t3"), ["g1_6"]); + }); + + test("genuinely re-run tool call (different content) is NEVER claimed", () => { + const oldCall = msg("tc1", "assistant", '{"status":"3 failed"}', { contentType: "tool-call", toolName: "build", toolCallId: "call_a|fc_0" }); + const rerun = msg("tc1-new", "assistant", '{"status":"all green"}', { contentType: "tool-call", toolName: "build", toolCallId: "call_b_fc_0" }); + const order = ["tc1"]; + const anchors = { tc1: anchorOf(oldCall) }; + const plan = planReconciliation(order, anchors, [rerun], new Set(order)); + assert.equal(plan.byTwin, 0); + assert.deepEqual(plan.unmatched, ["tc1"], "90%-similar-but-different content must not be asserted by a summary"); + assert.equal(plan.idRewriteSuspects, 0); + }); + + test("duplicate families pair k-th to k-th in conversation order", () => { + const mk = (id: string, callId: string): CoreMessage => + msg(id, "user", "ok", { contentType: "tool-result", toolName: "probe", toolCallId: callId }); + const old = [mk("a1", "call_a|1"), msg("mid", "user", "between"), mk("a2", "call_a|2")]; + const next = [mk("b1", "call_b_1"), msg("mid", "user", "between"), mk("b2", "call_b_2")]; + const plan = planFor(old, next, undefined); + assert.equal(plan.byTwin, 2); + assert.deepEqual(plan.claims.get("a1"), ["b1"], "first anchor → first twin"); + assert.deepEqual(plan.claims.get("a2"), ["b2"], "second anchor → second twin"); + }); +}); + +describe("escalation semantics (#2915) — the #2821 death spiral stays closed", () => { + test("three consecutive re-mint drifts: folds keep repairing, never escalate, never reap", () => { + const originals = history(); + const session = fakeSession(originals.map((m) => m.id as string)); + const logs: { level: string; msg: string }[] = []; + const log = (level: string, m: string) => logs.push({ level, msg: m }); + reconcileFoldCoverage(session, originals, opts(log)); // steady: seeds anchors/order/positions + + for (const gen of ["g1", "g2", "g3"]) { + const drifted = remint(originals, gen, true, `new request on ${gen}`); + const r = reconcileFoldCoverage(session, drifted, opts(log)); + assert.equal(r.kind, "reanchored"); + assert.equal(r.byPos, 11, `${gen}: order carries the whole re-mint`); + assert.equal(r.unmatched, 1, `${gen}: only the changed preamble id is honest loss`); + assert.equal(r.reaped, 0, `${gen}: no zombie reap`); + } + assert.ok(!logs.some((l) => l.msg.includes("compression substrate appears destroyed")), + "the #2821 verdict must never fire for re-mint churn"); + assert.ok(session.metadata.foldDriftEscalated === undefined, "escalation latch never sets"); + assert.ok(session.metadata.foldDriftStreak === undefined || session.metadata.foldDriftStreak === 0, + "claims every pass keep the streak reset"); + assert.ok(logs.some((l) => l.msg.includes("order break(s) resynced")), "resync is observable in the drift line"); + // the fold rides the newest generation's ids — coverage survived three model switches + const covered = (session.state.blocks as { effectiveMessageIds: string[] }[])[0].effectiveMessageIds; + for (let i = 1; i < 12; i++) assert.ok(covered.includes(`g3_${i}`), `g3_${i} covered`); + assert.ok(covered.includes("pre"), "the honestly-lost preamble id stays listed (warn, not destroy)"); + }); + + test("pre-latched escalation (old-build session crossing the upgrade) un-latches on the first recovered pass", () => { + const originals = history(); + const session = fakeSession(originals.map((m) => m.id as string)); + reconcileFoldCoverage(session, originals, opts()); // steady: seeds anchors/order/positions + // A session that latched destruction under an OLDER build (where re-mint + // churn could not recover) persists exactly this triple across the + // upgrade; the first recovered pass must clear it or every LATER + // episode stays silently latched (#2297 contract). + session.metadata.foldDriftStreak = 3; + session.metadata.foldDriftSince = Date.now(); + session.metadata.foldDriftEscalated = true; + const r = reconcileFoldCoverage(session, remint(originals, "g1", true, "req g1"), opts()); + assert.equal(r.kind, "reanchored", "the recovery itself works"); + assert.equal(r.byPos, 11); + assert.ok(session.metadata.foldDriftEscalated === undefined, "stale latch cleared"); + assert.ok(session.metadata.foldDriftStreak === undefined, "streak resets with the latch"); + assert.ok(session.metadata.foldDriftSince === undefined, "episode start resets with the latch"); + }); + + test("without the positional copy (pre-#2480 upgrade path) the twin pass alone prevents the spiral", () => { + const originals = history(); + const session = fakeSession(originals.map((m) => m.id as string)); + const logs: { level: string; msg: string }[] = []; + reconcileFoldCoverage(session, originals, opts()); + delete (session.metadata as Record).foldPositions; + // h1 rides the twin pass (no stored copy); its own pass SAVES a fresh + // {ids, canon} copy, so h2/h3 upgrade to the positional path — the + // order-centric system self-heals its evidence after one churn. + const r1 = reconcileFoldCoverage(session, remint(originals, "h1", true, "req h1"), opts((l, m) => logs.push({ level: l, msg: m }))); + assert.equal(r1.byTwin, 3, "h1: all tool anchors recovered via twins"); + assert.equal(r1.unmatched, 1, "h1: only the changed preamble"); + for (const gen of ["h2", "h3"]) { + const r = reconcileFoldCoverage(session, remint(originals, gen, true, `req ${gen}`), opts((l, m) => logs.push({ level: l, msg: m }))); + assert.equal(r.byPos, 11, `${gen}: positional copy regenerated — order carries the re-mint`); + assert.equal(r.unmatched, 1, `${gen}: only the changed preamble`); + } + assert.ok(!logs.some((l) => l.msg.includes("compression substrate appears destroyed"))); + }); +}); + +describe("gap resolution tiers (owner rule 2026-10-12: any channel → flanks → verified order)", () => { + test("simhash tier: formatting churn the exact normalizers miss still corroborates at the same index", () => { + const old = history(); + old[7] = msg("m4", "assistant", + "The quick brown fox jumps over the lazy dog near the river bank at dawn while birds sing"); + const next = remint(old, "g1"); + // one word varied in a long sentence: measured ≈6 bits on the 64-bit + // word-shingle sketch — re-rendered wording, not a different message. + // (Negation flips measure ≈11 and word drops ≈13: OUTSIDE the gate.) + next[7] = msg("g1_7", "assistant", + "The quick brown fox jumps over the lazy dog near the river bank at dawn while birds chirp"); + const plan = planFor(old, next, storedOf(old)); + assert.equal(plan.bySim, 1, "the re-worded-but-same message corroborates by simhash"); + assert.equal(plan.byOrder, 0, "simhash runs before the bracket tier"); + assert.deepEqual(plan.unmatched, []); + assert.deepEqual(plan.claims.get("m4"), ["g1_7"]); + }); + + test("mid-column deletion (#2915 review gap ③): flanks absorb the shift, the deleted id stays honestly unmatched", () => { + const old = history(); + const kept = remint(old.filter((_, i) => i !== 6), "g1"); // drop t3 (index 6) + const plan = planFor(old, kept, storedOf(old)); + // head claims 0-5 at the same index; the tail scan walks old 11→7 + // onto new 10→6 (order is the carrier); old_6 has no inbound cell. + assert.equal(plan.byPos, 11); + assert.equal(plan.byOrder, 0, "no bracket claim — the would-be slot cell is consumed by the verified tail"); + assert.deepEqual(plan.unmatched, ["t3"], "deletion is honest loss, never a guess"); + assert.deepEqual(plan.claims.get("end"), ["g1_10"]); + assert.deepEqual(plan.claims.get("m4"), ["g1_6"]); + }); + + test("delete + insert at the same index: the bracket claims the slot (owner-sanctioned stale-cell tradeoff)", () => { + const old = history(); + const next = remint(old, "g1"); + next[5] = msg("g1_5", "user", "a genuinely different replacement turn"); // m3 replaced in place + const plan = planFor(old, next, storedOf(old)); + assert.equal(plan.byPos, 11); + assert.equal(plan.byOrder, 1, "same index, verified claims on BOTH sides → assert the slot"); + assert.deepEqual(plan.unmatched, []); + assert.deepEqual(plan.claims.get("m3"), ["g1_5"], "coverage rides the slot — documented tradeoff, not a guess"); + }); +}); + +describe("escalation latch recovery (review gap ②) — upgrade-in-place heals", () => { + test("3 total-loss passes latch once; a healthy resend clears the latch; post-recovery churn still repairs", () => { + _setStoreForTest(new SessionStore({ enabled: false })); // escalation reaps via markDirty + const originals = history(); + const session = fakeSession(originals.map((m) => m.id as string)); + const logs: { level: string; msg: string }[] = []; + const log = (level: string, m: string) => logs.push({ level, msg: m }); + reconcileFoldCoverage(session, originals, opts(log)); // steady seed + + // total drift: EVERY cell's content rewritten (not just ids) — no + // channel, no simhash corroboration, and no verified flank ⇒ no tier + // may claim. Exactly the pre-#2915 death spiral, now latched once. + const scramble = (gen: string): CoreMessage[] => + originals.map((m, i) => msg(`${gen}_${i}`, m.role, `unrelated replacement words ${gen} ${i}`, { + contentType: m.contentType, toolName: m.toolName, toolCallId: m.toolCallId !== undefined ? `call_${gen}_${i}` : undefined, + })); + for (const gen of ["x1", "x2", "x3"]) { + const r = reconcileFoldCoverage(session, scramble(gen), opts(log)); + assert.equal(r.kind, "unmatched", `${gen}: total drift stays honest`); + } + assert.equal(session.metadata.foldDriftEscalated, true, "latched after 3 consecutive total-drift passes"); + assert.equal(session.metadata.foldDriftStreak, 3); + assert.equal(logs.filter((l) => l.msg.includes("compression substrate appears destroyed")).length, 1, + "#2297: exactly ONE error line per episode"); + + // healthy resend of the ORIGINAL bytes: missing=0 → noop, and the + // streak reset clears the latch — the upgrade-in-place recovery path. + const healthy = reconcileFoldCoverage(session, originals, opts(log)); + assert.equal(healthy.kind, "noop"); + assert.ok(session.metadata.foldDriftEscalated === undefined, "latch cleared on recovery"); + assert.ok(session.metadata.foldDriftStreak === undefined, "streak cleared on recovery"); + + // Escalation reaped the destroyed block (#2695 zombie-carrier reap) — + // so after the latch clears the ledger is a CLEAN SLATE, not a repair + // target: nothing is covered, nothing missing, and the next fold + // cycle starts fresh. That is the upgrade-in-place contract: a + // v0.1.192→v0.1.196 upgrade with a latched ledger heals instead of + // dead-looping the doomed preflight. + assert.equal((session.state.blocks as unknown[]).length, 0, "destroyed substrate reaped at escalation"); + const r = reconcileFoldCoverage(session, remint(originals, "g9"), opts(log)); + assert.equal(r.kind, "noop", "clean slate — the lane lives, folds rebuild on the next compress cycle"); + }); +});