Skip to content

feat(strategy): fail loudly on invalid store topology; merge adjacency in store order - #124

Merged
antra-tess merged 1 commit into
mainfrom
fix/topology-fail-loud
Sep 26, 2026
Merged

antra-tess merged 1 commit into
mainfrom
fix/topology-fail-loud

Conversation

@antra-tess

@antra-tess antra-tess commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Two ways a crossed node (a summary whose leaves are not contiguous among owned messages) got minted, and nothing that said so afterwards:

A second large production store carries 7 crossed L2/L3s of the #95 shape. Nothing at load or compile time reported either.

Changes

Fail loudly at load. initialize audits every summary: leaves (via chunk-linked L1s — superseded legacy generations own nothing) positioned among owned messages in store order; a hole is always another live representation, never-chunked messages occupy no position. topologyPolicy: 'reject' (default) throws StoreTopologyError naming the summaries, spans, hole counts and the interleaved owners, so ContextManager.open refuses the store. 'report' logs the same at error level and surfaces getCompressionDebt().topologyViolations (state critical). A kv-unified config that explicitly opts into gap handling (preserveGapBearingSummaries / treeifyNonContiguousSummaries) defaults to 'report' — those stores are known to carry gaps. ContextManager.open now releases a store it opened when initialize rejects (it leaked the lock behind the rejected promise). scripts/audit-topology.ts <store> [--namespace ns] [--json] runs the audit read-only, exit 2 on violations.

Never mint a crossed node. One position index (mergePositionIndex: chunk members ∪ live L1 sourceIds, store order; chunk members appended since the last listing go after it; hidden/pruned L1 sources occupy no position) drives contiguousMergeCandidates, enqueueMergeForRange (now split into strictly adjacent runs, longest run enqueued, the rest logged) and a new gate in executeMerge: sources must be one level below the target, resolvable, and strictly adjacent. A failing group is refused before any model call — dequeued into the merge quarantine with outcome topology_violation, console.error, merge:topology-refused in the compression log, getCompressionDebt().topologyRefusals and state critical.

Validation

  • npm test: 877/877.
  • Production store A (8.9k messages): the audit reports exactly the 7 summaries the kv-unified canonical forest rejects.
  • Production store B (75.7k messages): 7 violations, each a late-landing L1 inside an L2's span (Demand-path merges (enqueueMergeForRange, produce level ≥ 2) still bypass the strict-adjacency grammar — can mint a non-contiguous L2 that kv-unified then rejects forever #95 shape).
  • Across 13 production stores: crossed ownership in 9, all real on inspection, all dating from July (including L1-level cases where a peeled opening message shares a chunk with frontier messages).
  • New test/topology-guard.test.ts (11 tests): store-order adjacency vs record order, degrade without a listing, demand-path hole split, mint-time refusal (no model call, quarantined, critical), adjacent pass-through, audit (crossed / clean / never-chunked gap / reject vs report / kv-unified gap opt-in), and ContextManager.open rejecting a persisted crossed L2 then opening under 'report'.

⚠️ Deploy note

This is fail-closed by default: a store that already carries crossed ownership refuses to open on this CM until it is repaired or its config sets topologyPolicy: 'report'. Run node dist/scripts/audit-topology.js on a copy of every store before deploying. A general repair tool follows in a separate PR.

🤖 Generated with Claude Code

…y in store order

Every load audits the summary archive for crossed ownership — a summary
whose leaves are not contiguous among owned messages (chunk members ∪
chunk-linked L1 sourceIds) in store order. topologyPolicy 'reject'
(default) throws StoreTopologyError from initialize, so ContextManager.open
refuses the store until it is repaired; 'report' logs at error level and
surfaces the count through getCompressionDebt().topologyViolations (state
critical). A kv-unified config that opts into gap handling
(preserveGapBearingSummaries / treeifyNonContiguousSummaries) defaults to
'report'. ContextManager.open releases a store it opened when initialize
rejects. scripts/audit-topology.ts runs the same audit read-only.

Merge adjacency is judged in store order instead of chunk-record order, so
a chunk minted late over an early message (#122's head ratchet) no longer
joins the frontier's run; demand-path merges (#95) are split into strictly
adjacent runs; executeMerge refuses any group that is not one level below
the target and strictly adjacent in store order — no model call, the entry
moves to the merge quarantine with outcome topology_violation, and
getCompressionDebt().topologyRefusals / state critical say so. A crossed
node is never minted.

Calibrated on Linn's store copy (the 7 summaries the kv-unified forest
rejects) and on the synthetic corpus (7 crossed L2/L3s).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@antra-tess
antra-tess merged commit 7b6a129 into main Sep 26, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant