feat(strategy): fail loudly on invalid store topology; merge adjacency in store order - #124
Merged
Merged
Conversation
…y in store order Every load audits the summary archive for crossed ownership — a summary whose leaves are not contiguous among owned messages (chunk members ∪ chunk-linked L1 sourceIds) in store order. topologyPolicy 'reject' (default) throws StoreTopologyError from initialize, so ContextManager.open refuses the store until it is repaired; 'report' logs at error level and surfaces the count through getCompressionDebt().topologyViolations (state critical). A kv-unified config that opts into gap handling (preserveGapBearingSummaries / treeifyNonContiguousSummaries) defaults to 'report'. ContextManager.open releases a store it opened when initialize rejects. scripts/audit-topology.ts runs the same audit read-only. Merge adjacency is judged in store order instead of chunk-record order, so a chunk minted late over an early message (#122's head ratchet) no longer joins the frontier's run; demand-path merges (#95) are split into strictly adjacent runs; executeMerge refuses any group that is not one level below the target and strictly adjacent in store order — no model call, the entry moves to the merge quarantine with outcome topology_violation, and getCompressionDebt().topologyRefusals / state critical say so. A crossed node is never minted. Calibrated on Linn's store copy (the 7 summaries the kv-unified forest rejects) and on the synthetic corpus (7 crossed L2/L3s). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two ways a crossed node (a summary whose leaves are not contiguous among owned messages) got minted, and nothing that said so afterwards:
contiguousMergeCandidatesjudged adjacency in chunk-record order, where a chunk minted late over an early message sits next to the open frontier. One production store has exactly this — five single-message L1s over the opening messages, minted weeks later in reverse sequence order, merged into cross-era L2s → L3s → an L4 covering thousands of leaves. The kv-unified forest rejects the store; onpreserveGapBearingSummaries:truethe packed solver goes into buffered mode and runs away (exceeded ceiling 100000 at 135238, ~1.35× any ceiling) — a hard-down.enqueueMergeForRangetook every unmerged source inside the range, holes included.A second large production store carries 7 crossed L2/L3s of the #95 shape. Nothing at load or compile time reported either.
Changes
Fail loudly at load.
initializeaudits every summary: leaves (via chunk-linked L1s — superseded legacy generations own nothing) positioned among owned messages in store order; a hole is always another live representation, never-chunked messages occupy no position.topologyPolicy: 'reject'(default) throwsStoreTopologyErrornaming the summaries, spans, hole counts and the interleaved owners, soContextManager.openrefuses the store.'report'logs the same at error level and surfacesgetCompressionDebt().topologyViolations(statecritical). A kv-unified config that explicitly opts into gap handling (preserveGapBearingSummaries/treeifyNonContiguousSummaries) defaults to'report'— those stores are known to carry gaps.ContextManager.opennow releases a store it opened when initialize rejects (it leaked the lock behind the rejected promise).scripts/audit-topology.ts <store> [--namespace ns] [--json]runs the audit read-only, exit 2 on violations.Never mint a crossed node. One position index (
mergePositionIndex: chunk members ∪ live L1 sourceIds, store order; chunk members appended since the last listing go after it; hidden/pruned L1 sources occupy no position) drivescontiguousMergeCandidates,enqueueMergeForRange(now split into strictly adjacent runs, longest run enqueued, the rest logged) and a new gate inexecuteMerge: sources must be one level below the target, resolvable, and strictly adjacent. A failing group is refused before any model call — dequeued into the merge quarantine with outcometopology_violation,console.error,merge:topology-refusedin the compression log,getCompressionDebt().topologyRefusalsand statecritical.Validation
npm test: 877/877.test/topology-guard.test.ts(11 tests): store-order adjacency vs record order, degrade without a listing, demand-path hole split, mint-time refusal (no model call, quarantined, critical), adjacent pass-through, audit (crossed / clean / never-chunked gap / reject vs report / kv-unified gap opt-in), andContextManager.openrejecting a persisted crossed L2 then opening under'report'.This is fail-closed by default: a store that already carries crossed ownership refuses to open on this CM until it is repaired or its config sets
topologyPolicy: 'report'. Runnode dist/scripts/audit-topology.json a copy of every store before deploying. A general repair tool follows in a separate PR.🤖 Generated with Claude Code