fix(sync): remove 64k _meta ceiling + materialize catch-up snapshots (fifa-class CGs)

[branarakic]

What ships (step 2 — fifa-class Context Graphs)

Two stacked fixes that must ship as a pair; either alone is insufficient:

1. Remove the 64k _meta snapshot ceiling (issue bug(sync): SWM meta lane permanently unsyncable once _meta reaches 64,000 rows (oversized-snapshot fallback disabled for TTL-filtered sessions) #1847, canonical PR fix(sync): serve TTL-filtered SWM meta past the 64,000-row snapshot ceiling #1868): the responder applied its 64,000-row budget to the raw meta graph before TTL filtering and passed the fallback flag positionally-wrong, so any long-lived CG (e.g. fifa-world-cup-2026: ~4,600 data quads, 64k+ meta rows) became permanently unsyncable to every peer. Replaced with a bounded session-plan lane: budget binds on served rows, no store-side global sort, count- and digest-verified pages. 10/15 mainnet cores emit these refusals today.
2. Materialize catch-up snapshots (canonical PR fix(swm): materialize verified public snapshots on catch-up #1842): the public catch-up lane fetched and verified content snapshots, cached them — and never wrote them to the store, so a node that missed live gossip stayed empty forever. Now materialized under the same per-KA write lock live gossip uses, with in-lock version ordering and a digest-bound already-materialized guard.

Meta arrives + content lands. Base: current origin/testnet-canary (includes #1876 and the #1879 merge-main; its cli conflicts hand-resolved here).

Gate evidence (6-node devnet, full sync suite)

Final head: verify-fixes 8/10 (only the sanctioned #1780-409 and #1779-markdown knowns), proof 32/34 (only the two known watermark checks), and the fifa-class proof itself:

resilience/holdout: node absent during publication reconstructs the FULL corpus — 100/100 quads in 3009ms

Two earlier full-green runs on the pre-absorb head (32/34 + 32/34; holdout 100/100 in 6078ms and 9164ms). Agent unit suite 1249 passing.

Known issues (documented, non-blocking)

• Reconcile watermark counter does not advance on subscribers — delivery verified independently; not a health signal.
• Markdown-imported KA body replication (canary lineage) — tracked separately.
• fix(publish): let a curator VM-publish a member-shared rootless KA (#1778) #1780 curator-publish seal timing.

Neighbors #1869/#1870/#1873 are compatible, not competing. Real-world acceptance test: the fifa-world-cup-2026 CG itself — a testnet node syncing its full corpus is the field confirmation.

🤖 Generated with Claude Code

[branarakic]

Review: approve, with one required cleanup and three observations. I verified this against the branch head (70fc28325) rather than the PR text, and independently reproduced the test evidence: all 43 tests across the three new/modified suites pass locally against real OxigraphStores, including the 64k+-row ceiling scenarios (swm-snapshot-materializer 15, swm-public-snapshot-materialization 8, sync-responder-swm-meta-ceiling 20).

What I verified holds

• The plan-lane ordering contract is sound. The plan sorts graphs and subjects with compareCodePoint, and compareRows is compareCodePoint on (g, s, p, o) — so the plan's prefix sums genuinely agree with served row order, which is what the whole pagination scheme rests on. The set-equivalence and reassembly tests then prove it empirically at hostile page sizes (4-row pages vs 5- and 11-row groups).
• The crash windows in the materializer self-heal. Graph replace → head swap → meta append: a crash after replace leaves content newer than the head (next round: digest matches → needsRepair collapse); a crash after the swap but before the meta append leaves no head at all, and the next round's own verified-meta insert restores it. Each intermediate state converges. The replaceHeadMetadata foreign-operation kaUal ASK guard correctly mirrors the recovery lane's replaceMetaForGraphAssets join.
• The digest guard closes a real trap. All versions of a graph-scoped KA share one assertion-graph URI, so an equal-count older version passes any count-only check — the real-store test proving v1→v2 catch-up with same-count payloads (and the LIMIT-1 resolveKnowledgeAssetWorkspaceHead resolving unambiguously afterwards) is exactly the right evidence, and the second-round no-op doubles as proof the digest survives the store round-trip.
• Budget lifecycle checks out. admit→release (evictable retention), touch+release on memo hit, remove with reasons on expiry/LRU/replacement — all match the snapshot-budget API surface, and the time-based-expiry test is correctly isolated from the LRU test (roomy maxEntries so only the clock can evict). The controlPlane bypass of per-snapshot caps is well-argued: shrinking the per-snapshot budget is precisely how a session is forced into plan-paged mode, and rejecting the plan there would turn the degradation back into a refusal — while global-budget charging still bounds retained plans. The named-options refactor of readResponderRowsPage (killing the positional boolean that caused the production defect) is the right structural fix, not just a patch.
• The pair-shipping claim is real: the responder fix alone leaves requesters caching-but-never-writing snapshots; the materializer alone never receives meta from a 64k+-row responder. Either alone leaves fifa-class CGs broken.

Required cleanup

Drop packages/evm-module/deployments/localhost_contracts.json from the PR. It's 289 lines of regenerated dev-deploy artifact (the build rewrites it — this repo's known churn trap), it points the recorded deployment at an unrelated feat/rpc-usage-metrics commit, and it downgrades the recorded PublishingConviction version 10.0.8 → 10.0.7. Pure noise in a sync fix, and a guaranteed conflict magnet.

Observations (non-blocking)

1. Sessionless requesters on the TTL lane rebuild the plan every page. prepareResponderSession returns undefined without a syncSessionId, so createSessionPlanGetter gets no cache key and calls loadPlan per page: per-page discovery + chunked GROUP BY cost (bounded, and still far cheaper than the deleted global-sort query), but also no requireExisting protection and a fresh digest sidecar per plan object — offset>0 pages against a mutating store can skip/duplicate for such requesters. This is the same exposure the old OFFSET query had, so it's not a regression — but a code comment stating "sessionless TTL paging is best-effort" would prevent someone later mistaking the digest guard as covering that case. Worth confirming current fleet requesters always send syncSessionId on the SWM meta phase.
2. The paged lane serves discovery's admission; only the snapshot lane re-applies filterSwmMetaSnapshotRows. The doc honestly scopes agreement to "canonical typed-literal meta writes," and the set-equivalence test proves that scope. Non-canonical rows (e.g., an untyped publishedAt literal, where SPARQL dateTime comparison and the JS filter may disagree) could be served by paged mode and dropped by snapshot mode. Requester-side verification still gates everything, so this is a consistency footnote, not a hole.
3. Charging asymmetry: TTL meta plans are now charged to the global responder budget, but the exact-graph and fresh-SWM-data plan memos remain uncharged control-plane retention. Their per-entry footprint is much smaller (per-graph, not per-subject), but it's the same class of state — fine as a follow-up.
4. Minor: readStoredHead's MAX(?v) is numeric for the canonical xsd:integer writes; a legacy plain-string assertionVersion would compare lexicographically ("9" > "10"). Identical semantics to the pre-PR MAX read, so noting only for completeness.

The test engineering here deserves explicit credit: real >64k-row stores at default production budgets, mutation-kill coverage aimed at the exact positional-boolean defect, the aggregate-across-subgraphs cardinality cap, the same-count split-subject hybrid repro from the #1868 review, and store-query spies that structurally forbid the deleted global-sort shape from ever reaching the store. This is the standard sync-lane changes should be held to.

[branarakic]

Review addressed in 588628e:

Required cleanup — done. localhost_contracts.json restored byte-identical to the base (the churn also mispointed the recorded deployment and downgraded the recorded PublishingConviction version, as you noted — third time this cycle the regenerated artifact has tried to sneak into a PR).

Observation 1 — comment added on createSessionPlanGetter stating sessionless TTL paging is best-effort and explicitly NOT covered by the per-subject digest guard, so nobody later mistakes the guard for offset-page consistency protection. Worth a fleet check that current requesters always send syncSessionId on the SWM meta phase — agreed, tracking as follow-up.

Observations 2–4 — acknowledged as follow-ups: non-canonical literal admission asymmetry (consistency footnote, requester verification gates), charging the exact-graph/fresh-data plan memos to the global budget (same class, smaller footprint), and the legacy plain-string assertionVersion lexicographic MAX (pre-existing semantics, canonical writes are typed).

Build clean on the new head. Thank you for reproducing the test evidence independently — clear to merge.