Skip to content

Verify: export and snapshot drop data a restore needs #458

Closed

Description

@possiblyneal

Part of #448

Question

Does fork issue #443 reproduce (Memory relations lost on export and restore)? Also check rohitg00#1782 (restored image refs untracked), rohitg00#1555 (5000 import limit vs MAX_OBS_PER_SESSION) and rohitg00#1792 (audit export).

Activity

  1. self-assigned this
    on Nov 18, 2026
  2. possiblyneal commented on Nov 18, 2026

    @possiblyneal
    OwnerAuthor

    Verdict: PARTIAL (branch verify/443-relations, worktree tmp/wt/verify-443)

    Apply the relations fix. Reject the consolidate-crash claim, rohitg00#1555 and rohitg00#1792. rohitg00#1782 is not applied.

    Relations lost (applies)

    Repro (worktree tmp/wt/verify-443), fail 2/2 on base fdc3d45:

    • test/export-import.test.ts "round-trips Memory relations through export and a replace import": export has no relations, so replace import leaves the target without them
    • test/snapshot.test.ts "restores Memory relations and drops ones added after the snapshot": the relation added later survives restore and the captured one is not brought back
      Cause: DURABLE_STORES (src/functions/export-import.ts) has no KV.relations entry. Export, snapshot create and snapshot restore all derive from it.

    Complication found: mem::relate and mem::evolve stored relations under random generateId("rel") keys, while mesh already used source:target:type. A relation row carries no id, so export/restore cannot recover the original key. Left as is, a restore would write a second row under the content key next to the old rel_ row.

    Fix:

    • relationKey(rel) added to src/state/schema.ts, used by relate, evolve, mesh (2 sites), DURABLE_STORES and import. All writers now share the content key.
    • relations entry in DURABLE_STORES, so export and snapshot create/restore pick it up.
    • ExportData.relations?, and nonEmpty(stores.relations) in export.
    • Import writes relations (validated for sourceId/targetId/type, skip honoured). replace clears them first.
      Side effect: relating the same pair and type twice now overwrites one row instead of adding a second.
      Legacy rel_* rows are not migrated. Live store has 0, so there is no impact; a pre-fix row would be left in place and duplicated on the first replace-restore.
      After the fix: export-import, snapshot (3 files), relations, mesh, confidence and consistency tests pass (96); typecheck 0 errors.

    Live store: mem:relations has 0 rows. Nothing has been lost, so the defect is latent.

    Imported Memories missing concepts/files crash consolidate (rejected)

    The cited consolidate.ts:108,165 read obs.concepts and o.files, which are CompressedObservation fields, not Memory fields. consolidate never reads Memory.concepts or files. The test "imports Memory rows without concepts or files and leaves consolidate able to run" imports a bare Memory and runs consolidate; it PASSES on base. chunking already guards with Array.isArray. Retry if: a stack trace from a real import shows a Memory.concepts or Memory.files read throwing.

    Other upstream claims

    Bar (tmp/verify/443-verify.log)

    • typecheck: 0 errors. npm test: 3,119 passed, 1 skipped (272 files). eval:gate and eval:gate:rules: pass.
    • eval:suite fast, against base-fdc3d459: every metric unchanged (coding-life search, session-start and prompt-submit; PrecisionMemBench; LongMemEval recall@5 0.896 → 0.896).
    • The export version stays 0.9.29. The relations field is optional, so older export files still import.
    • Docs to update when this is applied: whichever docs list what export and snapshot carry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions