feat(store): journaled multi-note transactions and omind recover (v8.0.0)

AGENTS.md

@@ -30,22 +30,29 @@ replication (`mesh.py`, `merge.py`). Know which subsystem you are in.
    and the caller falls back to the older, dumber path. A missing model, a
    corrupt index, a locked database, a missing FTS5 build — all degrade search;
    none may break it. Test the failure branch, not just the happy one.
-3. **All note writes go through `OmiStore`** (`notes.upsert_note` for external
+3. **Multi-note writes go through `omind.txn`.** One note at a time is already
+   atomic (same-dir temp + `os.replace`). Two or more is not, and a crash
+   between them used to leave partial state with no way back. Any new operation
+   that writes several notes must journal its pre-images through
+   `txn.Transaction` under `store.write_lock()`, or it reintroduces the gap
+   `omind recover` exists to close. Recovery refuses to overwrite a note edited
+   after the crash — that edit is newer than the pre-image.
+4. **All note writes go through `OmiStore`** (`notes.upsert_note` for external
    writers) — flock, atomic rename, Lamport `Rev:` stamping, soft delete.
    Deletes archive (`Disabled: true`); only `omind mesh purge` truly removes.
-4. **`OmiStore.safe_name` guards every read and write.** Path traversal must stay
+5. **`OmiStore.safe_name` guards every read and write.** Path traversal must stay
    impossible; there are tests enforcing this. Don't route around them.
-5. **`store.py` stays framework-free.** No FastAPI, no MCP. The CLI and the web
+6. **`store.py` stays framework-free.** No FastAPI, no MCP. The CLI and the web
    app both build on it.
-6. **Credential notes are de-prioritised** in search and in the gate's
+7. **Credential notes are de-prioritised** in search and in the gate's
    suggestions unless the query is itself about credentials
    (`retrieve._CREDENTIAL_PENALTY`). The gate must never steer an agent into the
    secrets notes. This is load-bearing, not decoration.
-7. **No MCP tool may return unbounded output.** Every list-shaped tool pages
+8. **No MCP tool may return unbounded output.** Every list-shaped tool pages
    (`limit`, `offset`, `total`, `has_more`) via `server._page`. `list-notes` once
    returned ~90,800 tokens in a single result; `tests/test_server.py::
    test_every_list_tool_is_bounded` exists so a new tool cannot regress that.
-8. **Reserved files are not memories.** `index.md` and `Memory Template.md` are
+9. **Reserved files are not memories.** `index.md` and `Memory Template.md` are
    scaffolding; reading one does *not* clear the consult gate
    (`paths.NON_CONSULT_FILENAMES` — an anti-dodge measure, issue #109).
 

BACKLOG.md

@@ -66,24 +66,14 @@ quantized int8 vectors + RRF beats their JSON BM25 index), on multi-machine repl
 (they have none), on enforcement, and on shipping a real MCP server. What follows are the
 five places their design is genuinely better and the idea transfers._
 
-- [ ] **Journaled plan→apply→recover transactions for multi-note operations** ([#194](https://github.com/CryptoJones/omind/issues/194)) — _enhancement (durability)_ —
-  omind has atomic per-file replace, an advisory write lock, and version
-  preconditions, but no journal and no rollback, so an interrupted multi-note
-  operation leaves partial state. `store.create_and_disable_sources` concedes it in
-  its own docstring ("a process crash can still leave extra recoverable copies").
-  Generalize what `omind consolidate` already prototypes into a store-level
-  primitive plus `omind recover`. Skip their `approved_plan_sha256` handshake.
-- [x] **Typed confidence + symmetric `Conflicts with:` provenance** ([#195](https://github.com/CryptoJones/omind/issues/195)) — _enhancement (memory shape)_ —
-  shipped as two optional note fields that round-trip through Markdown, CLI, MCP, and
-  mesh merge like `Supersedes:`. A conflict binds both notes even when one side declared
-  it; retrieval surfaces the disagreement rather than resolving it; lint flags broken and
-  one-sided claims. No research-grade ledger. Original description follows.
-  their claim ledger types authority, assessment, confidence, and evidence relation
-  (`supports`/`contradicts`/`context`), keeping contradictions visible. omind's
-  `references:` is free text and `Supersedes:` only expresses clean ordered
-  replacement — there is no way to say "these two memories disagree" or "this was
-  never verified". Add two optional fields that round-trip like `Supersedes:` does;
-  do not grow a research-grade ledger.
+- [x] **Journaled plan→apply→recover transactions for multi-note operations** ([#194](https://github.com/CryptoJones/omind/issues/194)) — _enhancement (durability)_ —
+  shipped as `omind.txn` + `omind recover`. Pre-images captured and fsynced before the
+  first write, atomic per-file replace, a commit record, deterministic rollback. Recovery
+  refuses to overwrite a note edited after the crash — that edit is newer than the
+  pre-image — reporting a conflict and keeping the journal instead. In-process failures
+  roll themselves back. `create_and_disable_sources` migrated; the docstring that conceded
+  "a process crash can still leave extra recoverable copies" is gone. Skipped their
+  `approved_plan_sha256` handshake as planned.
 - [ ] **Machine-readable capability contract verified by `doctor`** ([#196](https://github.com/CryptoJones/omind/issues/196)) — _hardening_ —
   they declare every capability's tier, read/write scope, network need, and
   destructiveness in `config/capabilities.json`, verify it, and state explicitly

CHANGELOG.md

@@ -5,6 +5,64 @@ All notable changes to this project are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [8.0.0] - 2026-08-02
+
+### Added
+- **Journaled multi-note transactions and `omind recover`**
+  ([#194](https://github.com/CryptoJones/omind/issues/194)). Multi-file updates
+  cannot be truly atomic on the filesystems omind runs on, and this does not
+  pretend otherwise. It provides the contract that *can* be kept: every target's
+  pre-image is captured and fsynced before the first write, the writes go
+  through the store's existing atomic per-file replace, a commit record marks
+  the point of no return, and an interrupted run is rolled back deterministically.
+
+  omind had the first half of this already — same-dir temp + `os.replace`, an
+  advisory write lock, version preconditions — and no journal, so an interrupted
+  multi-note operation left partial state with no recovery path.
+  `store.create_and_disable_sources` (the `omind consolidate --apply` write path)
+  conceded it in its own docstring: *"a process crash can still leave extra
+  recoverable copies."* That fails toward keeping data, but a human still had to
+  notice and reconcile by hand.
+
+  `omind recover` rolls back anything that did not reach its commit record, and
+  is a no-op on a clean journal. `--dry-run` reports without touching the vault.
+
+  **Recovery refuses to clobber a later edit.** A pre-image is restored only when
+  the file still holds either that pre-image (nothing to do) or exactly the bytes
+  the interrupted run intended to write (ours to undo). Anything else means
+  someone edited the note after the crash — newer information than the pre-image
+  — so it is reported as a conflict, left alone, and its journal is kept for
+  inspection. `omind recover` exits 1 when that happens. Blind rollback would be
+  data loss wearing recovery's clothes.
+
+  An in-process failure rolls itself back, so `create_and_disable_sources` needs
+  no manual `recover` at all; the journal is for the case where the process dies.
+
+  Pre-images are restored **byte for byte**, through a binary atomic write
+  rather than the store's text-mode one. Routing them through a text writer
+  re-translates line endings: on Windows a `b"old A\r\n"` pre-image came back as
+  `b"old A\r\r\n"`, so every rollback silently grew a blank line in the note it
+  was restoring.
+
+  Content identity is hashed over line-ending-normalized text, not raw bytes.
+  `_atomic_write` writes in text mode, so on Windows every `\n` reaches the disk
+  as `\r\n` — hashing bytes meant a file never matched what had just been written
+  to it, every note read as a foreign edit, and recovery rolled back *nothing*.
+  Both Windows CI legs caught it before merge.
+
+  Journals live in the state dir, never the vault: they describe this machine's
+  interrupted filesystem work, are meaningless to a mesh peer, and must not
+  replicate (invariant 1).
+
+  Deliberately not copied from the source design: its `approved_plan_sha256`
+  handshake, which would put a hash-copying step in front of routine memory
+  writes. `consolidate` keeps plan/apply; everything else just gets the journal.
+
+### Changed
+- `AGENTS.md` gains invariant 3: any new operation writing several notes must
+  journal through `txn.Transaction` under `store.write_lock()`, or it
+  reintroduces the gap `omind recover` exists to close.
+
 ## [7.0.0] - 2026-08-02
 
 _The last of the 2026-08-01 review and 2026-08-02 comparison backlogs, released

README.md

@@ -53,6 +53,12 @@ reads and writes as long-term memory. `omind` does two things with it:
   French, Arabic, Russian, Chinese), including right-to-left layout for Arabic.
   Its API is unauthenticated by design — the localhost bind is the security
   boundary. Read [docs/serve.md](docs/serve.md) before exposing the port.
+- **`omind recover`** — roll back a multi-note write that was interrupted
+  mid-apply (a killed `consolidate --apply`, a power loss). Multi-file updates
+  can't be truly atomic, so omind journals every target's prior bytes before the
+  first write and can put them back. A note edited *after* the interruption is
+  reported as a conflict and left alone — that edit is newer than anything the
+  journal could restore.
 - **`omind doctor`** — diagnose the wiring in one shot: Claude CLI + git on
   `PATH`, the `omi` MCP server registered at user scope with the right command,
   the OMI folder readable, mesh health (node identity, merge driver, per-peer

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "omind"
-version = "7.0.0"
+version = "8.0.0"
 description = "Reproduce the OMI/Obsidian memory integration for AI agents, plus a local web app to view, edit, and add memory entries."
 readme = "README.md"
 requires-python = ">=3.10"

src/omind/__init__.py

@@ -2,4 +2,4 @@
 # Copyright 2026 Aaron K. Clark
 """omind — OMI/Obsidian memory tooling for AI agents."""
 
-__version__ = "7.0.0"
+__version__ = "8.0.0"

src/omind/cli.py

@@ -66,7 +66,7 @@ def build_parser() -> argparse.ArgumentParser:
     parser.add_argument("--version", action="version", version=f"omind {__version__}")
     sub = parser.add_subparsers(
         dest="command",
-        metavar="{help,setup,quickstart,serve,doctor,self-update,backup,ai,export,import,reindex,note,rollup,hook}",
+        metavar="{help,setup,quickstart,serve,doctor,self-update,backup,ai,export,import,reindex,note,rollup,recover,hook}",
     )
 
     help_p = sub.add_parser(
@@ -477,6 +477,27 @@ def build_parser() -> argparse.ArgumentParser:
     for gp in (g_neighbors, g_path, g_orphans, g_dangling, g_stats, g_frontier, g_export):
         _add_vault_args(gp)
 
+    recover = sub.add_parser(
+        "recover",
+        help="roll back a multi-note write that was interrupted mid-apply",
+        description=(
+            "Roll back any journaled multi-note transaction that did not reach its\n"
+            "commit record — an `omind consolidate --apply` killed mid-write, a\n"
+            "power loss, an OOM. A no-op when the journal is clean, which is the\n"
+            "normal case.\n"
+            "\n"
+            "A note whose bytes match neither the pre-image nor what the interrupted\n"
+            "run intended to write was edited after the crash, and is reported as a\n"
+            "conflict and left alone: that edit is newer than anything this could\n"
+            "restore. Its journal is kept so you can inspect it."
+        ),
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+    )
+    recover.add_argument(
+        "--dry-run", action="store_true", help="list what would be rolled back, change nothing"
+    )
+    _add_vault_args(recover)
+
     checkpoint = sub.add_parser(
         "checkpoint",
         help="summarize recent activity (journal + compliance log) into a daily "
@@ -1218,6 +1239,38 @@ def _run_graph(args: argparse.Namespace) -> int:
     return 0
 
 
+def _run_recover(args: argparse.Namespace) -> int:
+    """``omind recover``: roll back interrupted multi-note transactions."""
+    from omind import txn
+    from omind.store import OmiStore
+
+    omi_dir = (args.vault / args.folder).expanduser()
+    store = OmiStore(omi_dir)
+    if not txn.pending(omi_dir):
+        print("nothing to recover: no interrupted transactions")
+        return 0
+    # Take the same write lock a normal write takes, so recovery cannot race a
+    # concurrent MCP/web/cron writer touching the very notes it is restoring.
+    with store.write_lock():
+        reports = txn.recover(omi_dir, dry_run=args.dry_run)
+    conflicts = 0
+    for report in reports:
+        print(("would roll back " if args.dry_run else "") + report.format())
+        for name in report.conflicts:
+            print(f"  CONFLICT: {name} changed after the interruption — left as-is")
+            conflicts += 1
+    if conflicts:
+        print(
+            "\nSome notes were edited after the interrupted write. Their journals are kept;\n"
+            "inspect the notes, then delete the journal directory when you are satisfied.",
+            file=sys.stderr,
+        )
+        return 1
+    if not args.dry_run:
+        store.update_index()
+    return 0
+
+
 def _run_checkpoint(args: argparse.Namespace) -> int:
     from omind import checkpoint
 
@@ -1535,6 +1588,8 @@ def main(argv: list[str] | None = None) -> int:
         return _run_bench(args)
     if args.command == "lint":
         return _run_lint(args)
+    if args.command == "recover":
+        return _run_recover(args)
     if args.command == "graph":
         return _run_graph(args)
     if args.command == "checkpoint":

src/omind/paths.py

@@ -111,3 +111,12 @@ def access_state_path(omi_dir: Path) -> Path:
 def consolidation_dir(omi_dir: Path) -> Path:
     """Machine-local proposal/draft storage for one vault's reviewed merges."""
     return state_dir() / f"consolidate-{_omi_dir_digest(omi_dir)}"
+
+
+def transaction_dir(omi_dir: Path) -> Path:
+    """Journal + pre-image storage for one vault's multi-note transactions.
+
+    Machine-local and outside the vault: it records what the *filesystem* was
+    doing, is meaningless on another peer, and must never be mesh-synced.
+    """
+    return state_dir() / f"txn-{_omi_dir_digest(omi_dir)}"

src/omind/server.py

@@ -471,7 +471,7 @@ def _graph_query(
         if operation == "stats":
             return dict(graph.stats(graph_for()))
         if operation == "frontier":
-            # Paged like every other list-shaped result (invariant 7): the caller
+            # Paged like every other list-shaped result (invariant 8): the caller
             # asks for a page, the ranking is computed over the whole graph.
             ranked = [
                 {

src/omind/store.py

@@ -27,7 +27,7 @@
 
 import yaml
 
-from omind import filelock
+from omind import filelock, txn
 from omind.clock import Rev, next_rev
 from omind.paths import (
     INDEX_FILENAME,
@@ -1273,8 +1273,13 @@ def create_and_disable_sources(
         The source tuples are ``(filename, expected_version)``. All versions and
         the target's nonexistence are checked before the first write, closing
         the gap where another OmiStore writer could change the second source
-        between a separate create and two archive calls. A process crash can
-        still leave extra recoverable copies, never a hard-deleted source.
+        between a separate create and two archive calls.
+
+        The writes run inside a journaled transaction (:mod:`omind.txn`), so an
+        interrupted apply is rolled back by ``omind recover`` instead of leaving
+        the vault half-merged. This used to concede here that "a process crash
+        can still leave extra recoverable copies" — it fails toward keeping
+        data, but a human still had to notice and reconcile by hand (#194).
         """
         if not fields.title.strip():
             raise NoteError("a note requires a title")
@@ -1305,12 +1310,22 @@ def create_and_disable_sources(
             content = render_fields(fields)
             if self.node_id is not None:
                 content = self._stamped(target, content)
-            _atomic_write(target, content)
+            transaction = txn.Transaction(self.omi_dir)
+            transaction.write(target, content)
             for path, _expected in source_paths:
                 archived = _with_disabled(_read_text(path), True)
                 if self.node_id is not None:
                     archived = self._stamped(path, archived)
-                _atomic_write(path, archived)
+                transaction.write(path, archived)
+            transaction.prepare()
+            try:
+                transaction.apply(_atomic_write)
+            except BaseException:
+                # Put back what we managed to change before re-raising, so the
+                # in-process failure needs no `omind recover` at all.
+                transaction.rollback()
+                raise
+            transaction.commit()
             self._write_index()
         self._signal_write()
         return target.name