Ledger: restore records the migration invalidated for missing evidence (#39)

docs/ledger.md

@@ -31,7 +31,7 @@ This is the one definition of ledger states. Other docs and code link here.
 | `pending` | The bundle is hashed and recorded; it is not yet archived. | `seal`, `fail`, `invalidate` |
 | `sealed` | Every artifact and the bundle's tree object are stored in the evidence archive and were verified against the recorded manifest. | `invalidate` |
 | `failed` | The parser failed, a check before sealing failed, or sealing failed. Never becomes sealed. | terminal |
-| `invalidated` | Withdrawn after the fact (see [replay.md](replay.md)). The record is kept. | terminal |
+| `invalidated` | Withdrawn after the fact (see [replay.md](replay.md)). The record is kept. | terminal, except a record the migration invalidated for missing evidence, which `reverify` can seal again (below) |
 
 - **Sealed is integrity, not delivery.** A sealed record says nothing about
   whether any downstream target has received it. Delivery is a separate set
@@ -151,12 +151,30 @@ Migrating a pre-#12 (version 0) ledger:
 |------------|---------|
 | `pending`, `failed`, `invalidated` | Same state |
 | `committed`, bundle archivable and matching its manifest (from disk, or already in the archive) | `sealed` |
-| `committed`, bundle missing, changed or unsafe | `invalidated`, with the reason in `error` — the ledger can no longer vouch for it |
+| `committed`, bundle missing, changed or unsafe | `invalidated`, with the reason in `error` — the ledger can no longer vouch for it. The error says `evidence missing` (neither on disk nor archived) or `evidence mismatched` (anything else) |
 | `source_hash` that is the digest of exactly one Snapshot in the archive | Kept, with its `source_kind` |
 | any other `source_hash` | Moved to `legacy_source_hash` |
 | `source_path` | Kept, and recorded as a Source in the archive |
 | parser identity and config | Unknown: `NULL` |
 
+**Restoring missing evidence.** Invalidating is permanent, but evidence the
+archive never held (a pre-#16 ledger) may still exist on disk. After migration,
+an operator who has the files can seal such a record again:
+
+```bash
+python -m stele.ledger.reverify ledger.db archive/ <record_id> --artifact-dir /where/the/files/are
+```
+
+This is never done by migration, and it is narrow. The record must have been
+invalidated *by the migration* for *missing* evidence: its error says so and
+its only event is the migration's import, so a record an operator or a replay
+invalidated stays invalidated, as does one whose evidence was `mismatched`. It
+must have no deliveries (invalidating withdrew the delivered data). The files
+are hashed against the recorded manifest and archived exactly as sealing does;
+any difference leaves the record invalidated. On success the record is sealed,
+its error cleared, and a `record.reverified` event is chained, so the history
+reads: imported as invalidated, reverified.
+
 Two records for one `run_id` cannot be represented; migration stops with
 `LedgerMigrationError` and changes nothing. A ledger newer than this Stele
 raises `LedgerSchemaError`.
@@ -171,6 +189,7 @@ the two can't diverge:
 |---|---|
 | `record.created` | `create_pending`: run, artifact hash, input Snapshot, Source, parser identity and config, backend |
 | `record.sealed` / `record.failed` / `record.invalidated` | each state transition, with its error or reason |
+| `record.reverified` | an operator restored a record the migration invalidated for missing evidence (below) |
 | `delivery.opened` / `delivery.event` | each delivery and each intent, receipt, failure or removal in the delivery log |
 | `replay.logged` | each replay and its outcome |
 | `*.imported` | the state found when a ledger was migrated to version 5 |
@@ -217,6 +236,7 @@ if there is any problem.
 - `stele/ledger/delivery.py` — the delivery log (#13)
 - `tests/test_ledger.py` — state machine, hashing, per-run records
 - `tests/test_ledger_provenance.py` — Snapshot provenance, parser identity, sealing, migration
+- `tests/test_reverify.py` — restoring records invalidated for missing evidence
 - `tests/test_event_log.py` — chaining, verification, tamper detection, anchors, migration, CLI
 
 ## Completion criteria

stele/ledger/events.py

@@ -4,7 +4,8 @@
 Every fact the ledger learns is appended here as an event, in the same
 database transaction as the change it describes, so the two cannot diverge:
 
-  record.created / record.sealed / record.failed / record.invalidated
+  record.created / record.sealed / record.failed / record.invalidated /
+  record.reverified
   delivery.opened / delivery.event        (the #13 delivery log)
   replay.logged                           (the #14 replay log)
   *.imported                              (state found when a ledger was migrated)
@@ -217,6 +218,7 @@ def ok(self) -> bool:
     "record.sealed": "sealed",
     "record.failed": "failed",
     "record.invalidated": "invalidated",
+    "record.reverified": "sealed",     # an operator restored missing evidence (ledger.reverify)
 }
 
 

stele/ledger/migration.py

@@ -23,7 +23,10 @@
   archive and verified against the recorded manifest (from disk, or from the
   archive if it already holds the bundle). A committed record whose bundle
   cannot be verified becomes INVALIDATED with the reason: the ledger can no
-  longer vouch for that content.
+  longer vouch for that content. The reason says whether the evidence was
+  missing (neither on disk nor archived) or mismatched; only a record with
+  missing evidence can be restored later, by ledger.reverify, from files that
+  still hash as recorded.
 - source_hash is kept only when it is the digest of exactly one Snapshot in
   the archive. Otherwise it moves to legacy_source_hash, which is unverified
   audit data and never used as provenance.
@@ -55,6 +58,11 @@
 )
 
 
+# Why a committed record was invalidated by migration (prefix of its error).
+EVIDENCE_MISSING = "evidence missing"
+EVIDENCE_MISMATCHED = "evidence mismatched"
+
+
 class LedgerMigrationError(Exception):
     """Raised when an existing ledger cannot be migrated; nothing is changed."""
 
@@ -162,9 +170,13 @@ def _migrate_row(old: dict, archive: BlobStore) -> dict:
         ) as exc:
             row["state"] = "invalidated"
             row["finalized_at"] = datetime.now(timezone.utc).isoformat()
+            # Missing evidence (neither on disk nor archived) can be restored
+            # by an operator who still has the files (ledger.reverify);
+            # anything else means the content is not what was recorded.
+            kind = EVIDENCE_MISSING if isinstance(exc, MissingArtifactError) else EVIDENCE_MISMATCHED
             row["error"] = (
-                "INVALIDATED: migration — committed bundle could not be archived "
-                f"and verified: {exc!r}"
+                f"INVALIDATED: migration — {kind}: committed bundle could not be "
+                f"archived and verified: {exc!r}"
             )
         else:
             row["state"] = "sealed"

stele/ledger/reverify.py

@@ -0,0 +1,148 @@
+"""
+Restoring a record the migration invalidated for missing evidence (#39).
+
+Migrating a pre-#12 ledger invalidates a committed record whose bundle can be
+neither archived from disk nor found in the archive. Often the evidence was
+never archived (the archive did not exist yet) and the files are still
+somewhere: invalidating is safe, but it is permanent, so such a record could
+never be sealed again.
+
+reverify_record() is the explicit operator step that restores it. It never
+happens during migration, and it is narrow:
+
+- the record must be INVALIDATED *by the migration*, for *missing* evidence.
+  Its error says so, and its event history is exactly the migration's import
+  of it: an invalidation by an operator or a divergent replay has a
+  record.invalidated event (or another error) and stays invalidated, as does
+  a record whose evidence was found to be different (mismatched);
+- it must have no deliveries, since invalidation withdraws delivered data;
+- the files in the given directory are hashed against the recorded manifest
+  and archived exactly as sealing does. Any difference, or a missing file,
+  leaves the record INVALIDATED.
+
+On success the record is SEALED again, its error cleared, and a
+`record.reverified` event is appended to the hash chain in the same
+transaction; the earlier events stay, so the history reads: imported as
+invalidated, reverified.
+
+    python -m stele.ledger.reverify <ledger.db> <archive-dir> <record_id> --artifact-dir DIR
+"""
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+from ..archive import BlobStore
+from .events import EventLog
+from .migration import EVIDENCE_MISSING
+from .models import ArtifactRecord, ArtifactState
+from .store import (
+    ArtifactDriftError,
+    InvalidStateTransitionError,
+    LedgerStore,
+    MissingArtifactError,
+    RecordNotFoundError,
+    archive_bundle,
+)
+
+_MIGRATION_PREFIX = "INVALIDATED: migration — "
+
+
+class ReverifyRefused(Exception):
+    """The record cannot be restored; nothing was changed."""
+
+
+def reverifiable_problem(ledger: LedgerStore, record: ArtifactRecord) -> str | None:
+    """Why the record may not be reverified; None if it may."""
+    if record.state is not ArtifactState.INVALIDATED:
+        return f"the record is {record.state.value}, not invalidated"
+    error = record.error or ""
+    if not error.startswith(f"{_MIGRATION_PREFIX}{EVIDENCE_MISSING}:"):
+        return "it was not invalidated by the migration for missing evidence"
+    log = EventLog(ledger)
+    try:
+        events = log.for_subject(record.record_id)
+    finally:
+        log.close()
+    if [e.kind for e in events] != ["record.imported"] or events[0].body.get("state") != "invalidated":
+        return "its history is not just the migration's import, so it was invalidated for another reason"
+    deliveries = ledger._conn.execute(
+        "SELECT COUNT(*) FROM deliveries WHERE record_id=?", (record.record_id,)
+    ).fetchone()[0]
+    if deliveries:
+        return "it has deliveries, and invalidating withdrew the delivered data"
+    return None
+
+
+def reverify_record(
+    ledger: LedgerStore, record_id: str, artifact_dir: Path | str | None = None
+) -> ArtifactRecord:
+    """Seal an invalidated record again from files that hash as recorded.
+
+    artifact_dir defaults to the directory the record names. Raises
+    ReverifyRefused (nothing changed) for a record that may not be restored
+    or whose files do not match.
+    """
+    try:
+        record = ledger.get(record_id)
+    except RecordNotFoundError:
+        raise ReverifyRefused(f"no record {record_id}") from None
+    problem = reverifiable_problem(ledger, record)
+    if problem is not None:
+        raise ReverifyRefused(f"record {record_id} cannot be reverified: {problem}")
+    directory = Path(artifact_dir) if artifact_dir is not None else Path(record.artifact_dir)
+    try:
+        archive_bundle(ledger.archive, directory, record.artifact_manifest, record.artifact_hash)
+    except (MissingArtifactError, ArtifactDriftError) as exc:
+        raise ReverifyRefused(
+            f"record {record_id}: the files in {directory} are not the recorded bundle: {exc}"
+        ) from exc
+    try:
+        ledger._transition(
+            record_id,
+            from_states=(ArtifactState.INVALIDATED,),
+            action="reverify",
+            set_sql="state='sealed', finalized_at=?, error=NULL",
+            params=(_now_iso(),),
+            event=("record.reverified", {
+                "artifact_hash": record.artifact_hash,
+                "previous_error": record.error,
+            }),
+        )
+    except InvalidStateTransitionError as exc:
+        raise ReverifyRefused(str(exc)) from exc
+    return ledger.get(record_id)
+
+
+def _now_iso() -> str:
+    from datetime import datetime, timezone
+
+    return datetime.now(timezone.utc).isoformat()
+
+
+def main(argv: list[str] | None = None) -> int:
+    ap = argparse.ArgumentParser(
+        prog="python -m stele.ledger.reverify",
+        description="Seal a migration-invalidated record again from files that hash as recorded.",
+    )
+    ap.add_argument("ledger", type=Path)
+    ap.add_argument("archive", type=Path)
+    ap.add_argument("record_id")
+    ap.add_argument("--artifact-dir", type=Path, default=None,
+                    help="where the record's files are now (default: the directory it names)")
+    args = ap.parse_args(argv)
+    ledger = LedgerStore(args.ledger, BlobStore(args.archive))
+    try:
+        record = reverify_record(ledger, args.record_id, args.artifact_dir)
+    except ReverifyRefused as exc:
+        print(str(exc), file=sys.stderr)
+        return 1
+    finally:
+        ledger.close()
+    print(f"record {record.record_id} is sealed again (artifact_hash {record.artifact_hash})")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())