Skip to content

Verify every chained field; verifiers never migrate; harden ChatGPT ids - #51

Draft
Deep-Sixed wants to merge 1 commit into
mainfrom
claude/optimistic-mendel-82kq2z
Draft

Deep-Sixed wants to merge 1 commit into
mainfrom
claude/optimistic-mendel-82kq2z

Conversation

@Deep-Sixed

Copy link
Copy Markdown
Owner

Fixes the two P2s and the P3s from the review of main at 999fe3e. Each was reproduced against main before the fix.

P2: verify_ledger compared only some of the fields the chain records

record.created chains source_kind, source_id and backend, but verify_ledger never compared them with the table. Hand edits to any of them verified clean.

  • Reproduced: editing source_kind (file → tree, which makes replay ask the archive for the wrong Snapshot kind), source_id, backend or source_path left verify_ledger(...).ok == True.
  • Fix: every field a record or replay event carries is now compared.
    • Records: also the failure or invalidation reason, against the record's error.
    • Replays: also reason, policy and platform.
    • source_path is checked against the archived Source that source_id names.
    • Each report line names the field and both values.
  • artifact_dir is deliberately not checked. It says where the run's output was on disk. That is a location, not evidence. The docs say so.
  • created_at: observation references are named by the record's creation time, which the chain never recorded. record.created and record.imported now chain it. Records created before this change have their creation time on the row alone, and the docs say that too.
  • Observation references: resolving one rebuilt the observation from the row alone. It now also requires the row's source fields, and created_at where chained, to match the record's creation event.

P2: the verifiers migrated the ledger they were verifying

Every verifier opened the ledger with LedgerStore(...), which migrates on open.

  • Reproduced: python -m stele.ledger.events was pointed at a schema-6 ledger whose record had been flipped from invalidated back to sealed by hand. It migrated the ledger to schema 8 and chained the edited tables as record.imported. Then it reported "ok": true, and that false certification was written into the ledger for later verifications to trust.
  • Fix: LedgerStore(..., migrate=False) creates and migrates nothing. It raises LedgerSchemaError for a missing, foreign, older or newer ledger, and the message says why.
    • Used by stele.ledger.events, stele.identity, stele.extraction, and the report, sign-off, check and list commands of stele.verification.
    • Only verification run, which records runs, still migrates.
    • An older ledger is refused and left exactly as it was. The message says to migrate it the normal way first. There is no chain to verify what it held before then.

P3: ChatGPT adapter

  • Chunk-id collisions: the #<n> suffix for duplicated conversations could collide with a real id X#n, and "a:b" + "c" could collide with "a" + "b:c". A collision made the dispatcher reject the whole export. Conversation and node ids are now percent-encoded in chunk ids. UUIDs, which ChatGPT uses, are unchanged. Metadata node_id stays raw, since extraction anchors point into mapping by it.
  • Non-object author: a message whose author isn't an object crashed with AttributeError. It now raises MalformedExportError.
  • Overstated docs: the docstring and docs/adapter.md claimed the adapter never holds more than one conversation. That holds for iter_chunks(), not for transform(), which the dispatcher calls. Both now say so.

Tests

  • tests/test_event_log.py:
    • Each edited field is reported, with the expected message.
    • An intact ledger still verifies.
    • An older or missing ledger is refused and not migrated, through the API and all four CLIs.
  • tests/test_identity.py: an observation rebuilt from an edited created_at or source_id is refused.
  • tests/test_chatgpt_adapter.py: the two collision cases, UUID ids unchanged, and the malformed author.
  • Two existing tests in test_event_log.py pinned the old, coarser messages and now expect the new ones.
  • 22 of the 24 new tests fail without the fixes. The other two check that an intact ledger verifies and that UUID ids come through encoding unchanged, so they pass either way.
  • Locally: 856 passed, 121 skipped.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QRK7hqAaNLkH2JhWanY5VM


Generated by Claude Code

The P2s and P3s from the review of main at 999fe3e.

- verify_ledger compared only part of what the chain commits to.
  record.created carries source_kind, source_id and backend, but none was
  compared, so hand edits to them verified clean. It now compares every
  field a record or replay event carries: also a record's failure or
  invalidation reason (against its error column) and a replay's reason,
  policy and platform. record.created and record.imported now chain
  created_at too, which observations are named by; older events leave it
  unchecked. source_path is checked against the archived Source its
  source_id names. artifact_dir is a location, not evidence, and stays
  unchecked (documented).
- ObservationRef resolution rebuilt the observation from the row alone.
  It now also requires the row's source_hash, source_kind, source_id and
  (where chained) created_at to match the record's creation event.
- The verifiers migrated the ledger they verified: opening a ledger from
  before the event log imported its current, possibly edited tables into
  a new chain and then reported ok. LedgerStore gains migrate=False, which
  creates and migrates nothing and raises LedgerSchemaError for a missing,
  foreign, older or newer ledger. python -m stele.ledger.events,
  stele.identity, stele.extraction and the report/sign-off/check/list
  commands of stele.verification use it; only `verification run`, which
  records runs, still migrates.
- ChatGPT adapter: chunk ids percent-encode the conversation and node ids,
  so the "#n" suffix of a duplicated conversation cannot collide with a real
  id and "a:b"+"c" cannot collide with "a"+"b:c" (UUIDs are unchanged). A
  non-object author is MalformedExportError instead of AttributeError. The
  docstring and docs no longer claim transform() holds one conversation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QRK7hqAaNLkH2JhWanY5VM
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants