Skip to content

feat: persist fork provenance in the trace bundle - #41

Open
how2how2how2-arch wants to merge 1 commit into
utsab345:mainfrom
how2how2how2-arch:feat/persist-fork-provenance
Open

how2how2how2-arch wants to merge 1 commit into
utsab345:mainfrom
how2how2how2-arch:feat/persist-fork-provenance

Conversation

@how2how2how2-arch

Copy link
Copy Markdown
Contributor

Implements part of #31, the half that names a defect: a forked trace is saved without the provenance that says it is one.

The defect

fork_trace builds a ForkInfo and then throws it away. It calls forked_trace.save(output_path) and never passes the metadata it just constructed, so a bundle written by stepfork fork is byte-for-byte indistinguishable from a recorded run. A reader holding only the saved bundle cannot tell that outputs were replaced, where the fork point was, or what it was forked from — which is what #31's second scope item asks for.

What this does

Fork provenance is persisted as an optional bundle file, fork.json:

{
  "fork_info": { "parent_trace_path": ..., "parent_run_id": ..., "fork_point_event_id": ..., "fork_point_step": 3, "fork_timestamp": ..., "overrides_applied": 1 },
  "overridden_event_ids": ["evt_002"]
}

Three decisions in that shape, and why:

A separate file rather than a manifest field. manifest.json carries schema_version: "0.1" and the models are extra="forbid", so a new manifest field is either a version bump or a breaking change. I raised that trade-off on #33 and did not want to settle it on your behalf here. The bundle already has a precedent for adding a file: integrity.json was added after the initial v0.1 work, and older bundles stay loadable. fork.json follows it — present only on a forked bundle, and a bundle without it loads and verifies exactly as before. schema_version is unchanged.

overridden_event_ids alongside ForkInfo. ForkInfo.overrides_applied is a count, and a count cannot answer "was this output recorded or replaced". The ids can, so they are recorded next to it rather than folded into the count. ForkInfo itself is untouched.

Covered by the integrity record when it exists. integrity.json now hashes fork.json when present, so editing provenance is reported as a mismatch instead of passing. A bundle whose fork.json exists but is not in the record is reported as integrity_unlisted_file, so the file cannot be kept while its digest is quietly dropped. read_fork_provenance() returns the record, or None for a bundle that was not forked.

The honest limit, also written into docs/trace-format.md: deleting both the file and its digest leaves a bundle that verifies as an ordinary recorded run. Provenance is evidence about a bundle, not proof of its origin, and the doc says so rather than implying more.

Not done, deliberately

This PR is one logical change: persist the provenance that already exists in memory, cover it with the record, document what that does and does not prove.

Tests

tests/unit/test_fork_provenance.py — 6 tests: the fork's parent, fork point and replaced events round-trip through the bundle; a bundle written without a fork has no provenance and still verifies with exactly the three required files; provenance is named in the record and editing it is a mismatch; an unrecorded fork.json is reported; a recorded-but-deleted one is a missing file; a malformed one raises rather than reading as absent.

Two plants, both run against the committed tree:

  • Dropping the write_fork_provenance call (today's behaviour) → 4 failed, 2 passed.
  • Writing provenance but not re-covering it in the record → 3 failed, 3 passed.

Reverted, the suite is green.

Checks

pytest 722 passed (716 without the new file), ruff check clean, ruff format --check 204 files, mypy clean across 138 source files, mkdocs build --strict exit 0, and the three demos (quickstart, booking_agent, refund_agent) all exit 0.

One note on process: #31 asks for a design comment before a PR. I have kept the change to the item that names a defect and put the design here, since the reasoning is what a reviewer reads anyway — say the word if you would rather I have split it out first and I will not repeat the shape.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant