Repository navigation
feat(fork): report recorded outputs an override may have invalidated - #42
Open
how2how2how2-arch wants to merge 1 commit into
Open
how2how2how2-arch wants to merge 1 commit into
how2how2how2-arch wants to merge 1 commit into
Conversation
A fork keeps every event at or before the fork point, and an override replaces one of those outputs. Recorded outputs captured after the overridden step were produced under the upstream state the override changed, so replay substituting them is not evidence for the modified execution path, yet they were indistinguishable from an ordinary substitution: the only per-call classification was substituted vs executed, and nothing on the fork or the manifest said which recorded outputs an override upstream had invalidated. fork_trace() now reports the overridden steps and the retained recorded outputs that follow the earliest of them, and the CLI fork report shows both, with a note that replaying them is not evidence for the modified path. The classification is an upper bound rather than a dependency analysis: it marks every recorded output after the earliest overridden step whether or not that output's inputs changed, and it does not mark the event the override replaced, because that value is the override rather than a reused recording. What it does not do, and docs/fork.md now says so: replay does not consult the classification yet, so a call reported as substituted is not by itself proof that it belongs to the modified path, and the classification is not persisted into a bundle, so it is available only from the fork_trace() call that produced it. The approach follows the design proposed in the issue thread on 2026-10-28: the classification needs the set of overridden steps, which no arithmetic on overrides_applied can recover. Refs utsab345#33 Tests: tests/unit/test_fork_staleness.py covers the stale case, the still-matching control, the override target itself, two overrides in one fork, and that tool calls are never classified as outputs.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #33.
What this changes
A fork copies every event at or before the fork point, and an override replaces one of those outputs. A recorded output captured after the overridden step was produced under the upstream state the override changed, so replay substituting it is not evidence for the modified execution path. Today those outputs are indistinguishable from an ordinary substitution:
MatchedCall.actionis onlysubstitutedorexecuted, and nothing on the fork says which recorded outputs an override upstream invalidated.fork_trace()now reports both facts on its result:overridden_steps— which steps were replaced (a set, not the existingoverrides_appliedcount)stale_event_ids— the retained recorded outputs that follow the earliest of themThe CLI fork report shows them, and
docs/fork.mdgained a section on what the classification does and does not prove.The limitation is stated, not implied
This is an upper bound, not dependency analysis: every recorded output after the earliest overridden step is reported, whether or not its inputs actually changed. The event the override replaced is not reported as stale, because its value is the override rather than a reused recording.
Replay does not consult the classification yet, so a call reported as
substitutedis still not by itself proof that it belongs to the modified path. Neither is the classification persisted into a bundle, so it is available only from thefork_trace()call that produced it. Both are written into the docs rather than left to be discovered.Tests
tests/unit/test_fork_staleness.py, offline and deterministic:The plant: replacing
overridden_steps[0]with0in the classifier — classifying from "an override happened" instead of "which step it was" — failstest_only_outputs_after_the_earliest_override_are_stalewith exactly one extra id, and passes again once restored.Compatibility
No trace or receipt format change: the new fields are additive on
ForkResultwith defaults, and existing traces load, validate and replay unchanged.ruff check,ruff format --checkandmypy srcare clean, and the full suite is 722 passed.Design note
This follows the approach I proposed in the issue thread on 2026-10-28: the classification needs the set of overridden steps, and no arithmetic on
overrides_appliedcan recover it.