Skip to content

fix(test): reject mutation-harness false positives and honor fixture trust - #201

Open
wolfv wants to merge 2 commits into
pre1/io-cachefrom
pre1/mutation-harness
Open

wolfv wants to merge 2 commits into
pre1/io-cachefrom
pre1/mutation-harness

Conversation

@wolfv

@wolfv wolfv commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

Stack: follows #200.

  • Require successful unmodified verification before attempting mutations.
  • Treat crashes, timeouts, usage/setup errors and unrecognized rejection diagnostics as harness errors, not passing mutations.
  • Pass supplied trusted-root files to the Rust verifier.
  • Add offline --trusted-root support to the verification example; custom TUF instances now require --tuf-root rather than silently trusting a writable cache.
  • Add standard-library Python regression tests.

Validation: Python unittest checks, real offline baseline plus signature-bit mutation, and strict example Clippy. /usr/bin/false now exits the harness with status 2 rather than reporting every mutation passed. External verifier diagnostic changes intentionally require updating rejection markers.

…r failures

Signed-off-by: Wolf Vollprecht <w.vollprecht@gmail.com>
…ed hints

Signed-off-by: Wolf Vollprecht <w.vollprecht@gmail.com>
@wolfv

wolfv commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator Author

Added the small fixture/expectation changes previously in #205 here: stricter baseline checks now come with an offline CI fixture set in the same PR. This avoids leaving this PR dependent on a later CI repair.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant