Repository navigation
Fail every expired exemption, and judge mutation survivors from outcomes.json - #57
Merged
Merged
Conversation
mutation.rs parses cargo-mutants 27.1.0's outcomes.json and refuses any other version or shape, then sorts the missed mutants by the live mutation exemptions that cover them (Exemption::covers). Timeouts and unviable mutants are counted, not survivors. The fixture is a real run. Not wired into the gate yet. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
gate_survivors.rs plants a recorded cargo-mutants 27.1.0 outcomes.json through a cargo stub and judges explained and unexplained survivors, the max, timeouts, unviable mutants and an unknown shape. gate_exemptions.rs checks that an expired or not-a-day record fails the gate for any layer, on or off, at any max-tier, and that check only warns. A second recorded run holds the shape of a crate under a workspace. Most of these fail until the gate is wired to read outcomes.json and every record. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The mutation layer now removes any mutants.out/outcomes.json before cargo mutants runs, treats exit 0, 2 and 3 as a finished run, and reads the file that run wrote. Survivors are matched to live layer: mutation records and judged against unexplained_survivors_max; the row's judged carries the counts. An unreadable or other-release file fails the layer. Every exemption record is checked whatever its layer, on or off, at any max-tier: an expired or not-a-day date is a problem and exit 1. check prints the same lines as warnings and exits 0. Carried from the base review: a row with a missing tool or a dry run words the base without an outcome; the closing fail line counts commands that ran as the count line does; origin/HEAD naming a tree reads "not a commit"; the report is written before the summary, and a failed write ends the output with fail:; the per-layer verdict moves to judge.rs; the help and the report schema say what exit 2 covers. PROTOCOL 8.3 and 19 say how the ledger and the records are read. The cargo stubs of gate_base and laws_gate write what 27.1.0 writes for a run with no mutants, as the real tool always does. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The help and the report schema keep their "the config is invalid" opening and then say what else exit 2 covers. The loop over every layer and both tiers took longer than the 20 second CI limit under nextest; it is four tests now, each over four layers at one tier. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The four tests shared one scratch name, so threaded cargo test had them remove each other's directories. Each now names its own. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e prefix take scored a CRAP above the new-function threshold; the baseline check, the outcome kind and the counting are separate decisions now. The gate's own run found that nothing tested crate_prefix against a real workspace: a test names a crate under crates/ by its directory and a crate at the root by none. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…cord B1: real cargo-mutants 27.1.0 exits 0 and writes no mutants.out for a diff of docs, comments, tests or CI. The gate now passes --output <report dir>, so the file is where the gate looks whatever an adopter sets, and reads exit 0 with no file as zero mutants only when the run printed the line 27.1.0 prints for it (recorded from real runs and pinned). Any other missing file fails. The test stubs write nothing, as the real tool does, and a new test runs the real cargo-mutants over a README-only and a comment-only diff. B2: a survivor with function: null (a const, static, type alias or field type) is explained only by a record with no symbol. A real 27.1.0 run over such a crate is the new fixture. S2: a symbol-free whole-tree mutation record lifts the floor and explains no survivor; section 19 says so. S1: the laws and above-range verdicts are decided in judge.rs. S3: the exit 2 text names usage errors and the lock only when a layer in range needs a pin. S4: a wrong or missing total_mutants, exit 2 with nothing missed and exit 3 with no timeout each fail naming both facts. N1: a survivor is named once. N2: a failed cargo mutants says what its exit means. N4: the expiry day is UTC. N5: the explained-survivor list is rewritten to the one rule. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…it note The gate's own mutants run found two survivors: the guard that tells a missing program from another spawn error in run_mutants, and the laws row verdict now decided in judge.rs. Tests kill both. exit_note scored a CRAP just over the threshold; the exit meaning is its own small function. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…from skipping B3, S6: a symbol-free layer: mutation record is the whole tree when its glob matches every Rust source file the mutation scope examines (the .rs files under src/, and build.rs), or is made only of * and ** components, or is the crate root. One function, tiers::whole_tree, answers that for both check (the floor lift) and the gate (a record that explains no survivor), so /**, **/, **/**, *.rs, **/*.rs, */** and a single-file crate's only file are held to the same rule as **. Every spelling is tested at both ends. S7: the coverage and CRAP floors keep their own list of whole-crate paths, as before; * is not on it. S8: the tops-gate step sets RUST_TOPS_REQUIRE_MUTANTS, so the real cargo-mutants test cannot skip in CI. N6: the no-mutants line must be a whole stderr line (colour codes aside), not a phrase inside another line. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
B4, S9, N8: telling the whole tree from a part of it depended on how cargo-mutants walks files, and three reviews each found another shape that let one record silence the ledger. A survivor is now explained per function. One inside a function is explained only by a live mutation record whose symbol names that function and whose glob covers its file; one in no function (function: null) only by a record with no symbol whose glob covers its file. No record explains every survivor, so unexplained_survivors_max always decides. scope::examined_files, tiers::whole_tree and check_with's file list are gone: tiers.rs and scope.rs are as at main, so check's floor lift for the max, and the coverage and CRAP lists, are unchanged. The expectation that a symbol-free record covers every function in its glob is replaced by the per-function rule. PROTOCOL 8.3 and 19 say it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Slice 1.5: every expired exemption fails the gate, and the mutation layer judges its survivors.
--max-tier. Each record is named inproblems.checkprints the records as warnings and exits 0.cargo mutantsruns with any oldmutants.out/outcomes.jsonremoved first. Exits 0, 2 and 3 count as a finished run. The file that run wrote is read, pinned to cargo-mutants 27.1.0. An unknown shape, another release, or a missing file fails the layer and namesoutcomes.json.layer: mutationrecords with the exemption cover rule; a record with no symbol covers every function in its glob.unexplained_survivors_maxis judged and the failure names only the unexplained survivors. Timeouts and unviable mutants are counted, not treated as survivors.kill_rate_min_pctstays declarative.judgedcarries caught, missed, explained, unexplained, timeouts, unviable and max besidebase.judge.rsand rendered inreport.rs, with no expectation changed. The report is written before the summary, and a failed write ends the output withfail:.--help, the schema and the docs say what exit 2 covers.PROTOCOL.md§8.3 and §19.Gate at the head: nextest 560 passed;
117 mutants tested in 7m: 101 caught, 16 unviable;ok: 8 command(s) passed. The fixtures intests/fixtures/mutants-27.1.0/come from a real 27.1.0 run.🤖 Generated with Claude Code