Skip to content

Fail every expired exemption, and judge mutation survivors from outcomes.json - #57

Merged
VirtualMachinist merged 12 commits into
mainfrom
hn/g2-ledger
Nov 20, 2026
Merged

VirtualMachinist merged 12 commits into
mainfrom
hn/g2-ledger

Conversation

@VirtualMachinist

Copy link
Copy Markdown
Member

Slice 1.5: every expired exemption fails the gate, and the mutation layer judges its survivors.

  • Expiry. Every expired or not-a-day exemption fails the gate with exit 1, whatever its layer, whether that layer is on, and whatever the --max-tier. Each record is named in problems. check prints the records as warnings and exits 0.
  • Survivors ledger. cargo mutants runs with any old mutants.out/outcomes.json removed first. Exits 0, 2 and 3 count as a finished run. The file that run wrote is read, pinned to cargo-mutants 27.1.0. An unknown shape, another release, or a missing file fails the layer and names outcomes.json.
  • Matching. Missed mutants are matched to live layer: mutation records with the exemption cover rule; a record with no symbol covers every function in its glob. unexplained_survivors_max is judged and the failure names only the unexplained survivors. Timeouts and unviable mutants are counted, not treated as survivors. kill_rate_min_pct stays declarative.
  • Report. The mutation row's judged carries caught, missed, explained, unexplained, timeouts, unviable and max beside base.
  • Tidy-ups. Per-layer verdicts are decided in judge.rs and rendered in report.rs, with no expectation changed. The report is written before the summary, and a failed write ends the output with fail:. --help, the schema and the docs say what exit 2 covers.
  • Spec. PROTOCOL.md §8.3 and §19.

Gate at the head: nextest 560 passed; 117 mutants tested in 7m: 101 caught, 16 unviable; ok: 8 command(s) passed. The fixtures in tests/fixtures/mutants-27.1.0/ come from a real 27.1.0 run.

🤖 Generated with Claude Code

VirtualMachinist and others added 11 commits November 19, 2026 23:02
mutation.rs parses cargo-mutants 27.1.0's outcomes.json and refuses any
other version or shape, then sorts the missed mutants by the live
mutation exemptions that cover them (Exemption::covers). Timeouts and
unviable mutants are counted, not survivors. The fixture is a real run.
Not wired into the gate yet.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
gate_survivors.rs plants a recorded cargo-mutants 27.1.0 outcomes.json
through a cargo stub and judges explained and unexplained survivors, the
max, timeouts, unviable mutants and an unknown shape. gate_exemptions.rs
checks that an expired or not-a-day record fails the gate for any layer,
on or off, at any max-tier, and that check only warns. A second recorded
run holds the shape of a crate under a workspace. Most of these fail until
the gate is wired to read outcomes.json and every record.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The mutation layer now removes any mutants.out/outcomes.json before
cargo mutants runs, treats exit 0, 2 and 3 as a finished run, and reads
the file that run wrote. Survivors are matched to live layer: mutation
records and judged against unexplained_survivors_max; the row's judged
carries the counts. An unreadable or other-release file fails the layer.

Every exemption record is checked whatever its layer, on or off, at any
max-tier: an expired or not-a-day date is a problem and exit 1. check
prints the same lines as warnings and exits 0.

Carried from the base review: a row with a missing tool or a dry run
words the base without an outcome; the closing fail line counts commands
that ran as the count line does; origin/HEAD naming a tree reads "not a
commit"; the report is written before the summary, and a failed write
ends the output with fail:; the per-layer verdict moves to judge.rs; the
help and the report schema say what exit 2 covers. PROTOCOL 8.3 and 19
say how the ledger and the records are read.

The cargo stubs of gate_base and laws_gate write what 27.1.0 writes for
a run with no mutants, as the real tool always does.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The help and the report schema keep their "the config is invalid"
opening and then say what else exit 2 covers. The loop over every layer
and both tiers took longer than the 20 second CI limit under nextest; it
is four tests now, each over four layers at one tier.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The four tests shared one scratch name, so threaded cargo test had them
remove each other's directories. Each now names its own.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e prefix

take scored a CRAP above the new-function threshold; the baseline check,
the outcome kind and the counting are separate decisions now. The gate's
own run found that nothing tested crate_prefix against a real workspace:
a test names a crate under crates/ by its directory and a crate at the
root by none.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…cord

B1: real cargo-mutants 27.1.0 exits 0 and writes no mutants.out for a
diff of docs, comments, tests or CI. The gate now passes --output <report
dir>, so the file is where the gate looks whatever an adopter sets, and
reads exit 0 with no file as zero mutants only when the run printed the
line 27.1.0 prints for it (recorded from real runs and pinned). Any other
missing file fails. The test stubs write nothing, as the real tool does,
and a new test runs the real cargo-mutants over a README-only and a
comment-only diff.

B2: a survivor with function: null (a const, static, type alias or field
type) is explained only by a record with no symbol. A real 27.1.0 run over
such a crate is the new fixture.

S2: a symbol-free whole-tree mutation record lifts the floor and explains
no survivor; section 19 says so. S1: the laws and above-range verdicts are
decided in judge.rs. S3: the exit 2 text names usage errors and the lock
only when a layer in range needs a pin. S4: a wrong or missing
total_mutants, exit 2 with nothing missed and exit 3 with no timeout each
fail naming both facts. N1: a survivor is named once. N2: a failed cargo
mutants says what its exit means. N4: the expiry day is UTC. N5: the
explained-survivor list is rewritten to the one rule.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…it note

The gate's own mutants run found two survivors: the guard that tells a
missing program from another spawn error in run_mutants, and the laws row
verdict now decided in judge.rs. Tests kill both. exit_note scored a CRAP
just over the threshold; the exit meaning is its own small function.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…from skipping

B3, S6: a symbol-free layer: mutation record is the whole tree when its
glob matches every Rust source file the mutation scope examines (the .rs
files under src/, and build.rs), or is made only of * and ** components,
or is the crate root. One function, tiers::whole_tree, answers that for
both check (the floor lift) and the gate (a record that explains no
survivor), so /**, **/, **/**, *.rs, **/*.rs, */** and a single-file
crate's only file are held to the same rule as **. Every spelling is
tested at both ends.

S7: the coverage and CRAP floors keep their own list of whole-crate
paths, as before; * is not on it.

S8: the tops-gate step sets RUST_TOPS_REQUIRE_MUTANTS, so the real
cargo-mutants test cannot skip in CI. N6: the no-mutants line must be a
whole stderr line (colour codes aside), not a phrase inside another line.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@NebuHedron
NebuHedron self-requested a review as a code owner November 20, 2026 08:27
B4, S9, N8: telling the whole tree from a part of it depended on how
cargo-mutants walks files, and three reviews each found another shape
that let one record silence the ledger. A survivor is now explained per
function. One inside a function is explained only by a live mutation
record whose symbol names that function and whose glob covers its file;
one in no function (function: null) only by a record with no symbol whose
glob covers its file. No record explains every survivor, so
unexplained_survivors_max always decides.

scope::examined_files, tiers::whole_tree and check_with's file list are
gone: tiers.rs and scope.rs are as at main, so check's floor lift for the
max, and the coverage and CRAP lists, are unchanged. The expectation that
a symbol-free record covers every function in its glob is replaced by the
per-function rule. PROTOCOL 8.3 and 19 say it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@VirtualMachinist
VirtualMachinist merged commit 9e705f5 into main Nov 20, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant