Repository navigation
feat: a mutation report on the lines a pull request changes - #157
Merged
Merged
Conversation
gremlins and Stryker mutate the diff, and one pull request comment lists the mutants that survived and those no test reaches. It never fails the build.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A reusable workflow that shows, on a pull request, which mutants survive on the lines it changes. A surviving mutant is a line a test runs but never checks: the code changed, and no test failed. This is the diff-scoped mutation report from the estate testing standard (ADR-0022).
mutation-report.yml. The caller passesgo-directory,web-directory, or both.gremlins unleash --diff origin/<base>(gremlins v0.6.0 by default).--mutateset to the changed source files undersrc. Specs, generated clients and*.gen.tsare left out.mise.toml. Stryker runs through the caller's own devDependencies, so it uses the project's vitest.continue-on-error, so the report never fails the build. Whether it becomes a gate, and at which threshold, is decided later.scripts/mutation_report.pyturns gremlins' JSON (LIVED, NOT COVERED) and Stryker's mutation-testing-report JSON (Survived, NoCoverage) into that comment. Survivors are sorted by file and line, and mutants no test reaches get their own section. A missing report counts as no mutants, and an unreadable one is named in the comment. Its tests follow the testing standard: one behaviour per test, named as a sentence. CI holds the script to 100% line and branch coverage, likepath_buckets.py.Part of JorisJonkers-dev/workspace#128. #155 closes once identity calls a released version and a deliberately weak test shows up as a survivor.
Checks
tests/test_mutation_report.py: 14 tests, 100% line and branch coverage.tests/pass.