Repository navigation
ci(mutation): grade only the shipped products, not test/e2e-core - #337
Merged
Merged
Conversation
…ss, the bench and the CLI contract Per-file destinations for all 80 files, a four-layer stack (Mutation de-enrolment, bench, stream codec, harness and deletion), Stream K sequencing and the landing order of the open PRs that touch the package. ce-doc-review skipped: the root reviews this plan (r42).
…and the root's corrections Applies the root's six corrections of 22:25: MUTATED_PROJECTS is data at lockfile-closure-digest.integration.test.ts:29-34 and changes with mutation PROJECTS; stryker-js-cli-contract is not in PROJECTS, so the moved stream codec is ungraded until Stream K enrols it; the lint coupling at test/bench/oxlint.config.ts:1, test/bench/tsconfig.node.json:11, test/e2e/oxlint.config.ts:17 and test/e2e/AGENTS.md:21; kill-matrix.yml 9/15/82 and mutation.yml 36/43/64/118; the exact inventory (29 src modules plus mod.ts, 13 laws, 7 suites, 18 fixtures, 12 package files); and verified or [unverified]-marked pack and config claims. Adds Done and Does-not-count, the 28 outside files that name the package by layer, #287's overlap with PR D, #321's merge, and D6 for the three fixture schemas under tests-own-no-schemas.
…lution The root accepted D1, D2, D3 (guards checked first), D5 and D6, the Done grep, no-rename, no-shim, R6, R7, the inventory and the landing table. Its five review corrections are now in the plan: - no scratch-edit proofs: refusals are committed negative fixtures or a closed deliberate-red PR (bench lint ban, E2E-1's missing-journey case); - a contract codec is never mutation-graded, because Mutation selects *.workflow.ts only; PR C's codec is graded by its schema laws; - #306 and #327 fixed the ManifestDocument and CountedReportJson flakes, so the law moves unchanged with no follow-up unit; - D2's corpus change is timed before the post-#316 speed baseline, or recorded as a reviewed corpus change; - landing order A, then C at any time, B after #316 and #298, D last. PR A's evidence and its one-time cost (the first main Mutation run after merge re-verifies every mutant, because the cache path list changes) are named in the plan.
Mutation and the kill matrix mutated four projects: the three product packages and test/e2e-core, a private test-tooling package. Test tooling is not shipped, so grading it spends shard time on code no user runs (plan R7). - mutation.yml drops test/e2e-core from PROJECTS (:36), from the INCREMENTAL_REPORTS cache paths (:43), and from the incremental (:64) and build (:118) cache keys. - kill-matrix.yml drops it from both default project lists (:9, :15) and from its build cache key (:82), which stays byte-identical to mutation.yml's so the kill matrix still restores the mutation build. - lockfile-closure-digest.integration.test.ts drops it from MUTATED_PROJECTS, the test's copy of Mutation's project list, so its live-lockfile scenario checks the projects Mutation actually grades. The package itself, its mutation script and stryker.config.ts stay until the last layer deletes the package. One-time cost: a cache entry's version is a hash of its path list, so the first main Mutation run after this lands cannot restore the previous incremental reports and re-verifies every mutant once.
…lockfile test The lockfile-closure scenarios checked "every mutated project" against a hard-coded copy of Mutation's corpus, which nothing compared with the workflow. The test now reads env.PROJECTS from mutation.yml, so the two lists cannot drift apart (review run 20261013-084434-c89dc2ab, testing P2). CONST-W3: this PR edits .github/workflows/mutation.yml and kill-matrix.yml, which AGENTS.md:7 marks read-only and CONST-E9 treats as a judgment surface. Root assigned that edit when it accepted the plan (PR A). The edit only removes test/e2e-core from the corpus; no threshold, baseline or check changes. .stryker/budget-baseline.json stays as measured over four projects, and its owner decides whether to re-measure it. Plan 0900 supersedes 0218 with these review outcomes.
… change release.jsonc now gates intents on turbo's typecheck graph (#335), whose inputs include tests, so the test edit in this PR needs an intent. Nothing published changes.
…d MUTATED_PROJECTS Delta review run 20261013-090355-9173e728 (P3): the Problem Frame, the PR D ordering risk, the landing note and P9 still described the retired constant. They now give it as main's state at 381b3f5 and name the read of mutation.yml env.PROJECTS that replaces it.
systemfsoftware-maker
changed the title
ryanleecode
approved these changes
Oct 13, 2026
ryanleecode
left a comment
ryanleecode
left a comment
Contributor
There was a problem hiding this comment.
Root merge audit at 546bf50:
- CI is green on the head.
- Main since base 8f68cb5 (#322 and #336) shares no file with this PR.
- The diff grep is clean, and there are 0 threads.
- Full-path receipt 20261013-084434-c89dc2ab on 5a37a3f, plus a delta lens (20261013-090355-9173e728) over 6f2fb9c and 594b7e7, all on DeepSeek V4.1. The delta's P3 is fixed in 546bf50, which is plan text only.
- P1 #1 is the root-assigned workflow edit, declared under CONST-W3. #2 is fixed in 6f2fb9c. #3 is stated under Costs.
- I accept the residual, kill-matrix.yml's second copy of the corpus. Stream K owns that file.
- The lockfile scenarios read the corpus from mutation.yml, so the two can't drift.
This is PR A of the accepted e2e-core plan.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
Merging main brought #337, whose lockfile-closure-digest scenario reads .github/workflows/mutation.yml for the projects it grades. The full-path review of ba6a8b7 confirmed the read sat in no input: a workflow-only edit left stryker-js#test's hash unchanged (f3e3dc41 before and after). Declared as a $TURBO_ROOT$ input, the same edit moves the hash (2653559a -> 9d5ba277) and run --affected selects stryker-js#test alone. The plan moves to 2026-10-13-1028, superseding 0932. The changeset body drops internal file names.
Collaborator
Author
|
PR A's last Done line holds. The first |
This was referenced Oct 13, 2026
Merged
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
…lane After merging main, Mutation's and the kill matrix's project lists name only the three packages, so ADR-0004 no longer says test/e2e-core is still graded.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
…t of the next run's inputs Takes main through b4db6d8 (#313 root-bound dry-run coverage, #314, #317, #321, #322, #326, #330, #331, #332, #334, #335, #337). Merge resolutions: - dry-run-choice: main's current-root candidate filter over the one incremental file this branch reads; prior statuses come from the verdict store, flaky ids from the kept coverage. - dry-run.cell, deferrable-dry-run.cell, plan-request.cell: pass that one report text into main's choice; the plan prices from the kept coverage. - typescript-checker: main moved the runtime into typescript-checker-runtime.ts; it carries this branch's config/program digest scope. - e2e StrykerCliRunner: main split service from layer; runScript and runInFork move into microsandbox-stryker-cli-runner.ts. - Tests: main's flaky-root CompileError scenario writes the incremental file instead of the removed incrementalSources; the relocated-project multi-report scenario and dry-run-coverage-across-roots go with incrementalSources; the version-gate test uses layerStage. - The solution doc keeps both the store key and the root binding. Fix found while validating the merge: "A run killed after it scored its first mutant spares the next run its initial test run" failed 2 of 4 runs. The fixture kills the CLI at the first mutant event (run/mutant-run.ts:264-265), which can land inside the checkpoint's atomic write of the incremental file (mutation-reporting.service.ts:754). SIGKILL skips the temp file's cleanup (replace-file-atomically.ts:44), so reports/.main.json.<hex>.tmp stays. The crawl excluded only main.json (stryker-outputs.ts, read-project.cell.ts:116-124), so the temp file became a project file and the next run refused its dry run with closureChanged. replace-file-atomically.ts now owns the temp name and the rule matching it, and strykerOutputFilesOf excludes the incremental file's temp siblings. A new scenario plants that temp file between two runs of the built CLI; on the unfixed build the second run executed 2 tests, and it now executes none. The changeset says what holds for a killed run.
This was referenced Oct 13, 2026
ryanleecode
pushed a commit
that referenced
this pull request
Oct 13, 2026
#333) * build(mutation): mutate workflow files only in the three dogfood projects Ryan's ruling of 2026-10-12 22:47 EDT: mutation selects *.workflow.ts only, and *.schema.ts is never in a mutate set. The mutate globs of packages/stryker-js, stryker-js-vitest-runner and stryker-js-typescript-checker drop src/**/*.schema.ts. On main 2512017 that removes 62 schema files (53, 3, 6) and keeps 101 workflow files (86, 3, 12). Main's Mutation run 38338484710 scored 2,213 mutants in those schema files. Every project still has workflow files, so no config is deleted. The effect-schema-declarations ignorer stays: in that run it marked 744 mutants inside workflow files Ignored (tagged-class tags and fields, symbol descriptions, tagged-struct tags, class ids, annotations). Without it, those 744 would be scored as mutants no test can kill. ADR-0004 records the rule and supersedes ADR-0002's mutation-selection statements. ADR-0002 itself is unchanged. test/e2e-core/stryker.config.ts is untouched: Stream D's dissolution removes it from Mutation. * docs(adr): date the schema mutate glob correctly and scope ADR-0004 to its three projects Full-path review of #333 at 0a6e839 confirmed two findings. ADR-0004 said ADR-0002 widened the mutate set to schema files; git log -S shows the glob entered with the cell-architecture rewrite 1cf39f2, which also added ADR-0001, and ADR-0002 changed no config. ADR-0004 also stated its rule for every dogfood config while test/e2e-core, Mutation's fourth project, keeps its schema glob; the rule, driver and Confirmation now name the three covered projects. The changeset says the same in consumer terms. * docs(adr): record that #337 took test/e2e-core out of every mutation lane After merging main, Mutation's and the kill matrix's project lists name only the three packages, so ADR-0004 no longer says test/e2e-core is still graded. * docs(adr): name the two plans whose mutate-set premise ADR-0004 changes Full-path review of #333 at f95ba45 confirmed that two in-flight plans state the two-glob mutate set as fact (plan v3:45, stream-files plan:110). ADR-0004 now records both as a Neutral consequence for their owners to re-derive; it edits neither plan. * docs(adr): list every dated document that states the old mutate premise Full-path review of #333 at 8bb7669 confirmed that ADR-0004 named two of the dated documents stating the two-glob mutate set; git grep finds six documents at nine lines. The bullet now lists all of them and the grep that reproduces the list.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR A of the
test/e2e-coredissolution, and the bottom layer of its stack. Plan:docs/plans/2026-10-13-0900-refactor-dissolve-e2e-core-plan.md. It supersedes 0218, which root reviewed and accepted.Outcome
Mutation and the kill matrix now grade only the three shipped products:
packages/stryker-js,packages/stryker-js-typescript-checkerandpackages/stryker-js-vitest-runner.test/e2e-coreis test tooling that is about to be dissolved, and it no longer takes shard time. The lockfile-closure test takes its project list frommutation.yml, so it always checks the same corpus that Mutation grades.What changed
.github/workflows/mutation.yml:test/e2e-coreis removed fromPROJECTS(:36), fromINCREMENTAL_REPORTS(:43), from the incremental cache key (:64) and from the build cache key (:118)..github/workflows/kill-matrix.yml: it is removed from bothprojectsdefaults (:9,:15) and from the build cache key (:82). Both build keys stay byte-identical, so the kill matrix still restores the mutation lane's build.packages/stryker-js/tests/lockfile-closure-digest.integration.test.ts: the hard-codedMUTATED_PROJECTSis gone. Both "every mutated project" scenarios now readenv.PROJECTSfrommutation.yml, so the corpus and the test cannot drift apart..changeset/lockfile-scenarios-graded-projects.md: anoneintent. Since ci(repo): gate change intents on turbo's typecheck task graph #335, intents are gated on turbo'stypecheckgraph, whose inputs include tests. Nothing published changes.CONST-W3 declaration
AGENTS.md:7marks.github/workflows/read-only, andCONST-E9treats CI checks as a judgment surface. This PR edits two workflow files anyway, because root assigned exactly this change when it accepted the plan (PR A). The edit only removes a project from the corpus, and that project is being deleted. No threshold, baseline or check changes.Costs, stated
actions/toolkitcacheUtils.tsgetCacheVersion; restore sends that version,cache.ts), andINCREMENTAL_REPORTSis that list. So the firstmainMutation run after merge cannot restore the previous incremental reports, and it re-verifies every mutant once. Later runs reuse again..stryker/budget-baseline.json(561.078 s) was measured with four projects. It stays unchanged because it is a judgment surface. With three projects the gate has extra headroom equal to the removed project's share of the slowest shard, which was not measured. The baseline's owner decides whether to re-measure it.Evidence
lockfile-closure-digest.integration.test.ts: 12/12 pass with the three projects thatmutation.ymllists.stryker planwas not run locally, because it instruments every project and opens checker pools. The real-behaviour proof is the firstmainMutation run after merge: itsshard-planartifact should name only the three products.Review
Full path, run
20261013-084434-c89dc2abon5a37a3f85. Five lenses ran (correctness, project-standards, testing, adversarial, learnings), all served by DeepSeek v4.1 Flash. The merge leaf ran on Claude Haiku 4.5. No cross-model peer ran, because the review was limited to DeepSeek/MiniMax. Four findings went to the validator..github/workflows/, whichAGENTS.mdmarks read-onlyCONST-W3declaration in plan 0900 (A1), in commit6f2fb9c05and above. Root assigned the edit.6f2fb9c05: the test readsmutation.ymlenv.PROJECTS.actions/toolkitcache.ts/cacheUtils.ts: the version hashes the path list, so the fallback cannot match.One correctness P3 (the plan's D4 wording) was suppressed at anchor 50. Learnings pack ledger: 5 rules, 5 comply, 0 violate.
Delta, correctness only, run
20261013-090355-9173e728over6f2fb9c05and594b7e7ae, DeepSeek v4.1 Flash. One P3: plan 0900 still citedMUTATED_PROJECTSin four places. Fixed in546bf505b.Residual risks (not fixed):
kill-matrix.yml'sprojectsdefaults (:9,:15) are a second copy of the corpus.mutation.ymlcalls the kill matrix with noprojects:input, and no gate checks that the two copies are equal. The test covers onlymutation.yml.mutatedProjectssplitsPROJECTSon,without trimming. A spaced list would fail the scenarios loudly; it would not pass them silently.Overlap: #289 and #271 (Stream K's verdict store) also touch
mutation.yml/kill-matrix.yml. Root's landing order puts this PR first, and those PRs mergemainafter it (plan 0900, landing table).Notes
main. C, B and D follow per the plan, and D cannot merge before this PR.