Repository navigation
feat(repo): reuse CompileError verdicts keyed on the checker's whole program - #228
Merged
Merged
Conversation
…l file (#155) A dry-run-only run with incremental mode on now writes its dry-run coverage into the incremental file, keeping any verdicts already there, so a later run that reads the file through incrementalSources skips its own dry run. The run-inputs digest no longer covers dryRunOnly, so a dry-run-only preflight and the runs that reuse its coverage share one digest; existing caches are refused once with runInputsChanged. (cherry picked from eafe8f3) Verdict-Semantics: unchanged
The TypeScript checker reports the digest of the program it loaded: every source file the program holds, hashed, the tsconfig files it was built from, the TypeScript version, the checker plugin version and the checker's options. The engine stamps that digest beside every CompileError verdict it records, and a later incremental run reuses such a verdict only when the digest it recomputes is byte-equal. A missing digest, a changed file anywhere in the program, a changed tsconfig, toolchain or option all send the mutant back to the checker. Hashing the program needs the checker running, so the saving is the per-mutant check Holding the digest means a plugin interface that can answer `digest`, and a `programChanged` refusal reason on the reuse stream line. Verdicts the engine did not remember keep their previous keys and refusals Verdict-Semantics: unchanged
The reuse stream's refused counts gained a `programChanged` member, and the generated contract document listed it as required, so a consumer built against the committed stream would have refused every line an earlier engine wrote. The count is optional, the way the plan line's `projects` array is: a stream line without it still decodes Verdict-Semantics: unchanged
Local builds render the Plugin namespace export as Node and CI renders Node_2 as Node (docs/solutions/api-extractor-node-alias-nondeterminism.md). #227 committed the local rendering and api:check failed in run 37626449206; main's rendering is the one CI accepts Verdict-Semantics: unchanged
Every checker worker must answer the new digest RPC, so the plugin interface and the engine take a major bump per BREAK-1 Verdict-Semantics: unchanged
… CI and local builds Run 37626449206 failed api:check on a report regenerated locally; run 37649609332 passed with main's rendering Verdict-Semantics: unchanged
The digest covered the root tsconfig and its project references but not the configs reached through "extends", so editing an extended base left the program key unchanged and CompileError verdicts were reused across a real program change. The chain is now followed the way TypeScript resolves it - relative, absolute and node_modules package specifiers, string or array form - and every file in it is hashed. An unresolvable or unparseable config fails the digest, which forces a re-check. The engine-level scenario needs a fixture checker that digests the same chain, which the program-digesting fixture now does Verdict-Semantics: unchanged
Every digest entry was an absolute path, so the checker's sandbox directory (.stryker-tmp/sandbox-<random>) put a different key on the same program on every run and CompileError verdicts were never reused on CI. Paths are now named relative to the root tsconfig's directory; files outside it, such as node_modules realpaths, become deterministic "../" paths Verdict-Semantics: unchanged
The options JSON was re-encoded in insertion order, so semantically equal
option sets produced different keys and never matched; and a value that is not
JSON was silently replaced with {}, keying a configuration the checker never
ran. Object keys are now sorted before hashing and an unencodable option set
fails the digest, which forces a re-check
Verdict-Semantics: unchanged
A refusal caused only by a failed closure analysis was reported as closureChanged, and a keyed CompileError whose program still matched was reported the same way - neither names the input that was actually missing. The stream now carries a closureAnalysisFailed count, and that reason is chosen whenever the closure analysis failed and no earlier gate outranks it Verdict-Semantics: unchanged
cacheKeyOf and programKeyOf differed only in the digest they read, so one keyOf now serves both. readTypescriptVersion and readCheckerVersion share one reader whose failure returns no version and so no digest. The program digest borrows the existing sha256 and optional-field helpers instead of local copies, and reads program files with bounded concurrency Verdict-Semantics: unchanged
The reuse-refusal objects these scenarios compare gained a closureAnalysisFailed count, so each zero-refusal fixture now names it Verdict-Semantics: unchanged
The published stream schema gains the optional closureAnalysisFailed count that the reuse report now emits alongside the other refusal counts Verdict-Semantics: unchanged
The digest helpers now branch through Option and Boolean match, so no function exceeds the complexity budget, the key-order canonicaliser sorts schema keys without assertions, and the closure walker resolves extends candidates without a loop. The optional-field builder moves to the reuse module as a dual export, which keeps it out of a cell file and off the pipeable-signature rule Verdict-Semantics: unchanged
systemfsoftware-maker
added this pull request to stack #229
October 7, 2026 17:17
…ners Run 37664668214 timed out e2e sabotage and rest-1 at 19m59s with no failing test, on a tree identical to d82f758, which passed in run 37658299516 Verdict-Semantics: unchanged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This layer goes on top of #227. An incremental run now reuses
CompileErrorverdicts instead of type-checking every one again. On main,CompileErroris an ephemeral status (Mutant.schema.ts:25), so every run re-checks 3,888 mutants: 3,547 in stryker-js, 238 in typescript-checker and 103 in vitest-runner. That re-check is the largest fixed cost per shard; run 37622289361 ondb180fc3, with nothing to re-test, still spent about 100 s per shard on stryker-js.Reuse key (operator rule, cycle 116)
A remembered
CompileErroris reused only when the reuse key matches. The key covers every input of the type check:Program.getSourceFiles()). That includes lib.d.tsfiles andnode_modulesdeclarations, sorted by normalised path so enumeration order cannot matter;The checker computes this digest and serves it through a new
digestRPC. The engine stamps it beside eachCompileErrorit writes. A missing or different digest means the mutant is re-checked, and the refusal reason isprogramChanged. Hashing the current program needs the checker running, so this saves the per-mutant checks, not checker startup.Stream
ReuseReported.reusedcounts reusedCompileErrorverdicts.ReuseRefusalsgainsprogramChanged, added as an optional key so stream lines from earlier engines still decode. The schema version is unchanged.Breaking (BREAK-1)
Every checker worker must answer the
digestRPC. The plugin interface and stryker-js therefore take major bumps; the TypeScript checker and the cli-contract take minor bumps.Validation
packages/stryker-js/tests/compile-error-reuse.integration.test.tswas run in a scratch worktree on origin/main and failed 4 of 4 scenarios. For example, "an unchanged rerun reuses the CompileError verdicts and checks nothing" gotsecondRanNothing: 4(expected 0) andsecondReusedAll: false(expected true).compile-error-reuse.integration.test.ts): an unchanged rerun reuses every verdict with zero checks; editing a transitively imported.d.tsre-checks; changing tsconfig re-checks; editing an unrelated test file keeps every verdict.program-digest.integration.test.ts): runs the same cases against the checker's program digest.identify-program.workflow.property.test.tschecks that a permuted file set gives the same key, and that changing the TS version, checker version, options, an added file, or the tsconfig moves the key.check:ci(112/112, 17/17) pass, the stryker-js and typescript-checker suites pass, and the changeset check passes with intents for all 4 packages.Verdict-Semantics: unchanged: a verdict is reused only when the inputs equal those of the check that produced it, so statuses equal a from-scratch run (EXACT).Review (correctness, adversarial and simplify lenses, before opening)
Fixed, each with a test that fails before the fix:
extendschain, both relative paths and package specifiers, is now hashed into the digest. The checker and engine suites each gain a test that edits only the base config and expects a re-check..stryker-tmp/sandbox-<random>/, so with absolute paths the key would never match on CI. A new test copies the same fixture into two directories and gets the same digest.Dismissed:
@noble/hashes. It is the repo convention, and the PLUG-1 build gate passes.Nothing was run with stryker.
Borrowed / rejected
.d.tsandnode_modules, so that key would be too narrow.