Repository navigation
fix(e2e): bake each fixture with npm ci from a committed lock - #311
Merged
Merged
Conversation
Requirements-only plan for fix/e2e-locked-fixtures: one npm ci per fixture from a committed package-lock.json, closure entries overlaid from packed manifests, no age gate, timeout or second install. Answers the root P0's four questions with registry, stalled-log and npm ci probe evidence.
Adds the planning contract (KTD1-KTD7), units U1-U4, verification contract and definition of done to the brainstorm plan; R4 copies npm's manifest-sourced lock fields and R9 counts tests and docs, per the root's rulings.
Adds the pure closure-lock functions the bake and the lock script share: filledLockOf copies npm's manifest-sourced lock fields onto each file: tarball entry from its packed manifest, committableLockOf drops the integrity and version a source change or release moves, and withClosureDevDependencies lists the closure on the fixture root. ClosureInstall gains the closure as a name-to-spec record.
The bake no longer resolves: it stages each fixture with the packed closure as root devDependencies and the committed package-lock.json filled from the freshly packed manifests, then runs one npm ci per fixture with no timeout, budget or age gate. A drifted or missing lock fails with npm ci's own output and exit code plus a line naming fixtures:lock. Closure tarballs pack under version-free names (pnpm pack --out), so a release does not move the specs the locks record. The staging path is exported as BakedFixtureCache.stageProgram for the lock script.
A closure entry is filled with version, which npm ci compares against the ideal tree (npm/lib/utils/validate-lockfile.js), and the four fields arborist turns into edges (dependencies, optionalDependencies, peerDependencies, peerDependenciesMeta: @npmcli/arborist/lib/node.js), so a new dependency fails npm ci with Missing ... from lock file. Every other committed field is kept as written.
Covers AE3 (a stale closure entry takes the packed manifest's version and dependencies, other fields kept), AE4's precondition (a dependency the lock lacks lands in the filled entry), and a registry entry passing through unchanged with its integrity.
pnpm --filter @systemfsoftware/stryker-e2e fixtures:lock runs BakedFixtureCache.lockProgram: it stages every fixture the way the bake does, runs npm install --package-lock-only over the filled committed lock and writes it back with each closure tarball's integrity and version stripped. The script is the harness entry point under tsx, which test/e2e now declares. The seven locks are regenerated from #285's, which pin effect, @effect/platform-node, @effect/platform-node-shared and @effect/opentelemetry at 4.0.0, as pnpm-lock.yaml does.
…ill-in The sub-conductor ruling narrows R4 to the fields npm ci checks, the test-discipline lint puts the property suite in-source, the lock loop moves into the harness as lockProgram, and U4 drops the hang lesson.
npm links bins from the lock's package metadata, not from the extracted
tarball (arborist rebuild.js binLinks({ pkg: node.package })), so a
closure member that changed its bin installed with the stale link. The
fill-in now copies bin with the version and edge fields.
README names the drift npm ci reports (a fixture range or a closure edge) and the bin the bake copies from the packed manifest. The lane's AGENTS.md and the alias lesson's guard bullet now describe the npm ci bake instead of the positional-spec install.
ryanleecode
approved these changes
Oct 12, 2026
ryanleecode
left a comment
Contributor
There was a problem hiding this comment.
Verified at 4696b21: all 5 e2e legs, check, checker-parity and Nix green; up to date with main; 0 open threads; ~250 hand-written lines plus 7 committed fixture locks; no skips, disables, caps or force flags. Only red is the bench enterprise leg's setup-timed-out, which fails on every PR and leaves PRs with Stream G's bench-main-only.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 12, 2026
…ke's lock Merging #311 put a committed package-lock.json in the enterprise fixture. The bench copies that directory from side B, so npm install started from the bake's lock: an npm probe kept its 86 registry entries at their locked versions and dropped the 18 closure entries, which point at the bake's ../../packs layout. Versions then came from the e2e lock instead of each side's pnpm-lock overrides and --before cutoff, and withheldPins no longer described the install. The bench now removes the lock from its own copy before installing, so the enterprise install is the one the A/A runs 38140987766 and 38143012318 measured. The fixture, its lock and the bake are untouched.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 12, 2026
Main's bake (#311) runs one npm ci per testResources fixture from its committed package-lock.json and fails the whole bake when one is missing. The diff-scope fixture this branch adds had none, and its .gitignore kept the lock out of git. The lock is seeded from calc-fixture's, which declares the same three devDependencies, and normalized with fixtures:lock, so it pins effect and @effect/platform-node at 4.0.0 like the seven locks on main. A fresh resolution would have floated them to 4.0.3. The fixture's .gitignore no longer lists package-lock.json; the guest's two commits keep the lock unchanged, so the A-to-B diff still holds only src/target.ts. Plan: docs/plans/2026-10-09-1850-feat-mutant-quality-plan.md (L2).
This was referenced Oct 12, 2026
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 12, 2026
…d fixture bakes, LPT planner Takes #303's per-project lockfile closure digest, #311's npm-ci fixture bakes from committed locks, #301's shard planner, and #293's lawful planMutantTests generators. - The incremental cache version moves to 7. Main and this branch each moved it from 5 to 6 for different layouts, so neither may read the other's file. runInputsDigestOf keeps #303's lockfile closure input, and the store key reaches it through the same function. #303's closure files, tests and oracle fixtures are byte-identical to main. - incremental-reuse.integration.test.ts keeps this branch's version: #303's only edit there was the version number in the incrementalSources scenario this branch deletes. - plan-request.schema.ts keeps #301's PlanShardsRefused and drops the two schemas this branch retired. - The planMutantTests law file takes #293's lawfulSubsumptionOf and drops this branch's untestedOf, which fixed the same draw. - The e2e closure install is per fixture, so only fixtures that request the S3 driver install it, and it carries #311's devDependency map. The harness stages each fixture's planned dependencies into its manifest, and the bake runs npm ci. fixtures:lock regenerated the locks: the verdict-store fixture gains its own lock, and the other seven gain the yaml dependency #303 added to the engine. - The docs say a store is shared only between checkouts at the same absolute path, because mutant ids hash the absolute file path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The e2e bake stopped resolving dependencies. Each fixture now installs with one
npm cifrom its committedpackage-lock.json, so an upstream npm publish can no longer stall or change the lane. Main's e2e has been stuck since@effect/platform-node@4.0.3was published: it requires peereffect@^4.0.3, which does not exist, and a fresh resolution of^4.0.0picks it.Plan:
docs/plans/2026-10-11-1203-fix-e2e-locked-fixtures-plan.md.What changes
bake-fixtures.shruns onenpm ci --no-audit --no-fund --loglevel=errorper fixture. The twonpm installcalls,--min-release-age,timeoutand kill wrapper are gone. On failure the script passes npm's output and exit code through unchanged and adds one line namingfixtures:lock.integrityandversion. At bake time the harness copiesversion,bin,dependencies,optionalDependencies,peerDependenciesandpeerDependenciesMetaonto every closure entry from that tarball's freshly packed manifest (filledLockOfin@systemfsoftware/stryker-e2e-core). A source edit or a release therefore installs without regenerating any lock.versionand the edge fields are what npm ci validates (npm/lib/utils/validate-lockfile.js; arborist builds edges from those four fields innode.jsloadDeps).binis there because npm links bins from the lock's metadata, not from the extracted tarball (arborist/lib/arborist/rebuild.jscallsbinLinks({ pkg: node.package })). Without it, a changed bin would install the stale link with exit 0.pnpm pack --out <packs>/%s.tgzwrites tarballs without a version in the file name. The staged fixture manifest lists the whole closure, aliases included, asfile:../../packs/<name>.tgzdevDependencies (ClosureInstall.dependencies).pnpm --filter @systemfsoftware/stryker-e2e fixtures:lock(BakedFixtureCache.lockProgram) stages every fixture the way the bake does, runsnpm install --package-lock-only --ignore-scripts, and writes the lock back with closureintegrityandversionremoved.test/e2e/testResources/*/package-lock.jsonfiles are seeded from fix(repo): resolve the e2e fixture bake against a pinned registry snapshot #285's locks and regenerated by that script. Re-running it leaves all seven byte-identical (checked after thebinchange too).Done (c): the
@effectpins equalpnpm-lock.yamlEach row is
jq -r '[.packages["node_modules/effect"].version, .packages["node_modules/@effect/platform-node"].version, .packages["node_modules/@effect/platform-node-shared"].version]|join(" ")'over that fixture's lock.No closure entry in any lock keeps
integrityorversion.jq '[.packages[]|select((.resolved//"")|startswith("file:"))|select(has("integrity") or has("version"))]|length'prints0for all seven.Done (d): drift fails with npm's own error
This ran in a scratch copy outside the worktree, with no VM.
calc-fixturewas staged the way the bake stages it: catalog specs resolved, closure devDependencies written, lock filled from 17 freshly packed tarballs. The controlnpm ciexited 0 (added 149 packages in 1s), andnode_modules/.bin/strykerpointed to../@systemfsoftware/stryker-js/dist/main.mjs. Each case below then ran the bake script against that directory.Lock deleted:
effectbumped from^4.0.0to^4.0.2in the stagedpackage.json:Both failed in under a second, with exit code 1 from npm.
Why npm hung instead of failing with ETARGET
The cause is unknown. In a scratch install, a packed tarball depending on
@effect/platform-node@^4.0.0, installed with no lock, printed nothing and ignored SIGTERM until it was killed at 125 s. The same edges declared in the root manifest failed withERESOLVEin 2 s.Known gap: regeneration does not pin to
pnpm-lock.yamlThe root ruled that a fixture lock and
pnpm-lock.yamlmust record the same version wherever both resolve a package. This PR does not enforce that yet.fixtures:lockcopies each fixture's directory, committed lock included, into staging (test/e2e/src/Harness/fixture-cache.service.ts:301,fs.copy). It then runsnpm install --package-lock-onlythere (:738,LOCK_ONLY_INSTALLat:114). npm keeps every locked version that still satisfies its range, so the 4.0.0 pins survive only because fix(repo): resolve the e2e fixture bake against a pinned registry snapshot #285's locks carried them.^4.0.0resolves fresh and picks@effect/platform-node@4.0.3, the same release that stalled the bake.pnpm-lock.yamland the seven locks resolve, 53 already differ in version (e.g.@redis/client6.3.0 vs 6.2.1). None of them is one of the@effectpackages above.pnpm-lock.yaml'scatalogs.defaultsection, which records one version per name, and would apply it the same way at bake time sonpm cistill matches. The root has not yet decided whether that lands in this PR (as more commits on this branch) or in a follow-up.Known risks
hasInstallScriptfrom the lock, and the fill-in does not refresh it. None of the 17 packed closure manifests has apreinstall,installorpostinstallscript (scanned withtar -xOzf <tgz> package/package.json), so nothing changes today.committableLockOfhas no unit test. Its output is the seven committed locks; thejqcheck above shows no closure entry keepsintegrityorversion.withClosureDevDependencieshas no unit test. Only the bake covers it, and only indirectly. In the scratch copy, a staged manifest without the closure devDependencies still passednpm ci(exit 0,added 39 packages), because npm prunes the lock's extra closure entries instead of refusing them. The fixture then has nonode_modules/.bin/stryker, so the e2e legs fail at the first CLI run rather than at the bake.npm cihas no bound. The plan and the root ruled out any timeout, kill wrapper or retry. The[bake] <fixture>: npm ciline printed before each install still names the fixture.file:tarball dependency would break. The strip removesintegrityandversionfrom everyfile:*.tgzentry, but only closure tarballs are refilled. None of the seven locks has such an entry.@x/yandx-ywould both pack tox-y.tgz. That cannot happen while every closure member is@systemfsoftware-scoped.filledLockOf,withClosureDevDependencies). A local baked entry can outlive a change to them, as before this PR; CI starts cold.Tests
test/e2e-core/src/closure-lock.ts. It sits in-source because the repo's test-discipline lint rejects asrc/__tests__property file for a module that is not a workflow.∀m_StaleClosureEntry_≡ManifestVersionDependenciesAndBin: a stale closure entry takes the manifest's version, dependencies and bin, and nothing else (the manifest'slicensemust not be copied).∀d_DependencyAbsentFromLock_∈FilledEntry: a dependency the lock lacks lands in the filled entry.∀r_RegistryEntry_≡Unchanged: registry entries pass through unchanged.bin, the stale-entry property was refuted (2 shrinks).pnpm --filter @systemfsoftware/stryker-e2e-core testat670048b92: 45 files, 349 tests pass. After thebinchange the property suite passes 3/3, and typecheck, oxlint and tsgo lint are clean for e2e-core. Typecheck and lint are also clean for e2e.pnpm format:checkexits 0.Size
git diff --numstat origin/main...HEAD, excludingdocs/plansand thepackage-lock.jsonfiles. 254 hand-written lines; the 3pnpm-lock.yamllines are generated.docs/solutions/build-errors/e2e-bake-resolved-an-npm-alias-edge-from-the-registry.mddocs/solutions/build-errors/e2e-lane-packed-a-subset-of-its-workspace-closure.mdpnpm-lock.yaml(generatedtsxedge)test/e2e-core/src/closure-lock.tstest/e2e-core/src/install-closure.workflow.tstest/e2e-core/src/mod.tstest/e2e/AGENTS.mdtest/e2e/README.mdtest/e2e/package.jsontest/e2e/scripts/lock-fixtures.tstest/e2e/src/Harness/bake-key.schema.tstest/e2e/src/Harness/fixture-cache.service.tstest/e2e/tests/__fixtures__/bake-fixtures.shReview notes
bdff834ea.4696b2199.ci/e2e-postsubmit's edits tobake-fixtures.shand the README's install paragraph.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Compound pack check (report-only, ruled by the sub-conductor)
Against the changed files: 7 rules pass, 2 fail, 22 don't apply. Both failures are declared here, not fixed:
rich-type-over-foreign-encoded:test/e2e-core/src/closure-lock.tsreads npm lock fields as plain JSON with no decoded type. This is deliberate: the unit's contract rules out npm wire schemas (plan KTD1).pin-dependency-semantics: the tarball naming ofpnpm pack --out(fixture-cache.service.ts:126) and the lock fieldsnpm cireads (closure-lock.ts) have no dedicated test that runs pnpm or npm. Two things exercise them for real instead: the five e2e legs, which pack the closure and runnpm ciagainst the filled locks, and the Done (d) scratch runs above. A pnpm or npm upgrade that changes either behaviour fails an e2e leg. No extra test was added, to stay within the line limit.Simplify pass: done with reason. Two reviewers reported suggestions that kept or added lines, and the third was stopped. The code review was report-only and lighter than the full reviewer set.