diff --git a/docs/solutions/build-errors/e2e-bake-resolved-an-npm-alias-edge-from-the-registry.md b/docs/solutions/build-errors/e2e-bake-resolved-an-npm-alias-edge-from-the-registry.md new file mode 100644 index 000000000..c5a0d829f --- /dev/null +++ b/docs/solutions/build-errors/e2e-bake-resolved-an-npm-alias-edge-from-the-registry.md @@ -0,0 +1,77 @@ +--- +title: The e2e bake resolved an npm-alias edge from the registry, so the release PR failed with ETARGET +date: 2026-10-08 +category: build-errors +module: e2e-lane +problem_type: build_error +component: tooling +severity: high +symptoms: + - "Every e2e job on the release PR fails in the fixture bake with `npm error notarget No matching version found for @systemfsoftware/stryker-js-vitest-runner@^9.0.0`" + - "The runner tarball carrying exactly that version is in the same `npm install`" + - "e2e on `main` passes, because the runner's version there is also the registry's latest" +root_cause: wrong_api +resolution_type: code_fix +tags: [e2e-lane, pnpm-pack, workspace-protocol, npm-alias, npm-install, etarget, tarball-closure] +--- + +# The e2e bake resolved an npm-alias edge from the registry + +## Problem & Observable Boundary + +The `@systemfsoftware/stryker-js` manifest names the runner behind `testRunner: 'vm'` through a workspace alias, `"@systemfsoftware/stryker-js-vm-runner": "workspace:@systemfsoftware/stryker-js-vitest-runner@^"`. It needs the alias because the same manifest lists `@systemfsoftware/stryker-js-vitest-runner` as a `catalog:stryker` devDependency (the dogfood pin, START-6). The bake installed `npm install /packs/*.tgz`. On `main`, npm downloaded the published runner into the alias slot and the bake copied the packed runner over it. On the release commit the runner version was ahead of the registry, and the install failed with ETARGET before the copy. + +The closure was complete: the runner is an entry package, so its tarball was always in the install. The defect was in how the install consumed the closure. + +## Mechanism & Failure Modes + +Let $T$ be the set of packed tarballs handed to one `npm install`, and let $e = (a \to \texttt{npm:}t\texttt{@}r)$ be an edge of a member, where $a$ is the alias name and $t$ is a closure member. + +1. **Pack-time rewrite.** `pnpm pack` performs $\rho: \texttt{workspace:}t\texttt{@\^{}} \mapsto \texttt{npm:}t\texttt{@\^{}}v_t$. +2. **Name-keyed placement.** npm places a tarball for $t$ at `node_modules/t`. Edge $e$ needs `node_modules/a`. Since $a \neq t$, no member of $T$ fills that slot, and npm resolves $\texttt{npm:}t\texttt{@}r$ against the registry $R$. +3. **Failure condition.** $v_t \notin R \Rightarrow$ ETARGET. When $v_t \in R$ the install succeeds, but the alias slot holds the published $t$, not the packed one. Nothing fails, so the defect is silent until a release. + +## Architectural Invariants + +### I1: Every closure edge is satisfied by a local spec + +_For every edge npm installs for a packed member (`dependencies`, `optionalDependencies`, and `peerDependencies` not marked optional in `peerDependenciesMeta`) whose target, by name or through `npm:`, is a closure member, the same `npm install` carries a spec that fills that edge's slot from the member's tarball._ + +```ts +specs = [...tarballs, ...aliasEdges.map((e) => `${e.alias}@file:${tarballOf(e.target)}`)] +``` + +npm places `alias@file:` at `node_modules/alias` and records the tarball's real `name`. The dependent's `npm:` edge then validates against it with no registry request. + +### I2: A workspace target outside the plan refuses it + +_The plan fails global setup, naming the edge, when:_ + +- _a packed member's installed edge targets a workspace package with no tarball in the closure (`UnpackedWorkspaceDependency`);_ +- _two edges give one alias name different targets, or an alias name is also a member's own name (`ConflictingAliasTargets`), because npm keeps whichever spec comes last;_ +- _a fixture manifest (`dependencies`, `devDependencies`, required peers, `optionalDependencies`) names a workspace package (`FixtureNamesWorkspacePackage`), because the fixture's own `npm install` runs before the closure specs and resolves that edge from the registry._ + +A registry fallback for a workspace package is never a valid outcome. An optional peer npm will not install is not an edge. + +### I3: Tarball lookup is exact + +_A member's tarball is `--.tgz`, not any file that starts with `--`._ With prefix matching, `@systemfsoftware/stryker-js` could select `systemfsoftware-stryker-js-vitest-runner-*.tgz`. That would drop the CLI's own tarball from the bake cache key, and now that the install is planned from the looked-up tarballs, from the install too. + +`installClosure` and its refusals in `@systemfsoftware/stryker-e2e-core` implement I1 and I2. The harness plans the install only when a fixture needs baking, and passes the plan to `bake-fixtures.sh` as arguments. The step that copied the packed runner over the alias directory is deleted. + +## Anti-Pattern Code Smells + +- **Glob install with an alias in the closure.** `npm install /packs/*.tgz` assumes every edge is name-keyed. Any `npm:` spec in a packed manifest breaks that assumption. +- **Post-install repair.** Copying a package over a directory npm already filled from the registry hides the registry dependency and leaves the lockfile lying about provenance. +- **Prefix tarball lookup.** `startsWith(prefix) && endsWith('.tgz')` across sibling packages whose names share a prefix. + +## Verification & Prevention + +- `install-closure.integration.test.ts` in `@systemfsoftware/stryker-e2e-core` runs under `pnpm test`. It packs a closure with the production alias shape and installs the plan with `npm install --package-lock-only --offline`, an empty cache and an unreachable registry. It then requires `file:` provenance for every workspace package in the lockfile. Any registry fallback for a closure member fails the install. Its other scenarios cover each I2 refusal and the optional-peer exemption. +- Two-sided check (scratch evidence, not a committed fixture): the same offline install given the tarball list alone exits `ENOTCACHED` for `@systemfsoftware/stryker-js-vitest-runner`, and given the plan it exits 0. + +## Related + +- `docs/solutions/build-errors/e2e-lane-packed-a-subset-of-its-workspace-closure.md`: the closure-completeness fix this one extends to alias edges +- `docs/plans/2026-09-25-0809-refactor-vm-runner-runs-vitest-plan.md`: why the CLI names the runner through an alias +- Failing run: https://github.com/systemfsoftware/stryker-js-effect/actions/runs/37744072050 diff --git a/test/e2e-core/src/install-closure.schema.ts b/test/e2e-core/src/install-closure.schema.ts new file mode 100644 index 000000000..921078835 --- /dev/null +++ b/test/e2e-core/src/install-closure.schema.ts @@ -0,0 +1,29 @@ +import * as S from 'effect/Schema' + +const EdgeSpecs = S.optional(S.Record(S.String, S.String)) + +const PeerDependenciesMeta = S.optional(S.Record(S.String, S.Struct({ optional: S.optional(S.Boolean) }))) + +export const PackedManifest = S.Struct({ + name: S.String, + dependencies: EdgeSpecs, + peerDependencies: EdgeSpecs, + peerDependenciesMeta: PeerDependenciesMeta, + optionalDependencies: EdgeSpecs, +}) +export type PackedManifest = typeof PackedManifest.Type + +export const PackedMember = S.Struct({ tarballPath: S.String, manifest: PackedManifest }) +export type PackedMember = typeof PackedMember.Type + +export const FixtureManifest = S.Struct({ + dependencies: EdgeSpecs, + devDependencies: EdgeSpecs, + peerDependencies: EdgeSpecs, + peerDependenciesMeta: PeerDependenciesMeta, + optionalDependencies: EdgeSpecs, +}) +export type FixtureManifest = typeof FixtureManifest.Type + +export const StagedFixtureManifest = S.Struct({ path: S.String, manifest: FixtureManifest }) +export type StagedFixtureManifest = typeof StagedFixtureManifest.Type diff --git a/test/e2e-core/src/install-closure.workflow.ts b/test/e2e-core/src/install-closure.workflow.ts new file mode 100644 index 000000000..d1341e598 --- /dev/null +++ b/test/e2e-core/src/install-closure.workflow.ts @@ -0,0 +1,216 @@ +import { Workflow } from '@systemfsoftware/effect-cell-types' +import * as Arr from 'effect/Array' +import * as Boolean from 'effect/Boolean' +import { pipe } from 'effect/Function' +import * as Option from 'effect/Option' +import * as Rec from 'effect/Record' +import * as Result from 'effect/Result' +import * as S from 'effect/Schema' + +import { type PackedManifest, PackedMember, StagedFixtureManifest } from './install-closure.schema.js' + +export class InstallClosureCommand extends S.TaggedClass()('InstallClosureCommand', { + members: S.Array(PackedMember), + fixtures: S.Array(StagedFixtureManifest), + workspace: S.Array(S.String), +}) { + static readonly [Workflow.InstrumentationBrand] = {} as const +} + +const ClosureInstallTypeId: unique symbol = Symbol.for('@systemfsoftware/stryker-e2e-core/ClosureInstall') +type ClosureInstallTypeId = typeof ClosureInstallTypeId + +export class ClosureInstall extends S.TaggedClass()('ClosureInstall', { + specs: S.Array(S.String), +}) { + readonly [ClosureInstallTypeId] = ClosureInstallTypeId +} + +export class UnpackedWorkspaceDependency extends S.TaggedError()( + 'UnpackedWorkspaceDependency', + { + dependent: S.String, + dependency: S.String, + target: S.String, + }, +) { + override get message(): string { + return `${this.dependent} depends on the workspace package ${this.target} as "${this.dependency}", but the packed closure carries no tarball for it, so npm would resolve it from the registry` + } +} + +export class ConflictingAliasTargets extends S.TaggedError()('ConflictingAliasTargets', { + dependency: S.String, + targets: S.Array(S.String), +}) { + override get message(): string { + return `the packed closure installs "${this.dependency}" as more than one package (${ + this.targets.join(', ') + }); npm keeps whichever spec comes last, so every other dependent would get the wrong package` + } +} + +export class FixtureNamesWorkspacePackage extends S.TaggedError()( + 'FixtureNamesWorkspacePackage', + { + fixture: S.String, + dependency: S.String, + target: S.String, + }, +) { + override get message(): string { + return `${this.fixture} depends on the workspace package ${this.target} as "${this.dependency}", but the fixture's own npm install resolves it from the registry; the bake installs the packed closure into every fixture, so drop the edge` + } +} + +export const InstallClosureFailure = S.Union([ + UnpackedWorkspaceDependency, + ConflictingAliasTargets, + FixtureNamesWorkspacePackage, +]) +export type InstallClosureFailure = typeof InstallClosureFailure.Type + +interface ClosureEdge { + readonly dependent: string + readonly dependency: string + readonly target: string +} + +type EdgeSpecs = Readonly> + +type PeerEdges = Pick + +const NPM_ALIAS = /^npm:((?:@[^/@]+\/)?[^/@]+)(?:@.*)?$/ + +const NO_EDGES: EdgeSpecs = {} + +const specsOf = (specs: EdgeSpecs | undefined): EdgeSpecs => + Option.getOrElse(Option.fromUndefinedOr(specs), () => NO_EDGES) + +const targetOf = (dependency: string, spec: string): string => + Option.getOrElse( + Option.flatMap(Option.fromNullishOr(NPM_ALIAS.exec(spec)), (match) => Arr.get(match, 1)), + () => dependency, + ) + +const optionalPeer = (manifest: PeerEdges, dependency: string): boolean => + pipe( + Option.fromUndefinedOr(manifest.peerDependenciesMeta), + Option.flatMap((meta) => Rec.get(meta, dependency)), + Option.flatMap((entry) => Option.fromUndefinedOr(entry.optional)), + Option.getOrElse(() => false), + ) + +const installedPeers = (manifest: PeerEdges): EdgeSpecs => + Rec.filter(specsOf(manifest.peerDependencies), (_, dependency) => Boolean.not(optionalPeer(manifest, dependency))) + +const edgesIn = (dependent: string, fields: ReadonlyArray): ReadonlyArray => + fields.flatMap((specs) => + Object.entries(specs).map(([dependency, spec]) => ({ dependent, dependency, target: targetOf(dependency, spec) })) + ) + +const memberEdgesOf = (manifest: PackedManifest): ReadonlyArray => + edgesIn(manifest.name, [ + specsOf(manifest.dependencies), + installedPeers(manifest), + specsOf(manifest.optionalDependencies), + ]) + +const fixtureEdgesOf = (fixture: StagedFixtureManifest): ReadonlyArray => + edgesIn(fixture.path, [ + specsOf(fixture.manifest.dependencies), + specsOf(fixture.manifest.devDependencies), + installedPeers(fixture.manifest), + specsOf(fixture.manifest.optionalDependencies), + ]) + +const tarballOf = (command: InstallClosureCommand, name: string): Option.Option => + Option.map( + Arr.findFirst(command.members, (member) => member.manifest.name === name), + (member) => member.tarballPath, + ) + +const unpacked = (command: InstallClosureCommand, edge: ClosureEdge): boolean => + Boolean.and(command.workspace.includes(edge.target), Option.isNone(tarballOf(command, edge.target))) + +const isAlias = (edge: ClosureEdge): boolean => edge.target !== edge.dependency + +const targetsOf = ( + command: InstallClosureCommand, + aliases: ReadonlyArray, + dependency: string, +): ReadonlyArray => + Arr.dedupe([ + ...aliases.filter((edge) => edge.dependency === dependency).map((edge) => edge.target), + ...Option.toArray(Option.map(tarballOf(command, dependency), () => dependency)), + ]).sort() + +const fixtureRefusal = (command: InstallClosureCommand): Result.Result => + Option.match( + Arr.findFirst(command.fixtures.flatMap(fixtureEdgesOf), (edge) => command.workspace.includes(edge.target)), + { + onNone: () => Result.succeed(undefined), + onSome: (edge) => + Result.fail( + FixtureNamesWorkspacePackage.make({ + fixture: edge.dependent, + dependency: edge.dependency, + target: edge.target, + }), + ), + }, + ) + +const unpackedRefusal = ( + command: InstallClosureCommand, + edges: ReadonlyArray, +): Result.Result => + Option.match(Arr.findFirst(edges, (edge) => unpacked(command, edge)), { + onNone: () => Result.succeed(undefined), + onSome: (edge) => Result.fail(UnpackedWorkspaceDependency.make(edge)), + }) + +const conflictRefusal = ( + command: InstallClosureCommand, + aliases: ReadonlyArray, +): Result.Result => + Option.match( + Arr.findFirst( + Arr.dedupe(aliases.map((edge) => edge.dependency)), + (dependency) => targetsOf(command, aliases, dependency).length > 1, + ), + { + onNone: () => Result.succeed(undefined), + onSome: (dependency) => + Result.fail(ConflictingAliasTargets.make({ dependency, targets: targetsOf(command, aliases, dependency) })), + }, + ) + +const aliasSpecOf = (command: InstallClosureCommand, edge: ClosureEdge): ReadonlyArray => + Option.toArray(Option.map(tarballOf(command, edge.target), (path) => `${edge.dependency}@file:${path}`)) + +const installOf = (command: InstallClosureCommand, aliases: ReadonlyArray): ClosureInstall => + ClosureInstall.make({ + specs: [ + ...command.members.map((member) => member.tarballPath), + ...Arr.dedupe(aliases.flatMap((edge) => aliasSpecOf(command, edge))).sort(), + ], + }) + +const decide = (command: InstallClosureCommand): Result.Result => { + const edges = command.members.flatMap((member) => memberEdgesOf(member.manifest)) + const aliases = edges.filter(isAlias) + return pipe( + fixtureRefusal(command), + Result.flatMap((): Result.Result => unpackedRefusal(command, edges)), + Result.flatMap((): Result.Result => conflictRefusal(command, aliases)), + Result.map(() => installOf(command, aliases)), + ) +} + +export const installClosure = Workflow.make({ + command: InstallClosureCommand, + decision: ClosureInstall, + error: InstallClosureFailure, + decide, +}) diff --git a/test/e2e-core/src/mod.ts b/test/e2e-core/src/mod.ts index cce0ba6ab..d17590aa7 100644 --- a/test/e2e-core/src/mod.ts +++ b/test/e2e-core/src/mod.ts @@ -4,6 +4,8 @@ export * from './closure.schema.js' export * from './confirm-annotations.workflow.js' export * from './decode-report.workflow.js' export * from './decode-stream.workflow.js' +export * from './install-closure.schema.js' +export * from './install-closure.workflow.js' export * from './key-material.js' export * from './match-annotations.workflow.js' export * from './match.schema.js' diff --git a/test/e2e-core/tests/__fixtures__/npm-closure.schema.ts b/test/e2e-core/tests/__fixtures__/npm-closure.schema.ts new file mode 100644 index 000000000..2e90c9190 --- /dev/null +++ b/test/e2e-core/tests/__fixtures__/npm-closure.schema.ts @@ -0,0 +1,14 @@ +import * as S from 'effect/Schema' + +export const NpmManifestJson = S.fromJsonString(S.Struct({ + name: S.String, + version: S.String, + private: S.optional(S.Boolean), + dependencies: S.optional(S.Record(S.String, S.String)), +})) + +export const NpmLockEntry = S.Struct({ name: S.optional(S.String), resolved: S.optional(S.String) }) +export type NpmLockEntry = typeof NpmLockEntry.Type + +export const NpmLockfileJson = S.fromJsonString(S.Struct({ packages: S.Record(S.String, NpmLockEntry) })) +export type NpmLockfile = typeof NpmLockfileJson.Type diff --git a/test/e2e-core/tests/install-closure.integration.test.ts b/test/e2e-core/tests/install-closure.integration.test.ts new file mode 100644 index 000000000..ad2c2fe5d --- /dev/null +++ b/test/e2e-core/tests/install-closure.integration.test.ts @@ -0,0 +1,285 @@ +import { NodeFileSystem, NodePath } from '@effect/platform-node' +import * as NodeChildProcessSpawner from '@effect/platform-node/NodeChildProcessSpawner' +import { Gherkin, Given, it, makeFeature, Then, When } from '@systemfsoftware/effect-gherkin-spec' +import * as Effect from 'effect/Effect' +import * as FileSystem from 'effect/FileSystem' +import * as Layer from 'effect/Layer' +import * as Path from 'effect/Path' +import { ChildProcess, ChildProcessSpawner } from 'effect/process' +import * as Result from 'effect/Result' +import * as S from 'effect/Schema' +import * as Stream from 'effect/Stream' + +import { + installClosure, + InstallClosureCommand, + InstallClosureFailure, + type PackedManifest, + type PackedMember, + type StagedFixtureManifest, +} from '@systemfsoftware/stryker-e2e-core' +import { type NpmLockfile, NpmLockfileJson, NpmManifestJson } from './__fixtures__/npm-closure.schema.js' + +const Feature = makeFeature({ it }) + +const FILE_AND_PATH = Layer.mergeAll(NodeFileSystem.layer, NodePath.layer) + +const PORTS = Layer.mergeAll(FILE_AND_PATH, NodeChildProcessSpawner.layer.pipe(Layer.provide(FILE_AND_PATH))) + +const CLI = '@systemfsoftware/stryker-js' +const RUNNER = '@systemfsoftware/stryker-js-vitest-runner' +const RUNNER_ALIAS = '@systemfsoftware/stryker-js-vm-runner' +const PLUGIN_INTERFACE = '@systemfsoftware/stryker-js-plugin-interface' +const CHECKER = '@systemfsoftware/stryker-js-typescript-checker' + +const WORKSPACE = [CLI, RUNNER, PLUGIN_INTERFACE, CHECKER] + +const VERSION_ABSENT_FROM_THE_REGISTRY = '9999.0.0' + +const manifestOf = (name: string, dependencies: Record): PackedManifest => ({ + name, + dependencies, +}) + +const CLOSURE: ReadonlyArray = [ + manifestOf(CLI, { + [RUNNER_ALIAS]: `npm:${RUNNER}@^${VERSION_ABSENT_FROM_THE_REGISTRY}`, + [PLUGIN_INTERFACE]: `^${VERSION_ABSENT_FROM_THE_REGISTRY}`, + }), + manifestOf(RUNNER, {}), + manifestOf(PLUGIN_INTERFACE, {}), +] + +const memberOf = (manifest: PackedManifest): PackedMember => ({ tarballPath: `/packs/${manifest.name}.tgz`, manifest }) + +const refusalOf = (members: ReadonlyArray, fixtures: ReadonlyArray) => + Effect.gen(function*() { + const planned = installClosure(InstallClosureCommand.make({ members, fixtures, workspace: WORKSPACE })) + return Result.isFailure(planned) ? yield* S.encodeEffect(InstallClosureFailure)(planned.failure) : null + }) + +interface CommandOutcome { + readonly exitCode: number + readonly stdout: string + readonly stderr: string +} + +const runCommand = (argv: readonly [string, ...Array], cwd: string) => + Effect.scoped(Effect.gen(function*() { + const spawner = yield* ChildProcessSpawner.ChildProcessSpawner + const [command, ...args] = argv + const handle = yield* spawner.spawn(ChildProcess.make(command, args, { cwd })) + const [stdout, stderr, exitCode] = yield* Effect.all( + [ + handle.stdout.pipe(Stream.decodeText, Stream.runCollect), + handle.stderr.pipe(Stream.decodeText, Stream.runCollect), + handle.exitCode, + ] as const, + { concurrency: 'unbounded' }, + ) + return { exitCode, stdout: stdout.join(''), stderr: stderr.join('') } satisfies CommandOutcome + })) + +const packMember = (root: string, manifest: PackedManifest) => + Effect.gen(function*() { + const fs = yield* FileSystem.FileSystem + const path = yield* Path.Path + const source = path.join(root, 'sources', manifest.name) + const packs = path.join(root, 'packs') + yield* fs.makeDirectory(source, { recursive: true }) + yield* fs.makeDirectory(packs, { recursive: true }) + const packageJson = yield* S.encodeEffect(NpmManifestJson)({ + name: manifest.name, + version: VERSION_ABSENT_FROM_THE_REGISTRY, + dependencies: { ...manifest.dependencies }, + }) + yield* fs.writeFileString(path.join(source, 'package.json'), packageJson) + const packed = yield* runCommand(['npm', 'pack', '--pack-destination', packs], source) + const fileName = packed.stdout.trim().split('\n').at(-1) ?? '' + return { tarballPath: path.join(packs, fileName), manifest } satisfies PackedMember + }) + +interface LockProvenance { + readonly name: string + readonly resolved: string +} + +const provenanceOf = (lockfile: NpmLockfile): ReadonlyArray => + Object.entries(lockfile.packages) + .filter(([location]) => location !== '') + .map(([location, entry]) => ({ + name: entry.name ?? location.slice(location.lastIndexOf('node_modules/') + 'node_modules/'.length), + resolved: entry.resolved ?? '', + })) + +interface InstallOutcome { + readonly exitCode: number + readonly stderr: string + readonly provenance: ReadonlyArray +} + +const installOffline = (manifests: ReadonlyArray) => + Effect.scoped(Effect.gen(function*() { + const fs = yield* FileSystem.FileSystem + const path = yield* Path.Path + const root = yield* fs.makeTempDirectoryScoped({ prefix: 'install-closure-' }) + const members = yield* Effect.forEach(manifests, (manifest) => packMember(root, manifest)) + const install = yield* Effect.fromResult( + installClosure(InstallClosureCommand.make({ members, fixtures: [], workspace: WORKSPACE })), + ) + const fixture = path.join(root, 'fixture') + yield* fs.makeDirectory(fixture) + const fixtureJson = yield* S.encodeEffect(NpmManifestJson)({ name: 'fixture', version: '0.0.0', private: true }) + yield* fs.writeFileString(path.join(fixture, 'package.json'), fixtureJson) + const outcome = yield* runCommand( + [ + 'npm', + 'install', + '--package-lock-only', + '--offline', + '--ignore-scripts', + '--no-audit', + '--no-fund', + `--cache=${path.join(root, 'npm-cache')}`, + `--userconfig=${path.join(root, 'npmrc')}`, + '--registry=http://127.0.0.1:9/', + ...install.specs, + ], + fixture, + ) + const lock = yield* fs.readFileString(path.join(fixture, 'package-lock.json')).pipe( + Effect.flatMap(S.decodeEffect(NpmLockfileJson)), + Effect.map(provenanceOf), + Effect.orElseSucceed((): ReadonlyArray => []), + ) + return { exitCode: outcome.exitCode, stderr: outcome.stderr, provenance: lock } satisfies InstallOutcome + })) + +const registryResolvedWorkspacePackages = (outcome: InstallOutcome): ReadonlyArray => + outcome.provenance + .filter((entry) => WORKSPACE.includes(entry.name) && !entry.resolved.startsWith('file:')) + .map((entry) => entry.name) + +const workspaceProvenance = (outcome: InstallOutcome): ReadonlyArray => + outcome.provenance + .filter((entry) => WORKSPACE.includes(entry.name)) + .map((entry) => entry.name) + .sort() + +Feature('Installing the packed workspace closure') + .withLayer(PORTS) + .live('npm packs a closure and installs the planned specs offline with an empty cache and an unreachable registry') + .body(({ scenario }) => { + scenario( + 'A member that names a sibling through an npm alias installs that sibling from its packed tarball', + Gherkin.Do.pipe( + Given('a closure whose CLI names the runner through an npm alias and the plugin interface by name')( + 'closure', + () => Effect.succeed(CLOSURE), + ), + When('npm installs the planned closure')('installed', (s) => installOffline(s.closure)), + Then('every workspace package in the lockfile, the aliased runner included, resolves from a packed tarball')( + (s, expect) => + expect({ + exitCode: s.installed.exitCode, + stderr: s.installed.exitCode === 0 ? '' : s.installed.stderr, + registryResolved: registryResolvedWorkspacePackages(s.installed), + installed: workspaceProvenance(s.installed), + }).toStrictEqual({ + exitCode: 0, + stderr: '', + registryResolved: [], + installed: [CLI, PLUGIN_INTERFACE, RUNNER, RUNNER].sort(), + }), + ), + ), + ) + + scenario( + 'A member that names a workspace package the closure did not pack refuses the install, naming the edge', + Gherkin.Do.pipe( + Given('the closure without the runner tarball')( + 'members', + () => Effect.succeed(CLOSURE.filter((manifest) => manifest.name !== RUNNER).map(memberOf)), + ), + When('the closure install is planned')('refused', (s) => refusalOf(s.members, [])), + Then('the plan names the CLI, its alias, and the unpacked runner')((s, expect) => + expect(s.refused).toStrictEqual({ + _tag: 'UnpackedWorkspaceDependency', + dependent: CLI, + dependency: RUNNER_ALIAS, + target: RUNNER, + }) + ), + ), + ) + + scenario( + 'Two members that alias one name to different workspace packages refuse the install, naming both targets', + Gherkin.Do.pipe( + Given('a closure where the plugin interface aliases the runner name to the checker')( + 'members', + () => + Effect.succeed([ + ...CLOSURE.filter((manifest) => manifest.name !== PLUGIN_INTERFACE), + manifestOf(PLUGIN_INTERFACE, { [RUNNER_ALIAS]: `npm:${CHECKER}@^${VERSION_ABSENT_FROM_THE_REGISTRY}` }), + manifestOf(CHECKER, {}), + ].map(memberOf)), + ), + When('the closure install is planned')('refused', (s) => refusalOf(s.members, [])), + Then('the plan names the alias and both packages it would install as')((s, expect) => + expect(s.refused).toStrictEqual({ + _tag: 'ConflictingAliasTargets', + dependency: RUNNER_ALIAS, + targets: [CHECKER, RUNNER].sort(), + }) + ), + ), + ) + + scenario( + 'A fixture that names a workspace package refuses the install, naming the fixture manifest', + Gherkin.Do.pipe( + Given('a fixture whose manifest names the CLI as a devDependency')( + 'fixtures', + () => + Effect.succeed>([ + { + path: 'calc-fixture/package.json', + manifest: { devDependencies: { [CLI]: `^${VERSION_ABSENT_FROM_THE_REGISTRY}` } }, + }, + ]), + ), + When('the closure install is planned')('refused', (s) => refusalOf(CLOSURE.map(memberOf), s.fixtures)), + Then('the plan names the fixture manifest and the workspace package')((s, expect) => + expect(s.refused).toStrictEqual({ + _tag: 'FixtureNamesWorkspacePackage', + fixture: 'calc-fixture/package.json', + dependency: CLI, + target: CLI, + }) + ), + ), + ) + + scenario( + 'An optional peer on a workspace package the closure did not pack plans the install', + Gherkin.Do.pipe( + Given('a closure whose CLI declares the unpacked checker as an optional peer')('members', () => + Effect.succeed( + [ + { + ...manifestOf(CLI, {}), + peerDependencies: { [CHECKER]: `^${VERSION_ABSENT_FROM_THE_REGISTRY}` }, + peerDependenciesMeta: { [CHECKER]: { optional: true } }, + }, + manifestOf(RUNNER, {}), + ].map(memberOf), + )), + When('the closure install is planned')('refused', (s) => refusalOf(s.members, [])), + Then('the plan is not refused, because npm never fetches an absent optional peer')((s, expect) => + expect(s.refused).toBeNull() + ), + ), + ) + }) diff --git a/test/e2e/README.md b/test/e2e/README.md index a49a423a2..9bb57cc51 100644 --- a/test/e2e/README.md +++ b/test/e2e/README.md @@ -9,6 +9,8 @@ The suite resolves the workspace closure of the CLI and its plugins, packs each - Full mutation runs against realistic test suites - Exit codes and typed machine-mode JSON error envelopes on failure +The closure installs in one `npm install` with no `@systemfsoftware` workspace package taken from the registry. `pnpm pack` rewrites a workspace alias (`"@systemfsoftware/stryker-js-vm-runner": "workspace:@systemfsoftware/stryker-js-vitest-runner@^"`) to `npm:@^`, and npm resolves an `npm:` spec from the registry even when the target's tarball is in the same install. So the install passes every packed tarball plus one `@file:` spec per alias edge between closure members (`installClosure` in `@systemfsoftware/stryker-e2e-core`). Global setup fails, naming the edge, instead of reaching the registry when a packed member depends on a workspace package the closure did not pack (`UnpackedWorkspaceDependency`), when one alias name would install two different packages (`ConflictingAliasTargets`), or when a fixture manifest names a workspace package itself (`FixtureNamesWorkspacePackage`). The closure is installed into every fixture, so a fixture never names it. + ## Running locally Local runs need hardware virtualization and nothing else: no Docker daemon, Podman socket, or container CLI. diff --git a/test/e2e/src/Harness/bake-key.schema.ts b/test/e2e/src/Harness/bake-key.schema.ts index 7015069d0..d038b5e49 100644 --- a/test/e2e/src/Harness/bake-key.schema.ts +++ b/test/e2e/src/Harness/bake-key.schema.ts @@ -16,11 +16,7 @@ export class MissingTarball extends Schema.TaggedClass()('Missin directory: Schema.String, }) {} -export class UnreadableVersion extends Schema.TaggedClass()('UnreadableVersion', { - fileName: Schema.String, -}) {} - -export const PackedPackageLookup = Schema.Union([FoundPackage, MissingTarball, UnreadableVersion]) +export const PackedPackageLookup = Schema.Union([FoundPackage, MissingTarball]) export type PackedPackageLookup = typeof PackedPackageLookup.Type diff --git a/test/e2e/src/Harness/fixture-cache.service.ts b/test/e2e/src/Harness/fixture-cache.service.ts index 3896d714a..6ff6ee278 100644 --- a/test/e2e/src/Harness/fixture-cache.service.ts +++ b/test/e2e/src/Harness/fixture-cache.service.ts @@ -25,12 +25,19 @@ import { type FileBytes, type FixtureInput, fixtureKeyBytes, + FixtureManifest, + installClosure, + InstallClosureCommand, missingFixtures as missingFixturesWorkflow, MissingFixturesCommand, + PackedManifest, + type PackedMember, + type PackedTree, type PackInput, packsKeyBytes, pruneStaleEntries as pruneStaleEntriesWorkflow, PruneStaleEntriesCommand, + type StagedFixtureManifest, } from '@systemfsoftware/stryker-e2e-core' import type { BakeOutcome, PackedPackage, PackedPackageLookup, TurboDryClosure } from './bake-key.schema.js' @@ -41,7 +48,6 @@ import { MissingTarball, TurboClosure, TurboDryRun, - UnreadableVersion, } from './bake-key.schema.js' import type { WorkspaceCatalogs } from './catalog-resolution.js' import { parseFixtureManifest, parseWorkspaceCatalogs, resolveCatalogSpecs } from './catalog-resolution.js' @@ -85,6 +91,7 @@ const STEP_BAKE = 'bake every fixture in the preparation microVM' const STEP_CLOSURE = 'build the packed workspace closure' const STEP_TARBALLS = 'read the packed tarballs' const STEP_KEY = 'derive the bake cache key' +const STEP_INSTALL_PLAN = 'plan the workspace closure install' const TREE_CONCURRENCY = 16 const UNPACK_CONCURRENCY = 4 @@ -99,7 +106,16 @@ const ENTRY_PACKAGES = [ type Argv = readonly [string, ...Array] -const PACKED_TARBALL_VERSION = /-(\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?)\.tgz$/ +const PACKED_VERSION = String.raw`\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?` + +const REGEXP_SPECIAL = /[.*+?^${}()|[\]\\]/g + +const packedFileNameOf = (prefix: string): RegExp => + new RegExp(`^${prefix.replace(REGEXP_SPECIAL, String.raw`\$&`)}(${PACKED_VERSION})\\.tgz$`) + +const PACKED_MANIFEST_PATH = 'package/package.json' + +const WorkspaceListing = Schema.fromJsonString(Schema.Array(Schema.Struct({ name: Schema.optional(Schema.String) }))) const runCommand = (argv: Argv, cwd?: string) => Effect.scoped(Effect.gen(function*() { @@ -404,17 +420,21 @@ const lookupOf = ( directory: string, ): PackedPackageLookup => { const prefix = `${packageName.slice(1).replace('/', '-')}-` + const packedFileName = packedFileNameOf(prefix) return Option.match( - Array.findFirst(fileNames, (candidate) => candidate.startsWith(prefix) && candidate.endsWith('.tgz')), + Array.findFirst( + fileNames, + (candidate) => + Option.map(Option.fromNullishOr(packedFileName.exec(candidate)?.[1]), (version) => ({ + fileName: candidate, + version, + })), + ), { onNone: (): PackedPackageLookup => MissingTarball.make({ prefix, directory }), - onSome: (fileName): PackedPackageLookup => - Option.match(Option.fromNullishOr(PACKED_TARBALL_VERSION.exec(fileName)?.[1]), { - onNone: (): PackedPackageLookup => UnreadableVersion.make({ fileName }), - onSome: (version): PackedPackageLookup => - FoundPackage.make({ - pack: { name: packageName, version, fileName, tarballPath: `${directory}/${fileName}` }, - }), + onSome: ({ fileName, version }): PackedPackageLookup => + FoundPackage.make({ + pack: { name: packageName, version, fileName, tarballPath: `${directory}/${fileName}` }, }), }, ) @@ -427,14 +447,7 @@ const packedTarballOf = (fileNames: ReadonlyArray, packageName: string, Effect.fail( new PackFailure({ step: STEP_TARBALLS, - detail: `pnpm pack wrote no ${lookup.prefix}*.tgz into ${lookup.directory}`, - }), - )), - Match.tag('UnreadableVersion', (lookup) => - Effect.fail( - new PackFailure({ - step: STEP_TARBALLS, - detail: `the packed tarball ${lookup.fileName} carries no parseable version`, + detail: `pnpm pack wrote no ${lookup.prefix}.tgz into ${lookup.directory}`, }), )), Match.exhaustive, @@ -533,6 +546,75 @@ const packsInputOf = ( return { baseImage: GuestJobs.BASE_IMAGE, bakeScript, packs: packedTrees } }).pipe(seamSpan(SpanNames.packsKey, { 'e2e.packs': packs.length })) +const workspacePackagesOf = (environment: BakeEnvironment) => + Effect.gen(function*() { + const listing = yield* runChecked( + STEP_INSTALL_PLAN, + ['pnpm', 'ls', '-r', '--depth', '-1', '--json'], + environment.repoRoot, + ) + const projects = yield* Schema.decodeEffect(WorkspaceListing)(listing.stdout).pipe( + Effect.mapError(() => + new PackFailure({ step: STEP_INSTALL_PLAN, detail: 'pnpm ls wrote no parseable workspace listing' }) + ), + ) + return projects.flatMap((project) => Option.toArray(Option.fromNullishOr(project.name))) + }) + +const packedMemberOf = (tree: PackedTree) => + Effect.gen(function*() { + const unreadable = new PackFailure({ + step: STEP_INSTALL_PLAN, + detail: `the packed tarball ${tree.fileName} carries no readable ${PACKED_MANIFEST_PATH}`, + }) + const manifestFile = yield* Option.match( + Array.findFirst(tree.files, (file) => file.relativePath === PACKED_MANIFEST_PATH), + { + onNone: () => Effect.fail(unreadable), + onSome: (found) => Effect.succeed(found), + }, + ) + const manifest = yield* Schema.decodeEffect(Schema.fromJsonString(PackedManifest))( + new TextDecoder().decode(manifestFile.bytes), + ).pipe(Effect.mapError(() => unreadable)) + return { tarballPath: `${GuestJobs.GUEST_PACKS_ROOT}/${tree.fileName}`, manifest } satisfies PackedMember + }) + +const stagedManifestsOf = (input: FixtureInput) => + Effect.forEach( + Array.filter(input.files, (file) => isManifestPath(file.relativePath)), + (file) => + Schema.decodeEffect(Schema.fromJsonString(FixtureManifest))(new TextDecoder().decode(file.bytes)).pipe( + Effect.map((manifest): StagedFixtureManifest => ({ + path: `${input.fixtureId}/${file.relativePath}`, + manifest, + })), + Effect.mapError(() => + new PackFailure({ + step: STEP_INSTALL_PLAN, + detail: `the fixture manifest ${input.fixtureId}/${file.relativePath} is not a readable package.json`, + }) + ), + ), + ) + +const closureInstallOf = ( + environment: BakeEnvironment, + packsInput: PackInput, + fixtureInputs: ReadonlyArray, +) => + Effect.gen(function*() { + const workspace = yield* workspacePackagesOf(environment) + const members = yield* Effect.forEach(packsInput.packs, packedMemberOf) + const fixtures = (yield* Effect.forEach(fixtureInputs, stagedManifestsOf)).flat() + const install = yield* Effect.fromResult( + installClosure(InstallClosureCommand.make({ members, fixtures, workspace })), + ).pipe( + Effect.mapError((failure) => new PackFailure({ step: STEP_INSTALL_PLAN, detail: failure.message })), + ) + return install.specs + }) + const fixtureInputsOf = ( environment: BakeEnvironment, fixtureIds: ReadonlyArray, @@ -596,8 +678,11 @@ const bakeMissing = ( root: string, missing: ReadonlyArray, catalogs: WorkspaceCatalogs, + packsInput: PackInput, + fixtureInputs: ReadonlyArray, ) => Effect.gen(function*() { + const install = yield* closureInstallOf(environment, packsInput, fixtureInputs) const fs = yield* FileSystem.FileSystem const path = yield* Path.Path const crypto = yield* Crypto.Crypto @@ -611,7 +696,7 @@ const bakeMissing = ( const bakeScript = yield* fs.readFileString(environment.bakeScriptPath) yield* jobs.requireCleanExit( STEP_BAKE, - jobs.job(['sh', '-c', bakeScript], [ + jobs.job(['sh', '-c', bakeScript, 'bake-fixtures', ...install], [ { host: stagingDir, guest: GuestJobs.GUEST_BAKED_ROOT }, { host: packsDir, guest: GuestJobs.GUEST_PACKS_ROOT }, ]), @@ -649,7 +734,7 @@ const bake = (environment: BakeEnvironment): Effect.Effect Effect.void, - onFalse: () => bakeMissing(environment, packsDir, root, missing, catalogs), + onFalse: () => bakeMissing(environment, packsDir, root, missing, catalogs, packsInput, fixtureInputs), }) return { root, keys: keysRecordOf(fixtures), lease } }).pipe(Effect.ensuring(fs.remove(scratch, { recursive: true, force: true }).pipe(Effect.orDie))) diff --git a/test/e2e/tests/__fixtures__/bake-fixtures.sh b/test/e2e/tests/__fixtures__/bake-fixtures.sh index f6aecfd28..923165140 100755 --- a/test/e2e/tests/__fixtures__/bake-fixtures.sh +++ b/test/e2e/tests/__fixtures__/bake-fixtures.sh @@ -7,15 +7,5 @@ for dir in /baked/*/; do echo "[bake] $id: npm install the registry dependencies" >&2 npm install --no-audit --no-fund --loglevel=error echo "[bake] $id: npm install the workspace closure tarballs" >&2 - npm install --no-audit --no-fund --loglevel=error /packs/*.tgz - for alias in \ - node_modules/@systemfsoftware/stryker-js-vm-runner \ - node_modules/@systemfsoftware/stryker-js/node_modules/@systemfsoftware/stryker-js-vm-runner - do - if [ -d "$alias" ] && [ -d node_modules/@systemfsoftware/stryker-js-vitest-runner ]; then - echo "[bake] $id: point $alias at the packed workspace runner" >&2 - rm -rf "$alias" - cp -R node_modules/@systemfsoftware/stryker-js-vitest-runner "$alias" - fi - done + npm install --no-audit --no-fund --loglevel=error "$@" done