From 6aa7bba3b6114ce148670ae4a954206958d58e89 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 15:36:56 +0100 Subject: [PATCH 01/12] fix(process-graph): declare stable peers as caret ranges Pinning every peer dependency to one exact version forced each consumer onto exactly those versions. A stable peer is now a caret range whose floor is the version the workspace tests run against; the oRPC betas, drizzle-orm's release candidate and the SQLite wasm build stay exact, since a prerelease has no compatible-range semantics. The published-manifest check validates peers against the catalog by satisfies rather than equality, requires the caret form for a stable peer and an exact version for a prerelease one, and keeps dependencies and devDependencies exact. The smoke consumer installs each peer at the floor its specifier declares. --- CONTRIBUTING.md | 2 +- README.md | 2 +- package.json | 2 + packages/targets/process-graph/README.md | 2 +- packages/targets/process-graph/package.json | 14 +-- .../test/consumer.global-setup.ts | 5 +- pnpm-lock.yaml | 17 ++++ pnpm-workspace.yaml | 4 +- scripts/check-published-manifests.ts | 62 ++++++++++++- .../check-published-manifests.unit.test.ts | 91 +++++++++++++++++-- syncpack.config.ts | 2 +- 11 files changed, 175 insertions(+), 28 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cb281b6..c5788ba 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -10,7 +10,7 @@ A new package is private and reaches consumers through a subpath of the publishe 4. Add dependencies with pnpm: a sibling as `workspace:*` (`pnpm add @exadev/process-graph-@workspace:*`), and a third-party library from the catalog in `pnpm-workspace.yaml` (`pnpm add --save-catalog @`, which adds the catalog entry when there is none). `pnpm run deps:lint` rejects anything else. 5. Add its `tsconfig.build.json` to the root `tsconfig.build.json` `references`. 6. If it is the first package in its layer, add that layer's workspace entry to `knip.config.ts` (an entry that matches nothing is a knip hint). -7. Expose it through `process-graph`, in `packages/targets/process-graph`: a module under `src/` that re-exports it, named for the subpath (`src/.ts`), an entry for that module in `tsdown.config.ts`, the subpath in the `exports` map, the module in the subpath list in `eslint.config.ts`, a case in `src/entrypoints.unit.test.ts` and an import of the subpath in `test/package.smoke.test.ts`. Add the package to `devDependencies` at its exact version (`pnpm add -D --save-exact @exadev/process-graph-@`), never `workspace:*`, since that manifest is published as written. Each third-party library the new subpath needs that the root entry does not goes in `peerDependencies`, marked optional in `peerDependenciesMeta`, at the catalog's exact version (`pnpm add --save-peer --save-exact @`). Where a layer's names collide with another layer's on the same subpath, namespace one layer rather than shadow a name. +7. Expose it through `process-graph`, in `packages/targets/process-graph`: a module under `src/` that re-exports it, named for the subpath (`src/.ts`), an entry for that module in `tsdown.config.ts`, the subpath in the `exports` map, the module in the subpath list in `eslint.config.ts`, a case in `src/entrypoints.unit.test.ts` and an import of the subpath in `test/package.smoke.test.ts`. Add the package to `devDependencies` at its exact version (`pnpm add -D --save-exact @exadev/process-graph-@`), never `workspace:*`, since that manifest is published as written. Each third-party library the new subpath needs that the root entry does not goes in `peerDependencies`, marked optional in `peerDependenciesMeta`, as a caret range on the catalog's version, `^`, or at the catalog's exact version when that is a prerelease, which has no compatible-range semantics (`pnpm add --save-peer --save-exact @`, then widen a stable peer's specifier to the caret range; `deps:lint` holds both forms). Where a layer's names collide with another layer's on the same subpath, namespace one layer rather than shadow a name. 8. Write its `README.md`, then `ln -s README.md AGENTS.md && ln -s README.md CLAUDE.md`. 9. Add it to the internal packages table in the root `README.md`, and the subpath to the `process-graph` README. 10. Verify with the repository scripts: `pnpm lint`, `pnpm typecheck`, `pnpm test`, `pnpm test:smoke`, `pnpm exec knip`, `pnpm run deps:lint`. diff --git a/README.md b/README.md index bb1672a..938151f 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ pnpm build pnpm test ``` -Dependencies are added with `pnpm add`, never by editing a version by hand. Every third-party version is written once, exactly, in the catalog in `pnpm-workspace.yaml`: a private package names it as `catalog:` (`pnpm add --save-catalog @`) and a sibling as `workspace:*`. The published `process-graph` manifest names exact literal versions instead, because `npm publish` ships a protocol specifier verbatim; `pnpm run deps:lint` (`syncpack` and `scripts/check-published-manifests.ts`) holds both rules. The install age gate in `pnpm-workspace.yaml` is never lowered; ExaDev's own packages are the only entries in `minimumReleaseAgeExclude`. +Dependencies are added with `pnpm add`, never by editing a version by hand. Every third-party version is written once, exactly, in the catalog in `pnpm-workspace.yaml`: a private package names it as `catalog:` (`pnpm add --save-catalog @`) and a sibling as `workspace:*`. The published `process-graph` manifest names literal versions instead, because `npm publish` ships a protocol specifier verbatim: its dependencies exactly, and its peer dependencies as caret ranges on the lowest tested version, which the catalog's version satisfies, except a prerelease peer (oRPC's betas, Drizzle's release candidate), which stays exact because a prerelease has no compatible-range semantics; `pnpm run deps:lint` (`syncpack` and `scripts/check-published-manifests.ts`) holds both rules. The install age gate in `pnpm-workspace.yaml` is never lowered; ExaDev's own packages are the only entries in `minimumReleaseAgeExclude`. ## Commands diff --git a/package.json b/package.json index 136ff9f..ca55896 100644 --- a/package.json +++ b/package.json @@ -56,6 +56,7 @@ "@semantic-release/release-notes-generator": "catalog:", "@stryker-mutator/api": "catalog:", "@types/node": "catalog:", + "@types/semver": "catalog:", "conventional-changelog-conventionalcommits": "catalog:", "eslint": "catalog:", "eslint-config-prettier": "catalog:", @@ -71,6 +72,7 @@ "lint-staged": "catalog:", "prettier": "catalog:", "semantic-release": "catalog:", + "semver": "catalog:", "syncpack": "catalog:", "turbo": "catalog:", "typescript": "catalog:", diff --git a/packages/targets/process-graph/README.md b/packages/targets/process-graph/README.md index 85751f8..9d369a6 100644 --- a/packages/targets/process-graph/README.md +++ b/packages/targets/process-graph/README.md @@ -72,7 +72,7 @@ The package is built from the repository's internal packages, which are bundled | `react-flow` | `react`, `react-dom`, `@xyflow/react`, `elkjs`, `@vanilla-extract/recipes` | | `cli` and the bin | `commander`, `@orpc/client`, `@orpc/contract`, `@orpc/server`, `drizzle-orm` | -The versions are the ones `package.json` declares under `peerDependencies`. +The versions are the ones `package.json` declares under `peerDependencies`. A peer with a stable version is a caret range whose floor is the version this package is tested against, so any later compatible release satisfies it. The oRPC packages, `drizzle-orm` and `@sqlite.org/sqlite-wasm` are prereleases, which have no compatible-range semantics (a range on one admits only later prereleases of the same version, which may break anything), so each is declared at the exact version tested. With Vite, exclude `@sqlite.org/sqlite-wasm` from dependency pre-bundling (`optimizeDeps.exclude`), since it loads its `.wasm` file relative to itself. A worker module serving the browser store is one call: diff --git a/packages/targets/process-graph/package.json b/packages/targets/process-graph/package.json index 67b28bc..4b36bcf 100644 --- a/packages/targets/process-graph/package.json +++ b/packages/targets/process-graph/package.json @@ -162,14 +162,14 @@ "@orpc/contract": "2.0.0-beta.41", "@orpc/server": "2.0.0-beta.41", "@sqlite.org/sqlite-wasm": "3.53.4-build2", - "@vanilla-extract/css": "1.21.2", - "@vanilla-extract/recipes": "0.5.7", - "@xyflow/react": "12.12.0", - "commander": "15.0.0", + "@vanilla-extract/css": "^1.21.2", + "@vanilla-extract/recipes": "^0.5.7", + "@xyflow/react": "^12.12.0", + "commander": "^15.0.0", "drizzle-orm": "1.0.0-rc.4", - "elkjs": "0.12.0", - "react": "19.3.0", - "react-dom": "19.3.0" + "elkjs": "^0.12.0", + "react": "^19.3.0", + "react-dom": "^19.3.0" }, "peerDependenciesMeta": { "@orpc/client": { diff --git a/packages/targets/process-graph/test/consumer.global-setup.ts b/packages/targets/process-graph/test/consumer.global-setup.ts index eb86412..53b60de 100644 --- a/packages/targets/process-graph/test/consumer.global-setup.ts +++ b/packages/targets/process-graph/test/consumer.global-setup.ts @@ -24,7 +24,7 @@ declare module "vitest" { const packageDirectory = fileURLToPath(new URL("..", import.meta.url)); /** - * Each peer dependency the manifest declares, as `name@version`, so the consumer installs every optional subpath's libraries at the versions the package was built against. + * Each peer dependency the manifest declares, as `name@version` at the lowest version its specifier admits, so the consumer installs every optional subpath's libraries at the floor the manifest claims was tested. A stable peer is a caret range on one version and a prerelease peer is that version exactly (`scripts/check-published-manifests.ts` holds both), so dropping a leading caret gives the floor. * @returns The install specifiers. */ const peerSpecifiers = (): string[] => { @@ -42,7 +42,8 @@ const peerSpecifiers = (): string[] => { } return Object.entries(manifest.peerDependencies).map( - ([name, version]: [string, unknown]) => `${name}@${String(version)}`, + ([name, version]: [string, unknown]) => + `${name}@${String(version).replace(/^\^/, "")}`, ); }; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 1a61fd6..b181fc8 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -254,6 +254,9 @@ catalogs: '@types/react-dom': specifier: 19.3.0 version: 19.3.0 + '@types/semver': + specifier: 7.8.0 + version: 7.8.0 '@vanilla-extract/css': specifier: 1.21.2 version: 1.21.2 @@ -344,6 +347,9 @@ catalogs: semantic-release: specifier: 25.0.9 version: 25.0.9 + semver: + specifier: 7.8.5 + version: 7.8.5 syncpack: specifier: 15.3.3 version: 15.3.3 @@ -430,6 +436,9 @@ importers: '@types/node': specifier: 'catalog:' version: 26.6.4 + '@types/semver': + specifier: 'catalog:' + version: 7.8.0 conventional-changelog-conventionalcommits: specifier: 'catalog:' version: 9.3.1 @@ -475,6 +484,9 @@ importers: semantic-release: specifier: 'catalog:' version: 25.0.9(supports-color@7.2.0)(typescript@6.0.3) + semver: + specifier: 'catalog:' + version: 7.8.5 syncpack: specifier: 'catalog:' version: 15.3.3 @@ -3983,6 +3995,9 @@ packages: '@types/resolve@1.20.2': resolution: {integrity: sha512-60BCwRFOZCQhDncwQdxxeOEEkbc5dIMccYLwbxsS4TUNeVECQ/pBJ0j09mrHOl/JJvpRPGwO9SvE4nR2Nb/a4Q==} + '@types/semver@7.8.0': + resolution: {integrity: sha512-1mAINjtQCXXeLkJ9ehXkwOcBpqtLxiVtKhpUf83DdRNdQKV0iXZpaHYqRr7nj+wvxuJzoAmAwXI+sCNMv1CzLQ==} + '@types/trusted-types@2.0.7': resolution: {integrity: sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw==} @@ -10273,6 +10288,8 @@ snapshots: '@types/resolve@1.20.2': {} + '@types/semver@7.8.0': {} + '@types/trusted-types@2.0.7': {} '@types/unist@3.0.3': {} diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 99a4209..2d457e2 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -10,7 +10,7 @@ packages: # - ui//: components that render the model. # - targets/: deployable entry points, deliberately flat. -# Every private package names its siblings with `workspace:*` and its third-party libraries with `catalog:`, so a version is written once, in the catalog below, and nothing installs a sibling from the registry. The published `process-graph` package is the exception: `@semantic-release/npm` publishes with `npm publish`, which ships a `workspace:` or `catalog:` specifier verbatim, so its manifest names exact versions, which `scripts/check-published-manifests.ts` holds equal to the catalog and to each sibling's own version. linkWorkspacePackages lets those exact sibling versions link the workspace copy rather than resolve from the registry. +# Every private package names its siblings with `workspace:*` and its third-party libraries with `catalog:`, so a version is written once, in the catalog below, and nothing installs a sibling from the registry. The published `process-graph` package is the exception: `@semantic-release/npm` publishes with `npm publish`, which ships a `workspace:` or `catalog:` specifier verbatim, so its manifest names literal versions, which `scripts/check-published-manifests.ts` holds to the catalog and to each sibling's own version: exact and equal in `dependencies` and `devDependencies`, and in `peerDependencies` a caret range the catalog satisfies, or the exact catalog version for a prerelease. linkWorkspacePackages lets those exact sibling versions link the workspace copy rather than resolve from the registry. linkWorkspacePackages: true saveExact: true @@ -64,6 +64,7 @@ catalog: "@types/node": 26.6.4 "@types/react": 19.3.0 "@types/react-dom": 19.3.0 + "@types/semver": 7.8.0 "@vanilla-extract/css": 1.21.2 "@vanilla-extract/recipes": 0.5.7 "@vanilla-extract/vite-plugin": 5.2.6 @@ -94,6 +95,7 @@ catalog: react: 19.3.0 react-dom: 19.3.0 semantic-release: 25.0.9 + semver: 7.8.5 syncpack: 15.3.3 trilean: 1.7.2 turbo: 2.11.7 diff --git a/scripts/check-published-manifests.ts b/scripts/check-published-manifests.ts index bb4bb49..ebcfc8d 100644 --- a/scripts/check-published-manifests.ts +++ b/scripts/check-published-manifests.ts @@ -4,10 +4,15 @@ import { discoverWorkspace, type WorkspacePackage, } from "@exadev/semantic-release-workspace"; +import { prerelease, satisfies } from "semver"; import { parse } from "yaml"; /** - * The half of `deps:lint` that syncpack cannot express. A published manifest ships every specifier exactly as written, because `@semantic-release/npm` publishes with `npm publish`, so it must name exact versions, never a `workspace:`, `catalog:`, `link:` or `file:` protocol; each third-party version the catalog names must equal the catalog's, and each sibling version the sibling's own, so the published package was built and tested against what it declares. Every catalog entry must be an exact version too, since the catalog is where the private packages take theirs. + * The half of `deps:lint` that syncpack cannot express. A published manifest ships every specifier exactly as written, because `@semantic-release/npm` publishes with `npm publish`, so it must never name a `workspace:`, `catalog:`, `link:` or `file:` protocol. Every catalog entry must be an exact version, since the catalog is where the private packages take theirs. + * + * `dependencies` and `devDependencies` name exact versions, each third-party one equal to the catalog's and each sibling one equal to the sibling's own, so the published package was built and tested against what it installs. + * + * `peerDependencies` are the consumer's own installs, so pinning one to a single version would force every consumer onto exactly that version. A peer with a stable version is declared as a caret range on one version, `^`, whose floor is the lowest version tested: the catalog's version, which every test in the workspace runs against, must satisfy it, and the smoke test installs the packed package beside each peer at its floor. A peer whose version is a prerelease (`2.0.0-beta.41`, `1.0.0-rc.4`) stays exact and equal to the catalog's, because a prerelease has no compatible-range semantics: a caret range on one matches only later prereleases of that same version, which may break anything, and no stable release. */ /** One rule a manifest or the catalog breaks. */ @@ -30,6 +35,12 @@ const protocolSpecifier = /^(?:workspace|catalog|link|file):/; /** An exact semantic version, prerelease and build metadata allowed, with no range operator. */ const exactVersion = /^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?(?:\+[0-9A-Za-z.-]+)?$/; +/** A caret range on one exact version, the only range a published peer may declare. */ +const caretRange = new RegExp(`^\\^${exactVersion.source.slice(1)}`); + +/** The dependency field whose ranges the consumer resolves against its own installs. */ +const peerField = "peerDependencies"; + /** * Checks the catalog's entries. * @param catalog - The default catalog, library name to version. @@ -45,6 +56,43 @@ export const checkCatalog = (catalog: ReadonlyMap): Finding[] => reason: "a catalog entry must be an exact version", })); +/** + * Checks one peer dependency's specifier. + * @param specifier - What the manifest declares. + * @param expected - The catalog's or the sibling's version, which the workspace's tests run against; `undefined` when neither names the peer. + * @param source - Which of the two `expected` is, for the message. + * @returns The reasons the specifier is wrong; empty when it is right. + */ +const checkPeer = ( + specifier: string, + expected: string | undefined, + source: string, +): string[] => { + const tested = expected ?? specifier.replace(/^\^/, ""); + if (prerelease(tested) !== null) { + if (!exactVersion.test(specifier)) { + return [ + "a prerelease peer must be an exact version, since a prerelease has no compatible-range semantics", + ]; + } + + return expected !== undefined && specifier !== expected + ? [`must equal ${source}, ${expected}`] + : []; + } + if (!caretRange.test(specifier)) { + return [ + "a stable peer must be a caret range on the lowest tested version, such as ^1.2.3", + ]; + } + + return expected !== undefined && !satisfies(expected, specifier) + ? [ + `must be satisfied by ${source}, ${expected}, which the tests run against`, + ] + : []; +}; + /** * Checks one published package's manifest against the catalog and its siblings. * @param pkg - The published package, as the release orchestrator discovers it. @@ -69,14 +117,18 @@ export const checkPublishedManifest = ( "npm publish ships a protocol specifier verbatim, which no consumer can install", ); } + const expected = siblings.get(name) ?? catalog.get(name); + const source = siblings.has(name) + ? "the sibling's own version" + : "the catalog entry"; + if (field === peerField) { + return checkPeer(specifier, expected, source).flatMap(finding); + } if (!exactVersion.test(specifier)) { return finding("a published dependency must be an exact version"); } - const expected = siblings.get(name) ?? catalog.get(name); if (expected !== undefined && specifier !== expected) { - return finding( - `must equal ${siblings.has(name) ? "the sibling's own version" : "the catalog entry"}, ${expected}`, - ); + return finding(`must equal ${source}, ${expected}`); } return []; diff --git a/scripts/check-published-manifests.unit.test.ts b/scripts/check-published-manifests.unit.test.ts index 4355644..f0fd811 100644 --- a/scripts/check-published-manifests.unit.test.ts +++ b/scripts/check-published-manifests.unit.test.ts @@ -62,15 +62,18 @@ describe("checkPublishedManifest", () => { ]); }); - test("rejects a range", () => { - expect( - checkPublishedManifest( - published("peerDependencies", { zod: "^4.6.5" }), - catalog, - siblings, - ).map((finding) => finding.reason), - ).toEqual(["a published dependency must be an exact version"]); - }); + test.each(["dependencies", "devDependencies"] as const)( + "rejects a range in %s", + (field) => { + expect( + checkPublishedManifest( + published(field, { zod: "^4.6.5" }), + catalog, + siblings, + ).map((finding) => finding.reason), + ).toEqual(["a published dependency must be an exact version"]); + }, + ); test("rejects a version that differs from the catalog's or the sibling's", () => { expect( @@ -99,6 +102,76 @@ describe("checkPublishedManifest", () => { }); }); +describe("checkPublishedManifest on peerDependencies", () => { + const peerCatalog = new Map([ + ["react", "19.3.0"], + ["@orpc/server", "2.0.0-beta.41"], + ]); + const peers = (entries: Readonly>) => + checkPublishedManifest( + published("peerDependencies", entries), + peerCatalog, + siblings, + ).map(({ name, reason }) => [name, reason]); + + test("accepts a caret range on a stable version the catalog satisfies, and an exact prerelease equal to the catalog", () => { + expect( + peers({ react: "^19.3.0", "@orpc/server": "2.0.0-beta.41" }), + ).toEqual([]); + }); + + test("accepts a caret range whose floor is below the catalog's version, since the catalog still satisfies it", () => { + expect(peers({ react: "^19.0.0" })).toEqual([]); + }); + + test("rejects a caret range the catalog's version does not satisfy", () => { + expect(peers({ react: "^19.4.0" })).toEqual([ + [ + "react", + "must be satisfied by the catalog entry, 19.3.0, which the tests run against", + ], + ]); + }); + + test.each(["19.3.0", "~19.3.0", ">=19.3.0", "19.x", "*"])( + "rejects %s for a stable peer, which must be a caret range", + (specifier) => { + expect(peers({ react: specifier })).toEqual([ + [ + "react", + "a stable peer must be a caret range on the lowest tested version, such as ^1.2.3", + ], + ]); + }, + ); + + test("rejects a range on a prerelease peer, which must stay exact", () => { + expect(peers({ "@orpc/server": "^2.0.0-beta.41" })).toEqual([ + [ + "@orpc/server", + "a prerelease peer must be an exact version, since a prerelease has no compatible-range semantics", + ], + ]); + }); + + test("rejects an exact prerelease peer that differs from the catalog's", () => { + expect(peers({ "@orpc/server": "2.0.0-beta.40" })).toEqual([ + ["@orpc/server", "must equal the catalog entry, 2.0.0-beta.41"], + ]); + }); + + test("checks a peer the catalog does not name by its own version", () => { + expect( + peers({ elkjs: "^0.12.0", vite: "8.3.2", next: "1.0.0-rc.1" }), + ).toEqual([ + [ + "vite", + "a stable peer must be a caret range on the lowest tested version, such as ^1.2.3", + ], + ]); + }); +}); + describe("checkCatalog", () => { test("accepts exact versions, prereleases included, and rejects a range", () => { expect( diff --git a/syncpack.config.ts b/syncpack.config.ts index 34b35a7..5522088 100644 --- a/syncpack.config.ts +++ b/syncpack.config.ts @@ -3,7 +3,7 @@ import type { RcFile } from "syncpack"; /** * One version per library, written once. The private packages, the workspace root included, take every third-party version from the catalog in `pnpm-workspace.yaml` (`catalog:`) and every sibling through the workspace protocol (`workspace:*`). * - * The published `process-graph` package cannot: `@semantic-release/npm` publishes with `npm publish`, which ships a `catalog:` or `workspace:` specifier verbatim, so its manifest names exact versions. syncpack's catalog policy would rewrite those into `catalog:`, so the package is left to `scripts/check-published-manifests.ts`, the other half of `deps:lint`, which holds each of its versions exact and equal to the catalog entry or the sibling's own version, and each catalog entry exact. Version groups match first to last, so that exemption comes before the two groups that would otherwise claim its dependencies. + * The published `process-graph` package cannot: `@semantic-release/npm` publishes with `npm publish`, which ships a `catalog:` or `workspace:` specifier verbatim, so its manifest names exact versions. syncpack's catalog policy would rewrite those into `catalog:`, so the package is left to `scripts/check-published-manifests.ts`, the other half of `deps:lint`, which holds its dependencies exact and equal to the catalog entry or the sibling's own version, its peer dependencies to a caret range the catalog entry satisfies (exact for a prerelease), and each catalog entry exact. Version groups match first to last, so that exemption comes before the two groups that would otherwise claim its dependencies. */ const config: RcFile = { versionGroups: [ From 8affb90d37407fcafd1ad1fa0754df6831065992 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 17:30:33 +0100 Subject: [PATCH 02/12] feat(store): export private data-file preparation and a host-owned data path prepareDataFile, which creates a store's directory 0700 and its database 0600 before SQLite opens it and narrows an owned store others can read, moves from the command line into sqlite-node, so process-graph/store/node exports it with its modes and options for any host that opens a store itself. nodePorts takes a dataPath: a data file the host owns, used as the default data path and given the same private treatment as the command line's own XDG data file, so a host embedding the command line keeps its store private. --- packages/adapters/graph/sqlite-node/README.md | 18 +++ .../graph/sqlite-node}/src/data-file.ts | 15 +- .../sqlite-node/src/data-file.unit.test.ts | 131 ++++++++++++++++++ .../adapters/graph/sqlite-node/src/index.ts | 8 +- packages/targets/graph-cli/README.md | 4 +- .../graph-cli/src/data-file.unit.test.ts | 59 ++++++-- packages/targets/graph-cli/src/node.ts | 18 ++- packages/targets/process-graph/README.md | 2 +- 8 files changed, 228 insertions(+), 27 deletions(-) rename packages/{targets/graph-cli => adapters/graph/sqlite-node}/src/data-file.ts (65%) create mode 100644 packages/adapters/graph/sqlite-node/src/data-file.unit.test.ts diff --git a/packages/adapters/graph/sqlite-node/README.md b/packages/adapters/graph/sqlite-node/README.md index 2902aa3..f5b7fc5 100644 --- a/packages/adapters/graph/sqlite-node/README.md +++ b/packages/adapters/graph/sqlite-node/README.md @@ -20,4 +20,22 @@ close(); `openNodeGraphStore(path, options)` opens the file (or `":memory:"`), passing `node:sqlite`'s own options through, and the store migrates the schema on its first operation with the migrations embedded in `graph-store-sqlite`, the same ones the browser driver applies. A file database is switched to write-ahead logging, so a reader in another process is not blocked by a writer; set `timeout` to have a write wait for another process's lock. The store's change feed sees only the writes made through it, so a subscriber in one process does not hear another process's changes. +`prepareDataFile(path, { ownsDirectory, platform })` makes a data file private before the store opens it, since a store holds synced work that other users on the host must not read. It creates the directory when missing, readable by its owner only (`privateDirectoryMode`, `0700`) when the caller owns it, and creates the database file owner-only (`privateFileMode`, `0600`) before SQLite opens it, so the `-wal` and `-shm` files SQLite derives from it are private too, whatever the umask. In a directory the caller owns, an existing directory or store file others can read is narrowed to those modes and the call returns one warning naming what changed, for the caller to show; in a directory someone else chose, the directory and an existing file are left as they are. On Windows, where file modes mean nothing, it only creates the directory. + +```ts +import { + openNodeGraphStore, + prepareDataFile, +} from "@exadev/process-graph-sqlite-node"; + +const path = join(appHome, "graph", "graph.sqlite3"); +for (const warning of prepareDataFile(path, { + ownsDirectory: true, + platform: process.platform, +})) { + console.warn(warning); +} +const { store, close } = openNodeGraphStore(path); +``` + The contract's conformance kit runs against an in-memory database in process and against a file database over a `MessageChannel`. diff --git a/packages/targets/graph-cli/src/data-file.ts b/packages/adapters/graph/sqlite-node/src/data-file.ts similarity index 65% rename from packages/targets/graph-cli/src/data-file.ts rename to packages/adapters/graph/sqlite-node/src/data-file.ts index 30dc8b8..8b683f8 100644 --- a/packages/targets/graph-cli/src/data-file.ts +++ b/packages/adapters/graph/sqlite-node/src/data-file.ts @@ -13,10 +13,10 @@ const permissionBits = 0o777; /** The bits that give the file's group or anyone else any access. */ const groupAndOtherBits = 0o077; -/** Where a store's data file lives, and how much of its path the command line owns. */ +/** Where a store's data file lives, and how much of its path the caller owns. */ export interface DataFileOptions { /** - * Whether the directory is the command line's own default, which it creates private and keeps private. A directory the user named (`--data` or `PROCESS_GRAPH_DATA`) is created when missing but never has its mode changed. + * Whether the directory is the caller's own, such as an application's default data directory or one a host keeps under its own home, which is created private and kept private. A directory someone else chose, such as a path a user names on a command line, is created when missing but never has its mode changed. */ readonly ownsDirectory: boolean; /** The platform; on Windows, file modes mean nothing and nothing is set. */ @@ -59,10 +59,13 @@ const tighten = (path: string, mode: number): boolean => { }; /** - * Makes a data file ready for SQLite to open. The directory is created when missing, private when it is the command line's own. The database file is created private when missing, before SQLite opens it, so the file and the sidecars SQLite derives from it are never readable by others, whatever the umask; a file that already exists is left alone unless the directory is the command line's own. In its own directory, an existing directory or store file that group or others can read is narrowed to the private mode. - * @param path - The database file. - * @param options - How much of the path the command line owns, and the platform. - * @returns One warning line naming what was narrowed, or nothing. + * Makes a data file ready for `openNodeGraphStore` to open, so a store holding synced work is never readable by other users on the host. Call it before opening the store; it does not open the database itself. + * + * The directory is created when missing (with any missing parents), with `privateDirectoryMode` when it is the caller's own. The database file is created with `privateFileMode` when missing, before SQLite opens it, so the file and the `-wal` and `-shm` files SQLite derives from it are never readable by others, whatever the umask. A file that already exists is left alone unless the directory is the caller's own; there, an existing directory or store file (database, `-wal` or `-shm`) that group or others can read is narrowed to the private mode. On Windows, where file modes mean nothing, only the directory is created. + * @param path - The database file; not `":memory:"`, which has no file. + * @param options - Whether the caller owns the directory, and the platform. + * @returns One warning line naming what was narrowed, for the caller to show, or nothing when nothing was. + * @throws Error from the file system when the directory or file cannot be created or its mode read or changed. */ export const prepareDataFile = ( path: string, diff --git a/packages/adapters/graph/sqlite-node/src/data-file.unit.test.ts b/packages/adapters/graph/sqlite-node/src/data-file.unit.test.ts new file mode 100644 index 0000000..d9f51ee --- /dev/null +++ b/packages/adapters/graph/sqlite-node/src/data-file.unit.test.ts @@ -0,0 +1,131 @@ +import { + chmodSync, + mkdirSync, + mkdtempSync, + rmSync, + statSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { afterEach, beforeEach, describe, expect, test } from "vitest"; +import { + prepareDataFile, + privateDirectoryMode, + privateFileMode, +} from "./data-file.ts"; +import { openNodeGraphStore } from "./open.ts"; + +/** The modes the usual umask gives a new directory and file, readable by group and others. */ +const looseDirectoryMode = 0o755; +const looseFileMode = 0o644; +/** The permission bits of a mode. */ +const permissionBits = 0o777; + +const modeOf = (path: string): number => statSync(path).mode & permissionBits; + +let directory: string; + +beforeEach(() => { + directory = mkdtempSync(join(tmpdir(), "process-graph-data-file-")); +}); + +afterEach(() => { + rmSync(directory, { recursive: true, force: true }); +}); + +/** Makes a directory and a file in it with modes others can read, as an earlier, careless run might have. */ +const looseStore = (path: string): void => { + mkdirSync(dirname(path), { recursive: true }); + chmodSync(dirname(path), looseDirectoryMode); + writeFileSync(path, ""); + chmodSync(path, looseFileMode); +}; + +describe("in a directory the caller owns", () => { + test("creates the missing directories and the database private, and the store SQLite opens there keeps its sidecars private", () => { + const path = join(directory, "home", "graph", "graph.sqlite3"); + + expect( + prepareDataFile(path, { ownsDirectory: true, platform: "linux" }), + ).toEqual([]); + const { store, close } = openNodeGraphStore(path); + close(); + + expect(store).toBeDefined(); + expect([ + modeOf(join(directory, "home")), + modeOf(dirname(path)), + modeOf(path), + ]).toEqual([privateDirectoryMode, privateDirectoryMode, privateFileMode]); + }); + + test("narrows an existing directory and store files others can read, and names them in one warning", () => { + const path = join(directory, "graph", "graph.sqlite3"); + looseStore(path); + writeFileSync(`${path}-wal`, ""); + chmodSync(`${path}-wal`, looseFileMode); + + expect( + prepareDataFile(path, { ownsDirectory: true, platform: "linux" }), + ).toEqual([ + `made ${dirname(path)}, ${path}, ${path}-wal readable by its owner only, since other users could read the synced work in it`, + ]); + expect([ + modeOf(dirname(path)), + modeOf(path), + modeOf(`${path}-wal`), + ]).toEqual([privateDirectoryMode, privateFileMode, privateFileMode]); + }); + + test("says nothing when the store is already private", () => { + const path = join(directory, "graph", "graph.sqlite3"); + prepareDataFile(path, { ownsDirectory: true, platform: "linux" }); + + expect( + prepareDataFile(path, { ownsDirectory: true, platform: "linux" }), + ).toEqual([]); + }); +}); + +describe("in a directory someone else chose", () => { + test("creates the database private but leaves the directory's mode as it is", () => { + const supplied = join(directory, "theirs"); + mkdirSync(supplied); + chmodSync(supplied, looseDirectoryMode); + const path = join(supplied, "graph.sqlite3"); + + expect( + prepareDataFile(path, { ownsDirectory: false, platform: "linux" }), + ).toEqual([]); + expect([modeOf(supplied), modeOf(path)]).toEqual([ + looseDirectoryMode, + privateFileMode, + ]); + }); + + test("leaves a database file that already exists as it is", () => { + const path = join(directory, "theirs", "graph.sqlite3"); + looseStore(path); + + expect( + prepareDataFile(path, { ownsDirectory: false, platform: "linux" }), + ).toEqual([]); + expect(modeOf(path)).toBe(looseFileMode); + }); +}); + +test("on Windows, where file modes mean nothing, creates the directory and sets or changes no mode", () => { + const path = join(directory, "xdg", "process-graph", "graph.sqlite3"); + looseStore(path); + + expect( + prepareDataFile(path, { ownsDirectory: true, platform: "win32" }), + ).toEqual([]); + expect(modeOf(path)).toBe(looseFileMode); + + const fresh = join(directory, "fresh", "graph.sqlite3"); + prepareDataFile(fresh, { ownsDirectory: true, platform: "win32" }); + + expect(statSync(dirname(fresh)).isDirectory()).toBe(true); +}); diff --git a/packages/adapters/graph/sqlite-node/src/index.ts b/packages/adapters/graph/sqlite-node/src/index.ts index a3b3689..17fb149 100644 --- a/packages/adapters/graph/sqlite-node/src/index.ts +++ b/packages/adapters/graph/sqlite-node/src/index.ts @@ -1,4 +1,10 @@ /** - * The SQLite graph store on Node's built-in `node:sqlite`, through `drizzle-orm/node-sqlite`: the driver for the command line and for servers. `openNodeGraphStore` opens (and on first use migrates) a database file and returns the `GraphStore` to serve with the contract's `createGraphStoreRouter`. + * The SQLite graph store on Node's built-in `node:sqlite`, through `drizzle-orm/node-sqlite`: the driver for the command line and for servers. `openNodeGraphStore` opens (and on first use migrates) a database file and returns the `GraphStore` to serve with the contract's `createGraphStoreRouter`. `prepareDataFile` makes a data file private to its owner before the store opens it. */ export { openNodeGraphStore, type NodeGraphStore } from "./open.ts"; +export { + prepareDataFile, + privateDirectoryMode, + privateFileMode, + type DataFileOptions, +} from "./data-file.ts"; diff --git a/packages/targets/graph-cli/README.md b/packages/targets/graph-cli/README.md index 222cfa9..50b2b21 100644 --- a/packages/targets/graph-cli/README.md +++ b/packages/targets/graph-cli/README.md @@ -55,7 +55,7 @@ Every command takes the global options: Without `--data` or `PROCESS_GRAPH_DATA`, the store is `process-graph/graph.sqlite3` under `XDG_DATA_HOME`, or under `~/.local/share` when that is not set. -The store holds synced work that other users on the host must not read. On Unix, the command line creates its default directory, and any missing directories above it, readable by its owner only (`0700`), and creates the database file the same way (`0600`) before SQLite opens it, so the `-wal` and `-shm` files SQLite derives from it are private too. A default store that already exists readable by others is narrowed to those modes, with one warning on standard error naming what changed. At a path given with `--data` or `PROCESS_GRAPH_DATA`, only a database file the command line creates is made private; the directory and an existing file are left as the user made them. Windows has no such modes, and nothing is set there. +The store holds synced work that other users on the host must not read. On Unix, the command line creates its default directory, and any missing directories above it, readable by its owner only (`0700`), and creates the database file the same way (`0600`) before SQLite opens it, so the `-wal` and `-shm` files SQLite derives from it are private too. A default store that already exists readable by others is narrowed to those modes, with one warning on standard error naming what changed. A host that keeps the store under its own home passes that file as `nodePorts({ dataPath })`, and it gets the same treatment as the default store. At a path given with `--data` or `PROCESS_GRAPH_DATA`, only a database file the command line creates is made private; the directory and an existing file are left as the user made them. Windows has no such modes, and nothing is set there. ## Output @@ -85,7 +85,7 @@ A sync merges everything it reads into the model in one write, so a record synce ## Library -`runCli(args, ports, identity)` runs the whole command surface as a function and resolves with the exit status; it never reads `process.argv`, writes to the process's streams or exits. `CliPorts` is everything it touches: `stdout` and `stderr`, `openStore(path)` (any `GraphStoreClient`, over any transport), `defaultDataPath`, `source(prefix)` (any `GraphSourceClient`), `readFile`, the `facts` a host supplies for decision conditions and input artefacts, and the abort `signal`. The model's stored fact record answers first (`runFacts` layers it over the host's facts with graph-domain's `layeredFacts`), so the host's facts answer only what nothing was recorded for. `nodePorts(options)` builds them for a Node process, with the environment, home directory, GitHub token port, `fetch`, clock, facts, signal and streams each replaceable; `runProcess(identity)` is what the bins run. +`runCli(args, ports, identity)` runs the whole command surface as a function and resolves with the exit status; it never reads `process.argv`, writes to the process's streams or exits. `CliPorts` is everything it touches: `stdout` and `stderr`, `openStore(path)` (any `GraphStoreClient`, over any transport), `defaultDataPath`, `source(prefix)` (any `GraphSourceClient`), `readFile`, the `facts` a host supplies for decision conditions and input artefacts, and the abort `signal`. The model's stored fact record answers first (`runFacts` layers it over the host's facts with graph-domain's `layeredFacts`), so the host's facts answer only what nothing was recorded for. `nodePorts(options)` builds them for a Node process, with the environment, home directory, host-owned data file (`dataPath`), GitHub token port, `fetch`, clock, facts, signal and streams each replaceable; `runProcess(identity)` is what the bins run. A host program imports the library from `process-graph/cli` and mounts the command line as a subcommand by forwarding its arguments: diff --git a/packages/targets/graph-cli/src/data-file.unit.test.ts b/packages/targets/graph-cli/src/data-file.unit.test.ts index fffceea..5aaeab2 100644 --- a/packages/targets/graph-cli/src/data-file.unit.test.ts +++ b/packages/targets/graph-cli/src/data-file.unit.test.ts @@ -12,10 +12,9 @@ import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; import { afterEach, beforeEach, describe, expect, test } from "vitest"; import { - prepareDataFile, privateDirectoryMode, privateFileMode, -} from "./data-file.ts"; +} from "@exadev/process-graph-sqlite-node"; import { exitCodes } from "./exit-codes.ts"; import { nodePorts } from "./node.ts"; import { runCli } from "./program.ts"; @@ -139,14 +138,52 @@ describe("a store at a path the user names", () => { }); }); -test("on Windows, where file modes mean nothing, no mode is set or changed", () => { - const path = join(directory, "xdg", "process-graph", "graph.sqlite3"); - mkdirSync(dirname(path), { recursive: true }); - writeFileSync(path, "", { mode: looseFileMode }); - chmodSync(path, looseFileMode); +describe("a data file the host owns", () => { + test("is the default data path, over PROCESS_GRAPH_DATA, and is created private with the directories above it", async () => { + const owned = join(directory, "host", "graph", "graph.sqlite3"); + let stderr = ""; + const ports = nodePorts({ + env: { PROCESS_GRAPH_DATA: join(directory, "elsewhere.sqlite3") }, + home: directory, + dataPath: owned, + stderr: (text) => { + stderr += text; + }, + }); + + expect(ports.defaultDataPath).toBe(owned); + await expect( + runCli(["add", "participant", "team", "Team"], ports, { + name: "sb graph", + version: "0.0.0", + }), + ).resolves.toBe(exitCodes.success); + expect([ + modeOf(join(directory, "host")), + modeOf(dirname(owned)), + modeOf(owned), + ]).toEqual([privateDirectoryMode, privateDirectoryMode, privateFileMode]); + expect(stderr).toBe(""); + }); + + test("is narrowed when it already exists readable by others, as the command line's own is", async () => { + const owned = join(directory, "host", "graph.sqlite3"); + mkdirSync(dirname(owned), { mode: looseDirectoryMode }); + chmodSync(dirname(owned), looseDirectoryMode); + writeFileSync(owned, ""); + chmodSync(owned, looseFileMode); + const opened = await nodePorts({ + home: directory, + dataPath: owned, + }).openStore(owned); + await opened.close(); - expect( - prepareDataFile(path, { ownsDirectory: true, platform: "win32" }), - ).toEqual([]); - expect(modeOf(path)).toBe(looseFileMode); + expect(opened.warnings).toEqual([ + `made ${dirname(owned)}, ${owned} readable by its owner only, since other users could read the synced work in it`, + ]); + expect([modeOf(dirname(owned)), modeOf(owned)]).toEqual([ + privateDirectoryMode, + privateFileMode, + ]); + }); }); diff --git a/packages/targets/graph-cli/src/node.ts b/packages/targets/graph-cli/src/node.ts index abe0a0d..3e5b75d 100644 --- a/packages/targets/graph-cli/src/node.ts +++ b/packages/targets/graph-cli/src/node.ts @@ -11,10 +11,12 @@ import { type GraphqlFetch, } from "@exadev/process-graph-source-github-issues"; import { githubTokenFromEnvironment } from "@exadev/process-graph-source-github-issues/node"; -import { openNodeGraphStore } from "@exadev/process-graph-sqlite-node"; +import { + openNodeGraphStore, + prepareDataFile, +} from "@exadev/process-graph-sqlite-node"; import { createGraphStoreRouter } from "@exadev/process-graph-store-contract"; import { createRouterClient } from "@orpc/server"; -import { prepareDataFile } from "./data-file.ts"; import type { CliPorts } from "./ports.ts"; import type { ExitCode } from "./exit-codes.ts"; import { runCli, type ProgramIdentity } from "./program.ts"; @@ -25,6 +27,10 @@ export interface NodePortsOptions { readonly env?: Readonly>; /** The home directory the default data path falls back on; the user's when absent. */ readonly home?: string; + /** + * A data file the host owns, such as one under its own home directory. It becomes the default data path in place of `PROCESS_GRAPH_DATA` and the XDG data file, and it gets the treatment the command line gives its own XDG data file: its directory, and any missing directories above it, are created readable by the owner only and kept that way, the database is created owner-only before SQLite opens it, and an existing store others can read is narrowed, with a warning on standard error (see `prepareDataFile` in `process-graph/store/node`). A path given with `--data` still wins, and is treated as the user's. The XDG data file when absent. + */ + readonly dataPath?: string; /** Supplies GitHub tokens; `GITHUB_TOKEN` or the GitHub CLI's token (`githubTokenFromEnvironment`) when absent. */ readonly token?: GitHubTokenPort; /** How requests reach GitHub; the global `fetch` when absent. */ @@ -109,7 +115,7 @@ export const packageVersion = (manifest: URL): string => { }; /** - * The ports for a Node process: the SQLite store over `node:sqlite` (creating the data file's directory when needed), the GitHub issues source for `github:` scopes, the file system and the process's streams. + * The ports for a Node process: the SQLite store over `node:sqlite` (creating the data file's directory when needed, private when the command line or the host owns it), the GitHub issues source for `github:` scopes, the file system and the process's streams. * @param options - Replacements for any of the defaults. * @returns The ports. */ @@ -117,8 +123,8 @@ export const nodePorts = (options: NodePortsOptions = {}): CliPorts => { const env = options.env ?? process.env; const token = options.token ?? githubTokenFromEnvironment({ env }); const home = options.home ?? homedir(); - // The command line's own data file, in the XDG data directory, whose directory it creates and keeps private; a path the user names is theirs. - const ownDataPath = xdgDataPath(env, home); + // The data file the command line owns, the host's or else its own in the XDG data directory, whose directory it creates and keeps private; a path the user names is theirs. + const ownDataPath = options.dataPath ?? xdgDataPath(env, home); return { stdout: @@ -131,7 +137,7 @@ export const nodePorts = (options: NodePortsOptions = {}): CliPorts => { ((text) => { process.stderr.write(text); }), - defaultDataPath: defaultDataPath(env, home), + defaultDataPath: options.dataPath ?? defaultDataPath(env, home), openStore: (path) => { const warnings = path === inMemory diff --git a/packages/targets/process-graph/README.md b/packages/targets/process-graph/README.md index 9d369a6..fcf4ef7 100644 --- a/packages/targets/process-graph/README.md +++ b/packages/targets/process-graph/README.md @@ -47,7 +47,7 @@ The package is built from the repository's internal packages, which are bundled | `process-graph/source/github` | The `GraphSource` port (contract, schemas, router, pagination and an in-memory fake) and the GitHub issues adapter, `createGitHubIssuesSource`. | Browser, Worker, Node | | `process-graph/source/github/node` | `githubTokenFromEnvironment`: a token port reading `GITHUB_TOKEN` or the GitHub CLI's token. | Node | | `process-graph/store` | The `GraphStore` port (contract, schemas, router, change feed, structural queries and an in-memory fake) and the SQLite store over any Drizzle SQLite database. | Browser, Worker, Node | -| `process-graph/store/node` | `openNodeGraphStore`: the SQLite store on Node's built-in `node:sqlite`. | Node | +| `process-graph/store/node` | `openNodeGraphStore`: the SQLite store on Node's built-in `node:sqlite`; `prepareDataFile`: makes its data file and directory private before it opens. | Node | | `process-graph/store/wasm` | `connectGraphStore`: connects a page to the store a dedicated worker serves. | Browser | | `process-graph/store/wasm/worker` | `serveGraphStoreInWorker`: serves the store from a dedicated worker on SQLite wasm and OPFS, shared between tabs with no special headers. | Dedicated worker | | `process-graph/react` | Headless React hooks and unstyled components: follow a stored model, derive its run, project, filter and select. | Browser | From 6da406c2a867275a1b2f4ae2c2e8e16689c3a243 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 17:31:19 +0100 Subject: [PATCH 03/12] fix(source-github-issues): report a failing token port as unauthorised A token port that threw anything but a GraphSourceError escaped the adapter as a plain error, which the router turned into an internal server error, so the command line exited with a general failure and the port's own error sat two causes deep. The adapter now wraps it in an unauthorised GraphSourceError naming the token port and the owner, with the port's error as its cause, and passes a GraphSourceError the port throws, or an abort, through unchanged. --- .../graph/source-github-issues/README.md | 2 +- .../graph/source-github-issues/src/graphql.ts | 35 ++++++++++- .../src/graphql.unit.test.ts | 58 ++++++++++++++++++- .../core/graph/source-contract/src/errors.ts | 2 +- 4 files changed, 92 insertions(+), 5 deletions(-) diff --git a/packages/adapters/graph/source-github-issues/README.md b/packages/adapters/graph/source-github-issues/README.md index 09463f3..327c8c3 100644 --- a/packages/adapters/graph/source-github-issues/README.md +++ b/packages/adapters/graph/source-github-issues/README.md @@ -79,7 +79,7 @@ A rate-limited sync resumes from the last page's `next` once the limit resets. ## Tokens -The token comes from a `GitHubTokenPort`, `(owner, signal) => Promise`, asked for the owner of each repository or board read, so different organisations can use different credentials, and with `owner` undefined for the viewer's own listing, which no owner names. The token is sent only in the `Authorization` header and never logged or put in an error. +The token comes from a `GitHubTokenPort`, `(owner, signal) => Promise`, asked for the owner of each repository or board read, so different organisations can use different credentials, and with `owner` undefined for the viewer's own listing, which no owner names. The token is sent only in the `Authorization` header and never logged or put in an error. A port that cannot supply a token ends the call as `unauthorised`, just as GitHub rejecting one would: a `GraphSourceError` it throws is passed through as it is, and any other error is wrapped in one whose message names the token port and the owner (`the GitHub token port gave no token for : `) and whose cause is the port's own error. `storedTokenPort({ list, used })` chooses among tokens a host keeps (a browser's token store, say), each scoped to some owners or to any: a token naming the owner wins, the one naming the fewest owners first and then the most recently used; otherwise the most recently used token for any owner; a request with no owner takes only a token for any owner. Logins match without regard to case. `list` is read on every request, `used` is told which token was chosen so the host can keep its recency, and no fitting token is `unauthorised`. `selectStoredToken` is the choice on its own. diff --git a/packages/adapters/graph/source-github-issues/src/graphql.ts b/packages/adapters/graph/source-github-issues/src/graphql.ts index a4a440d..52a811e 100644 --- a/packages/adapters/graph/source-github-issues/src/graphql.ts +++ b/packages/adapters/graph/source-github-issues/src/graphql.ts @@ -12,6 +12,7 @@ export const githubGraphqlEndpoint = "https://api.github.com/graphql"; * @param owner - The login of the account whose repositories or board the request reads, or `undefined` for a request about the token's own holder (the viewer and the organisations it belongs to), which no owner names. * @param signal - The caller's abort signal. * @returns The token. + * @throws GraphSourceError with `unauthorised` when no token can be had, with a message saying how to get one; any other error is reported by the adapter as `unauthorised` too, naming the token port and the owner, with that error as the cause. */ export type GitHubTokenPort = ( owner: string | undefined, @@ -151,13 +152,43 @@ export interface GraphqlRequest { readonly schema: Schema; } +/** + * Asks the token port for the token a request needs. A port that cannot supply one has failed to provide credentials, which ends the sync exactly as GitHub rejecting them would, so whatever it throws is reported as `unauthorised`: a `GraphSourceError` the port threw as it is, and anything else wrapped in one whose message names the token port and the owner and whose cause is the port's own error. An abort is passed through, since it is the caller's, not the port's. + * @param transport - The token port. + * @param owner - The login the request is about, or `undefined` for the token's own holder. + * @param signal - The caller's abort signal. + * @returns The token. + * @throws GraphSourceError with `unauthorised` when the port fails. + */ +const tokenFor = async ( + transport: GraphqlTransport, + owner: string | undefined, + signal: AbortSignal | undefined, +): Promise => { + try { + return await transport.token(owner, signal); + } catch (error: unknown) { + if (error instanceof GraphSourceError || signal?.aborted === true) { + throw error; + } + const reason = error instanceof Error ? error.message : String(error); + throw new GraphSourceError( + { + kind: "unauthorised", + message: `the GitHub token port gave no token${owner === undefined ? "" : ` for ${owner}`}: ${reason}`, + }, + { cause: error }, + ); + } +}; + /** * Sends one GraphQL request and classifies the answer. * @param transport - The token, fetch and clock ports. * @param request - The document, its variables, the owner whose token to use and the schema for `data`. * @param signal - The caller's abort signal, passed to `fetch`. * @returns The parsed data and any `NOT_FOUND` or `FORBIDDEN` problems beside it. - * @throws GraphSourceError with `unauthorised` for HTTP 401, and `rate-limited` (with the reset instant when GitHub gives one) for any form of rate limiting. + * @throws GraphSourceError with `unauthorised` for HTTP 401 or a token port that fails, and `rate-limited` (with the reset instant when GitHub gives one) for any form of rate limiting. * @throws Error for any other failure: an HTTP error, another GraphQL error type, or data that does not match the schema. */ export const graphql = async ( @@ -165,7 +196,7 @@ export const graphql = async ( request: GraphqlRequest, signal: AbortSignal | undefined, ): Promise>> => { - const token = await transport.token(request.owner, signal); + const token = await tokenFor(transport, request.owner, signal); const response = await transport.fetch(githubGraphqlEndpoint, { method: "POST", headers: { diff --git a/packages/adapters/graph/source-github-issues/src/graphql.unit.test.ts b/packages/adapters/graph/source-github-issues/src/graphql.unit.test.ts index 68c885f..0cf8861 100644 --- a/packages/adapters/graph/source-github-issues/src/graphql.unit.test.ts +++ b/packages/adapters/graph/source-github-issues/src/graphql.unit.test.ts @@ -1,6 +1,11 @@ +import { + createGraphSourceRouter, + GraphSourceError, +} from "@exadev/process-graph-source-contract"; import { inProcessClient } from "@exadev/process-graph-source-contract/conformance"; import { describe, expect, test } from "vitest"; -import type { GraphqlFetch } from "./graphql.ts"; +import type { GitHubTokenPort, GraphqlFetch } from "./graphql.ts"; +import { createGitHubIssuesSource } from "./source.ts"; import { fakeToken, fixedNow, stubbedRouter } from "./test-support/sources.ts"; const scope = ["github:ExaDev/document-schema.js"]; @@ -260,3 +265,54 @@ describe("requests", () => { ]); }); }); + +describe("a token port that fails", () => { + /** A fetch the request must never reach, since it has no token to send. */ + const unreachable: GraphqlFetch = async () => + Promise.reject(new Error("fetch was called without a token")); + + /** A page read through the router with this token port. */ + const pageWithToken = async (token: GitHubTokenPort) => + inProcessClient( + createGraphSourceRouter( + createGitHubIssuesSource({ + token, + fetch: unreachable, + now: () => fixedNow, + }), + ), + ).v1.sync.page({ scope }); + + test("is UNAUTHORIZED, naming the token port and the owner, with the port's own error as the cause of the source error", async () => { + const original = new Error("the keychain is locked"); + const failure = await pageWithToken(async () => + Promise.reject(original), + ).catch((error: unknown) => error); + + expect(failure).toMatchObject({ + code: "UNAUTHORIZED", + message: + "the GitHub token port gave no token for ExaDev: the keychain is locked", + }); + expect( + failure instanceof Error && + failure.cause instanceof GraphSourceError && + failure.cause.cause, + ).toBe(original); + }); + + test("passes a GraphSourceError the port throws through as it is", async () => { + const own = new GraphSourceError({ + kind: "unauthorised", + message: "no stored GitHub token is for ExaDev or for any owner", + }); + + await expect( + pageWithToken(async () => Promise.reject(own)), + ).rejects.toMatchObject({ + code: "UNAUTHORIZED", + message: "no stored GitHub token is for ExaDev or for any owner", + cause: own, + }); + }); +}); diff --git a/packages/core/graph/source-contract/src/errors.ts b/packages/core/graph/source-contract/src/errors.ts index 2a98410..27c39c8 100644 --- a/packages/core/graph/source-contract/src/errors.ts +++ b/packages/core/graph/source-contract/src/errors.ts @@ -3,7 +3,7 @@ import type { BadRequestReason } from "./schema.ts"; /** * The failures that end a sync call, as a closed union a caller can switch on exhaustively. What a source can still partly read is not a failure: it arrives as a `Gap` in the page. * - * - `unauthorised`: the source rejected the credentials outright. + * - `unauthorised`: the source rejected the credentials outright, or none could be had for it. * - `rate-limited`: the source refused the request until `resetAt` (an ISO 8601 instant) when it said so. Resume from the last page's `next` after that. * - `bad-request`: the cursor or a scope identifier is not one this source issued or understands. */ From 0863a3d2345cdf303ede144e39d3b95b49b934a5 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 17:32:04 +0100 Subject: [PATCH 04/12] feat(cli): report an interrupted sync's resume point as data The library sync operation ended its error with command-line wording ("resume with: sync ..."), which a host calling it as a library could only parse. It now throws SyncInterruptedError, a CliError carrying the source's failure as its message, the request that resumes the failing source, the scopes not reached and whether pages were saved; the command line renders those as the commands it prints. The error's cause is the error underneath the source failure, so a token port's own error is one level deep, and such a failure exits with the source-failure status. --- packages/targets/graph-cli/README.md | 2 + packages/targets/graph-cli/src/errors.ts | 18 ++ packages/targets/graph-cli/src/index.ts | 2 + packages/targets/graph-cli/src/program.ts | 31 +++- packages/targets/graph-cli/src/render.ts | 22 ++- packages/targets/graph-cli/src/sync.ts | 66 +++++-- .../targets/graph-cli/src/sync.unit.test.ts | 164 ++++++++++++++++++ 7 files changed, 283 insertions(+), 22 deletions(-) create mode 100644 packages/targets/graph-cli/src/sync.unit.test.ts diff --git a/packages/targets/graph-cli/README.md b/packages/targets/graph-cli/README.md index 50b2b21..16e20a9 100644 --- a/packages/targets/graph-cli/README.md +++ b/packages/targets/graph-cli/README.md @@ -101,6 +101,8 @@ process.exitCode = await runCli(args, nodePorts(), { The operation behind each command is exported on its own and returns plain data rather than text, for hosts that want results, such as tools served over MCP: `sync`, `ready`, `blocked`, `impactOf`, `path`, `cycles`, `handoffsOf`, `why`, `show`, `add`, `setState`, `setArtefact`, `setFact`, `factsOf`, `validateStored` and `validateDocument`, each taking a `GraphSession` (a store client, a model id, the host's facts and a signal). The emitters `flowchartToMermaid`, `swimlaneToMermaid`, `stateMachineToMermaid`, `sequenceToMermaid` and their `...ToDot` counterparts are pure functions of a projection. +When a source fails part way, `sync` throws a `SyncInterruptedError`: a `CliError` with the source's failure as its message and exit status (a token port that cannot supply a token is a source failure, status 5), the error underneath it as its `cause`, and where to carry on as data rather than text: `resume`, the `SyncRequest` that continues the failing source after the cursor of its last page read, `remaining`, the scopes of the sources not reached, and `saved`, whether the pages read before the failure were written. The command line renders those as the `resume with:` and `then:` commands it prints. + `add` reads the model and saves it whole, so a write another process makes between the two is overwritten; the store checks every save, so a duplicate identifier or a dangling reference is rejected and nothing changes. ## Commands diff --git a/packages/targets/graph-cli/src/errors.ts b/packages/targets/graph-cli/src/errors.ts index 4b77684..b9230d7 100644 --- a/packages/targets/graph-cli/src/errors.ts +++ b/packages/targets/graph-cli/src/errors.ts @@ -1,5 +1,6 @@ import { CycleError, UnknownNodeError } from "@exadev/process-graph-dag"; import { ModelParseError } from "@exadev/process-graph-schema"; +import { GraphSourceError } from "@exadev/process-graph-source-contract"; import { ORPCError } from "@orpc/server"; import { CommanderError } from "commander"; import { exitCodes, type ExitCode } from "./exit-codes.ts"; @@ -56,6 +57,23 @@ const issueLines = (data: unknown): readonly string[] => { ); }; +/** + * The error underneath a source's failure, for a failure's cause. A source called in process through oRPC fails with an `ORPCError` whose cause is the `GraphSourceError` the source threw, whose own cause, when it has one, is what went wrong underneath, such as a token port's error. Those two layers say nothing more than the failure's message and exit status already do, so the innermost error is the one worth keeping. + * @param error - What the source call threw. + * @returns The `GraphSourceError`'s cause when it has one, otherwise that `GraphSourceError`; anything else as it is. + */ +export const underlyingCause = (error: unknown): unknown => { + const sourceError = + error instanceof ORPCError && error.cause instanceof GraphSourceError + ? error.cause + : error; + if (!(sourceError instanceof GraphSourceError)) { + return error; + } + + return sourceError.cause === undefined ? sourceError : sourceError.cause; +}; + /** The exit status and message an error is reported with. */ export interface Failure { readonly exitCode: ExitCode; diff --git a/packages/targets/graph-cli/src/index.ts b/packages/targets/graph-cli/src/index.ts index f361349..083b425 100644 --- a/packages/targets/graph-cli/src/index.ts +++ b/packages/targets/graph-cli/src/index.ts @@ -73,7 +73,9 @@ export { } from "./validate.ts"; export { sync, + SyncInterruptedError, type SourceLookup, + type SyncInterruption, type SyncRequest, type SyncResult, } from "./sync.ts"; diff --git a/packages/targets/graph-cli/src/program.ts b/packages/targets/graph-cli/src/program.ts index c2d4a05..05dd6eb 100644 --- a/packages/targets/graph-cli/src/program.ts +++ b/packages/targets/graph-cli/src/program.ts @@ -16,7 +16,7 @@ import { type Addition, type ArtefactState, } from "./edit.ts"; -import { failureOf } from "./errors.ts"; +import { CliError, failureOf } from "./errors.ts"; import { exitCodes, type ExitCode } from "./exit-codes.ts"; import type { CliPorts } from "./ports.ts"; import { @@ -41,6 +41,7 @@ import { renderSetArtefact, renderSetFact, renderSync, + renderSyncInterruption, renderValidate, renderWhy, } from "./render.ts"; @@ -52,7 +53,7 @@ import { type FormatName, type ViewName, } from "./show.ts"; -import { sync } from "./sync.ts"; +import { sync, SyncInterruptedError } from "./sync.ts"; import { terminalJson, terminalText } from "./terminal-text.ts"; import { validateDocument, validateStored } from "./validate.ts"; @@ -246,13 +247,25 @@ export const runCli = async ( ) => { const { after } = options; await withSession(async (session) => { - const result = await sync( - session, - ports.source, - typeof after === "string" ? { scopes, after } : { scopes }, - ); - - return { json: result, text: renderSync(result) }; + try { + const result = await sync( + session, + ports.source, + typeof after === "string" ? { scopes, after } : { scopes }, + ); + + return { json: result, text: renderSync(result) }; + } catch (error) { + // The library reports where to carry on as data; the command line says it as the commands to run. + if (error instanceof SyncInterruptedError) { + throw new CliError( + error.exitCode, + renderSyncInterruption(error.message, error), + { cause: error }, + ); + } + throw error; + } })(command); }, ); diff --git a/packages/targets/graph-cli/src/render.ts b/packages/targets/graph-cli/src/render.ts index 01357ba..a7eab18 100644 --- a/packages/targets/graph-cli/src/render.ts +++ b/packages/targets/graph-cli/src/render.ts @@ -17,7 +17,7 @@ import type { WhyResult, } from "./queries.ts"; import type { ElementSummary } from "./session.ts"; -import type { SyncResult } from "./sync.ts"; +import type { SyncInterruption, SyncResult } from "./sync.ts"; import type { ValidateResult } from "./validate.ts"; /** @@ -173,6 +173,26 @@ export const renderSync = (result: SyncResult): string => ), ]); +/** + * A sync a source interrupted: the source's failure, whether the pages read before it were saved, and the commands that carry on, `resume with: sync --after ` for the failing source and `then: sync ` for the sources not reached. + * @param message - The source's failure. + * @param interruption - Where the sync carries on. + * @returns The lines for standard error, without a final line break. + */ +export const renderSyncInterruption = ( + message: string, + interruption: SyncInterruption, +): string => { + const { resume, remaining, saved } = interruption; + + return [ + message, + ...(saved ? ["the pages read before the failure are saved"] : []), + `resume with: sync ${resume.scopes.join(" ")}${resume.after === undefined ? "" : ` --after ${resume.after}`}`, + ...(remaining.length === 0 ? [] : [`then: sync ${remaining.join(" ")}`]), + ].join("\n"); +}; + /** `validate`: `valid`, or each issue as its code, its JSON pointer when it has one, and its message. */ export const renderValidate = (result: ValidateResult): string => result.valid diff --git a/packages/targets/graph-cli/src/sync.ts b/packages/targets/graph-cli/src/sync.ts index 4c813d6..4570539 100644 --- a/packages/targets/graph-cli/src/sync.ts +++ b/packages/targets/graph-cli/src/sync.ts @@ -5,7 +5,7 @@ import { type SyncPage, } from "@exadev/process-graph-source-contract"; import type { BoundActivity } from "@exadev/process-graph-store-contract"; -import { CliError, failureOf } from "./errors.ts"; +import { CliError, failureOf, underlyingCause } from "./errors.ts"; import { exitCodes } from "./exit-codes.ts"; import { callOptions, type GraphSession } from "./session.ts"; @@ -20,6 +20,46 @@ export interface SyncRequest { readonly after?: string; } +/** Where a sync a source interrupted carries on, as data a host acts on; the command line renders it as the commands to run. */ +export interface SyncInterruption { + /** The request that continues the failing source: its scopes, after the cursor of the last page read from it, or from the start when none was read. */ + readonly resume: SyncRequest; + /** The scopes of the sources after the failing one, which were not read; sync them afresh once `resume` has finished. */ + readonly remaining: readonly string[]; + /** Whether pages read before the failure were written to the model. */ + readonly saved: boolean; +} + +/** + * Thrown by `sync` when a source fails part way. Its exit status and message are the source's failure, classified as `failureOf` classifies it; its cause is the error underneath that failure, such as the token port's own error; and what to run next is data, in `resume` and `remaining`, never text. + */ +export class SyncInterruptedError extends CliError implements SyncInterruption { + readonly resume: SyncRequest; + + readonly remaining: readonly string[]; + + readonly saved: boolean; + + /** + * @param exitCode - The exit status the source's failure maps to. + * @param message - The source's failure. + * @param interruption - Where the sync carries on. + * @param options - The underlying error, kept as the cause. + */ + constructor( + exitCode: CliError["exitCode"], + message: string, + interruption: SyncInterruption, + options?: ErrorOptions, + ) { + super(exitCode, message, options); + this.name = "SyncInterruptedError"; + this.resume = interruption.resume; + this.remaining = interruption.remaining; + this.saved = interruption.saved; + } +} + /** What a sync wrote. */ export interface SyncResult { readonly modelId: string; @@ -82,12 +122,12 @@ const boundActivities = (pages: readonly SyncPage[]) => { /** * Reads every page of the scopes from their sources and merges them into the session's model in one write, creating the model when the store has none. Records already in the model are replaced by identifier and the rest are kept, so syncing again refreshes what changed. * - * When a source fails part way, the pages read before the failure are still written, and the error says how to resume: the failing source's scopes with `--after` and the cursor of its last page read. + * When a source fails part way, the pages read before the failure are still written, and the error carries where to carry on: the request that resumes the failing source after the cursor of its last page read, and the scopes of the sources not reached. * @param session - The store and model. * @param sources - The source for each identifier prefix. * @param request - The scopes, and a cursor to resume after. * @returns What was written, and the gaps the sources reported. - * @throws CliError with `usage` for a scope with no source prefix, a prefix no source serves, or `after` with scopes of more than one source; and with the source's failure, the resume command added, when a source refuses the sync. + * @throws CliError with `usage` for a scope with no source prefix, a prefix no source serves, or `after` with scopes of more than one source; and SyncInterruptedError, with the source's failure and where to carry on, when a source fails part way. */ export const sync = async ( session: GraphSession, @@ -155,16 +195,18 @@ export const sync = async ( if (saved) { await write(); } - const later = clients.slice(index + 1).flatMap((next) => next.scopes); - throw new CliError( + throw new SyncInterruptedError( failure.exitCode, - [ - failure.message, - ...(saved ? ["the pages read before the failure are saved"] : []), - `resume with: sync ${group.scopes.join(" ")}${cursor === undefined ? "" : ` --after ${cursor}`}`, - ...(later.length === 0 ? [] : [`then: sync ${later.join(" ")}`]), - ].join("\n"), - { cause: error }, + failure.message, + { + resume: + cursor === undefined + ? { scopes: group.scopes } + : { scopes: group.scopes, after: cursor }, + remaining: clients.slice(index + 1).flatMap((next) => next.scopes), + saved, + }, + { cause: underlyingCause(error) }, ); } await readFrom(index + 1); diff --git a/packages/targets/graph-cli/src/sync.unit.test.ts b/packages/targets/graph-cli/src/sync.unit.test.ts new file mode 100644 index 0000000..4a87f32 --- /dev/null +++ b/packages/targets/graph-cli/src/sync.unit.test.ts @@ -0,0 +1,164 @@ +import { + createFakeGraphSource, + createGraphSourceRouter, + GraphSourceError, + type GraphSourcePager, +} from "@exadev/process-graph-source-contract"; +import { + createGitHubIssuesSource, + type GraphqlFetch, +} from "@exadev/process-graph-source-github-issues"; +import { + createFakeGraphStore, + createGraphStoreRouter, +} from "@exadev/process-graph-store-contract"; +import { createRouterClient } from "@orpc/server"; +import { afterEach, describe, expect, test } from "vitest"; +import { exitCodes } from "./exit-codes.ts"; +import type { GraphSession } from "./session.ts"; +import { sync, SyncInterruptedError, type SourceLookup } from "./sync.ts"; +import { githubLikeSource } from "./test-support/fixtures.ts"; +import { createHarness, type Harness } from "./test-support/harness.ts"; + +/** A session over an in-memory store. */ +const session = (): GraphSession => ({ + store: createRouterClient(createGraphStoreRouter(createFakeGraphStore())), + modelId: "default", + facts: () => ({ found: false }), + signal: undefined, +}); + +/** Serves each pager's scopes through the real oRPC router, in process. */ +const lookup = + (...pagers: readonly GraphSourcePager[]): SourceLookup => + (prefix) => { + const pager = pagers.find((candidate) => candidate.source === prefix); + + return pager === undefined + ? undefined + : createRouterClient(createGraphSourceRouter(pager)); + }; + +/** The fixture's GitHub-like source, rate limited after its first page. */ +const rateLimitedAfterOnePage = (): { + readonly pager: GraphSourcePager; + readonly failure: GraphSourceError; +} => { + const fake = createFakeGraphSource(githubLikeSource); + const failure = new GraphSourceError({ + kind: "rate-limited", + message: "rate limited", + resetAt: undefined, + }); + let reads = 0; + + return { + failure, + pager: { + source: fake.source, + page: async (request) => { + reads += 1; + if (reads > 1) { + throw failure; + } + + return fake.page(request); + }, + }, + }; +}; + +/** The GitHub issues adapter with a token port that fails with `original`, and a fetch the sync must never reach. */ +const githubWithFailingToken = (original: Error): GraphSourcePager => { + const unreachable: GraphqlFetch = async () => + Promise.reject(new Error("fetch was called without a token")); + + return createGitHubIssuesSource({ + token: async () => Promise.reject(original), + fetch: unreachable, + now: () => 0, + }); +}; + +/** Runs `sync` and returns what it threw. */ +const interruption = async ( + sources: SourceLookup, + scopes: readonly string[], +): Promise => + sync(session(), sources, { scopes }).then( + () => { + throw new Error("the sync did not fail"); + }, + (error: unknown) => error, + ); + +describe("a sync a source interrupts, called as a library", () => { + test("carries where to carry on as data, and only the source's failure as its message", async () => { + const { pager, failure } = rateLimitedAfterOnePage(); + const jira = createFakeGraphSource({ + source: "jira", + scopes: { "jira:ACME": [] }, + }); + + const error = await interruption(lookup(pager, jira), [ + "github:acme/app", + "jira:ACME", + "github:acme/web", + ]); + + expect(error).toBeInstanceOf(SyncInterruptedError); + expect(error).toMatchObject({ + exitCode: exitCodes.sourceFailure, + message: "rate limited", + resume: { + scopes: ["github:acme/app", "github:acme/web"], + after: "github%3Aacme%2Fapp@0", + }, + remaining: ["jira:ACME"], + saved: true, + cause: failure, + }); + }); + + test("reports a token port that fails as a source failure naming the token port, with the port's own error as the cause", async () => { + const original = new Error("the keychain is locked"); + + const error = await interruption(lookup(githubWithFailingToken(original)), [ + "github:acme/app", + ]); + + expect(error).toBeInstanceOf(SyncInterruptedError); + expect(error).toMatchObject({ + exitCode: exitCodes.sourceFailure, + message: + "the GitHub token port gave no token for acme: the keychain is locked", + resume: { scopes: ["github:acme/app"] }, + remaining: [], + saved: false, + }); + expect(error instanceof Error && error.cause).toBe(original); + expect(error).not.toHaveProperty(["resume", "after"]); + }); +}); + +describe("a sync a source interrupts, run as a command", () => { + let harness: Harness | undefined; + + afterEach(() => { + harness?.remove(); + }); + + test("exits with the source-failure status and prints the command that resumes it", async () => { + harness = createHarness( + githubWithFailingToken(new Error("the keychain is locked")), + ); + + expect(await harness.run("sync", "github:acme/app")).toEqual({ + code: exitCodes.sourceFailure, + stdout: "", + stderr: + "the GitHub token port gave no token for acme: the keychain is locked\n" + + "resume with: sync github:acme/app\n", + }); + }); +}); From 46c1f69d6f9fd90d6529a486e04f1969bc93c383 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 17:40:58 +0100 Subject: [PATCH 05/12] feat(process-graph): export the manifest and a build-time version constant A consumer had no way to read the package's version. The exports map now carries ./package.json, and process-graph/cli exports version, which tsdown writes into the bundle through define from the manifest when the package is built, so reading it touches no file at run time. Vitest applies the same define, and the bin's --version uses the constant too. The smoke test imports both from the packed tarball. --- packages/targets/process-graph/README.md | 3 ++- .../targets/process-graph/build-constants.ts | 25 ++++++++++++++++++ packages/targets/process-graph/package.json | 3 ++- packages/targets/process-graph/src/bin.ts | 10 +++---- .../process-graph/src/build-constants.d.ts | 2 ++ packages/targets/process-graph/src/cli.ts | 3 ++- .../src/entrypoints.unit.test.ts | 6 ++++- packages/targets/process-graph/src/version.ts | 2 ++ .../process-graph/src/version.unit.test.ts | 12 +++++++++ .../process-graph/test/package.smoke.test.ts | 26 +++++++++++++++++++ .../targets/process-graph/tsdown.config.ts | 4 +++ .../targets/process-graph/vitest.config.ts | 3 +++ 12 files changed, 90 insertions(+), 9 deletions(-) create mode 100644 packages/targets/process-graph/build-constants.ts create mode 100644 packages/targets/process-graph/src/build-constants.d.ts create mode 100644 packages/targets/process-graph/src/version.ts create mode 100644 packages/targets/process-graph/src/version.unit.test.ts diff --git a/packages/targets/process-graph/README.md b/packages/targets/process-graph/README.md index fcf4ef7..9c4f09b 100644 --- a/packages/targets/process-graph/README.md +++ b/packages/targets/process-graph/README.md @@ -55,7 +55,8 @@ The package is built from the repository's internal packages, which are bundled | `process-graph/ui/style.css` | The compiled styles for `process-graph/ui`. | Browser | | `process-graph/react-flow` | React Flow views of a flowchart, swimlane or state machine, laid out by `process-graph/layout`. | Browser | | `process-graph/react-flow/style.css` | The compiled styles for `process-graph/react-flow`, React Flow's own included. | Browser | -| `process-graph/cli` | The command line as a library: `runCli` over injected ports, `nodePorts` for a Node process, and each command's operation returning plain data. | Node | +| `process-graph/cli` | The command line as a library: `runCli` over injected ports, `nodePorts` for a Node process, each command's operation returning plain data, and `version`, this package's own. | Node | +| `process-graph/package.json` | The package's manifest, for reading its version or anything else it declares. | Any | `zod` and `trilean`, which the root entry needs, are dependencies. The libraries only other subpaths need are optional peer dependencies, so installing `process-graph` for its root entry pulls in nothing else; install the peers of each subpath you import beside it: diff --git a/packages/targets/process-graph/build-constants.ts b/packages/targets/process-graph/build-constants.ts new file mode 100644 index 0000000..186645b --- /dev/null +++ b/packages/targets/process-graph/build-constants.ts @@ -0,0 +1,25 @@ +import { readFileSync } from "node:fs"; + +/** The manifest's version, read once when a build or test run is configured. */ +const manifestVersion = (): string => { + const manifest: unknown = JSON.parse( + readFileSync(new URL("package.json", import.meta.url), "utf8"), + ); + if ( + typeof manifest !== "object" || + manifest === null || + !("version" in manifest) || + typeof manifest.version !== "string" + ) { + throw new Error("package.json has no version"); + } + + return manifest.version; +}; + +/** + * The identifiers the bundler replaces at build time, each with the JavaScript expression it becomes, so the bundle carries the values rather than reading them at run time. `tsdown.config.ts` and `vitest.config.ts` both pass these as `define`, so source run under test sees what the bundle carries; `src/build-constants.d.ts` declares each one. + */ +export const buildConstants: Readonly> = { + PROCESS_GRAPH_VERSION: JSON.stringify(manifestVersion()), +}; diff --git a/packages/targets/process-graph/package.json b/packages/targets/process-graph/package.json index 4b36bcf..4165ee2 100644 --- a/packages/targets/process-graph/package.json +++ b/packages/targets/process-graph/package.json @@ -65,7 +65,8 @@ "./cli": { "types": "./dist/cli.d.ts", "import": "./dist/cli.js" - } + }, + "./package.json": "./package.json" }, "types": "./dist/index.d.ts", "files": [ diff --git a/packages/targets/process-graph/src/bin.ts b/packages/targets/process-graph/src/bin.ts index df6af4a..d1a532e 100644 --- a/packages/targets/process-graph/src/bin.ts +++ b/packages/targets/process-graph/src/bin.ts @@ -6,6 +6,7 @@ import { missingPeers, missingPeersMessage, } from "./missing-peers.ts"; +import { version } from "./version.ts"; /** This package's manifest, beside `dist/`. */ const manifest = new URL("../package.json", import.meta.url); @@ -25,8 +26,8 @@ const declaredPeers = (): Map => { return new Map( Object.entries(parsed.peerDependencies).flatMap( - ([name, version]: [string, unknown]) => - typeof version === "string" ? [[name, version] as const] : [], + ([name, specifier]: [string, unknown]) => + typeof specifier === "string" ? [[name, specifier] as const] : [], ), ); }; @@ -62,10 +63,9 @@ if (missing.length > 0) { process.exitCode = exitCodes.missingPeers; } else { // Imported only once its peers are known to resolve, so a missing one is reported above rather than as an import failure. - const { packageVersion, runProcess } = - await import("@exadev/process-graph-cli"); + const { runProcess } = await import("@exadev/process-graph-cli"); process.exitCode = await runProcess({ name: "process-graph", - version: packageVersion(manifest), + version, }); } diff --git a/packages/targets/process-graph/src/build-constants.d.ts b/packages/targets/process-graph/src/build-constants.d.ts new file mode 100644 index 0000000..06f11af --- /dev/null +++ b/packages/targets/process-graph/src/build-constants.d.ts @@ -0,0 +1,2 @@ +/** The `process-graph` manifest's version, replaced with a string literal by the bundler's `define` (see `build-constants.ts`). */ +declare const PROCESS_GRAPH_VERSION: string; diff --git a/packages/targets/process-graph/src/cli.ts b/packages/targets/process-graph/src/cli.ts index 1c90461..4d88b05 100644 --- a/packages/targets/process-graph/src/cli.ts +++ b/packages/targets/process-graph/src/cli.ts @@ -1,2 +1,3 @@ -// The `process-graph/cli` entry point: the command line as a library. `runCli` runs the whole command surface against injected ports, `nodePorts` wires those ports for a Node process, and the operations behind each command and the Mermaid and DOT emitters return plain data. Node-only, which is why the root entry point does not carry it. +// The `process-graph/cli` entry point: the command line as a library. `runCli` runs the whole command surface against injected ports, `nodePorts` wires those ports for a Node process, and the operations behind each command and the Mermaid and DOT emitters return plain data. `version` is this package's own version, for a host that presents it. Node-only, which is why the root entry point does not carry it. export * from "@exadev/process-graph-cli"; +export { version } from "./version.ts"; diff --git a/packages/targets/process-graph/src/entrypoints.unit.test.ts b/packages/targets/process-graph/src/entrypoints.unit.test.ts index 81fb4b5..6faf975 100644 --- a/packages/targets/process-graph/src/entrypoints.unit.test.ts +++ b/packages/targets/process-graph/src/entrypoints.unit.test.ts @@ -19,6 +19,7 @@ import * as githubNodeEntry from "./source/github/node.ts"; import * as storeEntry from "./store.ts"; import * as storeNodeEntry from "./store/node.ts"; import * as uiEntry from "./ui.ts"; +import * as versionModule from "./version.ts"; /** One subpath entry point and the bundled packages whose exports it carries flat. */ interface Subpath { @@ -78,7 +79,10 @@ const subpaths: readonly Subpath[] = [ { name: "cli", entry: cliEntry, - layers: [["cli", cliPackage]], + layers: [ + ["cli", cliPackage], + ["version", versionModule], + ], }, ]; diff --git a/packages/targets/process-graph/src/version.ts b/packages/targets/process-graph/src/version.ts new file mode 100644 index 0000000..99fa4c2 --- /dev/null +++ b/packages/targets/process-graph/src/version.ts @@ -0,0 +1,2 @@ +/** The version of the installed `process-graph` package, as its manifest names it. The bundler writes it in when the package is built, so reading it touches no file. */ +export const version: string = PROCESS_GRAPH_VERSION; diff --git a/packages/targets/process-graph/src/version.unit.test.ts b/packages/targets/process-graph/src/version.unit.test.ts new file mode 100644 index 0000000..53fee5d --- /dev/null +++ b/packages/targets/process-graph/src/version.unit.test.ts @@ -0,0 +1,12 @@ +import { readFileSync } from "node:fs"; +import { expect, test } from "vitest"; +import { version } from "./version.ts"; + +test("version is the manifest's version", () => { + const manifest: unknown = JSON.parse( + readFileSync(new URL("../package.json", import.meta.url), "utf8"), + ); + + expect(manifest).toHaveProperty("version", version); + expect(version).toMatch(/^\d+\.\d+\.\d+/); +}); diff --git a/packages/targets/process-graph/test/package.smoke.test.ts b/packages/targets/process-graph/test/package.smoke.test.ts index 0783a58..cccd3e5 100644 --- a/packages/targets/process-graph/test/package.smoke.test.ts +++ b/packages/targets/process-graph/test/package.smoke.test.ts @@ -201,6 +201,32 @@ describe("each subpath, imported from the installed package", () => { }); }); + test("process-graph/package.json and process-graph/cli's version name the installed package's version", () => { + const installedManifest: unknown = JSON.parse( + readFileSync(join(installed, "package.json"), "utf8"), + ); + if ( + typeof installedManifest !== "object" || + installedManifest === null || + !("version" in installedManifest) || + typeof installedManifest.version !== "string" + ) { + throw new Error("the installed manifest has no version"); + } + + expect( + evaluate(` + import manifest from "process-graph/package.json" with { type: "json" }; + import { version } from "process-graph/cli"; + console.log(JSON.stringify({ name: manifest.name, manifest: manifest.version, cli: version })); + `), + ).toEqual({ + name: "process-graph", + manifest: installedManifest.version, + cli: installedManifest.version, + }); + }); + test("process-graph/cli runs a command through the Node ports", () => { expect( evaluate(` diff --git a/packages/targets/process-graph/tsdown.config.ts b/packages/targets/process-graph/tsdown.config.ts index 2c0b52d..bcbac78 100644 --- a/packages/targets/process-graph/tsdown.config.ts +++ b/packages/targets/process-graph/tsdown.config.ts @@ -1,5 +1,6 @@ import { fileURLToPath } from "node:url"; import { defineConfig } from "tsdown"; +import { buildConstants } from "./build-constants.ts"; /** Where a bundled UI package's compiled stylesheet is, resolved through that package's own `./style.css` export. */ const stylesheetOf = (name: string): string => @@ -10,6 +11,8 @@ const stylesheetOf = (name: string): string => * * The bundler carries no CSS across from the UI packages, which compile their Vanilla Extract styles into a `style.css` of their own, so each stylesheet is copied beside its subpath and exported as `/style.css`. * + * `define` writes build-time values into the bundle, such as the package's version for `process-graph/cli`, so nothing reads the manifest at run time. + * * `onlyBundle` is the guarantee rather than a hint: the build fails if anything from `node_modules` other than a workspace package would be inlined, so every third-party library stays an import resolved from the consumer's own install, as declared in `package.json`. Node builtins stay imports too, since the neutral platform leaves them unresolved; the entries that reach them (`cli`, `bin`, `store/node`, `source/github/node`) are the Node-only ones. */ export default defineConfig({ @@ -43,6 +46,7 @@ export default defineConfig({ to: "dist/react-flow", }, ], + define: { ...buildConstants }, tsconfig: "tsconfig.node.json", clean: true, }); diff --git a/packages/targets/process-graph/vitest.config.ts b/packages/targets/process-graph/vitest.config.ts index dd5e6f7..dc106bc 100644 --- a/packages/targets/process-graph/vitest.config.ts +++ b/packages/targets/process-graph/vitest.config.ts @@ -1,7 +1,10 @@ import { playwright } from "@vitest/browser-playwright"; import { defineConfig } from "vitest/config"; +import { buildConstants } from "./build-constants.ts"; export default defineConfig({ + // The values tsdown writes into the bundle, so source under test reads what the built package carries. + define: { ...buildConstants }, test: { projects: [ { From f24ff4c3d7db794ade8ebbcabdffe68d64ff6cae Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 17:47:25 +0100 Subject: [PATCH 06/12] test(store): check prepareDataFile under each umask where it lives The umask probe opened the command line's default store through nodePorts, loading the whole command line to test a guarantee that belongs to prepareDataFile. It now lives in sqlite-node and prepares and opens a store directly, with no per-test clock bound, since a spawned process's duration is the machine's scheduling; the command line keeps an in-process check that it hands its XDG data file to prepareDataFile as one it owns. --- .../sqlite-node/src/data-file.unit.test.ts | 47 ++++++++++++++++ .../graph/sqlite-node}/test/umask-probe.ts | 29 +++++----- .../graph-cli/src/data-file.unit.test.ts | 54 ++++++------------- 3 files changed, 80 insertions(+), 50 deletions(-) rename packages/{targets/graph-cli => adapters/graph/sqlite-node}/test/umask-probe.ts (51%) diff --git a/packages/adapters/graph/sqlite-node/src/data-file.unit.test.ts b/packages/adapters/graph/sqlite-node/src/data-file.unit.test.ts index d9f51ee..d5ec0f2 100644 --- a/packages/adapters/graph/sqlite-node/src/data-file.unit.test.ts +++ b/packages/adapters/graph/sqlite-node/src/data-file.unit.test.ts @@ -6,8 +6,10 @@ import { statSync, writeFileSync, } from "node:fs"; +import { spawnSync } from "node:child_process"; import { tmpdir } from "node:os"; import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; import { afterEach, beforeEach, describe, expect, test } from "vitest"; import { prepareDataFile, @@ -21,6 +23,15 @@ const looseDirectoryMode = 0o755; const looseFileMode = 0o644; /** The permission bits of a mode. */ const permissionBits = 0o777; +/** A umask that leaves new files readable by group and others, the usual default, so a test fails if the code relies on the umask for privacy. */ +const permissiveUmask = 0o022; +/** A umask that removes nothing, so every mode the code asks for is the mode it gets. */ +const openUmask = 0o000; +/** The radix the probe reads its umask argument in. */ +const octal = 8; + +/** The probe that prepares and opens a store under a given umask, in a process of its own, since only a main thread may set the umask and Vitest runs tests in worker threads. */ +const probe = fileURLToPath(new URL("../test/umask-probe.ts", import.meta.url)); const modeOf = (path: string): number => statSync(path).mode & permissionBits; @@ -42,6 +53,42 @@ const looseStore = (path: string): void => { chmodSync(path, looseFileMode); }; +/** No per-test time limit for the probe: it starts a Node process, so how long it takes is the machine's scheduling, not a property of this code, and a clock bound only fails it under load. The CI job's own timeout still ends a hang. */ +const unbounded = 0; + +describe( + "a store prepared in a process with its own umask", + { timeout: unbounded }, + () => { + test.each([ + { name: "the usual 022", umask: permissiveUmask }, + { name: "000, which narrows nothing", umask: openUmask }, + ])( + "under a umask of $name, creates the missing directories, the database and the -wal and -shm files SQLite derives from it private", + ({ umask }) => { + const result = spawnSync( + process.execPath, + [probe, umask.toString(octal), directory], + { encoding: "utf8" }, + ); + if (result.status !== 0) { + throw new Error(result.stderr); + } + const modes: unknown = JSON.parse(result.stdout); + + expect(modes).toEqual({ + warnings: [], + home: privateDirectoryMode, + graph: privateDirectoryMode, + database: privateFileMode, + wal: privateFileMode, + shm: privateFileMode, + }); + }, + ); + }, +); + describe("in a directory the caller owns", () => { test("creates the missing directories and the database private, and the store SQLite opens there keeps its sidecars private", () => { const path = join(directory, "home", "graph", "graph.sqlite3"); diff --git a/packages/targets/graph-cli/test/umask-probe.ts b/packages/adapters/graph/sqlite-node/test/umask-probe.ts similarity index 51% rename from packages/targets/graph-cli/test/umask-probe.ts rename to packages/adapters/graph/sqlite-node/test/umask-probe.ts index ec0f61d..5a7be39 100644 --- a/packages/targets/graph-cli/test/umask-probe.ts +++ b/packages/adapters/graph/sqlite-node/test/umask-probe.ts @@ -1,10 +1,11 @@ import { statSync } from "node:fs"; import { dirname, join } from "node:path"; import { emptyModel } from "@exadev/process-graph-store-contract"; -import { nodePorts } from "../src/node.ts"; +import { prepareDataFile } from "../src/data-file.ts"; +import { openNodeGraphStore } from "../src/open.ts"; /** - * Run as its own process by `data-file.unit.test.ts`, because only a main thread may set the umask and Vitest (under Stryker especially) runs tests in worker threads. Arguments: the umask as an octal string, and a directory to use as the XDG data directory's parent. It sets the umask, opens the command line's own default store there, writes to it, and prints the permission bits of each directory and file it made, and the warnings the port returned, as one JSON object, while the store is still open so SQLite's `-wal` and `-shm` files exist. + * Run as its own process by `data-file.unit.test.ts`, because only a main thread may set the umask and Vitest (under Stryker especially) runs tests in worker threads. Arguments: the umask as an octal string, and a directory to hold the store. It sets the umask, prepares a data file two missing directories deep in a directory it owns, opens the store there, writes to it, and prints the permission bits of each directory and file it made, and the warnings `prepareDataFile` returned, as one JSON object, while the store is still open so SQLite's `-wal` and `-shm` files exist. */ const [umask, directory] = process.argv.slice(2); @@ -18,25 +19,29 @@ const octal = 8; const permissionBits = 0o777; process.umask(Number.parseInt(umask, octal)); -const ports = nodePorts({ - env: { XDG_DATA_HOME: join(directory, "xdg") }, - home: directory, +const path = join(directory, "home", "graph", "graph.sqlite3"); +const warnings = prepareDataFile(path, { + ownsDirectory: true, + platform: process.platform, }); -const path = ports.defaultDataPath; -const opened = await ports.openStore(path); +const { store, close } = openNodeGraphStore(path); try { - await opened.client.v1.models.save({ modelId: "default", model: emptyModel }); + await store.saveModel({ + modelId: "default", + model: emptyModel, + signal: undefined, + }); const modeOf = (file: string): number => statSync(file).mode & permissionBits; process.stdout.write( JSON.stringify({ - warnings: opened.warnings, - xdg: modeOf(join(directory, "xdg")), - app: modeOf(dirname(path)), + warnings, + home: modeOf(join(directory, "home")), + graph: modeOf(dirname(path)), database: modeOf(path), wal: modeOf(`${path}-wal`), shm: modeOf(`${path}-shm`), }), ); } finally { - await opened.close(); + close(); } diff --git a/packages/targets/graph-cli/src/data-file.unit.test.ts b/packages/targets/graph-cli/src/data-file.unit.test.ts index 5aaeab2..e019cdb 100644 --- a/packages/targets/graph-cli/src/data-file.unit.test.ts +++ b/packages/targets/graph-cli/src/data-file.unit.test.ts @@ -6,10 +6,8 @@ import { statSync, writeFileSync, } from "node:fs"; -import { spawnSync } from "node:child_process"; import { tmpdir } from "node:os"; import { dirname, join } from "node:path"; -import { fileURLToPath } from "node:url"; import { afterEach, beforeEach, describe, expect, test } from "vitest"; import { privateDirectoryMode, @@ -19,13 +17,7 @@ import { exitCodes } from "./exit-codes.ts"; import { nodePorts } from "./node.ts"; import { runCli } from "./program.ts"; -/** A umask that leaves new files readable by group and others, the usual default, so a test fails if the code relies on the umask for privacy. */ -const permissiveUmask = 0o022; -/** A umask that removes nothing, so every mode the code asks for is the mode it gets. */ -const openUmask = 0o000; -/** The radix the probe reads its umask argument in. */ -const octal = 8; -/** The modes such a umask gives a new directory and file. */ +/** The modes the usual umask gives a new directory and file, readable by group and others. */ const looseDirectoryMode = 0o755; const looseFileMode = 0o644; /** The permission bits of a mode. */ @@ -43,9 +35,6 @@ afterEach(() => { rmSync(directory, { recursive: true, force: true }); }); -/** The probe that opens a default store under a given umask, in a process of its own, since only a main thread may set the umask and Vitest runs tests in worker threads. */ -const probe = fileURLToPath(new URL("../test/umask-probe.ts", import.meta.url)); - /** The Node ports with the XDG data directory inside the test's directory, so the default store is the command line's own. */ const portsIn = (stderr: (text: string) => void = () => undefined) => nodePorts({ @@ -55,32 +44,21 @@ const portsIn = (stderr: (text: string) => void = () => undefined) => }); describe("the command line's own default store", () => { - test.each([ - { name: "the usual 022", umask: permissiveUmask }, - { name: "000, which narrows nothing", umask: openUmask }, - ])( - "under a umask of $name, is created private, with the directories it creates, the database and the -wal and -shm files SQLite derives from it", - ({ umask }) => { - const result = spawnSync( - process.execPath, - [probe, umask.toString(octal), directory], - { encoding: "utf8" }, - ); - if (result.status !== 0) { - throw new Error(result.stderr); - } - const modes: unknown = JSON.parse(result.stdout); - - expect(modes).toEqual({ - warnings: [], - xdg: privateDirectoryMode, - app: privateDirectoryMode, - database: privateFileMode, - wal: privateFileMode, - shm: privateFileMode, - }); - }, - ); + // Whatever the umask, `prepareDataFile` makes these modes; its own tests in sqlite-node set the umask in a probe process. This checks the command line hands it its XDG data file as one it owns. + test("is created private, with the XDG data directory above it", async () => { + const ports = portsIn(); + const path = ports.defaultDataPath; + const opened = await ports.openStore(path); + await opened.close(); + + expect(path).toBe(join(directory, "xdg", "process-graph", "graph.sqlite3")); + expect(opened.warnings).toEqual([]); + expect([ + modeOf(join(directory, "xdg")), + modeOf(dirname(path)), + modeOf(path), + ]).toEqual([privateDirectoryMode, privateDirectoryMode, privateFileMode]); + }); test("is narrowed when it already exists readable by others, and the command line says so on standard error", async () => { let stderr = ""; From c3cdd52ed184a6c51fb4f1b2b963ae003f5f2903 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 17:53:50 +0100 Subject: [PATCH 07/12] test(cli): lift the clock bound on the spawned bin smoke test The test starts the built bin as a process three times, so how long it takes is the machine's scheduling rather than a property of the code, and the default per-test bound failed it on a loaded host. It now runs unbounded, as the projection package's work-counting cases do, leaving a hang to the CI job's own timeout. --- .../graph-cli/test/package.smoke.test.ts | 39 +++++++++++-------- 1 file changed, 23 insertions(+), 16 deletions(-) diff --git a/packages/targets/graph-cli/test/package.smoke.test.ts b/packages/targets/graph-cli/test/package.smoke.test.ts index 27f75d0..fb2336f 100644 --- a/packages/targets/graph-cli/test/package.smoke.test.ts +++ b/packages/targets/graph-cli/test/package.smoke.test.ts @@ -45,20 +45,27 @@ test("the built library runs a command against a SQLite file through the Node po expect(stdout).toBe("added activity build at revision 1\nbuild\tBuild\n"); }); -test("the built bin runs as a process, creating its data directory, and reports its exit status", () => { - const data = join(directory, "nested", "graph.sqlite3"); +/** No per-test time limit for the bin: each run starts a Node process, so how long it takes is the machine's scheduling, not a property of this code, and a clock bound only fails it under load. The CI job's own timeout still ends a hang. */ +const unbounded = 0; - expect(runBin(["add", "activity", "build", "Build"], data).status).toBe( - exitCodes.success, - ); - const ready = runBin(["ready", "--json"], data); - expect(ready.status).toBe(exitCodes.success); - expect(JSON.parse(ready.stdout)).toEqual({ - elements: [{ id: "build", label: "Build" }], - }); - expect(runBin(["blocked", "nothing"], data)).toMatchObject({ - status: exitCodes.notFound, - stdout: "", - stderr: 'no element "nothing" in the model\n', - }); -}); +test( + "the built bin runs as a process, creating its data directory, and reports its exit status", + { timeout: unbounded }, + () => { + const data = join(directory, "nested", "graph.sqlite3"); + + expect(runBin(["add", "activity", "build", "Build"], data).status).toBe( + exitCodes.success, + ); + const ready = runBin(["ready", "--json"], data); + expect(ready.status).toBe(exitCodes.success); + expect(JSON.parse(ready.stdout)).toEqual({ + elements: [{ id: "build", label: "Build" }], + }); + expect(runBin(["blocked", "nothing"], data)).toMatchObject({ + status: exitCodes.notFound, + stdout: "", + stderr: 'no element "nothing" in the model\n', + }); + }, +); From 1617206317c76143a0605b16157f8a66adf7815f Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 17:57:13 +0100 Subject: [PATCH 08/12] docs: say an install age gate must exempt trilean and trilean-regex A consumer with an install age gate exempted process-graph but still could not install it, because trilean and its dependency trilean-regex are ExaDev packages released alongside it and often younger than the gate. The root README gains an install section saying so, and the package README points to it. --- README.md | 8 ++++++++ packages/targets/process-graph/README.md | 2 +- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 938151f..dcbcc74 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,14 @@ A process and dependency graph model, with the reasoning, command line and React TypeScript, ESM, pnpm workspaces, Turborepo, Vitest, ESLint with `@exadev/eslint-config`, semantic-release. +## Install + +```sh +pnpm add process-graph +``` + +The root entry needs `zod` and `trilean`, which install with it; each other subpath names the optional peers to install beside it in the [`process-graph` README](packages/targets/process-graph/README.md). A project with an install age gate, such as pnpm's `minimumReleaseAge`, must exempt `trilean` and its own dependency `trilean-regex` as well as `process-graph`, since both are ExaDev packages released alongside it and are often younger than the gate; with pnpm, add all three to `minimumReleaseAgeExclude` in `pnpm-workspace.yaml`. + ## Getting started Node and pnpm versions are pinned in `.tool-versions` and `package.json`'s `packageManager`; install through mise. diff --git a/packages/targets/process-graph/README.md b/packages/targets/process-graph/README.md index 9c4f09b..5b61d79 100644 --- a/packages/targets/process-graph/README.md +++ b/packages/targets/process-graph/README.md @@ -58,7 +58,7 @@ The package is built from the repository's internal packages, which are bundled | `process-graph/cli` | The command line as a library: `runCli` over injected ports, `nodePorts` for a Node process, each command's operation returning plain data, and `version`, this package's own. | Node | | `process-graph/package.json` | The package's manifest, for reading its version or anything else it declares. | Any | -`zod` and `trilean`, which the root entry needs, are dependencies. The libraries only other subpaths need are optional peer dependencies, so installing `process-graph` for its root entry pulls in nothing else; install the peers of each subpath you import beside it: +`zod` and `trilean`, which the root entry needs, are dependencies; a project with an install age gate exempts `trilean` and `trilean-regex` beside `process-graph` (see [Install](../../../README.md#install)). The libraries only other subpaths need are optional peer dependencies, so installing `process-graph` for its root entry pulls in nothing else; install the peers of each subpath you import beside it: | Subpath | Peer dependencies | | ------------------- | ---------------------------------------------------------------------------- | From 11a5f1592df1939081e35d5478be39fe1ce5580c Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 18:05:32 +0100 Subject: [PATCH 09/12] feat(ui): add a theme entry without the components The theme contract vars was only exported beside the components, so a Vanilla Extract stylesheet that themed them loaded the whole component library and React. The ./theme entry carries vars, the light and dark theme classes and their values on their own; the library build emits it as a separate file sharing one chunk with the components' entry. --- packages/ui/graph/ui/README.md | 4 ++-- packages/ui/graph/ui/eslint.config.ts | 2 ++ packages/ui/graph/ui/package.json | 4 ++++ packages/ui/graph/ui/src/theme.ts | 5 +++++ .../ui/graph/ui/test/package.smoke.test.tsx | 22 ++++++++++++++++++- packages/ui/graph/ui/vite.config.ts | 4 ++-- 6 files changed, 36 insertions(+), 5 deletions(-) create mode 100644 packages/ui/graph/ui/src/theme.ts diff --git a/packages/ui/graph/ui/README.md b/packages/ui/graph/ui/README.md index 762412a..0b13cc4 100644 --- a/packages/ui/graph/ui/README.md +++ b/packages/ui/graph/ui/README.md @@ -18,11 +18,11 @@ import { ElementCard, GraphTheme, Legend } from "@exadev/process-graph-ui"; `vars` is the theme contract (`createThemeContract`): one CSS custom property for each value in `ThemeValues` (colours, each status's colour, spacing, radii, type and stroke widths). Every style reads the contract and nothing else. `lightTheme` and `darkTheme` are the classes that assign `lightValues` and `darkValues`; `GraphTheme` puts one of them on a `div` with `surfaceRoot`, which sets the page colours and type the components inherit. -A consumer restyles the components without forking them by assigning the contract itself, in its own Vanilla Extract build: +A consumer restyles the components without forking them by assigning the contract itself, in its own Vanilla Extract build. The `./theme` entry (`process-graph/ui/theme`) carries `vars`, the two theme classes and their values without the components, so a `.css.ts` file that imports it loads no React: ```ts import { createTheme } from "@vanilla-extract/css"; -import { vars, lightValues } from "@exadev/process-graph-ui"; +import { vars, lightValues } from "@exadev/process-graph-ui/theme"; export const brandTheme = createTheme(vars, { ...lightValues, diff --git a/packages/ui/graph/ui/eslint.config.ts b/packages/ui/graph/ui/eslint.config.ts index 5b6a264..0e62eb6 100644 --- a/packages/ui/graph/ui/eslint.config.ts +++ b/packages/ui/graph/ui/eslint.config.ts @@ -11,5 +11,7 @@ export default defineConfig( isomorphic: true, browserGlobals: true, react: { version }, + // The `./theme` entry point, a pure re-export of the theme contract and values beside the components' `src/index.ts`. + barrelEntries: ["src/theme.ts"], }), ); diff --git a/packages/ui/graph/ui/package.json b/packages/ui/graph/ui/package.json index 4eda018..1702550 100644 --- a/packages/ui/graph/ui/package.json +++ b/packages/ui/graph/ui/package.json @@ -18,6 +18,10 @@ "types": "./dist/index.d.ts", "import": "./dist/index.js" }, + "./theme": { + "types": "./dist/theme.d.ts", + "import": "./dist/theme.js" + }, "./style.css": "./dist/style.css" }, "types": "./dist/index.d.ts", diff --git a/packages/ui/graph/ui/src/theme.ts b/packages/ui/graph/ui/src/theme.ts new file mode 100644 index 0000000..c7c3308 --- /dev/null +++ b/packages/ui/graph/ui/src/theme.ts @@ -0,0 +1,5 @@ +/** + * The `./theme` entry: the theme contract `vars`, the `lightTheme` and `darkTheme` classes and the values they assign, without the components. A Vanilla Extract stylesheet that themes the components with `createTheme(vars, values)` or `assignVars` imports this entry, so building it loads no React and no component code. The compiled rules for these classes are in `style.css`, the same stylesheet the components use. + */ +export { vars, lightTheme, darkTheme } from "./theme.css.ts"; +export { lightValues, darkValues, type ThemeValues } from "./theme-values.ts"; diff --git a/packages/ui/graph/ui/test/package.smoke.test.tsx b/packages/ui/graph/ui/test/package.smoke.test.tsx index acb0df8..390dc01 100644 --- a/packages/ui/graph/ui/test/package.smoke.test.tsx +++ b/packages/ui/graph/ui/test/package.smoke.test.tsx @@ -3,7 +3,16 @@ import { readFileSync } from "node:fs"; import { join } from "node:path"; import { afterEach, beforeAll, expect, test } from "vitest"; // Deliberately imports the built dist/, not src/: this is the check that catches a wrong exports map, a build that left Vanilla Extract calls in the JavaScript, or a stylesheet the build failed to emit. -import { GraphTheme, StatusBadge, lightValues } from "../dist/index.js"; +import { + GraphTheme, + StatusBadge, + darkTheme, + darkValues, + lightTheme, + lightValues, + vars, +} from "../dist/index.js"; +import * as theme from "../dist/theme.js"; afterEach(cleanup); @@ -27,3 +36,14 @@ test("the built package's components take their colours from the compiled styles lightValues.colour.status.blocked, ); }); + +test("the built theme entry carries the same contract, classes and values as the components' entry, and only those", () => { + expect({ ...theme }).toEqual({ + vars, + lightTheme, + darkTheme, + lightValues, + darkValues, + }); + expect(theme.vars.colour.status.blocked).toMatch(/^var\(--/); +}); diff --git a/packages/ui/graph/ui/vite.config.ts b/packages/ui/graph/ui/vite.config.ts index 48f2180..c32e223 100644 --- a/packages/ui/graph/ui/vite.config.ts +++ b/packages/ui/graph/ui/vite.config.ts @@ -10,9 +10,9 @@ export default defineConfig({ oxc: { jsx: { runtime: "automatic" } }, build: { lib: { - entry: "src/index.ts", + // The theme entry is its own file so that importing it loads no component code; what the two share is emitted once as a chunk. + entry: { index: "src/index.ts", theme: "src/theme.ts" }, formats: ["es"], - fileName: "index", cssFileName: "style", }, rolldownOptions: { From ed019f979c1369b0d9042c9e7f931b8d3b118bb6 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 18:06:01 +0100 Subject: [PATCH 10/12] feat(process-graph): add the process-graph/ui/theme subpath process-graph/ui/theme exports the theme contract, the theme classes and their values without the components, for a .css.ts file that themes process-graph/ui. The smoke test imports it in the consumer project that has none of the optional peers, where React does not resolve, and checks it carries the same contract and values as process-graph/ui. --- packages/targets/process-graph/README.md | 36 ++++++++++--------- .../targets/process-graph/eslint.config.ts | 1 + packages/targets/process-graph/package.json | 4 +++ .../src/entrypoints.unit.test.ts | 7 ++++ .../targets/process-graph/src/ui/theme.ts | 2 ++ .../process-graph/test/package.smoke.test.ts | 30 ++++++++++++++-- .../targets/process-graph/tsdown.config.ts | 1 + 7 files changed, 62 insertions(+), 19 deletions(-) create mode 100644 packages/targets/process-graph/src/ui/theme.ts diff --git a/packages/targets/process-graph/README.md b/packages/targets/process-graph/README.md index 5b61d79..4986dc1 100644 --- a/packages/targets/process-graph/README.md +++ b/packages/targets/process-graph/README.md @@ -40,23 +40,24 @@ deriveStatuses(model, { submit: "done" }, facts); // Map { "submit" => "done", " The package is built from the repository's internal packages, which are bundled into it and not published on their own; each internal package's README documents its functions and rules. Each subpath loads only the code it reaches. -| Import | What it is | Runs in | -| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | -| `process-graph` | The model (`defineModel`, `parseModel`, `validate`), its projections and inverses, and the reasoning over a run (statuses, ready set, blocked-by, impact, what-if, critical path). | Browser, Worker, Node | -| `process-graph/layout` | `layout`: positions for a projected flowchart, swimlane or state machine, in the model's `views` shape, through elkjs. | Browser, Worker, Node | -| `process-graph/source/github` | The `GraphSource` port (contract, schemas, router, pagination and an in-memory fake) and the GitHub issues adapter, `createGitHubIssuesSource`. | Browser, Worker, Node | -| `process-graph/source/github/node` | `githubTokenFromEnvironment`: a token port reading `GITHUB_TOKEN` or the GitHub CLI's token. | Node | -| `process-graph/store` | The `GraphStore` port (contract, schemas, router, change feed, structural queries and an in-memory fake) and the SQLite store over any Drizzle SQLite database. | Browser, Worker, Node | -| `process-graph/store/node` | `openNodeGraphStore`: the SQLite store on Node's built-in `node:sqlite`; `prepareDataFile`: makes its data file and directory private before it opens. | Node | -| `process-graph/store/wasm` | `connectGraphStore`: connects a page to the store a dedicated worker serves. | Browser | -| `process-graph/store/wasm/worker` | `serveGraphStoreInWorker`: serves the store from a dedicated worker on SQLite wasm and OPFS, shared between tabs with no special headers. | Dedicated worker | -| `process-graph/react` | Headless React hooks and unstyled components: follow a stored model, derive its run, project, filter and select. | Browser | -| `process-graph/ui` | Styled components over `process-graph/react`, the SVG sequence diagram and the theme contract with light and dark themes. | Browser | -| `process-graph/ui/style.css` | The compiled styles for `process-graph/ui`. | Browser | -| `process-graph/react-flow` | React Flow views of a flowchart, swimlane or state machine, laid out by `process-graph/layout`. | Browser | -| `process-graph/react-flow/style.css` | The compiled styles for `process-graph/react-flow`, React Flow's own included. | Browser | -| `process-graph/cli` | The command line as a library: `runCli` over injected ports, `nodePorts` for a Node process, each command's operation returning plain data, and `version`, this package's own. | Node | -| `process-graph/package.json` | The package's manifest, for reading its version or anything else it declares. | Any | +| Import | What it is | Runs in | +| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | +| `process-graph` | The model (`defineModel`, `parseModel`, `validate`), its projections and inverses, and the reasoning over a run (statuses, ready set, blocked-by, impact, what-if, critical path). | Browser, Worker, Node | +| `process-graph/layout` | `layout`: positions for a projected flowchart, swimlane or state machine, in the model's `views` shape, through elkjs. | Browser, Worker, Node | +| `process-graph/source/github` | The `GraphSource` port (contract, schemas, router, pagination and an in-memory fake) and the GitHub issues adapter, `createGitHubIssuesSource`. | Browser, Worker, Node | +| `process-graph/source/github/node` | `githubTokenFromEnvironment`: a token port reading `GITHUB_TOKEN` or the GitHub CLI's token. | Node | +| `process-graph/store` | The `GraphStore` port (contract, schemas, router, change feed, structural queries and an in-memory fake) and the SQLite store over any Drizzle SQLite database. | Browser, Worker, Node | +| `process-graph/store/node` | `openNodeGraphStore`: the SQLite store on Node's built-in `node:sqlite`; `prepareDataFile`: makes its data file and directory private before it opens. | Node | +| `process-graph/store/wasm` | `connectGraphStore`: connects a page to the store a dedicated worker serves. | Browser | +| `process-graph/store/wasm/worker` | `serveGraphStoreInWorker`: serves the store from a dedicated worker on SQLite wasm and OPFS, shared between tabs with no special headers. | Dedicated worker | +| `process-graph/react` | Headless React hooks and unstyled components: follow a stored model, derive its run, project, filter and select. | Browser | +| `process-graph/ui` | Styled components over `process-graph/react`, the SVG sequence diagram and the theme contract with light and dark themes. | Browser | +| `process-graph/ui/theme` | The theme contract `vars`, `lightTheme`, `darkTheme`, `lightValues` and `darkValues` without the components, for a Vanilla Extract stylesheet that themes `process-graph/ui` without loading React. | Any | +| `process-graph/ui/style.css` | The compiled styles for `process-graph/ui`. | Browser | +| `process-graph/react-flow` | React Flow views of a flowchart, swimlane or state machine, laid out by `process-graph/layout`. | Browser | +| `process-graph/react-flow/style.css` | The compiled styles for `process-graph/react-flow`, React Flow's own included. | Browser | +| `process-graph/cli` | The command line as a library: `runCli` over injected ports, `nodePorts` for a Node process, each command's operation returning plain data, and `version`, this package's own. | Node | +| `process-graph/package.json` | The package's manifest, for reading its version or anything else it declares. | Any | `zod` and `trilean`, which the root entry needs, are dependencies; a project with an install age gate exempts `trilean` and `trilean-regex` beside `process-graph` (see [Install](../../../README.md#install)). The libraries only other subpaths need are optional peer dependencies, so installing `process-graph` for its root entry pulls in nothing else; install the peers of each subpath you import beside it: @@ -70,6 +71,7 @@ The package is built from the repository's internal packages, which are bundled | `store/wasm/worker` | `@orpc/contract`, `@orpc/server`, `drizzle-orm`, `@sqlite.org/sqlite-wasm` | | `react` | `react` | | `ui` | `react`, `@vanilla-extract/recipes`, `@vanilla-extract/css` (types) | +| `ui/theme` | `@vanilla-extract/css` (types) | | `react-flow` | `react`, `react-dom`, `@xyflow/react`, `elkjs`, `@vanilla-extract/recipes` | | `cli` and the bin | `commander`, `@orpc/client`, `@orpc/contract`, `@orpc/server`, `drizzle-orm` | diff --git a/packages/targets/process-graph/eslint.config.ts b/packages/targets/process-graph/eslint.config.ts index 9560b24..0c4d89d 100644 --- a/packages/targets/process-graph/eslint.config.ts +++ b/packages/targets/process-graph/eslint.config.ts @@ -12,6 +12,7 @@ const subpathEntries = [ "src/store/wasm/worker.ts", "src/react.ts", "src/ui.ts", + "src/ui/theme.ts", "src/react-flow.ts", "src/cli.ts", ]; diff --git a/packages/targets/process-graph/package.json b/packages/targets/process-graph/package.json index 4165ee2..a6b577e 100644 --- a/packages/targets/process-graph/package.json +++ b/packages/targets/process-graph/package.json @@ -56,6 +56,10 @@ "types": "./dist/ui.d.ts", "import": "./dist/ui.js" }, + "./ui/theme": { + "types": "./dist/ui/theme.d.ts", + "import": "./dist/ui/theme.js" + }, "./react-flow": { "types": "./dist/react-flow.d.ts", "import": "./dist/react-flow.js" diff --git a/packages/targets/process-graph/src/entrypoints.unit.test.ts b/packages/targets/process-graph/src/entrypoints.unit.test.ts index 6faf975..28068e0 100644 --- a/packages/targets/process-graph/src/entrypoints.unit.test.ts +++ b/packages/targets/process-graph/src/entrypoints.unit.test.ts @@ -9,6 +9,7 @@ import * as sqliteNodePackage from "@exadev/process-graph-sqlite-node"; import * as storeContractPackage from "@exadev/process-graph-store-contract"; import * as storeSqlitePackage from "@exadev/process-graph-store-sqlite"; import * as uiPackage from "@exadev/process-graph-ui"; +import * as uiThemePackage from "@exadev/process-graph-ui/theme"; import { describe, expect, test } from "vitest"; import * as cliEntry from "./cli.ts"; import * as layoutEntry from "./layout.ts"; @@ -19,6 +20,7 @@ import * as githubNodeEntry from "./source/github/node.ts"; import * as storeEntry from "./store.ts"; import * as storeNodeEntry from "./store/node.ts"; import * as uiEntry from "./ui.ts"; +import * as uiThemeEntry from "./ui/theme.ts"; import * as versionModule from "./version.ts"; /** One subpath entry point and the bundled packages whose exports it carries flat. */ @@ -71,6 +73,11 @@ const subpaths: readonly Subpath[] = [ entry: uiEntry, layers: [["ui", uiPackage]], }, + { + name: "ui/theme", + entry: uiThemeEntry, + layers: [["ui/theme", uiThemePackage]], + }, { name: "react-flow", entry: reactFlowEntry, diff --git a/packages/targets/process-graph/src/ui/theme.ts b/packages/targets/process-graph/src/ui/theme.ts new file mode 100644 index 0000000..39c3b4f --- /dev/null +++ b/packages/targets/process-graph/src/ui/theme.ts @@ -0,0 +1,2 @@ +// The `process-graph/ui/theme` entry point: the theme contract `vars`, the `lightTheme` and `darkTheme` classes and the values they assign, without the components, so a Vanilla Extract stylesheet that themes `process-graph/ui` loads no React. The rules for the classes are in `process-graph/ui/style.css`. +export * from "@exadev/process-graph-ui/theme"; diff --git a/packages/targets/process-graph/test/package.smoke.test.ts b/packages/targets/process-graph/test/package.smoke.test.ts index cccd3e5..5596d41 100644 --- a/packages/targets/process-graph/test/package.smoke.test.ts +++ b/packages/targets/process-graph/test/package.smoke.test.ts @@ -11,13 +11,14 @@ const installed = join(consumer, "node_modules", "process-graph"); /** * Runs an ES module in the consumer's project, as a consumer's own code would run, and parses the JSON it prints. * @param source - The module's source; it prints one JSON document. + * @param project - The project to run it in; the one with every peer installed when absent. * @returns The printed document. */ -const evaluate = (source: string): unknown => { +const evaluate = (source: string, project: string = consumer): unknown => { const result = spawnSync( process.execPath, ["--input-type=module", "--eval", source], - { cwd: consumer, encoding: "utf8" }, + { cwd: project, encoding: "utf8" }, ); if (result.status !== 0) { throw new Error(result.stderr); @@ -254,6 +255,31 @@ describe("each subpath, imported from the installed package", () => { }); }); +describe("process-graph/ui/theme, in a project with none of the optional peers", () => { + test("imports without React, and carries the same contract and values as process-graph/ui", () => { + const bareConsumer = inject("bareConsumer"); + const theme = ` + import * as theme from "process-graph/ui/theme"; + const resolves = (name) => { try { import.meta.resolve(name); return true; } catch { return false; } }; + console.log(JSON.stringify({ react: resolves("react"), exports: Object.keys(theme).sort(), vars: theme.vars, light: theme.lightValues })); + `; + const bare = evaluate(theme, bareConsumer); + const full = evaluate(` + import { vars, lightValues } from "process-graph/ui"; + console.log(JSON.stringify({ vars, light: lightValues })); + `); + + expect(bare).toMatchObject({ + react: false, + exports: ["darkTheme", "darkValues", "lightTheme", "lightValues", "vars"], + }); + if (typeof full !== "object" || full === null) { + throw new Error("process-graph/ui printed no theme"); + } + expect(bare).toMatchObject(full); + }); +}); + describe("the installed bin", () => { const bin = join(consumer, "node_modules", ".bin", "process-graph"); const manifest: unknown = JSON.parse( diff --git a/packages/targets/process-graph/tsdown.config.ts b/packages/targets/process-graph/tsdown.config.ts index bcbac78..f8bad10 100644 --- a/packages/targets/process-graph/tsdown.config.ts +++ b/packages/targets/process-graph/tsdown.config.ts @@ -27,6 +27,7 @@ export default defineConfig({ "store/wasm/worker": "src/store/wasm/worker.ts", react: "src/react.ts", ui: "src/ui.ts", + "ui/theme": "src/ui/theme.ts", "react-flow": "src/react-flow.ts", cli: "src/cli.ts", bin: "src/bin.ts", From 9d4ff60ae54239a6dadef19afe9e9b8ed6ae943d Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 18:01:15 +0100 Subject: [PATCH 11/12] fix(schema): declare modelSchema's type by name A consumer that embedded modelSchema in a schema of its own and emitted declarations had the whole model schema written out inline, trilean's JSON value included, which could exceed the length TypeScript will serialise (TS7056) and forced an annotation. modelSchema is now declared as the exported interface ModelSchema, whose unwrap still returns the strict object schema, so such a declaration refers to the name. A test compiles a consumer-style z.object({ model: modelSchema }) under declaration: true and checks it emits without diagnostics and names ModelSchema. --- packages/core/graph/schema/README.md | 2 +- .../schema/src/declaration-emit.unit.test.ts | 69 +++++++++++++++++++ packages/core/graph/schema/src/index.ts | 2 +- packages/core/graph/schema/src/schema.ts | 23 +++++-- 4 files changed, 88 insertions(+), 8 deletions(-) create mode 100644 packages/core/graph/schema/src/declaration-emit.unit.test.ts diff --git a/packages/core/graph/schema/README.md b/packages/core/graph/schema/README.md index 5b6261a..b0b2600 100644 --- a/packages/core/graph/schema/README.md +++ b/packages/core/graph/schema/README.md @@ -85,7 +85,7 @@ The two cover different things on purpose. - **Types** carry what only an author can get wrong before the model exists: a reference to an undeclared identifier, an identifier of the wrong space, an entity acting. They need the identifiers to be literals, so they only apply to a model written in source. - **The open schema** carries what a loaded document can get wrong: shape, vocabulary, unknown keys (every object is strict, so a misspelt field is an error and is never dropped), then the rules in `validate`. -`z.infer` is exactly `Model`, pinned by a type-level test, so a loaded model is usable anywhere an open-space model is. +`z.infer` is exactly `Model`, pinned by a type-level test, so a loaded model is usable anywhere an open-space model is. `modelSchema` is declared as the named interface `ModelSchema`, so a consumer that embeds it in a schema of its own, such as `z.object({ model: modelSchema })`, and emits declarations gets a reference to `ModelSchema` rather than the whole model schema written out, which can exceed the length TypeScript will serialise (TS7056). ## Extensions diff --git a/packages/core/graph/schema/src/declaration-emit.unit.test.ts b/packages/core/graph/schema/src/declaration-emit.unit.test.ts new file mode 100644 index 0000000..3ef24c4 --- /dev/null +++ b/packages/core/graph/schema/src/declaration-emit.unit.test.ts @@ -0,0 +1,69 @@ +import { join } from "node:path"; +import ts from "typescript"; +import { expect, test } from "vitest"; + +/** A consumer's module that embeds `modelSchema` in a schema of its own and exports it, so its declaration file has to write the schema's type. */ +const consumerSource = `import { z } from "zod"; +import { modelSchema } from "./index.ts"; + +export const envelope = z.object({ model: modelSchema }); +`; + +/** + * Emits the declaration file of a module beside this package's source, as a consumer compiling with `declaration: true` would. + * @param source - The module's source. + * @returns The emitted declaration file and every diagnostic the program and the emit reported, as `TS: `. + */ +const declarationsOf = ( + source: string, +): { + readonly declarations: string; + readonly diagnostics: readonly string[]; +} => { + const file = join(import.meta.dirname, "declaration-emit.consumer.ts"); + const options: ts.CompilerOptions = { + declaration: true, + emitDeclarationOnly: true, + strict: true, + exactOptionalPropertyTypes: true, + allowImportingTsExtensions: true, + module: ts.ModuleKind.NodeNext, + moduleResolution: ts.ModuleResolutionKind.NodeNext, + target: ts.ScriptTarget.ES2024, + skipLibCheck: true, + }; + const host = ts.createCompilerHost(options); + const getSourceFile = host.getSourceFile.bind(host); + const fileExists = host.fileExists.bind(host); + host.getSourceFile = (name, languageVersion, ...rest) => + name === file + ? ts.createSourceFile(name, source, languageVersion) + : getSourceFile(name, languageVersion, ...rest); + host.fileExists = (name) => name === file || fileExists(name); + let declarations = ""; + host.writeFile = (name, text) => { + if (name.endsWith("declaration-emit.consumer.d.ts")) { + declarations = text; + } + }; + const program = ts.createProgram([file], options, host); + const consumer = program.getSourceFile(file); + const emitted = program.emit(consumer); + const diagnostics = [ + ...ts.getPreEmitDiagnostics(program, consumer), + ...emitted.diagnostics, + ].map( + (diagnostic) => + `TS${String(diagnostic.code)}: ${ts.flattenDiagnosticMessageText(diagnostic.messageText, "\n")}`, + ); + + return { declarations, diagnostics }; +}; + +test("a consumer's schema embedding modelSchema emits declarations that name ModelSchema rather than writing it out", () => { + const { declarations, diagnostics } = declarationsOf(consumerSource); + + expect(diagnostics).toEqual([]); + expect(declarations).toMatch(/model: import\("[^"]+"\)\.ModelSchema;/); + expect(declarations).not.toContain("participants"); +}); diff --git a/packages/core/graph/schema/src/index.ts b/packages/core/graph/schema/src/index.ts index 395eced..6549c2f 100644 --- a/packages/core/graph/schema/src/index.ts +++ b/packages/core/graph/schema/src/index.ts @@ -47,5 +47,5 @@ export { type OverlayIssue, } from "./overlay.ts"; export type { OverlayStatus } from "./vocabulary.ts"; -export { modelSchema } from "./schema.ts"; +export { modelSchema, type ModelSchema } from "./schema.ts"; export { ModelParseError, parseModel, type ModelIssue } from "./parse.ts"; diff --git a/packages/core/graph/schema/src/schema.ts b/packages/core/graph/schema/src/schema.ts index f5ef8b7..895d1b7 100644 --- a/packages/core/graph/schema/src/schema.ts +++ b/packages/core/graph/schema/src/schema.ts @@ -138,12 +138,8 @@ const viewsSchema = z ) .readonly(); -/** - * The open (runtime) shape of a model: identifiers are plain strings, so references are not checked here. `validate` checks what the shape cannot, and `parseModel` runs both. - * - * Every object is strict. The inferred type is exactly `Model`, which a type-level test pins in both directions. - */ -export const modelSchema = record({ +/** The strict object schema inside `modelSchema`, before it is made read-only. */ +const modelObjectSchema = z.strictObject({ participants: z.array(participantSchema).readonly(), events: z.array(eventSchema).readonly(), actions: z.array(actionSchema).readonly(), @@ -154,3 +150,18 @@ export const modelSchema = record({ projections: z.array(projectionSchema).readonly(), views: viewsSchema.exactOptional(), }); + +/** + * The type of `modelSchema`, declared by name so that a declaration file that embeds the schema, such as a consumer's `z.object({ model: modelSchema })` under `declaration: true`, refers to `ModelSchema` rather than writing out the whole model schema with every record and trilean's JSON value inline, which can exceed the length the compiler will serialise (TS7056). + */ +export interface ModelSchema extends z.ZodReadonly { + /** The strict object schema inside, whose `shape` holds each record's own schema. */ + readonly unwrap: () => typeof modelObjectSchema; +} + +/** + * The open (runtime) shape of a model: identifiers are plain strings, so references are not checked here. `validate` checks what the shape cannot, and `parseModel` runs both. + * + * Every object is strict. The inferred type is exactly `Model`, which a type-level test pins in both directions. + */ +export const modelSchema: ModelSchema = modelObjectSchema.readonly(); From afdc3bffdff0ab14dd0f04e03de08c75e8da1f11 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sun, 11 Oct 2026 18:15:45 +0100 Subject: [PATCH 12/12] test(schema): lift the clock bound on the declaration emit test The test builds a TypeScript program over the schema package and zod's declarations, which under coverage instrumentation on a CI runner took past the default per-test bound. Its duration is the compiler's work on the machine at hand, so it runs unbounded, leaving a hang to the job's timeout. --- .../schema/src/declaration-emit.unit.test.ts | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/packages/core/graph/schema/src/declaration-emit.unit.test.ts b/packages/core/graph/schema/src/declaration-emit.unit.test.ts index 3ef24c4..44dd674 100644 --- a/packages/core/graph/schema/src/declaration-emit.unit.test.ts +++ b/packages/core/graph/schema/src/declaration-emit.unit.test.ts @@ -60,10 +60,17 @@ const declarationsOf = ( return { declarations, diagnostics }; }; -test("a consumer's schema embedding modelSchema emits declarations that name ModelSchema rather than writing it out", () => { - const { declarations, diagnostics } = declarationsOf(consumerSource); +/** No per-test time limit: the test builds a whole TypeScript program over this package and zod's declarations, so how long it takes is the compiler's work on the machine at hand, not a property of this code, and a clock bound only fails it under load or coverage instrumentation. The CI job's own timeout still ends a hang. */ +const unbounded = 0; - expect(diagnostics).toEqual([]); - expect(declarations).toMatch(/model: import\("[^"]+"\)\.ModelSchema;/); - expect(declarations).not.toContain("participants"); -}); +test( + "a consumer's schema embedding modelSchema emits declarations that name ModelSchema rather than writing it out", + { timeout: unbounded }, + () => { + const { declarations, diagnostics } = declarationsOf(consumerSource); + + expect(diagnostics).toEqual([]); + expect(declarations).toMatch(/model: import\("[^"]+"\)\.ModelSchema;/); + expect(declarations).not.toContain("participants"); + }, +);