diff --git a/README.md b/README.md index 3c067ed..2e6f9cc 100644 --- a/README.md +++ b/README.md @@ -56,6 +56,7 @@ Configuration lives at exactly one of two levels: in the package (`eslint.config | [`graph-source-github-issues`](packages/adapters/graph/source-github-issues/README.md) | The GitHub issues adapter for the `GraphSource` port: issues as bound activities, `blockedBy` as transitions, sub-issues, repositories, organisations and assignees, across organisations and repositories. | | [`graph-store-contract`](packages/core/graph/store-contract/README.md) | The persistence port as an oRPC contract: save, load and delete models, set run state, upsert items from a source, structural ready, blocked-chain and impact queries, and a change feed; with an in-memory fake and a conformance kit. | | [`graph-store-sqlite`](packages/adapters/graph/store-sqlite/README.md) | The store port on SQLite through Drizzle: one table definition, committed drizzle-kit migrations applied the same way in every runtime, recursive SQL for the structural queries, against Drizzle's async SQLite database so any driver plugs in. | +| [`graph-sqlite-node`](packages/adapters/graph/sqlite-node/README.md) | The SQLite store on Node's built-in `node:sqlite` through `drizzle-orm/node-sqlite`, for the command line and servers. | | [`process-graph`](packages/targets/process-graph/README.md) | The primary entry point: re-exports the schema, projection and domain APIs flat and the DAG API as the `dag` namespace. | ## Conventions diff --git a/packages/adapters/graph/sqlite-node/AGENTS.md b/packages/adapters/graph/sqlite-node/AGENTS.md new file mode 120000 index 0000000..42061c0 --- /dev/null +++ b/packages/adapters/graph/sqlite-node/AGENTS.md @@ -0,0 +1 @@ +README.md \ No newline at end of file diff --git a/packages/adapters/graph/sqlite-node/CLAUDE.md b/packages/adapters/graph/sqlite-node/CLAUDE.md new file mode 120000 index 0000000..42061c0 --- /dev/null +++ b/packages/adapters/graph/sqlite-node/CLAUDE.md @@ -0,0 +1 @@ +README.md \ No newline at end of file diff --git a/packages/adapters/graph/sqlite-node/README.md b/packages/adapters/graph/sqlite-node/README.md new file mode 100644 index 0000000..cf408f2 --- /dev/null +++ b/packages/adapters/graph/sqlite-node/README.md @@ -0,0 +1,21 @@ +# @exadev/process-graph-sqlite-node + +[![GitHub](https://img.shields.io/badge/GitHub-181717?logo=github&logoColor=white)](https://github.com/ExaDev/process-graph) [![npm](https://img.shields.io/badge/npm-CB3837?logo=npm&logoColor=white)](https://www.npmjs.com/package/@exadev/process-graph-sqlite-node) [![Release](https://img.shields.io/github/v/release/ExaDev/process-graph)](https://github.com/ExaDev/process-graph/releases/latest) [![CI](https://img.shields.io/github/actions/workflow/status/ExaDev/process-graph/ci.yml?branch=main)](https://github.com/ExaDev/process-graph/actions) + +The SQLite graph store from `@exadev/process-graph-store-sqlite` on Node's built-in `node:sqlite`, through `drizzle-orm/node-sqlite`. It is the driver for the command line and for servers, and needs no native module. + +```ts +import { createGraphStoreRouter } from "@exadev/process-graph-store-contract"; +import { openNodeGraphStore } from "@exadev/process-graph-sqlite-node"; +import { createRouterClient } from "@orpc/server"; + +const { store, close } = openNodeGraphStore("graph.sqlite3"); +const client = createRouterClient(createGraphStoreRouter(store)); + +await client.v1.query.ready({ modelId: "release" }); +close(); +``` + +`openNodeGraphStore(path, options)` opens the file (or `":memory:"`), passing `node:sqlite`'s own options through, and the store migrates the schema on its first operation with the migrations embedded in `graph-store-sqlite`, the same ones the browser driver applies. A file database is switched to write-ahead logging, so a reader in another process is not blocked by a writer; set `timeout` to have a write wait for another process's lock. The store's change feed sees only the writes made through it, so a subscriber in one process does not hear another process's changes. + +The contract's conformance kit runs against an in-memory database in process and against a file database over a `MessageChannel`. diff --git a/packages/adapters/graph/sqlite-node/eslint.config.ts b/packages/adapters/graph/sqlite-node/eslint.config.ts new file mode 100644 index 0000000..b9974ff --- /dev/null +++ b/packages/adapters/graph/sqlite-node/eslint.config.ts @@ -0,0 +1,10 @@ +import { defineConfig } from "eslint/config"; +import { packageLintConfig } from "../../../../eslint.shared.ts"; + +export default defineConfig( + packageLintConfig({ + tsconfigRootDir: import.meta.dirname, + // Node-only by design: it opens databases with node:sqlite. + projects: ["./tsconfig.json"], + }), +); diff --git a/packages/adapters/graph/sqlite-node/package.json b/packages/adapters/graph/sqlite-node/package.json new file mode 100644 index 0000000..3c23789 --- /dev/null +++ b/packages/adapters/graph/sqlite-node/package.json @@ -0,0 +1,77 @@ +{ + "name": "@exadev/process-graph-sqlite-node", + "version": "0.0.0", + "description": "The SQLite graph store on Node's built-in node:sqlite through drizzle-orm/node-sqlite, for the command line and servers.", + "type": "module", + "repository": { + "type": "git", + "url": "git+https://github.com/ExaDev/process-graph.git", + "directory": "packages/adapters/graph/sqlite-node" + }, + "homepage": "https://github.com/ExaDev/process-graph/tree/main/packages/adapters/graph/sqlite-node#readme", + "bugs": { + "url": "https://github.com/ExaDev/process-graph/issues" + }, + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "types": "./dist/index.d.ts", + "files": [ + "dist" + ], + "publishConfig": { + "access": "public", + "provenance": true, + "registry": "https://registry.npmjs.org/" + }, + "sideEffects": false, + "license": "MIT", + "scripts": { + "build": "turbo run _build", + "_build": "tsc -b tsconfig.build.json", + "lint": "turbo run _lint", + "_lint": "eslint . --cache --max-warnings 0", + "_lint:fix": "eslint . --fix --cache --max-warnings 0", + "typecheck": "turbo run _typecheck _typecheck:attw", + "_typecheck": "tsc -p tsconfig.json --noEmit", + "_typecheck:attw": "attw --pack --profile esm-only", + "test": "turbo run _test", + "_test": "vitest run --project unit", + "test:coverage": "turbo run _test:coverage", + "_test:coverage": "vitest run --project unit --coverage", + "test:smoke": "turbo run _test:smoke", + "_test:smoke": "vitest run --project smoke", + "test:mutation": "turbo run _test:mutation", + "_test:mutation": "stryker run stryker.config.ts", + "prepublishOnly": "pnpm run lint && pnpm run typecheck && pnpm run test && pnpm run build && publint && attw --pack --profile esm-only" + }, + "keywords": [ + "process-model", + "store", + "sqlite", + "node-sqlite", + "drizzle" + ], + "dependencies": { + "@exadev/process-graph-store-contract": "1.0.1", + "@exadev/process-graph-store-sqlite": "1.0.0", + "drizzle-orm": "1.0.0-rc.4" + }, + "devDependencies": { + "@arethetypeswrong/cli": "0.18.5", + "@orpc/client": "2.0.0-beta.41", + "@orpc/server": "2.0.0-beta.41", + "@stryker-mutator/core": "10.0.0", + "@stryker-mutator/typescript-checker": "10.0.0", + "@stryker-mutator/vitest-runner": "10.0.0", + "@types/node": "26.6.4", + "@vitest/coverage-v8": "5.0.3", + "eslint": "10.12.0", + "publint": "0.3.25", + "typescript": "6.0.3", + "vitest": "5.0.3" + } +} diff --git a/packages/adapters/graph/sqlite-node/src/index.ts b/packages/adapters/graph/sqlite-node/src/index.ts new file mode 100644 index 0000000..a3b3689 --- /dev/null +++ b/packages/adapters/graph/sqlite-node/src/index.ts @@ -0,0 +1,4 @@ +/** + * The SQLite graph store on Node's built-in `node:sqlite`, through `drizzle-orm/node-sqlite`: the driver for the command line and for servers. `openNodeGraphStore` opens (and on first use migrates) a database file and returns the `GraphStore` to serve with the contract's `createGraphStoreRouter`. + */ +export { openNodeGraphStore, type NodeGraphStore } from "./open.ts"; diff --git a/packages/adapters/graph/sqlite-node/src/open.ts b/packages/adapters/graph/sqlite-node/src/open.ts new file mode 100644 index 0000000..0d2de13 --- /dev/null +++ b/packages/adapters/graph/sqlite-node/src/open.ts @@ -0,0 +1,45 @@ +import { DatabaseSync, type DatabaseSyncOptions } from "node:sqlite"; +import type { GraphStore } from "@exadev/process-graph-store-contract"; +import { createSqliteGraphStore } from "@exadev/process-graph-store-sqlite"; +import { sql } from "drizzle-orm"; +import { drizzle, type NodeSQLiteDatabase } from "drizzle-orm/node-sqlite"; + +/** An open store and the handles behind it. */ +export interface NodeGraphStore { + /** The store, to serve with `createGraphStoreRouter`. */ + readonly store: GraphStore; + /** The Drizzle database the store runs on, for tools that read the tables directly. */ + readonly db: NodeSQLiteDatabase; + /** Closes the database. The store must not be used afterwards. */ + readonly close: () => void; +} + +/** The path `node:sqlite` reads as a private database in memory rather than a file. */ +const inMemory = ":memory:"; + +/** + * Opens a graph store on a SQLite database file, creating the file when it does not exist. The schema is migrated on the store's first operation. + * + * A file database is switched to write-ahead logging, so readers in other processes (a command line beside a server) are not blocked by a writer. Each process's change feed sees only its own writes. + * @param path - The database file, or `":memory:"` for a private database that lives as long as the store. + * @param options - `node:sqlite` options passed through unchanged, such as `timeout` for how long a write waits for another process's lock. + * @returns The store, its database and a way to close it. + */ +export const openNodeGraphStore = ( + path: string, + options: DatabaseSyncOptions = {}, +): NodeGraphStore => { + const client = new DatabaseSync(path, options); + const db = drizzle({ client }); + if (path !== inMemory) { + db.run(sql`PRAGMA journal_mode = WAL`); + } + + return { + store: createSqliteGraphStore({ db }), + db, + close: () => { + client.close(); + }, + }; +}; diff --git a/packages/adapters/graph/sqlite-node/src/open.unit.test.ts b/packages/adapters/graph/sqlite-node/src/open.unit.test.ts new file mode 100644 index 0000000..c85272f --- /dev/null +++ b/packages/adapters/graph/sqlite-node/src/open.unit.test.ts @@ -0,0 +1,92 @@ +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { MessageChannel } from "node:worker_threads"; +import { + createGraphStoreRouter, + type GraphStoreClient, +} from "@exadev/process-graph-store-contract"; +import { + everyFieldModel, + graphStoreConformance, +} from "@exadev/process-graph-store-contract/conformance"; +import { createORPCClient } from "@orpc/client"; +import { RPCLink } from "@orpc/client/message-port"; +import { createRouterClient } from "@orpc/server"; +import { RPCHandler } from "@orpc/server/message-port"; +import { sql } from "drizzle-orm"; +import { describe, expect, test } from "vitest"; +import { openNodeGraphStore } from "./open.ts"; + +graphStoreConformance({ + name: "node:sqlite in memory, in process", + createClient: async () => { + const { store, close } = openNodeGraphStore(":memory:"); + + return Promise.resolve({ + client: createRouterClient(createGraphStoreRouter(store)), + close: async () => { + close(); + + return Promise.resolve(); + }, + }); + }, +}); + +graphStoreConformance({ + name: "node:sqlite on a file, over a message port", + createClient: async () => { + const directory = mkdtempSync(join(tmpdir(), "graph-sqlite-node-")); + const { store, close } = openNodeGraphStore( + join(directory, "graph.sqlite3"), + ); + const { port1, port2 } = new MessageChannel(); + new RPCHandler(createGraphStoreRouter(store)).upgrade(port1); + port1.start(); + const client: GraphStoreClient = createORPCClient( + new RPCLink({ port: port2 }), + ); + port2.start(); + + return Promise.resolve({ + client, + close: async () => { + port1.close(); + port2.close(); + close(); + rmSync(directory, { recursive: true }); + + return Promise.resolve(); + }, + }); + }, +}); + +describe("openNodeGraphStore", () => { + test("puts a file database in write-ahead logging and keeps what was saved across a reopen", async () => { + const directory = mkdtempSync(join(tmpdir(), "graph-sqlite-node-")); + const path = join(directory, "graph.sqlite3"); + try { + const first = openNodeGraphStore(path); + await createRouterClient( + createGraphStoreRouter(first.store), + ).v1.models.save({ + modelId: "orders", + model: everyFieldModel, + }); + expect(first.db.values(sql`PRAGMA journal_mode`)).toEqual([["wal"]]); + first.close(); + + const second = openNodeGraphStore(path); + const loaded = await createRouterClient( + createGraphStoreRouter(second.store), + ).v1.models.load({ modelId: "orders" }); + second.close(); + + expect(loaded.model).toStrictEqual(everyFieldModel); + } finally { + rmSync(directory, { recursive: true }); + } + }); +}); diff --git a/packages/adapters/graph/sqlite-node/stryker.config.ts b/packages/adapters/graph/sqlite-node/stryker.config.ts new file mode 100644 index 0000000..6429b27 --- /dev/null +++ b/packages/adapters/graph/sqlite-node/stryker.config.ts @@ -0,0 +1,3 @@ +import { packageStrykerConfig } from "../../../../stryker.shared.ts"; + +export default packageStrykerConfig(); diff --git a/packages/adapters/graph/sqlite-node/test/package.smoke.test.ts b/packages/adapters/graph/sqlite-node/test/package.smoke.test.ts new file mode 100644 index 0000000..557a3df --- /dev/null +++ b/packages/adapters/graph/sqlite-node/test/package.smoke.test.ts @@ -0,0 +1,13 @@ +import { createGraphStoreRouter } from "@exadev/process-graph-store-contract"; +import { createRouterClient } from "@orpc/server"; +import { expect, test } from "vitest"; +// Deliberately imports the built dist/, not src/: this is the check that catches a wrong exports map or a build misconfiguration that a unit test importing straight from src/ never touches. +import { openNodeGraphStore } from "../dist/index.js"; + +test("the built package opens a store that answers through an oRPC client", async () => { + const { store, close } = openNodeGraphStore(":memory:"); + const client = createRouterClient(createGraphStoreRouter(store)); + + await expect(client.v1.models.list()).resolves.toEqual([]); + close(); +}); diff --git a/packages/adapters/graph/sqlite-node/tsconfig.build.json b/packages/adapters/graph/sqlite-node/tsconfig.build.json new file mode 100644 index 0000000..669bf2a --- /dev/null +++ b/packages/adapters/graph/sqlite-node/tsconfig.build.json @@ -0,0 +1,22 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "composite": true, + "noEmit": false, + "declaration": true, + "declarationMap": false, + "rootDir": "src", + "outDir": "dist", + "rewriteRelativeImportExtensions": true + }, + "include": ["src/**/*.ts"], + "exclude": ["src/**/*.test.ts"], + "references": [ + { + "path": "../../../core/graph/store-contract/tsconfig.build.json" + }, + { + "path": "../store-sqlite/tsconfig.build.json" + } + ] +} diff --git a/packages/adapters/graph/sqlite-node/tsconfig.json b/packages/adapters/graph/sqlite-node/tsconfig.json new file mode 100644 index 0000000..7d6d148 --- /dev/null +++ b/packages/adapters/graph/sqlite-node/tsconfig.json @@ -0,0 +1,8 @@ +{ + "extends": "../../../../tsconfig.base.json", + "compilerOptions": { + "lib": ["ES2024"], + "types": ["node"] + }, + "include": ["src/**/*.ts", "test/**/*.ts", "*.config.ts"] +} diff --git a/packages/adapters/graph/sqlite-node/vitest.config.ts b/packages/adapters/graph/sqlite-node/vitest.config.ts new file mode 100644 index 0000000..01b9391 --- /dev/null +++ b/packages/adapters/graph/sqlite-node/vitest.config.ts @@ -0,0 +1,16 @@ +import { defineConfig } from "vitest/config"; + +export default defineConfig({ + test: { + projects: [ + { + extends: true, + test: { name: "unit", include: ["src/**/*.test.ts"] }, + }, + { + extends: true, + test: { name: "smoke", include: ["test/package.smoke.test.ts"] }, + }, + ], + }, +}); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 8fd3707..338141b 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -306,6 +306,55 @@ importers: specifier: 5.0.3 version: 5.0.3(@types/node@26.6.4)(@vitest/coverage-v8@5.0.3)(vite@8.3.2(@types/node@26.6.4)(jiti@2.7.0)(yaml@2.9.1)) + packages/adapters/graph/sqlite-node: + dependencies: + '@exadev/process-graph-store-contract': + specifier: 1.0.1 + version: link:../../../core/graph/store-contract + '@exadev/process-graph-store-sqlite': + specifier: 1.0.0 + version: link:../store-sqlite + drizzle-orm: + specifier: 1.0.0-rc.4 + version: 1.0.0-rc.4(zod@4.6.5) + devDependencies: + '@arethetypeswrong/cli': + specifier: 0.18.5 + version: 0.18.5 + '@orpc/client': + specifier: 2.0.0-beta.41 + version: 2.0.0-beta.41 + '@orpc/server': + specifier: 2.0.0-beta.41 + version: 2.0.0-beta.41 + '@stryker-mutator/core': + specifier: 10.0.0 + version: 10.0.0(@types/node@26.6.4) + '@stryker-mutator/typescript-checker': + specifier: 10.0.0 + version: 10.0.0(@stryker-mutator/core@10.0.0(@types/node@26.6.4))(typescript@6.0.3) + '@stryker-mutator/vitest-runner': + specifier: 10.0.0 + version: 10.0.0(@stryker-mutator/core@10.0.0(@types/node@26.6.4))(vitest@5.0.3) + '@types/node': + specifier: 26.6.4 + version: 26.6.4 + '@vitest/coverage-v8': + specifier: 5.0.3 + version: 5.0.3(vitest@5.0.3) + eslint: + specifier: 10.12.0 + version: 10.12.0(jiti@2.7.0)(supports-color@7.2.0) + publint: + specifier: 0.3.25 + version: 0.3.25 + typescript: + specifier: 6.0.3 + version: 6.0.3 + vitest: + specifier: 5.0.3 + version: 5.0.3(@types/node@26.6.4)(@vitest/coverage-v8@5.0.3)(vite@8.3.2(@types/node@26.6.4)(jiti@2.7.0)(yaml@2.9.1)) + packages/adapters/graph/store-sqlite: dependencies: '@exadev/process-graph-domain': diff --git a/tsconfig.build.json b/tsconfig.build.json index db63477..9c492a3 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -28,6 +28,9 @@ { "path": "packages/adapters/graph/store-sqlite/tsconfig.build.json" }, + { + "path": "packages/adapters/graph/sqlite-node/tsconfig.build.json" + }, { "path": "packages/targets/process-graph/tsconfig.build.json" }