Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ Configuration lives at exactly one of two levels: in the package (`eslint.config
| [`graph-source-github-issues`](packages/adapters/graph/source-github-issues/README.md) | The GitHub issues adapter for the `GraphSource` port: issues as bound activities, `blockedBy` as transitions, sub-issues, repositories, organisations and assignees, across organisations and repositories. |
| [`graph-store-contract`](packages/core/graph/store-contract/README.md) | The persistence port as an oRPC contract: save, load and delete models, set run state, upsert items from a source, structural ready, blocked-chain and impact queries, and a change feed; with an in-memory fake and a conformance kit. |
| [`graph-store-sqlite`](packages/adapters/graph/store-sqlite/README.md) | The store port on SQLite through Drizzle: one table definition, committed drizzle-kit migrations applied the same way in every runtime, recursive SQL for the structural queries, against Drizzle's async SQLite database so any driver plugs in. |
| [`graph-sqlite-node`](packages/adapters/graph/sqlite-node/README.md) | The SQLite store on Node's built-in `node:sqlite` through `drizzle-orm/node-sqlite`, for the command line and servers. |
| [`process-graph`](packages/targets/process-graph/README.md) | The primary entry point: re-exports the schema, projection and domain APIs flat and the DAG API as the `dag` namespace. |

## Conventions
Expand Down
1 change: 1 addition & 0 deletions packages/adapters/graph/sqlite-node/AGENTS.md
1 change: 1 addition & 0 deletions packages/adapters/graph/sqlite-node/CLAUDE.md
21 changes: 21 additions & 0 deletions packages/adapters/graph/sqlite-node/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# @exadev/process-graph-sqlite-node

[![GitHub](https://img.shields.io/badge/GitHub-181717?logo=github&logoColor=white)](https://github.com/ExaDev/process-graph) [![npm](https://img.shields.io/badge/npm-CB3837?logo=npm&logoColor=white)](https://www.npmjs.com/package/@exadev/process-graph-sqlite-node) [![Release](https://img.shields.io/github/v/release/ExaDev/process-graph)](https://github.com/ExaDev/process-graph/releases/latest) [![CI](https://img.shields.io/github/actions/workflow/status/ExaDev/process-graph/ci.yml?branch=main)](https://github.com/ExaDev/process-graph/actions)

The SQLite graph store from `@exadev/process-graph-store-sqlite` on Node's built-in `node:sqlite`, through `drizzle-orm/node-sqlite`. It is the driver for the command line and for servers, and needs no native module.

```ts
import { createGraphStoreRouter } from "@exadev/process-graph-store-contract";
import { openNodeGraphStore } from "@exadev/process-graph-sqlite-node";
import { createRouterClient } from "@orpc/server";

const { store, close } = openNodeGraphStore("graph.sqlite3");
const client = createRouterClient(createGraphStoreRouter(store));

await client.v1.query.ready({ modelId: "release" });
close();
```

`openNodeGraphStore(path, options)` opens the file (or `":memory:"`), passing `node:sqlite`'s own options through, and the store migrates the schema on its first operation with the migrations embedded in `graph-store-sqlite`, the same ones the browser driver applies. A file database is switched to write-ahead logging, so a reader in another process is not blocked by a writer; set `timeout` to have a write wait for another process's lock. The store's change feed sees only the writes made through it, so a subscriber in one process does not hear another process's changes.

The contract's conformance kit runs against an in-memory database in process and against a file database over a `MessageChannel`.
10 changes: 10 additions & 0 deletions packages/adapters/graph/sqlite-node/eslint.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import { defineConfig } from "eslint/config";
import { packageLintConfig } from "../../../../eslint.shared.ts";

export default defineConfig(
packageLintConfig({
tsconfigRootDir: import.meta.dirname,
// Node-only by design: it opens databases with node:sqlite.
projects: ["./tsconfig.json"],
}),
);
77 changes: 77 additions & 0 deletions packages/adapters/graph/sqlite-node/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
{
"name": "@exadev/process-graph-sqlite-node",
"version": "0.0.0",
"description": "The SQLite graph store on Node's built-in node:sqlite through drizzle-orm/node-sqlite, for the command line and servers.",
"type": "module",
"repository": {
"type": "git",
"url": "git+https://github.com/ExaDev/process-graph.git",
"directory": "packages/adapters/graph/sqlite-node"
},
"homepage": "https://github.com/ExaDev/process-graph/tree/main/packages/adapters/graph/sqlite-node#readme",
"bugs": {
"url": "https://github.com/ExaDev/process-graph/issues"
},
"exports": {
".": {
"types": "./dist/index.d.ts",
"import": "./dist/index.js"
}
},
"types": "./dist/index.d.ts",
"files": [
"dist"
],
"publishConfig": {
"access": "public",
"provenance": true,
"registry": "https://registry.npmjs.org/"
},
"sideEffects": false,
"license": "MIT",
"scripts": {
"build": "turbo run _build",
"_build": "tsc -b tsconfig.build.json",
"lint": "turbo run _lint",
"_lint": "eslint . --cache --max-warnings 0",
"_lint:fix": "eslint . --fix --cache --max-warnings 0",
"typecheck": "turbo run _typecheck _typecheck:attw",
"_typecheck": "tsc -p tsconfig.json --noEmit",
"_typecheck:attw": "attw --pack --profile esm-only",
"test": "turbo run _test",
"_test": "vitest run --project unit",
"test:coverage": "turbo run _test:coverage",
"_test:coverage": "vitest run --project unit --coverage",
"test:smoke": "turbo run _test:smoke",
"_test:smoke": "vitest run --project smoke",
"test:mutation": "turbo run _test:mutation",
"_test:mutation": "stryker run stryker.config.ts",
"prepublishOnly": "pnpm run lint && pnpm run typecheck && pnpm run test && pnpm run build && publint && attw --pack --profile esm-only"
},
"keywords": [
"process-model",
"store",
"sqlite",
"node-sqlite",
"drizzle"
],
"dependencies": {
"@exadev/process-graph-store-contract": "1.0.1",
"@exadev/process-graph-store-sqlite": "1.0.0",
"drizzle-orm": "1.0.0-rc.4"
},
"devDependencies": {
"@arethetypeswrong/cli": "0.18.5",
"@orpc/client": "2.0.0-beta.41",
"@orpc/server": "2.0.0-beta.41",
"@stryker-mutator/core": "10.0.0",
"@stryker-mutator/typescript-checker": "10.0.0",
"@stryker-mutator/vitest-runner": "10.0.0",
"@types/node": "26.6.4",
"@vitest/coverage-v8": "5.0.3",
"eslint": "10.12.0",
"publint": "0.3.25",
"typescript": "6.0.3",
"vitest": "5.0.3"
}
}
4 changes: 4 additions & 0 deletions packages/adapters/graph/sqlite-node/src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
/**
* The SQLite graph store on Node's built-in `node:sqlite`, through `drizzle-orm/node-sqlite`: the driver for the command line and for servers. `openNodeGraphStore` opens (and on first use migrates) a database file and returns the `GraphStore` to serve with the contract's `createGraphStoreRouter`.
*/
export { openNodeGraphStore, type NodeGraphStore } from "./open.ts";
45 changes: 45 additions & 0 deletions packages/adapters/graph/sqlite-node/src/open.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import { DatabaseSync, type DatabaseSyncOptions } from "node:sqlite";
import type { GraphStore } from "@exadev/process-graph-store-contract";
import { createSqliteGraphStore } from "@exadev/process-graph-store-sqlite";
import { sql } from "drizzle-orm";
import { drizzle, type NodeSQLiteDatabase } from "drizzle-orm/node-sqlite";

/** An open store and the handles behind it. */
export interface NodeGraphStore {
/** The store, to serve with `createGraphStoreRouter`. */
readonly store: GraphStore;
/** The Drizzle database the store runs on, for tools that read the tables directly. */
readonly db: NodeSQLiteDatabase;
/** Closes the database. The store must not be used afterwards. */
readonly close: () => void;
}

/** The path `node:sqlite` reads as a private database in memory rather than a file. */
const inMemory = ":memory:";

/**
* Opens a graph store on a SQLite database file, creating the file when it does not exist. The schema is migrated on the store's first operation.
*
* A file database is switched to write-ahead logging, so readers in other processes (a command line beside a server) are not blocked by a writer. Each process's change feed sees only its own writes.
* @param path - The database file, or `":memory:"` for a private database that lives as long as the store.
* @param options - `node:sqlite` options passed through unchanged, such as `timeout` for how long a write waits for another process's lock.
* @returns The store, its database and a way to close it.
*/
export const openNodeGraphStore = (
path: string,
options: DatabaseSyncOptions = {},
): NodeGraphStore => {
const client = new DatabaseSync(path, options);
const db = drizzle({ client });
if (path !== inMemory) {
db.run(sql`PRAGMA journal_mode = WAL`);
}

return {
store: createSqliteGraphStore({ db }),
db,
close: () => {
client.close();
},
};
};
92 changes: 92 additions & 0 deletions packages/adapters/graph/sqlite-node/src/open.unit.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { MessageChannel } from "node:worker_threads";
import {
createGraphStoreRouter,
type GraphStoreClient,
} from "@exadev/process-graph-store-contract";
import {
everyFieldModel,
graphStoreConformance,
} from "@exadev/process-graph-store-contract/conformance";
import { createORPCClient } from "@orpc/client";
import { RPCLink } from "@orpc/client/message-port";
import { createRouterClient } from "@orpc/server";
import { RPCHandler } from "@orpc/server/message-port";
import { sql } from "drizzle-orm";
import { describe, expect, test } from "vitest";
import { openNodeGraphStore } from "./open.ts";

graphStoreConformance({
name: "node:sqlite in memory, in process",
createClient: async () => {
const { store, close } = openNodeGraphStore(":memory:");

return Promise.resolve({
client: createRouterClient(createGraphStoreRouter(store)),
close: async () => {
close();

return Promise.resolve();
},
});
},
});

graphStoreConformance({
name: "node:sqlite on a file, over a message port",
createClient: async () => {
const directory = mkdtempSync(join(tmpdir(), "graph-sqlite-node-"));
const { store, close } = openNodeGraphStore(
join(directory, "graph.sqlite3"),
);
const { port1, port2 } = new MessageChannel();
new RPCHandler(createGraphStoreRouter(store)).upgrade(port1);
port1.start();
const client: GraphStoreClient = createORPCClient(
new RPCLink({ port: port2 }),
);
port2.start();

return Promise.resolve({
client,
close: async () => {
port1.close();
port2.close();
close();
rmSync(directory, { recursive: true });

return Promise.resolve();
},
});
},
});

describe("openNodeGraphStore", () => {
test("puts a file database in write-ahead logging and keeps what was saved across a reopen", async () => {
const directory = mkdtempSync(join(tmpdir(), "graph-sqlite-node-"));
const path = join(directory, "graph.sqlite3");
try {
const first = openNodeGraphStore(path);
await createRouterClient(
createGraphStoreRouter(first.store),
).v1.models.save({
modelId: "orders",
model: everyFieldModel,
});
expect(first.db.values(sql`PRAGMA journal_mode`)).toEqual([["wal"]]);
first.close();

const second = openNodeGraphStore(path);
const loaded = await createRouterClient(
createGraphStoreRouter(second.store),
).v1.models.load({ modelId: "orders" });
second.close();

expect(loaded.model).toStrictEqual(everyFieldModel);
} finally {
rmSync(directory, { recursive: true });
}
});
});
3 changes: 3 additions & 0 deletions packages/adapters/graph/sqlite-node/stryker.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import { packageStrykerConfig } from "../../../../stryker.shared.ts";

export default packageStrykerConfig();
13 changes: 13 additions & 0 deletions packages/adapters/graph/sqlite-node/test/package.smoke.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import { createGraphStoreRouter } from "@exadev/process-graph-store-contract";
import { createRouterClient } from "@orpc/server";
import { expect, test } from "vitest";
// Deliberately imports the built dist/, not src/: this is the check that catches a wrong exports map or a build misconfiguration that a unit test importing straight from src/ never touches.
import { openNodeGraphStore } from "../dist/index.js";

test("the built package opens a store that answers through an oRPC client", async () => {
const { store, close } = openNodeGraphStore(":memory:");
const client = createRouterClient(createGraphStoreRouter(store));

await expect(client.v1.models.list()).resolves.toEqual([]);
close();
});
22 changes: 22 additions & 0 deletions packages/adapters/graph/sqlite-node/tsconfig.build.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"extends": "./tsconfig.json",
"compilerOptions": {
"composite": true,
"noEmit": false,
"declaration": true,
"declarationMap": false,
"rootDir": "src",
"outDir": "dist",
"rewriteRelativeImportExtensions": true
},
"include": ["src/**/*.ts"],
"exclude": ["src/**/*.test.ts"],
"references": [
{
"path": "../../../core/graph/store-contract/tsconfig.build.json"
},
{
"path": "../store-sqlite/tsconfig.build.json"
}
]
}
8 changes: 8 additions & 0 deletions packages/adapters/graph/sqlite-node/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"extends": "../../../../tsconfig.base.json",
"compilerOptions": {
"lib": ["ES2024"],
"types": ["node"]
},
"include": ["src/**/*.ts", "test/**/*.ts", "*.config.ts"]
}
16 changes: 16 additions & 0 deletions packages/adapters/graph/sqlite-node/vitest.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import { defineConfig } from "vitest/config";

export default defineConfig({
test: {
projects: [
{
extends: true,
test: { name: "unit", include: ["src/**/*.test.ts"] },
},
{
extends: true,
test: { name: "smoke", include: ["test/package.smoke.test.ts"] },
},
],
},
});
49 changes: 49 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading