diff --git a/README.md b/README.md index e89b627..bb1672a 100644 --- a/README.md +++ b/README.md @@ -53,23 +53,23 @@ The published package: The internal packages, private and bundled into `process-graph`: -| Package | What it is | -| -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [`graph-dag`](packages/core/graph/dag/README.md) | Model-independent dependency-graph primitives over a node set and a `needs` accessor: layers, ancestors, descendants, cycles, transitive reduction, critical path. | -| [`graph-schema`](packages/core/graph/schema/README.md) | The process model: typed identifier spaces, `defineModel` with compile-time reference checking, structural validation. | -| [`graph-projection`](packages/core/graph/projection/README.md) | Projections of the model into a flowchart, swimlane, state machine and sequence diagram, plus derived hand-offs, and the inverses that turn an edited view into a diff against the model. | -| [`graph-domain`](packages/core/graph/domain/README.md) | Reasoning over the model and a run overlay: control graph, illegal cycles, three-valued status, ready set, blocked-by, impact, what-if and critical path. | -| [`graph-layout`](packages/core/graph/layout/README.md) | Layered layout of a projected flowchart, swimlane or state machine into positions in the model's `views` shape. | -| [`graph-source-contract`](packages/core/graph/source-contract/README.md) | The `GraphSource` port as an oRPC contract: versioned, paged and streamed sync of bound activities, participants, transitions and run state, resumable by cursor, typed filters and scope listings, with an in-memory fake and a conformance kit. | -| [`graph-source-github-issues`](packages/adapters/graph/source-github-issues/README.md) | The GitHub issues adapter for the `GraphSource` port: issues, `blockedBy` and `blocking`, sub-issues and assignees across organisations, repositories and Projects v2 boards, with server-side filters, a scope listing and per-owner tokens. | -| [`graph-store-contract`](packages/core/graph/store-contract/README.md) | The persistence port as an oRPC contract: save, load and delete models, set run state, upsert items and relations from a source, ready, blocked-chain, impact and hierarchy queries, a change feed; with an in-memory fake and a conformance kit. | -| [`graph-store-sqlite`](packages/adapters/graph/store-sqlite/README.md) | The store port on SQLite through Drizzle: one table definition, committed drizzle-kit migrations applied the same way in every runtime, recursive SQL for the structural queries, against Drizzle's async SQLite database so any driver plugs in. | -| [`graph-sqlite-node`](packages/adapters/graph/sqlite-node/README.md) | The SQLite store on Node's built-in `node:sqlite` through `drizzle-orm/node-sqlite`, for the command line and servers. | -| [`graph-sqlite-wasm`](packages/adapters/graph/sqlite-wasm/README.md) | The SQLite store in the browser: SQLite wasm in a dedicated worker on OPFS `opfs-sahpool`, shared between tabs under a Web Lock with no special headers, served to the page over oRPC. | -| [`graph-react`](packages/ui/graph/react/README.md) | Headless React hooks and unstyled components: follow a stored model, derive statuses, the ready set, blocked chains, impact and explanations, project views, filter, select and navigate by keyboard. | -| [`graph-ui`](packages/ui/graph/ui/README.md) | Styled components over `graph-react` and an SVG sequence diagram, in Vanilla Extract with a typed theme contract and light and dark themes, shipped as compiled CSS. | -| [`graph-react-flow`](packages/ui/graph/react-flow/README.md) | React Flow views of the model: flowchart, swimlane with lanes as group nodes, and state machine, laid out by `graph-layout`, showing each status and the critical path; sequence projections drawn by `graph-ui`. | -| [`graph-cli`](packages/targets/graph-cli/README.md) | The command line and its library entry: sync GitHub issues into the SQLite store, ready, blocked, impact, critical path, cycles, hand-offs and status explanations, and every view drawn in Mermaid or Graphviz DOT. | +| Package | What it is | +| -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| [`graph-dag`](packages/core/graph/dag/README.md) | Model-independent dependency-graph primitives over a node set and a `needs` accessor: layers, ancestors, descendants, cycles, transitive reduction, critical path. | +| [`graph-schema`](packages/core/graph/schema/README.md) | The process model: typed identifier spaces, `defineModel` with compile-time reference checking, structural validation. | +| [`graph-projection`](packages/core/graph/projection/README.md) | Projections of the model into a flowchart, swimlane, state machine and sequence diagram, plus derived hand-offs, and the inverses that turn an edited view into a diff against the model. | +| [`graph-domain`](packages/core/graph/domain/README.md) | Reasoning over the model and a run overlay: control graph, illegal cycles, three-valued status, ready set, blocked-by, impact, what-if and critical path. | +| [`graph-layout`](packages/core/graph/layout/README.md) | Layered layout of a projected flowchart, swimlane or state machine into positions in the model's `views` shape. | +| [`graph-source-contract`](packages/core/graph/source-contract/README.md) | The `GraphSource` port as an oRPC contract: versioned, paged and streamed sync of bound activities, participants, transitions and run state, resumable by cursor, typed filters and scope listings, with an in-memory fake and a conformance kit. | +| [`graph-source-github-issues`](packages/adapters/graph/source-github-issues/README.md) | The GitHub issues adapter for the `GraphSource` port: issues, `blockedBy` and `blocking`, sub-issues and assignees across organisations, repositories and Projects v2 boards, with server-side filters, a scope listing and per-owner tokens. | +| [`graph-store-contract`](packages/core/graph/store-contract/README.md) | The persistence port as an oRPC contract: save, load and delete models, set run state and the facts a run reads (provided artefacts, condition values), upsert items and relations from a source, ready, blocked-chain, impact and hierarchy queries, a change feed; with an in-memory fake and a conformance kit. | +| [`graph-store-sqlite`](packages/adapters/graph/store-sqlite/README.md) | The store port on SQLite through Drizzle: one table definition, committed drizzle-kit migrations applied the same way in every runtime, recursive SQL for the structural queries, against Drizzle's async SQLite database so any driver plugs in. | +| [`graph-sqlite-node`](packages/adapters/graph/sqlite-node/README.md) | The SQLite store on Node's built-in `node:sqlite` through `drizzle-orm/node-sqlite`, for the command line and servers. | +| [`graph-sqlite-wasm`](packages/adapters/graph/sqlite-wasm/README.md) | The SQLite store in the browser: SQLite wasm in a dedicated worker on OPFS `opfs-sahpool`, shared between tabs under a Web Lock with no special headers, served to the page over oRPC. | +| [`graph-react`](packages/ui/graph/react/README.md) | Headless React hooks and unstyled components: follow a stored model, derive statuses, the ready set, blocked chains, impact and explanations, project views, filter, select and navigate by keyboard. | +| [`graph-ui`](packages/ui/graph/ui/README.md) | Styled components over `graph-react` and an SVG sequence diagram, in Vanilla Extract with a typed theme contract and light and dark themes, shipped as compiled CSS. | +| [`graph-react-flow`](packages/ui/graph/react-flow/README.md) | React Flow views of the model: flowchart, swimlane with lanes as group nodes, and state machine, laid out by `graph-layout`, showing each status and the critical path; sequence projections drawn by `graph-ui`. | +| [`graph-cli`](packages/targets/graph-cli/README.md) | The command line and its library entry: sync GitHub issues into the SQLite store, ready, blocked, impact, critical path, cycles, hand-offs and status explanations, and every view drawn in Mermaid or Graphviz DOT. | The web app, private and not bundled into `process-graph`: diff --git a/packages/adapters/graph/store-sqlite/README.md b/packages/adapters/graph/store-sqlite/README.md index 145530f..9e0a6b3 100644 --- a/packages/adapters/graph/store-sqlite/README.md +++ b/packages/adapters/graph/store-sqlite/README.md @@ -20,7 +20,7 @@ const store = createRouterClient( `src/tables.ts` is the one table definition. Every record a model declares is a row keyed by model id and record id, with a `position` column keeping the declared order; enumerated columns take their values from graph-schema's own vocabulary, and nested values are JSON text. Rows never reach the contract: `src/rows.ts` flattens a model into rows and reassembles it, parsing the result with graph-schema's `modelSchema`, so what leaves the store is checked to be a model. A unit test derives the row schemas with `drizzle-orm/zod` and asserts they agree with the model schema on every shared scalar field. -Relations kept beside a model (`child-of`, `assigned-to`, `part-of`) are rows of their own, keyed by model, kind and both ends so a relation sent twice is stored once, with a position keeping the order they were first stored. Each transition row records whether graph-domain's `loopTransitions` counts it as a decision's loop edge, so the structural queries can leave loop edges out without loading the model. No foreign keys are declared, because SQLite enforces them only on connections that enable them; the store writes and deletes a model's rows together in one transaction instead. +Relations kept beside a model (`child-of`, `assigned-to`, `part-of`) are rows of their own, keyed by model, kind and both ends so a relation sent twice is stored once, with a position keeping the order they were first stored. A model's fact record is two tables beside the overlay: `artefact_facts`, one row per artefact whose provision is known with a boolean `provided`, and `named_facts`, one row per named fact with its trilean value as JSON text, parsed with the contract's `factRecordSchema` on every read. An artefact or name with no row is unknown. Each transition row records whether graph-domain's `loopTransitions` counts it as a decision's loop edge, so the structural queries can leave loop edges out without loading the model. No foreign keys are declared, because SQLite enforces them only on connections that enable them; the store writes and deletes a model's rows together in one transaction instead. ## Migrations @@ -30,7 +30,7 @@ The build embeds them in the package (`scripts/embed-migrations.ts`, parsed by d ## Writes and queries -Every write runs `checkStored` from the contract before it commits, takes the next value of the store-wide revision sequence, rewrites the model's rows in one transaction and publishes the change to the store's `ChangeFeed` once it has committed. A failed check or an aborted signal rolls the transaction back. +Every write runs `checkStored` (and `factIssues` for the facts it keeps) from the contract before it commits, takes the next value of the store-wide revision sequence, rewrites the model's rows in one transaction and publishes the change to the store's `ChangeFeed` once it has committed. A failed check or an aborted signal rolls the transaction back. `query.ready`, `query.blockedChain` and `query.impact` are single `WITH RECURSIVE` queries through Drizzle's `sql` template over `transitions`, `elements` and `overlay`, with the contract's semantics (control elements passed through, loop edges left out, traversal stopping at done work). `query.descendants` and `query.ancestors` are the same kind of query over the `child-of` rows of `relations`, and `query.children` a plain one. The walks cap distances at the model's element count, which is longer than any shortest path and ends the recursion on a cycle no decision closes, or a cycle of `child-of` relations. diff --git a/packages/adapters/graph/store-sqlite/drizzle/20261011094259_facts/migration.sql b/packages/adapters/graph/store-sqlite/drizzle/20261011094259_facts/migration.sql new file mode 100644 index 0000000..d42ca6f --- /dev/null +++ b/packages/adapters/graph/store-sqlite/drizzle/20261011094259_facts/migration.sql @@ -0,0 +1,13 @@ +CREATE TABLE `artefact_facts` ( + `model_id` text NOT NULL, + `artefact` text NOT NULL, + `provided` integer NOT NULL, + CONSTRAINT `artefact_facts_pk` PRIMARY KEY(`model_id`, `artefact`) +); +--> statement-breakpoint +CREATE TABLE `named_facts` ( + `model_id` text NOT NULL, + `name` text NOT NULL, + `value` text NOT NULL, + CONSTRAINT `named_facts_pk` PRIMARY KEY(`model_id`, `name`) +); diff --git a/packages/adapters/graph/store-sqlite/drizzle/20261011094259_facts/snapshot.json b/packages/adapters/graph/store-sqlite/drizzle/20261011094259_facts/snapshot.json new file mode 100644 index 0000000..a5a7fe9 --- /dev/null +++ b/packages/adapters/graph/store-sqlite/drizzle/20261011094259_facts/snapshot.json @@ -0,0 +1,1011 @@ +{ + "version": "7", + "dialect": "sqlite", + "id": "14607925-52c0-4840-abf7-e479580f08cd", + "prevIds": ["730e6eb0-db67-4d79-bf62-9711195facf8"], + "ddl": [ + { + "name": "actions", + "entityType": "tables" + }, + { + "name": "artefact_facts", + "entityType": "tables" + }, + { + "name": "artefacts", + "entityType": "tables" + }, + { + "name": "elements", + "entityType": "tables" + }, + { + "name": "events", + "entityType": "tables" + }, + { + "name": "fragments", + "entityType": "tables" + }, + { + "name": "models", + "entityType": "tables" + }, + { + "name": "named_facts", + "entityType": "tables" + }, + { + "name": "overlay", + "entityType": "tables" + }, + { + "name": "participants", + "entityType": "tables" + }, + { + "name": "projections", + "entityType": "tables" + }, + { + "name": "relations", + "entityType": "tables" + }, + { + "name": "sequences", + "entityType": "tables" + }, + { + "name": "transitions", + "entityType": "tables" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "actions" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "actions" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "position", + "entityType": "columns", + "table": "actions" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "actions" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "artefact_facts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "artefact", + "entityType": "columns", + "table": "artefact_facts" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "provided", + "entityType": "columns", + "table": "artefact_facts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "artefacts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "artefacts" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "position", + "entityType": "columns", + "table": "artefacts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "artefacts" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "deliverable", + "entityType": "columns", + "table": "artefacts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "elements" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "position", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "kind", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "label", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "actor", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "action", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "source", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "affects", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "consumes", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "produces", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "parent", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "entry", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "exit", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "role", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "from_actor", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "to_actor", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "mode", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "event", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "extensions", + "entityType": "columns", + "table": "elements" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "events" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "events" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "position", + "entityType": "columns", + "table": "events" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "events" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "payload", + "entityType": "columns", + "table": "events" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "fragments" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "fragments" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "position", + "entityType": "columns", + "table": "fragments" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "operator", + "entityType": "columns", + "table": "fragments" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "operands", + "entityType": "columns", + "table": "fragments" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "models" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "revision", + "entityType": "columns", + "table": "models" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "views", + "entityType": "columns", + "table": "models" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "named_facts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "named_facts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "value", + "entityType": "columns", + "table": "named_facts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "overlay" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "element", + "entityType": "columns", + "table": "overlay" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "status", + "entityType": "columns", + "table": "overlay" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "participants" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "participants" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "position", + "entityType": "columns", + "table": "participants" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "label", + "entityType": "columns", + "table": "participants" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "kind", + "entityType": "columns", + "table": "participants" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "projections" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "position", + "entityType": "columns", + "table": "projections" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "projections" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "notation", + "entityType": "columns", + "table": "projections" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "subject", + "entityType": "columns", + "table": "projections" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "scenario", + "entityType": "columns", + "table": "projections" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "relations" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "kind", + "entityType": "columns", + "table": "relations" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "from_id", + "entityType": "columns", + "table": "relations" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "to_id", + "entityType": "columns", + "table": "relations" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "position", + "entityType": "columns", + "table": "relations" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "sequences" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "value", + "entityType": "columns", + "table": "sequences" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model_id", + "entityType": "columns", + "table": "transitions" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "transitions" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "position", + "entityType": "columns", + "table": "transitions" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "source", + "entityType": "columns", + "table": "transitions" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "target", + "entityType": "columns", + "table": "transitions" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "trigger", + "entityType": "columns", + "table": "transitions" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "guard", + "entityType": "columns", + "table": "transitions" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "condition", + "entityType": "columns", + "table": "transitions" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "effect", + "entityType": "columns", + "table": "transitions" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "loop", + "entityType": "columns", + "table": "transitions" + }, + { + "columns": ["model_id", "id"], + "nameExplicit": false, + "name": "actions_pk", + "entityType": "pks", + "table": "actions" + }, + { + "columns": ["model_id", "artefact"], + "nameExplicit": false, + "name": "artefact_facts_pk", + "entityType": "pks", + "table": "artefact_facts" + }, + { + "columns": ["model_id", "id"], + "nameExplicit": false, + "name": "artefacts_pk", + "entityType": "pks", + "table": "artefacts" + }, + { + "columns": ["model_id", "id"], + "nameExplicit": false, + "name": "elements_pk", + "entityType": "pks", + "table": "elements" + }, + { + "columns": ["model_id", "id"], + "nameExplicit": false, + "name": "events_pk", + "entityType": "pks", + "table": "events" + }, + { + "columns": ["model_id", "id"], + "nameExplicit": false, + "name": "fragments_pk", + "entityType": "pks", + "table": "fragments" + }, + { + "columns": ["model_id", "name"], + "nameExplicit": false, + "name": "named_facts_pk", + "entityType": "pks", + "table": "named_facts" + }, + { + "columns": ["model_id", "element"], + "nameExplicit": false, + "name": "overlay_pk", + "entityType": "pks", + "table": "overlay" + }, + { + "columns": ["model_id", "id"], + "nameExplicit": false, + "name": "participants_pk", + "entityType": "pks", + "table": "participants" + }, + { + "columns": ["model_id", "position"], + "nameExplicit": false, + "name": "projections_pk", + "entityType": "pks", + "table": "projections" + }, + { + "columns": ["model_id", "kind", "from_id", "to_id"], + "nameExplicit": false, + "name": "relations_pk", + "entityType": "pks", + "table": "relations" + }, + { + "columns": ["model_id", "id"], + "nameExplicit": false, + "name": "transitions_pk", + "entityType": "pks", + "table": "transitions" + }, + { + "columns": ["id"], + "nameExplicit": false, + "name": "models_pk", + "table": "models", + "entityType": "pks" + }, + { + "columns": ["name"], + "nameExplicit": false, + "name": "sequences_pk", + "table": "sequences", + "entityType": "pks" + }, + { + "columns": [ + { + "value": "model_id", + "isExpression": false + }, + { + "value": "kind", + "isExpression": false + }, + { + "value": "to_id", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "relations_by_to", + "entityType": "indexes", + "table": "relations" + }, + { + "columns": [ + { + "value": "model_id", + "isExpression": false + }, + { + "value": "target", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "transitions_by_target", + "entityType": "indexes", + "table": "transitions" + }, + { + "columns": [ + { + "value": "model_id", + "isExpression": false + }, + { + "value": "source", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "transitions_by_source", + "entityType": "indexes", + "table": "transitions" + } + ], + "renames": [] +} diff --git a/packages/adapters/graph/store-sqlite/src/rows.ts b/packages/adapters/graph/store-sqlite/src/rows.ts index e487b28..7ea4b02 100644 --- a/packages/adapters/graph/store-sqlite/src/rows.ts +++ b/packages/adapters/graph/store-sqlite/src/rows.ts @@ -6,13 +6,17 @@ import { type OpenIds, type Overlay, } from "@exadev/process-graph-schema"; +import type { FactRecord } from "@exadev/process-graph-domain"; +import { factRecordSchema } from "@exadev/process-graph-store-contract"; import type { actions, + artefactFacts, artefacts, elements, events, fragments, models, + namedFacts, overlay, participants, projections, @@ -271,3 +275,47 @@ export const overlayFromRows = ( overlaySchema.parse( Object.fromEntries(rows.map(({ element, status }) => [element, status])), ); + +/** + * Reassembles a fact record from its rows, parsing it with the contract's `factRecordSchema`, so every stored value is checked to be a trilean value. + * @param artefactRows - The model's artefact fact rows. + * @param valueRows - The model's named fact rows. + * @returns The fact record. + */ +export const factsFromRows = ( + artefactRows: readonly (typeof artefactFacts.$inferSelect)[], + valueRows: readonly (typeof namedFacts.$inferSelect)[], +): FactRecord => + factRecordSchema.parse({ + artefacts: Object.fromEntries( + artefactRows.map(({ artefact, provided }) => [artefact, provided]), + ), + values: Object.fromEntries( + valueRows.map(({ name, value }) => [name, value]), + ), + }); + +/** + * Flattens a fact record into rows for one model. + * @param modelId - The model the facts are about. + * @param facts - The record. + * @returns The artefact fact rows and the named fact rows. + */ +export const factRows = ( + modelId: string, + facts: FactRecord, +): { + readonly artefacts: readonly (typeof artefactFacts.$inferInsert)[]; + readonly values: readonly (typeof namedFacts.$inferInsert)[]; +} => ({ + artefacts: Object.entries(facts.artefacts).map(([artefact, provided]) => ({ + modelId, + artefact, + provided, + })), + values: Object.entries(facts.values).map(([name, value]) => ({ + modelId, + name, + value, + })), +}); diff --git a/packages/adapters/graph/store-sqlite/src/store.ts b/packages/adapters/graph/store-sqlite/src/store.ts index 297c767..8d10929 100644 --- a/packages/adapters/graph/store-sqlite/src/store.ts +++ b/packages/adapters/graph/store-sqlite/src/store.ts @@ -1,15 +1,23 @@ -import { loopTransitions } from "@exadev/process-graph-domain"; +import { + emptyFactRecord, + loopTransitions, + type FactRecord, +} from "@exadev/process-graph-domain"; import type { Model, OpenIds, Overlay } from "@exadev/process-graph-schema"; import { ChangeFeed, checkStored, emptyModel, + factIssues, + factsWithin, GraphStoreError, mergeBoundItems, + mergeFacts, mergeRelations, overlayWithin, relationIssues, relationsWithin, + withoutFacts, type ChangeKind, type GraphStore, type Relation, @@ -34,6 +42,8 @@ import { ready, } from "./queries.ts"; import { + factRows, + factsFromRows, fromRows, overlayFromRows, toRows, @@ -41,11 +51,13 @@ import { } from "./rows.ts"; import { actions, + artefactFacts, artefacts, elements, events, fragments, models, + namedFacts, overlay as overlayTable, participants, projections, @@ -67,6 +79,9 @@ const recordTables = [ relationsTable, ] as const; +/** The tables that hold a model's fact record, both keyed by `modelId`. */ +const factTables = [artefactFacts, namedFacts] as const; + /** The name of the store-wide revision sequence's row in `sequences`. */ const revisionSequence = "revision"; @@ -165,6 +180,40 @@ export const createSqliteGraphStore = ( .where(eq(relationsTable.modelId, modelId)) .orderBy(asc(relationsTable.position)); + const readFacts = async (modelId: string): Promise => + factsFromRows( + await db + .select() + .from(artefactFacts) + .where(eq(artefactFacts.modelId, modelId)) + .orderBy(asc(artefactFacts.artefact)), + await db + .select() + .from(namedFacts) + .where(eq(namedFacts.modelId, modelId)) + .orderBy(asc(namedFacts.name)), + ); + + /** Replaces a model's fact rows with a record's; run inside the caller's transaction. */ + const replaceFacts = async ( + modelId: string, + facts: FactRecord, + ): Promise => { + await sequentially( + factTables.map( + (table) => async () => + db.delete(table).where(eq(table.modelId, modelId)), + ), + ); + const rows = factRows(modelId, facts); + if (rows.artefacts.length > 0) { + await db.insert(artefactFacts).values([...rows.artefacts]); + } + if (rows.values.length > 0) { + await db.insert(namedFacts).values([...rows.values]); + } + }; + const read = async (modelId: string): Promise => { const [model] = await db .select() @@ -222,6 +271,7 @@ export const createSqliteGraphStore = ( revision: model.revision, model: fromRows(rows), overlay: await readOverlay(modelId), + facts: await readFacts(modelId), }; }; @@ -269,20 +319,25 @@ export const createSqliteGraphStore = ( readonly model: Model; readonly overlay: Overlay; readonly relations: readonly Relation[]; + readonly facts: FactRecord; readonly signal: AbortSignal | undefined; } - /** Replaces a model's rows, overlay and relations in one transaction and publishes the change once it commits. */ + /** Replaces a model's rows, overlay, relations and facts in one transaction and publishes the change once it commits. */ const write = async ({ modelId, kind, model, overlay, relations, + facts, signal, }: Write): Promise => { checkStored(model, overlay); - const issues = relationIssues(model, relations); + const issues = [ + ...relationIssues(model, relations), + ...factIssues(model, facts), + ]; if (issues.length > 0) { throw new GraphStoreError({ kind: "invalid", issues }); } @@ -319,6 +374,7 @@ export const createSqliteGraphStore = ( })), ); } + await replaceFacts(modelId, facts); signal?.throwIfAborted(); return taken; @@ -328,6 +384,38 @@ export const createSqliteGraphStore = ( return revision; }; + /** Replaces a model's fact record in one transaction, after checking it names only artefacts the model declares, and publishes the change once it commits. */ + const writeFacts = async ({ + modelId, + model, + facts, + signal, + }: { + readonly modelId: string; + readonly model: Model; + readonly facts: FactRecord; + readonly signal: AbortSignal | undefined; + }): Promise => { + const issues = factIssues(model, facts); + if (issues.length > 0) { + throw new GraphStoreError({ kind: "invalid", issues }); + } + const revision = await inTransaction(db, async () => { + const taken = await nextRevision(); + await db + .update(models) + .set({ revision: taken }) + .where(eq(models.id, modelId)); + await replaceFacts(modelId, facts); + signal?.throwIfAborted(); + + return taken; + }); + feed.publish({ modelId, kind: "facts", revision }); + + return revision; + }; + return { feed, listModels: async () => @@ -353,6 +441,10 @@ export const createSqliteGraphStore = ( model, stored ? await readRelations(modelId) : [], ), + facts: factsWithin( + model, + stored ? await readFacts(modelId) : emptyFactRecord, + ), signal, }); }), @@ -368,6 +460,7 @@ export const createSqliteGraphStore = ( await db .delete(overlayTable) .where(eq(overlayTable.modelId, modelId)); + await replaceFacts(modelId, emptyFactRecord); await db.delete(models).where(eq(models.id, modelId)); signal?.throwIfAborted(); @@ -412,11 +505,47 @@ export const createSqliteGraphStore = ( return revision; }), + readFacts: async ({ modelId }) => + operation(async () => { + const revision = await revisionOf(modelId); + if (revision === undefined) { + throw new GraphStoreError({ kind: "not-found", modelId }); + } + + return { modelId, revision, facts: await readFacts(modelId) }; + }), + setFacts: async ({ modelId, facts, signal }) => + operation(async () => { + const stored = await read(modelId); + + return writeFacts({ + modelId, + model: stored.model, + facts: mergeFacts(stored.facts, facts), + signal, + }); + }), + clearFacts: async ({ modelId, facts, signal }) => + operation(async () => { + const stored = await read(modelId); + + return writeFacts({ + modelId, + model: stored.model, + facts: withoutFacts(stored.facts, facts), + signal, + }); + }), upsertItems: async ({ modelId, items, signal }) => operation(async () => { const stored = (await revisionOf(modelId)) === undefined - ? { model: emptyModel, overlay: {}, relations: [] } + ? { + model: emptyModel, + overlay: {}, + relations: [], + facts: emptyFactRecord, + } : { ...(await read(modelId)), relations: await readRelations(modelId), @@ -427,6 +556,7 @@ export const createSqliteGraphStore = ( kind: "items", ...mergeBoundItems(stored.model, stored.overlay, items), relations: mergeRelations(stored.relations, items.relations ?? []), + facts: stored.facts, signal, }); }), diff --git a/packages/adapters/graph/store-sqlite/src/store.unit.test.ts b/packages/adapters/graph/store-sqlite/src/store.unit.test.ts index 7c88090..b83416c 100644 --- a/packages/adapters/graph/store-sqlite/src/store.unit.test.ts +++ b/packages/adapters/graph/store-sqlite/src/store.unit.test.ts @@ -33,7 +33,7 @@ graphStoreConformance({ }); describe("a store on a database file", () => { - test("keeps its models and revisions when the file is opened again", async () => { + test("keeps its models, run state, facts and revisions when the file is opened again", async () => { const directory = mkdtempSync(join(tmpdir(), "graph-store-sqlite-")); const path = join(directory, "graph.sqlite3"); try { @@ -51,6 +51,13 @@ describe("a store on a database file", () => { modelId: "orders", overlay: { ship: "done" }, }); + await writer.v1.facts.set({ + modelId: "orders", + facts: { + artefacts: { basket: true, receipt: false }, + values: { approved: { kind: "boolean", value: true } }, + }, + }); first.close(); const second = new DatabaseSync(path); @@ -68,9 +75,13 @@ describe("a store on a database file", () => { expect(stored).toStrictEqual({ modelId: "orders", - revision: revision + 1, + revision: revision + 2, model: everyFieldModel, overlay: { ship: "done" }, + facts: { + artefacts: { basket: true, receipt: false }, + values: { approved: { kind: "boolean", value: true } }, + }, }); expect(next.revision).toBeGreaterThan(stored.revision); } finally { diff --git a/packages/adapters/graph/store-sqlite/src/tables.ts b/packages/adapters/graph/store-sqlite/src/tables.ts index db3145e..ed8cbd2 100644 --- a/packages/adapters/graph/store-sqlite/src/tables.ts +++ b/packages/adapters/graph/store-sqlite/src/tables.ts @@ -193,3 +193,25 @@ export const relations = sqliteTable( index("relations_by_to").on(table.modelId, table.kind, table.to), ], ); + +/** What is known about whether an artefact is provided, one row per artefact the model's fact record names: `provided` is true when it is provided and false when it is known not to be. An artefact with no row is unknown. */ +export const artefactFacts = sqliteTable( + "artefact_facts", + { + modelId: text("model_id").notNull(), + artefact: text("artefact").notNull(), + provided: integer("provided", { mode: "boolean" }).notNull(), + }, + (table) => [primaryKey({ columns: [table.modelId, table.artefact] })], +); + +/** The value of each named fact a model's fact record holds, as JSON text, typed `unknown` because `rows.ts` checks it with trilean's value schema on every read. A name with no row is unknown. */ +export const namedFacts = sqliteTable( + "named_facts", + { + modelId: text("model_id").notNull(), + name: text("name").notNull(), + value: text("value", { mode: "json" }).notNull(), + }, + (table) => [primaryKey({ columns: [table.modelId, table.name] })], +); diff --git a/packages/core/graph/domain/README.md b/packages/core/graph/domain/README.md index 93ddf6d..1b7c74f 100644 --- a/packages/core/graph/domain/README.md +++ b/packages/core/graph/domain/README.md @@ -41,6 +41,9 @@ deriveStatuses(model, overlay, facts); // Map { "triage" => "done", ... } | `whatIfDone(model, overlay, facts, id)` | The elements that become ready and every status change if `id` were done. | | `criticalPath(model, overlay, facts, weight)` | The heaviest chain through the work still ahead. | | `unmetOutputs(model, overlay, facts)` | Each deliverable not yet produced, with the elements and missing inputs it still needs. | +| `factsFrom(record)` | The `Facts` port over a `FactRecord` of provided artefacts and named fact values. | +| `layeredFacts(...ports)` | A `Facts` port answering each key from the first port that finds it. | +| `factNamesIn(condition)` | The named facts a condition reads: the string key of each `reference` node, once each. | Every function that derives status throws `CycleError` (from `graph-dag`, carrying the cycles) when `illegalCycles` is not empty, `UnknownNodeError` when a transition names an element the model does not declare, and `UnsupportedConditionError` for a condition the facts port cannot serve. @@ -83,6 +86,8 @@ The `Facts` port is `(key: JsonValue) => Resolution`: an opaque key in, `{ found | `elementStatusKey(id)` | This package, from the overlay | The text `"done"` or `"open"`. Not found, so indeterminate, when the overlay says `unknown` or `id` is not an element. It reads the overlay, never the derived status, so no cycle is possible. | | `artefactKey(id)` | The port: the package asks it | The boolean `true` when the input artefact is provided. Not found or `false` means it is not. | +A `FactRecord` is the plain data a store keeps for one run: `artefacts` maps an artefact id to `true` (provided) or `false` (known not to be), and `values` maps a fact's name to its trilean value. Whatever it does not name is unknown. `factsFrom(record)` is the `Facts` port over it: a string key is a named fact, `artefactKey(id)` reads `artefacts`, and anything else is not found, so an artefact recorded `false` is definite in a condition that reads its key while one not recorded leaves that condition indeterminate. `layeredFacts(...ports)` answers each key from the first port that finds it, so stored facts can sit in front of facts a host supplies. + `lookup`, `some`, `every` and fold conditions need tables or collections, which the port does not serve; they throw `UnsupportedConditionError` rather than being read as an empty collection. ## Queries diff --git a/packages/core/graph/domain/src/facts.ts b/packages/core/graph/domain/src/facts.ts index 7dc89e6..f7a716f 100644 --- a/packages/core/graph/domain/src/facts.ts +++ b/packages/core/graph/domain/src/facts.ts @@ -1,4 +1,11 @@ -import type { JsonValue, Resolution, SyncResolvers } from "trilean"; +import type { + ComputedValue, + ExpressionNode, + JsonValue, + PredicateNode, + Resolution, + SyncResolvers, +} from "trilean"; import { createSyncEvaluator } from "trilean"; import type { IdSpace, Model, Overlay } from "@exadev/process-graph-schema"; import type { Branch } from "./status.ts"; @@ -49,6 +56,105 @@ export const isProvided = (facts: Facts, id: string): boolean => { return answer.found && answer.value.kind === "boolean" && answer.value.value; }; +/** + * What is known from outside the process about one run, as plain data a store can keep: which artefacts are provided, and the values of named facts that conditions read. + * + * Each part is three-valued by presence. An artefact mapped to `true` is provided and one mapped to `false` is known not to be; an artefact the record does not name is unknown, which `isProvided` treats as not provided but a condition reading `artefactKey` sees as indeterminate. A named fact holds a trilean value; a name the record does not hold is unknown, so a condition reading it is indeterminate. + */ +export interface FactRecord { + /** The artefacts whose provision is known, by artefact id. */ + readonly artefacts: Readonly>; + /** The values of named facts, by the name a condition's `reference` node gives as its key. */ + readonly values: Readonly>; +} + +/** A record that knows nothing: no artefact is provided and every named fact is unknown. */ +export const emptyFactRecord: FactRecord = { artefacts: {}, values: {} }; + +const isRecord = (value: JsonValue): value is Record => + typeof value === "object" && value !== null && !Array.isArray(value); + +/** The artefact id an `artefactKey` key names, or `undefined` for any other key. */ +const artefactOf = (key: JsonValue): string | undefined => { + if (!isRecord(key) || Object.keys(key).length !== 1) { + return undefined; + } + const id = key[artefactField]; + + return typeof id === "string" ? id : undefined; +}; + +/** + * Adapts a fact record to the `Facts` port. A string key is a named fact, answered with its value when the record holds it; a key built by `artefactKey` is answered with the boolean the record holds for that artefact; every other key, and any name or artefact the record does not hold, is not found. + * @param record - What is known. + * @returns A `Facts` port reading the record. + */ +export const factsFrom = (record: FactRecord): Facts => { + const values = new Map(Object.entries(record.values)); + const artefacts = new Map(Object.entries(record.artefacts)); + + return (key) => { + if (typeof key === "string") { + const value = values.get(key); + + return value === undefined ? { found: false } : { found: true, value }; + } + const id = artefactOf(key); + const provided = id === undefined ? undefined : artefacts.get(id); + + return provided === undefined + ? { found: false } + : { found: true, value: { kind: "boolean", value: provided } }; + }; +}; + +/** + * The named facts a condition reads: the string key of every `reference` node in it, each once, in the order first met walking the tree depth first. Reserved keys and other non-string keys are not names, so they are left out, and a reference's own key is never searched for further references. + * @param condition - A trilean predicate or expression, such as a transition's `condition`. + * @returns The names, each once. + */ +export const factNamesIn = ( + condition: PredicateNode | ExpressionNode, +): readonly string[] => { + const names = new Set(); + const pending: unknown[] = [condition]; + for (let node = pending.pop(); node !== undefined; node = pending.pop()) { + if (Array.isArray(node)) { + const items: readonly unknown[] = node; + pending.push(...[...items].reverse()); + } else if (typeof node === "object" && node !== null) { + if ("kind" in node && node.kind === "reference" && "key" in node) { + if (typeof node.key === "string") { + names.add(node.key); + } + } else { + const fields: readonly unknown[] = Object.values(node); + pending.push(...[...fields].reverse()); + } + } + } + + return [...names]; +}; + +/** + * Layers facts ports: each key is answered by the first port that finds it, so an earlier port overrides a later one only where it knows the answer. + * @param ports - The ports, most authoritative first. + * @returns A `Facts` port that is not found only when no port finds the key. + */ +export const layeredFacts = + (...ports: readonly Facts[]): Facts => + (key) => { + for (const port of ports) { + const answer = port(key); + if (answer.found) { + return answer; + } + } + + return { found: false }; + }; + /** * Thrown when a condition uses a node whose evaluation needs a table or a collection from the facts port, which answers single keys only. */ @@ -70,9 +176,6 @@ export class UnsupportedConditionError extends Error { const evaluator = createSyncEvaluator({}); -const isRecord = (value: JsonValue): value is Record => - typeof value === "object" && value !== null && !Array.isArray(value); - /** * Evaluates every transition's condition. * diff --git a/packages/core/graph/domain/src/facts.unit.test.ts b/packages/core/graph/domain/src/facts.unit.test.ts new file mode 100644 index 0000000..d28b406 --- /dev/null +++ b/packages/core/graph/domain/src/facts.unit.test.ts @@ -0,0 +1,182 @@ +import { describe, expect, it } from "vitest"; +import { deriveStatuses } from "./derive.ts"; +import { + artefactKey, + elementStatusKey, + emptyFactRecord, + factNamesIn, + factsFrom, + layeredFacts, +} from "./facts.ts"; +import { readySet } from "./queries.ts"; +import { fails, holds, noFacts } from "./test-support/conditions.ts"; +import { approval, pipeline } from "./test-support/models.ts"; + +const yes = { kind: "boolean", value: true } as const; +const no = { kind: "boolean", value: false } as const; + +describe("factsFrom", () => { + it("answers a named fact with its recorded value, of any kind", () => { + const facts = factsFrom({ + artefacts: {}, + values: { approved: yes, budget: { kind: "number", value: 3 } }, + }); + + expect(facts("approved")).toEqual({ found: true, value: yes }); + expect(facts("budget")).toEqual({ + found: true, + value: { kind: "number", value: 3 }, + }); + }); + + it("answers an artefact key with the recorded provision, false included", () => { + const facts = factsFrom({ + artefacts: { spec: true, bundle: false }, + values: {}, + }); + + expect(facts(artefactKey("spec"))).toEqual({ found: true, value: yes }); + expect(facts(artefactKey("bundle"))).toEqual({ found: true, value: no }); + }); + + it("does not find what the record does not hold", () => { + const facts = factsFrom({ + artefacts: { spec: true }, + values: { approved: yes }, + }); + + expect(facts("unrecorded")).toEqual({ found: false }); + expect(facts(artefactKey("release"))).toEqual({ found: false }); + expect(facts({ $artefact: "spec", extra: true })).toEqual({ + found: false, + }); + expect(facts({ $artefact: 1 })).toEqual({ found: false }); + expect(facts(["approved"])).toEqual({ found: false }); + expect(facts(null)).toEqual({ found: false }); + expect(facts(elementStatusKey("spec"))).toEqual({ found: false }); + }); + + it("reads a name and an artefact id as different keys", () => { + const facts = factsFrom({ artefacts: { spec: true }, values: {} }); + + expect(facts("spec")).toEqual({ found: false }); + }); + + it("does not treat an inherited property as a recorded fact", () => { + const facts = factsFrom(emptyFactRecord); + + expect(facts("toString")).toEqual({ found: false }); + expect(facts(artefactKey("constructor"))).toEqual({ found: false }); + }); +}); + +describe("layeredFacts", () => { + const stored = factsFrom({ artefacts: {}, values: { approved: no } }); + const host = factsFrom({ + artefacts: {}, + values: { approved: yes, region: { kind: "text", value: "eu" } }, + }); + + it("takes the first port that finds the key", () => { + expect(layeredFacts(stored, host)("approved")).toEqual({ + found: true, + value: no, + }); + }); + + it("falls through to a later port for what an earlier one lacks", () => { + expect(layeredFacts(stored, host)("region")).toEqual({ + found: true, + value: { kind: "text", value: "eu" }, + }); + }); + + it("finds nothing when no port does", () => { + expect(layeredFacts(stored, host)("other")).toEqual({ found: false }); + expect(layeredFacts()("approved")).toEqual({ found: false }); + }); +}); + +describe("a recorded fact drives the status derivation", () => { + it("blocks a consumer of an unproduced input until the record provides it", () => { + expect(deriveStatuses(pipeline, {}, noFacts).get("build")).toBe("blocked"); + expect( + deriveStatuses( + pipeline, + {}, + factsFrom({ artefacts: { spec: false }, values: {} }), + ).get("build"), + ).toBe("blocked"); + expect( + readySet( + pipeline, + {}, + factsFrom({ artefacts: { spec: true }, values: {} }), + ), + ).toEqual(["build"]); + }); + + it("takes the branch a recorded fact selects and skips the other", () => { + const statuses = (approved: boolean) => + deriveStatuses( + approval, + { triage: "done" }, + factsFrom({ + artefacts: {}, + values: { approved: { kind: "boolean", value: approved } }, + }), + ); + + expect(statuses(true).get("accept")).toBe("ready"); + expect(statuses(true).get("reject")).toBe("skipped"); + expect(statuses(false).get("accept")).toBe("skipped"); + expect(statuses(false).get("reject")).toBe("ready"); + }); + + it("leaves both branches unknown when the record does not hold the fact", () => { + const statuses = deriveStatuses( + approval, + { triage: "done" }, + factsFrom(emptyFactRecord), + ); + + expect(statuses.get("accept")).toBe("unknown"); + expect(statuses.get("reject")).toBe("unknown"); + }); +}); + +describe("factNamesIn", () => { + it("lists each named fact a condition reads once, in the order met, and no reserved key", () => { + expect( + factNamesIn({ + kind: "allOf", + operands: [ + holds("approved"), + { + kind: "anyOf", + operands: [ + fails("budgeted"), + holds("approved"), + { + kind: "textCompare", + op: "equals", + left: { kind: "reference", key: elementStatusKey("audit") }, + right: { kind: "textLiteral", value: "done" }, + }, + ], + }, + { + kind: "compare", + op: "gt", + left: { kind: "reference", key: "budget" }, + right: { kind: "numberLiteral", value: 0 }, + }, + ], + }), + ).toEqual(["approved", "budgeted", "budget"]); + }); + + it("finds nothing in a condition that reads no named fact", () => { + expect(factNamesIn({ kind: "booleanLiteral", value: true })).toEqual([]); + }); +}); diff --git a/packages/core/graph/domain/src/index.ts b/packages/core/graph/domain/src/index.ts index b182c86..1f0f665 100644 --- a/packages/core/graph/domain/src/index.ts +++ b/packages/core/graph/domain/src/index.ts @@ -5,8 +5,13 @@ export { statuses, type Branch, type Status } from "./status.ts"; export { artefactKey, elementStatusKey, + emptyFactRecord, evaluateBranches, + factNamesIn, + factsFrom, + layeredFacts, UnsupportedConditionError, + type FactRecord, type Facts, } from "./facts.ts"; export { controlDag, illegalCycles, loopTransitions } from "./control.ts"; diff --git a/packages/core/graph/store-contract/README.md b/packages/core/graph/store-contract/README.md index 8c40d11..abce6ba 100644 --- a/packages/core/graph/store-contract/README.md +++ b/packages/core/graph/store-contract/README.md @@ -4,7 +4,7 @@ A private workspace package, not published on its own: it is bundled into [`process-graph`](../../../targets/process-graph/README.md), which exposes it as `process-graph/store`, beside the SQLite store. -The persistence port for process models and their run overlays, as an [oRPC](https://orpc.dev) router contract. One definition runs in process (a router client called directly), across a `MessagePort` to a worker, or over HTTP, with no change to its consumers. Inputs and outputs are Zod schemas built from `@exadev/process-graph-schema`'s `modelSchema` and `overlaySchema`, so the port carries the model as it is and storage layout never reaches it. `src/` imports nothing Node-only. +The persistence port for process models, their run overlays and the facts known about each run, as an [oRPC](https://orpc.dev) router contract. One definition runs in process (a router client called directly), across a `MessagePort` to a worker, or over HTTP, with no change to its consumers. Inputs and outputs are Zod schemas built from `@exadev/process-graph-schema`'s `modelSchema` and `overlaySchema`, so the port carries the model as it is and storage layout never reaches it. `src/` imports nothing Node-only. ```ts import { createRouterClient } from "@orpc/server"; @@ -30,22 +30,25 @@ for await (const change of await store.v1.changes({ modelId: "release" })) { The contract is keyed by version: `graphStoreContract.v1`. A breaking change adds a `v2` beside it, so one server serves both while clients move across. -| Procedure | Input | Output | -| -------------------- | ----------------------------- | ----------------------------------------- | -| `models.list` | none | Each model's id and revision, by id | -| `models.save` | `modelId`, `model` | The revision the save took | -| `models.load` | `modelId` | `modelId`, `revision`, `model`, `overlay` | -| `models.delete` | `modelId` | Whether there was a model to delete | -| `overlay.set` | `modelId`, `overlay` | The revision the change took | -| `items.upsert` | `modelId`, `items` | The revision the change took | -| `relations.list` | `modelId` | The relations kept beside the model | -| `query.ready` | `modelId` | The ready elements, in model order | -| `query.blockedChain` | `modelId`, `element` | The unfinished work upstream | -| `query.impact` | `modelId`, `element` | The unfinished work downstream | -| `query.children` | `modelId`, `element` | The element's direct children | -| `query.descendants` | `modelId`, `element` | Everything beneath it, nearest first | -| `query.ancestors` | `modelId`, `element` | Everything above it, nearest first | -| `changes` | `modelId`, optionally `since` | An event iterator of changes | +| Procedure | Input | Output | +| -------------------- | ----------------------------- | -------------------------------------------------- | +| `models.list` | none | Each model's id and revision, by id | +| `models.save` | `modelId`, `model` | The revision the save took | +| `models.load` | `modelId` | `modelId`, `revision`, `model`, `overlay`, `facts` | +| `models.delete` | `modelId` | Whether there was a model to delete | +| `overlay.set` | `modelId`, `overlay` | The revision the change took | +| `facts.read` | `modelId` | `modelId`, `revision`, `facts` | +| `facts.set` | `modelId`, `facts` | The revision the change took | +| `facts.clear` | `modelId`, `facts` | The revision the change took | +| `items.upsert` | `modelId`, `items` | The revision the change took | +| `relations.list` | `modelId` | The relations kept beside the model | +| `query.ready` | `modelId` | The ready elements, in model order | +| `query.blockedChain` | `modelId`, `element` | The unfinished work upstream | +| `query.impact` | `modelId`, `element` | The unfinished work downstream | +| `query.children` | `modelId`, `element` | The element's direct children | +| `query.descendants` | `modelId`, `element` | Everything beneath it, nearest first | +| `query.ancestors` | `modelId`, `element` | Everything above it, nearest first | +| `changes` | `modelId`, optionally `since` | An event iterator of changes | Every call takes the caller's abort signal through the oRPC client options. Failures are the contract's typed errors: `NOT_FOUND` (with `modelId`, and `element` for a query about an element the model lacks) and `UNPROCESSABLE_CONTENT` (with the `issues` that `validate` and `validateOverlay` report). Input that does not match the schemas is oRPC's `BAD_REQUEST`. @@ -53,19 +56,24 @@ Every call takes the caller's abort signal through the oRPC client options. Fail Every write is atomic and checked: a change that would leave the model invalid, or the overlay naming an element the model does not declare, is rejected whole and the store is unchanged. A write whose signal has fired before it commits changes nothing. -- `models.save` replaces the model and keeps the run state only of elements the new model still declares, and the relations only of what it still declares. +- `models.save` replaces the model and keeps the run state only of elements the new model still declares, the relations only of what it still declares, and the facts about artefacts only of those it still declares. Named facts are kept. - `overlay.set` sets the run state of the elements it names and leaves the others. +- `facts.set` records each artefact's provision and each named fact's value it gives, and keeps the rest; `facts.clear` forgets the artefacts and named facts it lists, or the whole record with `{ all: true }`. - `items.upsert` merges items read from a source: participants, bound activities and transitions replace the record with the same id in its place, or are appended in their given order; overlay entries replace the run state of the elements they name; each relation not already stored is appended. It creates the model from an empty one when there is none. A bound activity's `source` is required and equals its id. ## Relations A relation is a structural link the model has no field for, kept beside it: `child-of` from a sub-task element to its parent element, `assigned-to` from an element to a person participant, and `part-of` from one participant to the participant that contains it. These are the relations a `GraphSource` reports, so a sync's `relations` go straight into `items.upsert`. A relation is identified by its kind and its two ends, and is stored once however often it is sent. Each end must name what the model declares in the space its kind requires, or the write is rejected whole with an `unknown-reference` issue. Relations carry no ordering: the dependency queries never read them, so a parent does not wait on its children unless a transition says so. +## Facts + +A model's fact record is what is known from outside the process about its run, kept beside the overlay: graph-domain's `FactRecord`. `artefacts` maps an artefact id to `true` when it is provided and `false` when it is known not to be; `values` maps a fact's name, the string key a condition's `reference` node gives, to its trilean value (`boolean`, `number`, `text`, `instant`, `duration` or `complex`). Whatever the record does not name is unknown, so the three states of an artefact are provided, absent and unknown, and a named fact is unknown until it is set and again once it is cleared. Each artefact named must be one the model declares, or the write is rejected whole with an `unknown-reference` issue; a fact's name is free. graph-domain's `factsFrom` adapts the record to the `Facts` port, so a consumer loads the model and reasons over `deriveStatuses(model, overlay, factsFrom(facts))`. A model deleted and saved again starts with no facts. + Each write takes the next revision from one store-wide sequence, so revisions only grow, a model's revision is the revision of its last change, and a model deleted and saved again never reuses an earlier one. ## Queries -The queries are structural: they read the stored transitions and run state, and nothing else, so a store can answer them in its own query language without loading the model or evaluating anything. For reasoning that follows conditions, artefacts and three-valued status, load the model and use `@exadev/process-graph-domain`. +The queries are structural: they read the stored transitions and run state, and nothing else (not the facts), so a store can answer them in its own query language without loading the model or evaluating anything. For reasoning that follows conditions, artefacts and three-valued status, load the model and use `@exadev/process-graph-domain`. - Transitions that close a legal decision loop (`loopTransitions` from `graph-domain`) are not dependencies. - Decisions and pseudo-states are control elements: traversals pass through them and never report them. @@ -77,7 +85,7 @@ The queries are structural: they read the stored transitions and run state, and ## Changes -`changes` yields every later change to one model as `{ modelId, kind, revision }`, with `kind` one of `saved`, `overlay`, `items` or `deleted`. The router subscribes before it returns, so a caller holding the iterator is already subscribed and misses nothing it writes next. Given `since`, a revision the caller last saw, the first event is `{ modelId, kind: "missed" }` when the model is no longer at that revision (it changed, or it has gone), and the caller should load it again. +`changes` yields every later change to one model as `{ modelId, kind, revision }`, with `kind` one of `saved`, `overlay`, `items`, `facts` or `deleted`. The router subscribes before it returns, so a caller holding the iterator is already subscribed and misses nothing it writes next. Given `since`, a revision the caller last saw, the first event is `{ modelId, kind: "missed" }` when the model is no longer at that revision (it changed, or it has gone), and the caller should load it again. A subscription's memory is bounded. A change that commits while the subscriber is waiting goes straight to it; one that commits while the subscriber is busy is held, up to the subscription's capacity. When a change arrives and the subscription already holds that many, the held changes are replaced by one `{ modelId, kind: "missed" }`, and later changes follow it in order. A subscriber that keeps reading therefore misses nothing, and one that stops reading costs the store a fixed amount of memory and learns that it must load the model again. The capacity is a `ChangeFeed` option (`new ChangeFeed({ capacity })`, or `subscribe(modelId, { capacity })` for one subscription); the default, `defaultSubscriptionCapacity`, holds one change, because the only thing a change asks of a subscriber is to read the model again, and the model it reads after one pending change already reflects every later one. The conformance kit checks that a subscriber that stops reading under sustained writes is told it missed whatever it did not receive. @@ -85,7 +93,7 @@ A store's `ChangeFeed` only sees the writes made through that store. Where sever ## Implementing a store -An adapter implements `GraphStore` (plain async functions taking a request object with the abort signal) and serves it with `createGraphStoreRouter`, which owns abort handling, error mapping and subscriptions. `checkStored`, `mergeBoundItems`, `overlayWithin`, `mergeRelations`, `relationsWithin` and `relationIssues` give every store the same write rules, and `structureOf`, `readyOf`, `blockedChainOf`, `impactOf`, `childrenOf`, `descendantsOf` and `ancestorsOf` are the reference queries `createFakeGraphStore` uses. +An adapter implements `GraphStore` (plain async functions taking a request object with the abort signal) and serves it with `createGraphStoreRouter`, which owns abort handling, error mapping and subscriptions. `checkStored`, `mergeBoundItems`, `overlayWithin`, `mergeRelations`, `relationsWithin`, `relationIssues`, `mergeFacts`, `withoutFacts`, `factsWithin` and `factIssues` give every store the same write rules, and `structureOf`, `readyOf`, `blockedChainOf`, `impactOf`, `childrenOf`, `descendantsOf` and `ancestorsOf` are the reference queries `createFakeGraphStore` uses. Every adapter runs the conformance kit, through a real oRPC client: diff --git a/packages/core/graph/store-contract/src/conformance-facts.ts b/packages/core/graph/store-contract/src/conformance-facts.ts new file mode 100644 index 0000000..31fecd0 --- /dev/null +++ b/packages/core/graph/store-contract/src/conformance-facts.ts @@ -0,0 +1,448 @@ +import { + deriveStatuses, + emptyFactRecord, + factsFrom, +} from "@exadev/process-graph-domain"; +import type { Model, OpenIds } from "@exadev/process-graph-schema"; +import { describe, expect, test } from "vitest"; +import { + type ClientFactory, + failureOf, + pairModel, + takeUntil, + withClient, +} from "./conformance-support.ts"; + +/** + * A model whose statuses depend on facts: `build` consumes `spec`, an input no activity produces, and the decision `approve` takes `ship` only when the named fact `approved` is true. + */ +const factsModel: Model = { + participants: [{ id: "team", label: "Team", kind: "role" }], + events: [], + actions: [], + artefacts: [ + { id: "spec", name: "Specification" }, + { id: "bundle", name: "Bundle", deliverable: true }, + ], + elements: [ + { + kind: "activity", + id: "build", + label: "Build", + actor: "team", + consumes: ["spec"], + produces: ["bundle"], + }, + { kind: "decision", id: "approve", label: "Approved?", actor: "team" }, + { kind: "activity", id: "ship", label: "Ship", actor: "team" }, + ], + transitions: [ + { id: "build-approve", source: "build", target: "approve" }, + { + id: "approve-ship", + source: "approve", + target: "ship", + condition: { + kind: "compare", + op: "eq", + left: { kind: "reference", key: "approved" }, + right: { kind: "booleanLiteral", value: true }, + }, + }, + ], + fragments: [], + projections: [], +}; + +/** A value of every kind a fact can hold, so a store that loses a kind or a field of one fails the round trip. */ +const everyKindOfValue = { + approved: { kind: "boolean", value: true }, + budget: { kind: "number", value: 2.5, unit: { m: 1, s: -1 } }, + region: { kind: "text", value: "eu-west \u{1f600}" }, + due: { kind: "instant", value: "2026-10-11T09:00:00Z" }, + window: { kind: "duration", value: 3, unit: "d" }, + phase: { kind: "complex", re: 1, im: -0.5 }, +} as const; + +/** + * The facts' share of the conformance kit: what `facts.set` and `facts.clear` record beside a model, how other writes keep the record consistent, how subscribers hear of it, and that the record read back drives graph-domain's reasoning. `graphStoreConformance` runs it inside its own `describe` block. + * @param createClient - Makes a client for an empty store under test. + */ +export const factConformance = (createClient: ClientFactory): void => { + describe("facts", () => { + test( + "a model starts with an empty fact record, read alone or with the model", + withClient(createClient, async (client) => { + const { revision } = await client.v1.models.save({ + modelId: "release", + model: factsModel, + }); + + await expect( + client.v1.facts.read({ modelId: "release" }), + ).resolves.toStrictEqual({ + modelId: "release", + revision, + facts: { artefacts: {}, values: {} }, + }); + await expect( + client.v1.models.load({ modelId: "release" }), + ).resolves.toMatchObject({ facts: emptyFactRecord }); + }), + ); + + test( + "set facts read back exactly, every value kind kept, and each set merges by name", + withClient(createClient, async (client) => { + const saved = await client.v1.models.save({ + modelId: "release", + model: factsModel, + }); + const first = await client.v1.facts.set({ + modelId: "release", + facts: { + artefacts: { spec: false, bundle: true }, + values: everyKindOfValue, + }, + }); + const second = await client.v1.facts.set({ + modelId: "release", + facts: { + artefacts: { spec: true }, + values: { approved: { kind: "boolean", value: false } }, + }, + }); + + expect(first.revision).toBeGreaterThan(saved.revision); + expect(second.revision).toBeGreaterThan(first.revision); + const expected = { + artefacts: { spec: true, bundle: true }, + values: { + ...everyKindOfValue, + approved: { kind: "boolean", value: false }, + }, + }; + await expect( + client.v1.facts.read({ modelId: "release" }), + ).resolves.toStrictEqual({ + modelId: "release", + revision: second.revision, + facts: expected, + }); + await expect( + client.v1.models.load({ modelId: "release" }), + ).resolves.toStrictEqual({ + modelId: "release", + revision: second.revision, + model: factsModel, + overlay: {}, + facts: expected, + }); + }), + ); + + test( + "a fact about an artefact the model does not declare is rejected whole", + withClient(createClient, async (client) => { + const { revision } = await client.v1.models.save({ + modelId: "release", + model: factsModel, + }); + + await expect( + failureOf( + client.v1.facts.set({ + modelId: "release", + facts: { + artefacts: { spec: true, invoice: true }, + values: { approved: { kind: "boolean", value: true } }, + }, + }), + ), + ).resolves.toMatchObject({ + code: "UNPROCESSABLE_CONTENT", + data: { + issues: [ + { + code: "unknown-reference", + message: 'facts refer to unknown artefact id "invoice"', + }, + ], + }, + }); + await expect( + client.v1.facts.read({ modelId: "release" }), + ).resolves.toStrictEqual({ + modelId: "release", + revision, + facts: { artefacts: {}, values: {} }, + }); + }), + ); + + test( + "reading, setting or clearing the facts of a model that does not exist is NOT_FOUND", + withClient(createClient, async (client) => { + await expect( + failureOf(client.v1.facts.read({ modelId: "missing" })), + ).resolves.toMatchObject({ + code: "NOT_FOUND", + data: { modelId: "missing" }, + }); + await expect( + failureOf( + client.v1.facts.set({ + modelId: "missing", + facts: { values: { approved: { kind: "boolean", value: true } } }, + }), + ), + ).resolves.toMatchObject({ + code: "NOT_FOUND", + data: { modelId: "missing" }, + }); + await expect( + failureOf( + client.v1.facts.clear({ modelId: "missing", facts: { all: true } }), + ), + ).resolves.toMatchObject({ + code: "NOT_FOUND", + data: { modelId: "missing" }, + }); + await expect(client.v1.models.list()).resolves.toEqual([]); + }), + ); + + test( + "clearing forgets the named facts, or with all, the whole record", + withClient(createClient, async (client) => { + await client.v1.models.save({ modelId: "release", model: factsModel }); + await client.v1.facts.set({ + modelId: "release", + facts: { + artefacts: { spec: true, bundle: false }, + values: everyKindOfValue, + }, + }); + const named = await client.v1.facts.clear({ + modelId: "release", + facts: { artefacts: ["spec", "never-set"], values: ["approved"] }, + }); + + const kept = Object.fromEntries( + Object.entries(everyKindOfValue).filter( + ([name]) => name !== "approved", + ), + ); + await expect( + client.v1.facts.read({ modelId: "release" }), + ).resolves.toStrictEqual({ + modelId: "release", + revision: named.revision, + facts: { artefacts: { bundle: false }, values: kept }, + }); + + const all = await client.v1.facts.clear({ + modelId: "release", + facts: { all: true }, + }); + + expect(all.revision).toBeGreaterThan(named.revision); + await expect( + client.v1.facts.read({ modelId: "release" }), + ).resolves.toStrictEqual({ + modelId: "release", + revision: all.revision, + facts: { artefacts: {}, values: {} }, + }); + }), + ); + + test( + "a save keeps named facts and drops facts about artefacts the new model no longer declares; overlay and items writes keep them all", + withClient(createClient, async (client) => { + await client.v1.models.save({ modelId: "release", model: factsModel }); + await client.v1.facts.set({ + modelId: "release", + facts: { + artefacts: { spec: true, bundle: false }, + values: { approved: { kind: "boolean", value: true } }, + }, + }); + await client.v1.overlay.set({ + modelId: "release", + overlay: { build: "done" }, + }); + + await expect( + client.v1.facts.read({ modelId: "release" }), + ).resolves.toMatchObject({ + facts: { + artefacts: { spec: true, bundle: false }, + values: { approved: { kind: "boolean", value: true } }, + }, + }); + + await client.v1.models.save({ + modelId: "release", + model: { + ...factsModel, + artefacts: factsModel.artefacts.filter(({ id }) => id === "spec"), + elements: factsModel.elements.map((element) => + element.kind === "activity" && element.id === "build" + ? { ...element, produces: [] } + : element, + ), + }, + }); + + await expect( + client.v1.facts.read({ modelId: "release" }), + ).resolves.toMatchObject({ + facts: { + artefacts: { spec: true }, + values: { approved: { kind: "boolean", value: true } }, + }, + }); + }), + ); + + test( + "a model deleted and saved again starts with no facts", + withClient(createClient, async (client) => { + await client.v1.models.save({ modelId: "release", model: factsModel }); + await client.v1.facts.set({ + modelId: "release", + facts: { + artefacts: { spec: true }, + values: { approved: { kind: "boolean", value: true } }, + }, + }); + await client.v1.models.delete({ modelId: "release" }); + await client.v1.models.save({ modelId: "release", model: factsModel }); + + await expect( + client.v1.facts.read({ modelId: "release" }), + ).resolves.toMatchObject({ facts: { artefacts: {}, values: {} } }); + }), + ); + + test( + "a facts write whose signal has already fired is refused and changes nothing", + withClient(createClient, async (client) => { + const { revision } = await client.v1.models.save({ + modelId: "release", + model: factsModel, + }); + const controller = new AbortController(); + controller.abort(); + + await expect( + client.v1.facts.set( + { modelId: "release", facts: { artefacts: { spec: true } } }, + { signal: controller.signal }, + ), + ).rejects.toThrow(); + await expect( + client.v1.facts.read({ modelId: "release" }), + ).resolves.toStrictEqual({ + modelId: "release", + revision, + facts: { artefacts: {}, values: {} }, + }); + }), + ); + + test( + "every subscriber to the model hears each facts write, and reads the new facts back", + withClient(createClient, async (client) => { + await client.v1.models.save({ modelId: "release", model: factsModel }); + const first = takeUntil( + await client.v1.changes({ modelId: "release" }), + ({ kind }) => kind === "deleted", + ); + const second = takeUntil( + await client.v1.changes({ modelId: "release" }), + ({ kind }) => kind === "deleted", + ); + + const set = await client.v1.facts.set({ + modelId: "release", + facts: { artefacts: { spec: true } }, + }); + const cleared = await client.v1.facts.clear({ + modelId: "release", + facts: { artefacts: ["spec"] }, + }); + const deleted = await client.v1.models.delete({ modelId: "release" }); + if (!deleted.deleted) { + throw new Error("the model was saved, so the delete removes it"); + } + const expected = [ + { modelId: "release", kind: "facts", revision: set.revision }, + { modelId: "release", kind: "facts", revision: cleared.revision }, + { modelId: "release", kind: "deleted", revision: deleted.revision }, + ]; + + await expect(first).resolves.toEqual(expected); + await expect(second).resolves.toEqual(expected); + }), + ); + + test( + "the facts read back drive the reasoning: an input set provided unblocks its consumer, and a stored fact picks the branch", + withClient(createClient, async (client) => { + await client.v1.models.save({ modelId: "release", model: factsModel }); + const statuses = async () => { + const { model, overlay, facts } = await client.v1.models.load({ + modelId: "release", + }); + + return deriveStatuses(model, overlay, factsFrom(facts)); + }; + + expect((await statuses()).get("build")).toBe("blocked"); + + await client.v1.facts.set({ + modelId: "release", + facts: { artefacts: { spec: true } }, + }); + await client.v1.overlay.set({ + modelId: "release", + overlay: { build: "done" }, + }); + + expect((await statuses()).get("ship")).toBe("unknown"); + + await client.v1.facts.set({ + modelId: "release", + facts: { values: { approved: { kind: "boolean", value: false } } }, + }); + + expect((await statuses()).get("ship")).toBe("skipped"); + + await client.v1.facts.set({ + modelId: "release", + facts: { values: { approved: { kind: "boolean", value: true } } }, + }); + + expect((await statuses()).get("ship")).toBe("ready"); + }), + ); + + test( + "facts are kept per model", + withClient(createClient, async (client) => { + await client.v1.models.save({ modelId: "release", model: factsModel }); + await client.v1.models.save({ modelId: "pair", model: pairModel }); + await client.v1.facts.set({ + modelId: "release", + facts: { values: { approved: { kind: "boolean", value: true } } }, + }); + + await expect( + client.v1.facts.read({ modelId: "pair" }), + ).resolves.toMatchObject({ facts: { artefacts: {}, values: {} } }); + }), + ); + }); +}; diff --git a/packages/core/graph/store-contract/src/conformance.ts b/packages/core/graph/store-contract/src/conformance.ts index 9044f9e..6da4f37 100644 --- a/packages/core/graph/store-contract/src/conformance.ts +++ b/packages/core/graph/store-contract/src/conformance.ts @@ -1,5 +1,6 @@ import type { Model, OpenIds } from "@exadev/process-graph-schema"; import { describe, expect, test } from "vitest"; +import { factConformance } from "./conformance-facts.ts"; import { relationConformance } from "./conformance-relations.ts"; import { type ClientFactory, @@ -198,6 +199,7 @@ export const graphStoreConformance = ( revision, model: everyFieldModel, overlay: {}, + facts: { artefacts: {}, values: {} }, }); }), ); @@ -289,6 +291,7 @@ export const graphStoreConformance = ( revision, model: smaller, overlay: { plan: "done" }, + facts: { artefacts: {}, values: {} }, }); }), ); @@ -410,6 +413,7 @@ export const graphStoreConformance = ( projections: [], }, overlay: issueItems.overlay, + facts: { artefacts: {}, values: {} }, }); }), ); @@ -498,6 +502,7 @@ export const graphStoreConformance = ( revision, model: pairModel, overlay: {}, + facts: { artefacts: {}, values: {} }, }); }), ); @@ -669,6 +674,8 @@ export const graphStoreConformance = ( relationConformance(options.createClient); + factConformance(options.createClient); + test( "a write whose signal has already fired is refused and changes nothing", withClient(options.createClient, async (client) => { diff --git a/packages/core/graph/store-contract/src/contract.ts b/packages/core/graph/store-contract/src/contract.ts index b38c125..8c9de4d 100644 --- a/packages/core/graph/store-contract/src/contract.ts +++ b/packages/core/graph/store-contract/src/contract.ts @@ -3,12 +3,15 @@ import { modelSchema, overlaySchema } from "@exadev/process-graph-schema"; import { z } from "zod"; import { boundItemsSchema, + factChangeSchema, + factClearingSchema, issueSchema, modelIdSchema, modelSummarySchema, reachSchema, relationSchema, revisionSchema, + storedFactsSchema, storedModelSchema, storeChangeSchema, } from "./schema.ts"; @@ -43,6 +46,7 @@ const written = z.strictObject({ revision: revisionSchema }).readonly(); * - `overlay.set` sets the run state of the elements it names. * - `items.upsert` merges items read from a source into a model by identifier, creating the model when it does not exist. * - `relations.list` returns the relations kept beside a model, which `items.upsert` adds to. + * - `facts.read`, `facts.set` and `facts.clear` keep what is known from outside the process about a model's run: which artefacts are provided and the values of named facts its conditions read. `models.load` returns the same record beside the overlay. * - `query.ready`, `query.blockedChain` and `query.impact` answer structural questions from the stored transitions and run state alone (see the package README for exactly what they read), so a store can answer them without loading the model or evaluating conditions. * - `query.children`, `query.descendants` and `query.ancestors` walk the stored `child-of` relations below and above an element. * - `changes` yields every later change to one model as an event iterator. A subscriber that falls behind by more than the store's subscription capacity gets one `missed` in place of the changes it let pile up, so a store's memory stays bounded and the subscriber knows to load the model again. @@ -81,6 +85,23 @@ const graphStoreV1Contract = oc.errors(graphStoreErrors).router({ ) .output(written), }, + facts: { + read: oc.input(modelRef).output(storedFactsSchema), + set: oc + .input( + z + .strictObject({ modelId: modelIdSchema, facts: factChangeSchema }) + .readonly(), + ) + .output(written), + clear: oc + .input( + z + .strictObject({ modelId: modelIdSchema, facts: factClearingSchema }) + .readonly(), + ) + .output(written), + }, items: { upsert: oc .input( diff --git a/packages/core/graph/store-contract/src/facts.ts b/packages/core/graph/store-contract/src/facts.ts new file mode 100644 index 0000000..254f0ed --- /dev/null +++ b/packages/core/graph/store-contract/src/facts.ts @@ -0,0 +1,92 @@ +import { emptyFactRecord, type FactRecord } from "@exadev/process-graph-domain"; +import type { Issue, Model, OpenIds } from "@exadev/process-graph-schema"; +import type { FactChange, FactClearing } from "./schema.ts"; + +/** + * The write rules for a model's fact record, shared by every store so each keeps facts the same way. Each function returns a new record and never changes the one it is given. + */ + +/** The entries of a record whose key passes a test. */ +const entriesWhere = ( + record: Readonly>, + keep: (key: string) => boolean, +): Readonly> => + Object.fromEntries(Object.entries(record).filter(([key]) => keep(key))); + +/** + * Keeps only the artefact entries of a record that name an artefact the model declares, so replacing a model drops what was known about artefacts it no longer has. Named facts belong to the run rather than to any record of the model, so they are all kept. + * @param model - The model the record is for. + * @param facts - The record to narrow. + * @returns The record with its artefacts narrowed to the model's. + */ +export const factsWithin = ( + model: Model, + facts: FactRecord, +): FactRecord => { + const declared = new Set(model.artefacts.map(({ id }) => id)); + + return { + artefacts: entriesWhere(facts.artefacts, (id) => declared.has(id)), + values: facts.values, + }; +}; + +/** + * Applies a change to a record: each artefact and named fact the change gives replaces what the record held for it, and the rest is kept. The result is not checked; see `factIssues`. + * @param facts - The stored record. + * @param change - The facts to record. + * @returns The merged record. + */ +export const mergeFacts = ( + facts: FactRecord, + change: FactChange, +): FactRecord => ({ + artefacts: { ...facts.artefacts, ...change.artefacts }, + values: { ...facts.values, ...change.values }, +}); + +/** + * Forgets facts, so what they named is unknown again: the whole record with `all`, otherwise the named artefacts and named facts. A name the record does not hold is ignored. + * @param facts - The stored record. + * @param clearing - What to forget. + * @returns The record without what was forgotten. + */ +export const withoutFacts = ( + facts: FactRecord, + clearing: FactClearing, +): FactRecord => { + if ("all" in clearing) { + return emptyFactRecord; + } + const artefacts = new Set(clearing.artefacts); + const values = new Set(clearing.values); + + return { + artefacts: entriesWhere(facts.artefacts, (id) => !artefacts.has(id)), + values: entriesWhere(facts.values, (name) => !values.has(name)), + }; +}; + +/** + * The issues with a change to a model's facts: an artefact the model does not declare is an `unknown-reference`, as an unknown reference anywhere else in a model is. Named facts are free names, since any `reference` key in a condition may read one. + * @param model - The model the facts are for. + * @param change - The facts to record. + * @returns One issue per undeclared artefact, empty when every artefact resolves. + */ +export const factIssues = ( + model: Model, + change: FactChange, +): readonly Issue[] => { + const declared = new Set(model.artefacts.map(({ id }) => id)); + + return Object.keys(change.artefacts ?? {}).flatMap((id) => + declared.has(id) + ? [] + : [ + { + code: "unknown-reference" as const, + message: `facts refer to unknown artefact id "${id}"`, + }, + ], + ); +}; diff --git a/packages/core/graph/store-contract/src/fake.ts b/packages/core/graph/store-contract/src/fake.ts index 68db092..28e0c00 100644 --- a/packages/core/graph/store-contract/src/fake.ts +++ b/packages/core/graph/store-contract/src/fake.ts @@ -1,5 +1,7 @@ +import { emptyFactRecord, type FactRecord } from "@exadev/process-graph-domain"; import type { Model, OpenIds, Overlay } from "@exadev/process-graph-schema"; import { GraphStoreError } from "./errors.ts"; +import { factIssues, factsWithin, mergeFacts, withoutFacts } from "./facts.ts"; import { ChangeFeed } from "./feed.ts"; import { ancestorsOf, @@ -30,6 +32,7 @@ interface Entry { readonly model: Model; readonly overlay: Overlay; readonly relations: readonly Relation[]; + readonly facts: FactRecord; readonly revision: number; } @@ -87,6 +90,7 @@ export const createFakeGraphStore = (): GraphStore => { readonly model: Model; readonly overlay: Overlay; readonly relations: readonly Relation[]; + readonly facts: FactRecord; readonly signal: AbortSignal | undefined; } @@ -96,16 +100,26 @@ export const createFakeGraphStore = (): GraphStore => { model, overlay, relations, + facts, signal, }: Write): number => { checkStored(model, overlay); - const issues = relationIssues(model, relations); + const issues = [ + ...relationIssues(model, relations), + ...factIssues(model, facts), + ]; if (issues.length > 0) { throw new GraphStoreError({ kind: "invalid", issues }); } signal?.throwIfAborted(); sequence += 1; - entries.set(modelId, { model, overlay, relations, revision: sequence }); + entries.set(modelId, { + model, + overlay, + relations, + facts, + revision: sequence, + }); feed.publish({ modelId, kind, revision: sequence }); return sequence; @@ -127,13 +141,14 @@ export const createFakeGraphStore = (): GraphStore => { model, overlay: overlayWithin(model, existing?.overlay ?? {}), relations: relationsWithin(model, existing?.relations ?? []), + facts: factsWithin(model, existing?.facts ?? emptyFactRecord), signal, }); }), loadModel: later(({ modelId }): StoredModel => { - const { model, overlay, revision } = entryOf(modelId); + const { model, overlay, facts, revision } = entryOf(modelId); - return { modelId, revision, model, overlay }; + return { modelId, revision, model, overlay, facts }; }), deleteModel: later(({ modelId, signal }) => { if (!entries.has(modelId)) { @@ -155,6 +170,34 @@ export const createFakeGraphStore = (): GraphStore => { model: entry.model, overlay: { ...entry.overlay, ...overlay }, relations: entry.relations, + facts: entry.facts, + signal, + }); + }), + readFacts: later(({ modelId }) => { + const { facts, revision } = entryOf(modelId); + + return { modelId, revision, facts }; + }), + setFacts: later(({ modelId, facts, signal }) => { + const entry = entryOf(modelId); + + return write({ + ...entry, + modelId, + kind: "facts", + facts: mergeFacts(entry.facts, facts), + signal, + }); + }), + clearFacts: later(({ modelId, facts, signal }) => { + const entry = entryOf(modelId); + + return write({ + ...entry, + modelId, + kind: "facts", + facts: withoutFacts(entry.facts, facts), signal, }); }), @@ -163,6 +206,7 @@ export const createFakeGraphStore = (): GraphStore => { model: emptyModel, overlay: {}, relations: [], + facts: emptyFactRecord, }; return write({ @@ -170,6 +214,7 @@ export const createFakeGraphStore = (): GraphStore => { kind: "items", ...mergeBoundItems(entry.model, entry.overlay, items), relations: mergeRelations(entry.relations, items.relations ?? []), + facts: entry.facts, signal, }); }), diff --git a/packages/core/graph/store-contract/src/index.ts b/packages/core/graph/store-contract/src/index.ts index 004c1df..f75c204 100644 --- a/packages/core/graph/store-contract/src/index.ts +++ b/packages/core/graph/store-contract/src/index.ts @@ -8,6 +8,10 @@ export { boundActivitySchema, boundItemsSchema, changeKinds, + factChangeSchema, + factClearingSchema, + factNameSchema, + factRecordSchema, issueSchema, modelIdSchema, modelSummarySchema, @@ -16,10 +20,13 @@ export { relationSchema, revisionSchema, storeChangeSchema, + storedFactsSchema, storedModelSchema, type BoundActivity, type BoundItems, type ChangeKind, + type FactChange, + type FactClearing, type ModelId, type ModelSummary, type Reach, @@ -27,6 +34,7 @@ export { type RelationKind, type Revision, type StoreChange, + type StoredFacts, type StoredModel, } from "./schema.ts"; export { GraphStoreError, type GraphStoreFailure } from "./errors.ts"; @@ -44,6 +52,7 @@ export { mergeBoundItems, overlayWithin, } from "./merge.ts"; +export { factIssues, factsWithin, mergeFacts, withoutFacts } from "./facts.ts"; export { ancestorsOf, childrenOf, diff --git a/packages/core/graph/store-contract/src/router.ts b/packages/core/graph/store-contract/src/router.ts index ed96810..a3cf911 100644 --- a/packages/core/graph/store-contract/src/router.ts +++ b/packages/core/graph/store-contract/src/router.ts @@ -92,6 +92,23 @@ export const createGraphStoreRouter = (store: GraphStore): GraphStoreRouter => })), ), }, + facts: { + read: os.v1.facts.read.handler(async ({ input, signal, errors }) => + call(signal, errors, async () => + store.readFacts({ ...input, signal }), + ), + ), + set: os.v1.facts.set.handler(async ({ input, signal, errors }) => + call(signal, errors, async () => ({ + revision: await store.setFacts({ ...input, signal }), + })), + ), + clear: os.v1.facts.clear.handler(async ({ input, signal, errors }) => + call(signal, errors, async () => ({ + revision: await store.clearFacts({ ...input, signal }), + })), + ), + }, items: { upsert: os.v1.items.upsert.handler(async ({ input, signal, errors }) => call(signal, errors, async () => ({ diff --git a/packages/core/graph/store-contract/src/schema.ts b/packages/core/graph/store-contract/src/schema.ts index c865fbe..0e69894 100644 --- a/packages/core/graph/store-contract/src/schema.ts +++ b/packages/core/graph/store-contract/src/schema.ts @@ -1,4 +1,6 @@ +import type { FactRecord } from "@exadev/process-graph-domain"; import { modelSchema, overlaySchema } from "@exadev/process-graph-schema"; +import { ComputedValueSchema } from "trilean"; import { z } from "zod"; /** @@ -56,16 +58,68 @@ export const boundItemsSchema = z }) .readonly(); -/** One stored model and the run state laid over it, at the revision of its last change. */ +/** The name of a fact a condition reads: the string key of its `reference` node. */ +export const factNameSchema = z.string().min(1); + +/** + * What is known from outside the process about one model's run, kept beside its overlay: which artefacts are provided (`true`) or known not to be (`false`), by artefact id, and the trilean value of each named fact, by name. What the record does not name is unknown. It is graph-domain's `FactRecord`, which `factsFrom` adapts to the `Facts` port the reasoning reads. + */ +export const factRecordSchema = z + .strictObject({ + artefacts: z.record(z.string().min(1), z.boolean()).readonly(), + values: z.record(factNameSchema, ComputedValueSchema).readonly(), + }) + .readonly() satisfies z.ZodType; + +/** One stored model, the run state laid over it and the facts known about its run, at the revision of its last change. */ export const storedModelSchema = z .strictObject({ modelId: modelIdSchema, revision: revisionSchema, model: modelSchema, overlay: overlaySchema, + facts: factRecordSchema, + }) + .readonly(); + +/** A model's fact record at the revision of the model's last change. */ +export const storedFactsSchema = z + .strictObject({ + modelId: modelIdSchema, + revision: revisionSchema, + facts: factRecordSchema, + }) + .readonly(); + +/** + * Facts to record for a model: each artefact's provision and each named fact's value replaces what the record held for it, and everything the change does not name is kept. Every artefact must be one the model declares. + */ +export const factChangeSchema = z + .strictObject({ + artefacts: z + .record(z.string().min(1), z.boolean()) + .readonly() + .exactOptional(), + values: z + .record(factNameSchema, ComputedValueSchema) + .readonly() + .exactOptional(), }) .readonly(); +/** + * Facts to forget, so they are unknown again: either the named artefacts and named facts, or, with `all`, the whole record. A name the record does not hold is ignored. + */ +export const factClearingSchema = z.union([ + z + .strictObject({ + artefacts: z.array(z.string().min(1)).readonly().exactOptional(), + values: z.array(factNameSchema).readonly().exactOptional(), + }) + .readonly(), + z.strictObject({ all: z.literal(true) }).readonly(), +]); + /** A stored model's name and the revision of its last change. */ export const modelSummarySchema = z .strictObject({ modelId: modelIdSchema, revision: revisionSchema }) @@ -77,7 +131,13 @@ export const reachSchema = z .readonly(); /** The kinds of change a store reports. */ -export const changeKinds = ["saved", "overlay", "items", "deleted"] as const; +export const changeKinds = [ + "saved", + "overlay", + "items", + "facts", + "deleted", +] as const; /** * A change to a model. `revision` is the revision the change took. `missed` says the subscriber missed one or more changes and should load the model again: it is sent at the start of a subscription that names a revision the model is no longer at (the model changed, or has gone), and in place of the changes a subscriber that reads too slowly has let pile up beyond its subscription's capacity. @@ -107,6 +167,9 @@ export type BoundItems = z.infer; export type Relation = z.infer; export type RelationKind = (typeof relationKinds)[number]; export type StoredModel = z.infer; +export type StoredFacts = z.infer; +export type FactChange = z.infer; +export type FactClearing = z.infer; export type ModelSummary = z.infer; export type Reach = z.infer; export type ChangeKind = (typeof changeKinds)[number]; diff --git a/packages/core/graph/store-contract/src/store.ts b/packages/core/graph/store-contract/src/store.ts index e2995dd..722db4d 100644 --- a/packages/core/graph/store-contract/src/store.ts +++ b/packages/core/graph/store-contract/src/store.ts @@ -2,10 +2,13 @@ import type { Model, OpenIds, Overlay } from "@exadev/process-graph-schema"; import type { ChangeFeed } from "./feed.ts"; import type { BoundItems, + FactChange, + FactClearing, ModelSummary, Reach, Relation, Revision, + StoredFacts, StoredModel, } from "./schema.ts"; @@ -37,19 +40,19 @@ export interface GraphStore { readonly signal: AbortSignal | undefined; }) => Promise; /** - * Stores a model under an id, replacing any model stored there and dropping the run state and relations of what the new model does not declare (see `relationsWithin`). + * Stores a model under an id, replacing any model stored there and dropping the run state, relations and artefact facts of what the new model does not declare (see `overlayWithin`, `relationsWithin` and `factsWithin`). * @returns The revision the save took. */ readonly saveModel: ( request: ModelRequest & { readonly model: Model }, ) => Promise; /** - * Reads a model back exactly as saved or merged, in its declared order, with its run state. - * @returns The model, its overlay and its revision. + * Reads a model back exactly as saved or merged, in its declared order, with its run state and its fact record. + * @returns The model, its overlay, its facts and its revision. */ readonly loadModel: (request: ModelRequest) => Promise; /** - * Removes a model, its run state and its relations. + * Removes a model, its run state, its relations and its facts. * @returns The revision the deletion took, or `undefined` when there was no such model and nothing changed. */ readonly deleteModel: ( @@ -62,6 +65,25 @@ export interface GraphStore { readonly applyOverlay: ( request: ModelRequest & { readonly overlay: Overlay }, ) => Promise; + /** + * A model's fact record. + * @returns The record and the model's revision. + */ + readonly readFacts: (request: ModelRequest) => Promise; + /** + * Records facts about a model's run (see `mergeFacts`). Every artefact named must be one the model declares (`factIssues`); otherwise nothing is written. + * @returns The revision the change took. + */ + readonly setFacts: ( + request: ModelRequest & { readonly facts: FactChange }, + ) => Promise; + /** + * Forgets facts about a model's run, so what they named is unknown again (see `withoutFacts`). + * @returns The revision the change took. + */ + readonly clearFacts: ( + request: ModelRequest & { readonly facts: FactClearing }, + ) => Promise; /** * Merges items read from a source into a model by id (see `mergeBoundItems`), and adds their relations (see `mergeRelations`), creating the model from an empty one when it does not exist. Every relation stored must name what the merged model declares (`relationIssues`); otherwise nothing is written. * @returns The revision the change took. diff --git a/packages/targets/graph-cli/README.md b/packages/targets/graph-cli/README.md index 383e457..222cfa9 100644 --- a/packages/targets/graph-cli/README.md +++ b/packages/targets/graph-cli/README.md @@ -17,26 +17,31 @@ process-graph show --view swimlane --format mermaid ## Commands -| Command | What it does | -| ------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | -| `sync [--after ]` | Reads the scopes from their sources and merges them into the model in one write; prints what it wrote and any gaps. | -| `ready` | The work that can start now. | -| `blocked ` | The open work an element is waiting on: the ends of its blocked-by chain. | -| `impact ` | The live work that depends on an element. | -| `path` | The critical path: the longest chain of work still ahead, each element counting one step. | -| `cycles` | The cycles no decision closes. | -| `handoffs` | The transitions from one actor's work to another's, such as a dependency across repositories or organisations. | -| `why ` | The rule that decided an element's status, its needs and artefacts, and the blocked-by chain behind it. | -| `show [--view ] [--format ]` | Draws the model. Views: `flowchart` (default), `swimlane`, `state`, `sequence`. Formats: `mermaid` (default), `dot`, `json` (the projection itself). | -| `add participant