Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,15 @@ Configuration lives at exactly one of two levels: in the package (`eslint.config

## Packages

| Package | What it is |
| -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| [`graph-dag`](packages/core/graph/dag/README.md) | Model-independent dependency-graph primitives over a node set and a `needs` accessor: layers, ancestors, descendants, cycles, transitive reduction, critical path. |
| [`graph-schema`](packages/core/graph/schema/README.md) | The process model: typed identifier spaces, `defineModel` with compile-time reference checking, structural validation. |
| [`graph-projection`](packages/core/graph/projection/README.md) | Projections of the model into a flowchart, swimlane, state machine and sequence diagram, plus derived hand-offs. |
| [`graph-domain`](packages/core/graph/domain/README.md) | Reasoning over the model and a run overlay: control graph, illegal cycles, three-valued status, ready set, blocked-by, impact, what-if and critical path. |
| [`graph-layout`](packages/core/graph/layout/README.md) | Layered layout of a projected flowchart, swimlane or state machine into positions in the model's `views` shape. |
| [`process-graph`](packages/targets/process-graph/README.md) | The primary entry point: re-exports the schema, projection and domain APIs flat and the DAG API as the `dag` namespace. |
| Package | What it is |
| ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`graph-dag`](packages/core/graph/dag/README.md) | Model-independent dependency-graph primitives over a node set and a `needs` accessor: layers, ancestors, descendants, cycles, transitive reduction, critical path. |
| [`graph-schema`](packages/core/graph/schema/README.md) | The process model: typed identifier spaces, `defineModel` with compile-time reference checking, structural validation. |
| [`graph-projection`](packages/core/graph/projection/README.md) | Projections of the model into a flowchart, swimlane, state machine and sequence diagram, plus derived hand-offs. |
| [`graph-domain`](packages/core/graph/domain/README.md) | Reasoning over the model and a run overlay: control graph, illegal cycles, three-valued status, ready set, blocked-by, impact, what-if and critical path. |
| [`graph-layout`](packages/core/graph/layout/README.md) | Layered layout of a projected flowchart, swimlane or state machine into positions in the model's `views` shape. |
| [`graph-source-contract`](packages/core/graph/source-contract/README.md) | The `GraphSource` port as an oRPC contract: versioned, paged and streamed sync of bound activities, participants, transitions and run state, resumable by cursor, with an in-memory fake and a conformance kit. |
| [`process-graph`](packages/targets/process-graph/README.md) | The primary entry point: re-exports the schema, projection and domain APIs flat and the DAG API as the `dag` namespace. |

## Conventions

Expand Down
1 change: 1 addition & 0 deletions packages/core/graph/source-contract/AGENTS.md
1 change: 1 addition & 0 deletions packages/core/graph/source-contract/CLAUDE.md
118 changes: 118 additions & 0 deletions packages/core/graph/source-contract/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# @exadev/process-graph-source-contract

[![GitHub](https://img.shields.io/badge/GitHub-181717?logo=github&logoColor=white)](https://github.com/ExaDev/process-graph) [![npm](https://img.shields.io/badge/npm-CB3837?logo=npm&logoColor=white)](https://www.npmjs.com/package/@exadev/process-graph-source-contract) [![Release](https://img.shields.io/github/v/release/ExaDev/process-graph)](https://github.com/ExaDev/process-graph/releases/latest) [![CI](https://img.shields.io/github/actions/workflow/status/ExaDev/process-graph/ci.yml?branch=main)](https://github.com/ExaDev/process-graph/actions)

The `GraphSource` port: how the process model from `@exadev/process-graph-schema` reads work from an external tracker. It is an [oRPC](https://orpc.dev) router contract (`@orpc/contract`), so one definition is called in process, across a `MessagePort` to a worker, or over HTTP, and a consumer never changes with the transport. The package also ships the router builder adapters use, an in-memory fake and the conformance kit every adapter must pass. `src/` imports nothing Node-only.

```ts
import { createRouterClient } from "@orpc/server";
import {
assembleSync,
createFakeGraphSource,
createGraphSourceRouter,
} from "@exadev/process-graph-source-contract";

const router = createGraphSourceRouter(
createFakeGraphSource({ source: "fake", scopes }),
);
const client = createRouterClient(router);

const pages = [];
for await (const page of await client.v1.sync.stream({
scope: ["fake:acme/alpha"],
})) {
pages.push(page);
}
const { model, overlay, relations, gaps } = assembleSync(pages);
```

## The contract

`graphSourceContract` is keyed by version. A breaking change adds `v2` beside `v1`, so one server serves both while clients move across.

| Procedure | Input | Output |
| ---------------- | ------------------- | -------------------------------------------- |
| `v1.describe` | none | `{ source }`: the prefix of every identifier |
| `v1.sync.page` | `{ scope, after? }` | one `SyncPage` |
| `v1.sync.stream` | `{ scope, after? }` | an event iterator of `SyncPage` |

`scope` lists source-qualified scope identifiers, read in order, each once: for GitHub, `github:org` for every repository an owner has and `github:org/repo` for one. `after` is a cursor from an earlier page's `next`.

A `SyncPage` is a slice of a model: bound activities, the participants they name, the transitions between them, relations, run state as an `Overlay`, and gaps. Its record schemas are taken from `modelSchema`, so a page can never drift from the model.

- **Bound activity.** A model `activity` whose `source` is required and equal to its `id`, a source-qualified identifier such as `github:org/repo#12`. The same record reached twice is the same element.
- **Transition.** One per dependency, from the prerequisite to the work that waits on it.
- **Relation.** A link the model has no field for: `child-of` (a sub-task to its parent), `assigned-to` (an activity to a person) and `part-of` (a repository to its organisation). Relations carry no ordering; a consumer decides whether a parent waits on its children.
- **Gap.** Part of the scope the source could not read (`not-found` or `forbidden`), reported as data so the rest of the sync still arrives.
- **`next`.** The cursor to resume after this page; absent on the last page.

## Resuming

Cursors are stable: the same scope and cursor give the same page while the source is unchanged, and a cursor holds no in-memory state, so a sync interrupted in one process resumes in another. `sync.stream` reads pages only as the consumer pulls them, so a slow consumer applies backpressure and closing the iterator stops the reads. Each event carries its `next` cursor as the event id, so over an event-stream transport a reconnecting client's `lastEventId` resumes there; an explicit `after` takes precedence.

## Errors

Failures that end a call are typed oRPC errors with their detail as data, using oRPC's common codes so an HTTP transport answers with the matching status.

| Code | Data | When |
| ------------------- | -------------------------------------------------------- | -------------------------------------------------------- |
| `UNAUTHORIZED` | `{}` | The source rejected the credentials. |
| `TOO_MANY_REQUESTS` | `{ resetAt? }` | The source is rate limiting; resume after `resetAt`. |
| `BAD_REQUEST` | `{ reason: "invalid-cursor" \| "invalid-scope", value }` | A cursor the source did not issue, or a malformed scope. |

What a source can partly read is not an error: it is a page with gaps. Every call takes the caller's abort signal through the oRPC client options and passes it to the source.

## Writing an adapter

An adapter implements `GraphSourcePager`, one page at a time, and throws `GraphSourceError` for the failures above. `createGraphSourceRouter(pager)` builds the whole port on it, so paging, streaming, resumption and error mapping are the same for every source.

```ts
const pager: GraphSourcePager = {
source: "linear",
page: async ({ scope, cursor, signal }) => readOnePage(scope, cursor, signal),
};
```

## Conformance

`@exadev/process-graph-source-contract/conformance` exports `graphSourceConformance`, a set of Vitest tests (Vitest is an optional peer dependency, needed only for this entry point). Call it from the adapter's own tests, once per transport; the tests talk only to the oRPC client.

```ts
import {
graphSourceConformance,
inProcessClient,
} from "@exadev/process-graph-source-contract/conformance";

graphSourceConformance({
name: "my adapter, in process",
createClient: () => inProcessClient(createGraphSourceRouter(pager)),
scope: ["linear:team-a"], // must span at least two pages
missingScope: "linear:no-such-team",
malformedScope: "team-a",
});
```

The entry point also exports `inProcessClient(router)` and `messagePortClient(router)`, the two transports this package's own tests run the kit over; a `messagePortClient` closes its channel after the test file.

It checks that paging and streaming agree; that a stream closed after any page resumes from that page's `next` with exactly the pages that follow; that a cursor never repeats; that a sync is deterministic; that the assembled model and overlay pass the schema's validation and every identifier carries the source prefix; that relations name what the sync returned; that an unknown scope is a gap beside the rest; that malformed scopes and foreign cursors are typed errors; and that an aborted signal ends both procedures.

## Functions

| Export | What it is |
| -------------------------------- | ------------------------------------------------------------------------------------------- |
| `graphSourceContract` | The router contract. |
| `GraphSourceClient` | The client type for the contract, over any transport. |
| `createGraphSourceRouter(pager)` | The implemented router over a pager. |
| `assembleSync(pages)` | Merges a sync's pages into one model, overlay, relation list and gap list; later pages win. |
| `paginate(first, read)` | An async iterable over a cursor-linked sequence, reading each page only as it is pulled. |
| `createFakeGraphSource(options)` | The reference in-memory pager. |
| `GraphSourceError` | What a pager throws to end a call with a typed failure. |

## Commands

```sh
pnpm build # tsc -b -> dist/ (ESM + .d.ts)
pnpm test # unit tests, including the conformance kit over the fake in process and over a message port
pnpm test:smoke # the built dist/ exercised as a real artefact
pnpm test:mutation # Stryker
```
9 changes: 9 additions & 0 deletions packages/core/graph/source-contract/eslint.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import { defineConfig } from "eslint/config";
import { packageLintConfig } from "../../../../eslint.shared.ts";

export default defineConfig(
packageLintConfig({
tsconfigRootDir: import.meta.dirname,
isomorphic: true,
}),
);
90 changes: 90 additions & 0 deletions packages/core/graph/source-contract/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
{
"name": "@exadev/process-graph-source-contract",
"version": "0.0.0",
"description": "The GraphSource port as an oRPC contract: versioned, paginated and streamed sync of bound activities, participants, transitions and overlay status, with an in-memory fake and a conformance kit every adapter must pass.",
"type": "module",
"repository": {
"type": "git",
"url": "git+https://github.com/ExaDev/process-graph.git",
"directory": "packages/core/graph/source-contract"
},
"homepage": "https://github.com/ExaDev/process-graph/tree/main/packages/core/graph/source-contract#readme",
"bugs": {
"url": "https://github.com/ExaDev/process-graph/issues"
},
"exports": {
".": {
"types": "./dist/index.d.ts",
"import": "./dist/index.js"
},
"./conformance": {
"types": "./dist/conformance.d.ts",
"import": "./dist/conformance.js"
}
},
"types": "./dist/index.d.ts",
"files": [
"dist"
],
"publishConfig": {
"access": "public",
"provenance": true,
"registry": "https://registry.npmjs.org/"
},
"sideEffects": false,
"license": "MIT",
"scripts": {
"build": "turbo run _build",
"_build": "tsc -b tsconfig.build.json",
"lint": "turbo run _lint",
"_lint": "eslint . --cache --max-warnings 0",
"_lint:fix": "eslint . --fix --cache --max-warnings 0",
"typecheck": "turbo run _typecheck _typecheck:attw",
"_typecheck": "tsc -p tsconfig.json --noEmit && tsc -p tsconfig.node.json --noEmit",
"_typecheck:attw": "attw --pack --profile esm-only",
"test": "turbo run _test",
"_test": "vitest run --project unit",
"test:coverage": "turbo run _test:coverage",
"_test:coverage": "vitest run --project unit --coverage",
"test:smoke": "turbo run _test:smoke",
"_test:smoke": "vitest run --project smoke",
"test:mutation": "turbo run _test:mutation",
"_test:mutation": "stryker run stryker.config.ts",
"prepublishOnly": "pnpm run lint && pnpm run typecheck && pnpm run test && pnpm run build && publint && attw --pack --profile esm-only"
},
"keywords": [
"process-model",
"orpc",
"contract",
"port",
"conformance"
],
"devDependencies": {
"@arethetypeswrong/cli": "0.18.5",
"@stryker-mutator/core": "10.0.0",
"@stryker-mutator/typescript-checker": "10.0.0",
"@stryker-mutator/vitest-runner": "10.0.0",
"@types/node": "26.6.4",
"@vitest/coverage-v8": "5.0.3",
"eslint": "10.12.0",
"publint": "0.3.25",
"typescript": "6.0.3",
"vitest": "5.0.3"
},
"dependencies": {
"@exadev/process-graph-schema": "1.0.2",
"@orpc/client": "2.0.0-beta.41",
"@orpc/contract": "2.0.0-beta.41",
"@orpc/server": "2.0.0-beta.41",
"trilean": "1.7.2",
"zod": "4.6.5"
},
"peerDependencies": {
"vitest": "5.0.3"
},
"peerDependenciesMeta": {
"vitest": {
"optional": true
}
}
}
70 changes: 70 additions & 0 deletions packages/core/graph/source-contract/src/assemble.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
import type { Model, OpenIds, Overlay } from "@exadev/process-graph-schema";
import type { Gap, Relation, SyncPage } from "./schema.ts";

/** A whole sync merged into one model, with what the model has no place for kept beside it. */
export interface AssembledSync {
/** The participants, bound activities and transitions of every page, each identifier once. */
readonly model: Model<OpenIds>;
/** The run state of every activity, keyed by its identifier. */
readonly overlay: Overlay<OpenIds>;
/** Every relation, each distinct one once. */
readonly relations: readonly Relation[];
/** Every gap, in page order. */
readonly gaps: readonly Gap[];
}

const relationKey = (relation: Readonly<Relation>): string =>
JSON.stringify([relation.kind, relation.from, relation.to]);

/** Keeps one entry per key, in first-seen order, holding the latest value seen for it. */
const latestById = <T>(
items: readonly T[],
keyOf: (item: T) => string,
): readonly T[] => {
const byKey = new Map<string, T>();
for (const item of items) {
byKey.set(keyOf(item), item);
}

return [...byKey.values()];
};

/**
* Merges the pages of a sync into one model. A record can appear on several pages, for example an issue in another repository reached through a dependency before its own repository is read; identity is the identifier, so it becomes one element, and where pages disagree the later page wins, being the more recent read.
* @param pages - The pages of one sync, in the order they arrived.
* @returns The model, overlay, relations and gaps of the whole sync.
*/
export const assembleSync = (pages: readonly SyncPage[]): AssembledSync => {
// Entries later in the list overwrite earlier ones, so the later page wins.
const overlay: Overlay<OpenIds> = Object.fromEntries(
pages.flatMap((page) => Object.entries(page.overlay)),
);

return {
model: {
participants: latestById(
pages.flatMap((page) => page.participants),
(participant) => participant.id,
),
events: [],
actions: [],
artefacts: [],
elements: latestById(
pages.flatMap((page) => page.activities),
(activity) => activity.id,
),
transitions: latestById(
pages.flatMap((page) => page.transitions),
(transition) => transition.id,
),
fragments: [],
projections: [],
},
overlay,
relations: latestById(
pages.flatMap((page) => page.relations),
relationKey,
),
gaps: pages.flatMap((page) => page.gaps),
};
};
Loading