Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,8 @@ Configuration lives at exactly one of two levels: in the package (`eslint.config
| [`graph-store-contract`](packages/core/graph/store-contract/README.md) | The persistence port as an oRPC contract: save, load and delete models, set run state, upsert items from a source, structural ready, blocked-chain and impact queries, and a change feed; with an in-memory fake and a conformance kit. |
| [`graph-store-sqlite`](packages/adapters/graph/store-sqlite/README.md) | The store port on SQLite through Drizzle: one table definition, committed drizzle-kit migrations applied the same way in every runtime, recursive SQL for the structural queries, against Drizzle's async SQLite database so any driver plugs in. |
| [`graph-sqlite-node`](packages/adapters/graph/sqlite-node/README.md) | The SQLite store on Node's built-in `node:sqlite` through `drizzle-orm/node-sqlite`, for the command line and servers. |
| [`process-graph`](packages/targets/process-graph/README.md) | The primary entry point: re-exports the schema, projection and domain APIs flat and the DAG API as the `dag` namespace. |
| [`graph-cli`](packages/targets/graph-cli/README.md) | The command line and its library entry: sync GitHub issues into the SQLite store, ready, blocked, impact, critical path, cycles, hand-offs and status explanations, and every view drawn in Mermaid or Graphviz DOT. |
| [`process-graph`](packages/targets/process-graph/README.md) | The primary entry point: re-exports the schema, projection and domain APIs flat and the DAG API as the `dag` namespace, and carries the `process-graph` bin. |

## Conventions

Expand Down
3 changes: 2 additions & 1 deletion knip.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ const config: KnipConfig = {
project: ["src/**/*.{ts,tsx}!"],
},
"packages/targets/*": {
entry: ["src/index.{ts,tsx}"],
// A target's bin is an entry point of its own, run as a process rather than imported.
entry: ["src/index.{ts,tsx}", "src/bin.ts"],
project: ["src/**/*.{ts,tsx}!"],
},
},
Expand Down
31 changes: 17 additions & 14 deletions packages/core/graph/domain/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,19 +25,20 @@ deriveStatuses(model, overlay, facts); // Map { "triage" => "done", ... }

## Functions

| Function | Result |
| --------------------------------------------- | --------------------------------------------------------------------------------------- |
| `controlDag(model)` | The `graph-dag` `Dag` over element ids, with loop edges removed. |
| `loopTransitions(model)` | The transitions that close a legal decision loop. |
| `illegalCycles(model)` | The cycles of the control graph, each with its path. |
| `evaluateBranches(model, overlay, facts)` | Each transition's `taken`, `not-taken` or `indeterminate`. |
| `deriveStatuses(model, overlay, facts)` | The `Status` of every element. |
| `readySet(model, overlay, facts)` | The elements that are `ready`. |
| `blockedBy(model, overlay, facts, id)` | The open leaves an element is waiting on. |
| `impact(model, overlay, facts, id)` | The live elements that transitively depend on `id`. |
| `whatIfDone(model, overlay, facts, id)` | The elements that become ready and every status change if `id` were done. |
| `criticalPath(model, overlay, facts, weight)` | The heaviest chain through the work still ahead. |
| `unmetOutputs(model, overlay, facts)` | Each deliverable not yet produced, with the elements and missing inputs it still needs. |
| Function | Result |
| --------------------------------------------- | ---------------------------------------------------------------------------------------- |
| `controlDag(model)` | The `graph-dag` `Dag` over element ids, with loop edges removed. |
| `loopTransitions(model)` | The transitions that close a legal decision loop. |
| `illegalCycles(model)` | The cycles of the control graph, each with its path. |
| `evaluateBranches(model, overlay, facts)` | Each transition's `taken`, `not-taken` or `indeterminate`. |
| `deriveStatuses(model, overlay, facts)` | The `Status` of every element. |
| `readySet(model, overlay, facts)` | The elements that are `ready`. |
| `blockedBy(model, overlay, facts, id)` | The open leaves an element is waiting on. |
| `impact(model, overlay, facts, id)` | The live elements that transitively depend on `id`. |
| `explainStatus(model, overlay, facts, id)` | The rule that decided `id`'s status, what it was decided from, and the blocked-by chain. |
| `whatIfDone(model, overlay, facts, id)` | The elements that become ready and every status change if `id` were done. |
| `criticalPath(model, overlay, facts, weight)` | The heaviest chain through the work still ahead. |
| `unmetOutputs(model, overlay, facts)` | Each deliverable not yet produced, with the elements and missing inputs it still needs. |

Every function that derives status throws `CycleError` (from `graph-dag`, carrying the cycles) when `illegalCycles` is not empty, `UnknownNodeError` when a transition names an element the model does not declare, and `UnsupportedConditionError` for a condition the facts port cannot serve.

Expand Down Expand Up @@ -84,7 +85,9 @@ The `Facts` port is `(key: JsonValue) => Resolution`: an opaque key in, `{ found

## Queries

`blockedBy` starts at an element's unfinished needs and follows each one's own unfinished needs to the elements that wait on nothing further: the minimal set of open work to finish first. Cut transitions and provided artefacts are not followed. An element that is not blocked by anything returns an empty list.
`blockedBy` starts at an element's unfinished needs and follows each one's own unfinished needs to the elements that wait on nothing further: the minimal set of open work to finish first. Cut transitions and provided artefacts are not followed. An element that is not blocked by anything returns an empty list, and so does an element that is `done` or `skipped`, even one the overlay marked done ahead of its needs.

`explainStatus` says why an element has its status. `rule` names the rule of the status table that decided it (`overlay-done`, `overlay-unknown`, `unknown-need`, `all-cut`, `needs-met` or `waiting`); `transitions` lists its incoming transitions other than loop edges, each with its branch, its source's status and whether it is cut; `artefacts` lists what it consumes, each with whether it is provided and its producers' statuses. `chain` is the blocked-by chain: the element, then every unfinished need reached from it, each with the needs it still waits on, so the links after the first that wait on nothing are exactly what `blockedBy` returns. The explanation is recorded by the derivation itself, so it never disagrees with `deriveStatuses`.

`impact` follows live needs in reverse, so a branch that was not taken, a skipped element and an element already `done` carry no impact; `id` itself is not included.

Expand Down
117 changes: 104 additions & 13 deletions packages/core/graph/domain/src/derive.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,68 @@ export interface Analysis<I extends IdSpace> {
* For each element, the elements its progress still depends on: the sources of its incoming transitions that are not cut (the condition is not definitely false and the source is not skipped) and the producers of the artefacts it consumes that are neither provided nor produced already, except producers that are skipped. Includes elements that are done.
*/
readonly liveNeeds: ReadonlyMap<I["element"], readonly I["element"][]>;
/** For each element, what its status was derived from, recorded by the derivation itself so an explanation never re-derives it. */
readonly basis: ReadonlyMap<I["element"], StatusBasis<I>>;
}

/** Which of the derivation's rules decided an element's status, in the order the rules are applied. */
export const statusRules = [
"overlay-done",
"overlay-unknown",
"unknown-need",
"all-cut",
"needs-met",
"waiting",
] as const;

/**
* Which rule decided a status.
*
* - `overlay-done`: the overlay says `done`.
* - `overlay-unknown`: the overlay says `unknown`.
* - `unknown-need`: a live transition's condition is indeterminate or its source is `unknown`, or a consumed artefact that is not provided and has no done producer has an `unknown` producer.
* - `all-cut`: the element has incoming transitions and every one is cut, so it is `skipped`.
* - `needs-met`: every live need is done and every consumed artefact is satisfied, so it is `ready`, or `done` for a decision or pseudo-state.
* - `waiting`: anything else, so it is `blocked`.
*/
export type StatusRule = (typeof statusRules)[number];

/** An incoming transition of an element, as the derivation saw it. Loop edges are not incoming transitions. */
export interface IncomingBasis<I extends IdSpace> {
readonly transition: I["transition"];
readonly source: I["element"];
/** What the transition's condition evaluated to. */
readonly branch: Branch;
/** The derived status of the source. */
readonly sourceStatus: Status;
/** Whether the transition is cut: its condition is definitely false or its source is skipped. A cut transition is not a need. */
readonly cut: boolean;
}

/** A producer of a consumed artefact and its derived status. */
export interface ProducerBasis<I extends IdSpace> {
readonly element: I["element"];
readonly status: Status;
}

/** A consumed artefact of an activity, as the derivation saw it. */
export interface ArtefactBasis<I extends IdSpace> {
readonly artefact: I["artefact"];
/** Whether the facts port reports it provided from outside the process. */
readonly provided: boolean;
/** The activities that produce it, in model order. Empty when none does. */
readonly producers: readonly ProducerBasis<I>[];
/** Whether it is provided or a producer is done. */
readonly satisfied: boolean;
}

/** What one element's status was derived from. */
export interface StatusBasis<I extends IdSpace> {
readonly rule: StatusRule;
/** The element's incoming transitions other than loop edges, in model order. */
readonly transitions: readonly IncomingBasis<I>[];
/** The artefacts the element consumes, in the order it lists them; empty for anything but an activity. */
readonly artefacts: readonly ArtefactBasis<I>[];
}

/**
Expand Down Expand Up @@ -84,26 +146,45 @@ export const analyse = <I extends IdSpace>({

const statuses = new Map<I["element"], Status>();
const liveNeeds = new Map<I["element"], readonly I["element"][]>();
const basis = new Map<I["element"], StatusBasis<I>>();
for (const id of topologicalLayers(dag).layers.flat()) {
const element = lookup(elements, id);
const edges = lookup(incoming, id).map((transitionId) => ({
source: lookup(transitions, transitionId).source,
branch: lookup(branches, transitionId),
}));
const live = edges.filter(
({ source, branch }) =>
branch !== "not-taken" && lookup(statuses, source) !== "skipped",
);
const edges = lookup(incoming, id).map((transitionId) => {
const source = lookup(transitions, transitionId).source;
const branch = lookup(branches, transitionId);
const sourceStatus = lookup(statuses, source);

return {
transition: transitionId,
source,
branch,
sourceStatus,
cut: branch === "not-taken" || sourceStatus === "skipped",
};
});
const live = edges.filter(({ cut }) => !cut);

const dataNeeds = new Set<I["element"]>();
const artefacts: ArtefactBasis<I>[] = [];
let dataUnknown = false;
let dataUnmet = false;
for (const artefact of element.kind === "activity"
? (element.consumes ?? [])
: []) {
const makers = producers.get(artefact) ?? [];
const made = makers.map((maker) => lookup(statuses, maker));
if (isProvided(facts, artefact) || made.includes("done")) {
const provided = isProvided(facts, artefact);
const satisfied = provided || made.includes("done");
artefacts.push({
artefact,
provided,
producers: makers.map((maker) => ({
element: maker,
status: lookup(statuses, maker),
})),
satisfied,
});
if (satisfied) {
continue;
}
dataUnmet = true;
Expand All @@ -120,38 +201,48 @@ export const analyse = <I extends IdSpace>({

const own = overlay[id];
let status: Status;
let rule: StatusRule;
if (own === "done") {
status = "done";
rule = "overlay-done";
} else if (own === "unknown") {
status = "unknown";
rule = "overlay-unknown";
} else if (
own === "unknown" ||
dataUnknown ||
live.some(
({ source, branch }) =>
branch === "indeterminate" || lookup(statuses, source) === "unknown",
({ branch, sourceStatus }) =>
branch === "indeterminate" || sourceStatus === "unknown",
)
) {
status = "unknown";
rule = "unknown-need";
} else if (edges.length > 0 && live.length === 0) {
status = "skipped";
rule = "all-cut";
} else if (
!dataUnmet &&
live.every(({ source }) => lookup(statuses, source) === "done")
live.every(({ sourceStatus }) => sourceStatus === "done")
) {
status =
element.kind === "decision" || element.kind === "pseudo"
? "done"
: "ready";
rule = "needs-met";
} else {
status = "blocked";
rule = "waiting";
}
statuses.set(id, status);
basis.set(id, { rule, transitions: edges, artefacts });
}

return {
dag,
branches,
statuses: new Map(dag.nodes.map((id) => [id, lookup(statuses, id)])),
liveNeeds,
basis,
};
};

Expand Down
86 changes: 86 additions & 0 deletions packages/core/graph/domain/src/explain.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
import type { IdSpace, Model, Overlay } from "@exadev/process-graph-schema";
import {
analyse,
type ArtefactBasis,
type IncomingBasis,
type StatusRule,
} from "./derive.ts";
import type { Facts } from "./facts.ts";
import { lookup } from "./lookup.ts";
import { waitingOn } from "./progress.ts";
import type { Status } from "./status.ts";

/** One element of a blocked-by chain and the unfinished needs it still waits on. */
export interface ChainLink<I extends IdSpace> {
readonly element: I["element"];
readonly status: Status;
/** The element's live needs that are not done, in model order. Empty for the open work at the end of the chain. */
readonly waitingOn: readonly I["element"][];
}

/** Why an element has the status it has. */
export interface StatusExplanation<I extends IdSpace> {
readonly element: I["element"];
readonly status: Status;
/** The rule that decided the status. */
readonly rule: StatusRule;
/** The element's incoming transitions other than loop edges, each with its branch, its source's status and whether it is cut, in model order. */
readonly transitions: readonly IncomingBasis<I>[];
/** The artefacts the element consumes, each with whether it is provided and its producers' statuses. */
readonly artefacts: readonly ArtefactBasis<I>[];
/**
* The blocked-by chain: the element, then every unfinished need reached from it by following unfinished needs, in model order. The links after the first that wait on nothing are the elements `blockedBy` returns. A done or skipped element waits on nothing, so its chain is the element alone.
*/
readonly chain: readonly ChainLink<I>[];
}

/**
* Explains an element's status: the rule that decided it, the transitions and artefacts it was decided from, and the chain of unfinished work behind it. The explanation is the derivation's own record, so it always agrees with `deriveStatuses` and `blockedBy`.
* @param model - The model to read.
* @param overlay - Run state by element id.
* @param facts - The injected source of facts conditions read, and of which input artefacts are provided.
* @param id - The element asked about.
* @returns The explanation.
* @throws UnknownNodeError when `id` is not an element.
* @throws CycleError when the control graph has an illegal cycle.
*/
export const explainStatus = <I extends IdSpace>(
model: Model<I>,
overlay: Readonly<Overlay<I>>,
facts: Facts,
id: I["element"],
): StatusExplanation<I> => {
const analysis = analyse({ model, overlay, facts });
const basis = lookup(analysis.basis, id);
const reached = new Map<I["element"], readonly I["element"][]>();
const pending = [id];
for (let next = pending.pop(); next !== undefined; next = pending.pop()) {
if (reached.has(next)) {
continue;
}
const needs = waitingOn(analysis, next);
reached.set(next, needs);
pending.push(...needs);
}
const link = (element: I["element"]): ChainLink<I> => ({
element,
status: lookup(analysis.statuses, element),
waitingOn: analysis.dag.nodes.filter((node) =>
lookup(reached, element).includes(node),
),
});

return {
element: id,
status: lookup(analysis.statuses, id),
rule: basis.rule,
transitions: basis.transitions,
artefacts: basis.artefacts,
chain: [
link(id),
...analysis.dag.nodes
.filter((node) => node !== id && reached.has(node))
.map(link),
],
};
};
Loading
Loading