From 283a29b10634ec10190526f3c42b8a5792979bd3 Mon Sep 17 00:00:00 2001 From: Sean Mauk Date: Fri, 19 Jun 2026 17:45:55 +0000 Subject: [PATCH] refactor(stargraph)!: reconcile plugin with real stargraph tool surface The harbor->stargraph rename was textually complete but the plugin documented a CLI/API/plugin surface that never existed in the real stargraph tool (KrakenNet/stargraph). Reconcile every command, agent, reference, and skill against the actual tool, verified against src/stargraph + docs/reference/. CLI: drop fictional `graph verify`, `plugins verify|reload|inspect`, `skills compile|list`, `facts list|get`; document the real 8 subcommands (run/serve/inspect/replay/respond/simulate/counterfactual/verify-audit) with correct flags. Validation: `graph verify` -> `run --inspect` / `simulate` / stargraph.ir.validate. Facts + checkpoints -> `inspect RUN_ID --db --step N --diff N M`. Plugins: directory plugins / plugin.toml / ~/.stargraph/plugins -> entry-point pip package + stargraph_plugin manifest factory (pii_guard archetype). Skills: markdown SKILL.md compiler / stargraph-md-skills -> Python stargraph.skills.Skill + skill bundle (Shipwright layout). Graph YAML: ad-hoc shape -> real IRDocument (ir_version/id/nodes[kind]/ rules[when,then]/governance/state_class). Project config stargraph.yaml -> stargraph.toml (graph IR *.yaml unchanged). Env: invented STARGRAPH_URL/_TOKEN/_GRAPHS_DIR -> real STARGRAPH_PROFILE/ _CONFIG_DIR/_TRACE_PLUGINS/_TOML_FILENAME. API: :9000 + /health + /checkpoints + /replay + /events -> :8000 + real /v1/* route set. Stores: Kuzu -> RyuGraph/cypher + real provider ids. Naming: trigger kinds manual/cron/webhook; run ids UUIDv7 / cf-; tool key ns.name@ver; packs slug+version PackMount. 37 files: 8 agents, 18 commands, 8 references, smart-stargraph skill, README, plugin.json (0.3.0 -> 0.4.0). smart-kraken left byte-identical (cross-project lint anchor). BREAKING CHANGE: command/agent behavior, env vars, config filename, and documented CLI/API all change to match the real stargraph tool; prior fictional surfaces are removed. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01RtJymPXFQKDQuYAvgiAMtS --- plugins/stargraph/.claude-plugin/plugin.json | 6 +- plugins/stargraph/README.md | 99 +++--- .../stargraph/agents/dir-plugin-builder.md | 161 +++++---- plugins/stargraph/agents/graph-builder.md | 67 +++- plugins/stargraph/agents/hitl-driver.md | 52 ++- plugins/stargraph/agents/md-skill-builder.md | 125 ++++--- plugins/stargraph/agents/node-builder.md | 60 +++- plugins/stargraph/agents/pack-builder.md | 49 ++- plugins/stargraph/agents/runner.md | 49 ++- plugins/stargraph/agents/skill-builder.md | 66 +++- plugins/stargraph/commands/checkpoints.md | 23 +- plugins/stargraph/commands/counterfactual.md | 61 ++-- plugins/stargraph/commands/inspect.md | 52 ++- plugins/stargraph/commands/new-dir-plugin.md | 89 +++-- plugins/stargraph/commands/new-graph.md | 54 ++- plugins/stargraph/commands/new-md-skill.md | 85 ++++- plugins/stargraph/commands/new-node.md | 43 ++- plugins/stargraph/commands/new-pack.md | 40 ++- plugins/stargraph/commands/new-skill.md | 61 +++- plugins/stargraph/commands/new-tool.md | 56 ++- plugins/stargraph/commands/new-trigger.md | 92 +++-- plugins/stargraph/commands/replay.md | 27 +- plugins/stargraph/commands/respond.md | 51 +-- plugins/stargraph/commands/run.md | 48 ++- plugins/stargraph/commands/serve.md | 35 +- plugins/stargraph/commands/simulate.md | 38 +- plugins/stargraph/commands/store.md | 45 ++- plugins/stargraph/commands/verify-graph.md | 37 +- plugins/stargraph/references/bosun-packs.md | 114 +++--- .../stargraph/references/graph-yaml-schema.md | 332 ++++++++++-------- plugins/stargraph/references/hitl-patterns.md | 164 ++++----- .../stargraph/references/provenance-facts.md | 128 +++---- .../references/stargraph-concepts.md | 12 +- plugins/stargraph/references/state-schema.md | 33 +- .../stargraph/references/store-protocols.md | 55 +-- plugins/stargraph/references/triggers.md | 178 ++++++---- .../stargraph/skills/smart-stargraph/SKILL.md | 160 +++++---- 37 files changed, 1790 insertions(+), 1057 deletions(-) diff --git a/plugins/stargraph/.claude-plugin/plugin.json b/plugins/stargraph/.claude-plugin/plugin.json index 0ec31c2..37634ca 100644 --- a/plugins/stargraph/.claude-plugin/plugin.json +++ b/plugins/stargraph/.claude-plugin/plugin.json @@ -1,9 +1,9 @@ { "name": "stargraph", - "version": "0.3.0", - "description": "Author + operate Stargraph — graphs, skills, nodes, tools, triggers, Bosun rule packs, markdown SKILL.md skills, directory-based plugins; run/replay/counterfactual against stargraph serve, drive HITL, inspect checkpoints.", + "version": "0.4.0", + "description": "Author + operate Stargraph — IRDocument graphs, nodes, @tool callables, Python Skill bundles, entry-point plugins, triggers, Bosun rule packs; run/inspect/replay/counterfactual against stargraph serve, drive HITL, verify audit logs.", "author": {"name": "kraken-networks"}, "repository": "https://github.com/KrakenNet/stargraph", "license": "Apache-2.0", - "keywords": ["stargraph","graphs","skills","tools","triggers","provenance","authoring","orchestration","hitl","counterfactual","bosun","skill-md","dir-plugin"] + "keywords": ["stargraph","graphs","skills","tools","triggers","provenance","authoring","orchestration","hitl","counterfactual","bosun","ir","plugins"] } diff --git a/plugins/stargraph/README.md b/plugins/stargraph/README.md index abe1b8e..39e1024 100644 --- a/plugins/stargraph/README.md +++ b/plugins/stargraph/README.md @@ -5,8 +5,8 @@ Slash commands, agents, and skills for authoring **and operating** framework with deterministic governance via Fathom. Covers the full Stargraph CLI surface (`run`, `serve`, `inspect`, `replay`, -`counterfactual`, `respond`, `simulate`) plus authoring (graphs, nodes, tools, -skills, triggers, Bosun rule packs) and store wiring. +`counterfactual`, `respond`, `simulate`, `verify-audit`) plus authoring (graphs, +nodes, tools, skills, triggers, Bosun rule packs) and store wiring. ## Commands @@ -14,47 +14,47 @@ skills, triggers, Bosun rule packs) and store wiring. | Command | Purpose | |---|---| -| `/stargraph:new-graph ` | Scaffold a graph (state.py, nodes/, rules/, stargraph.yaml, tests). | -| `/stargraph:new-node ` | Add a typed node (DSPy / ML / tool / retrieval / sub-graph). | -| `/stargraph:new-tool ` | Tool definition with JSON Schema + namespace + permissions + side-effects. | -| `/stargraph:new-skill ` | Skill bundle — tools, optional sub-graph, prompt fragment (pip-package format). | -| `/stargraph:new-md-skill ` | Lightweight `SKILL.md` (YAML + markdown) compiled by `stargraph-md-skills` to a typed `Skill`. | -| `/stargraph:new-dir-plugin ` | Drop-in directory plugin under `~/.stargraph/plugins/` discovered by `stargraph-dir-plugins`. | -| `/stargraph:new-pack [--flavor routing\|governance]` | New Bosun-compatible rule pack. | -| `/stargraph:new-trigger --type manual\|cron\|webhook` | Wire a trigger and verify scheduler pickup. | -| `/stargraph:store add ` | Wire a Store provider (vector/graph/doc/memory/fact). | -| `/stargraph:verify-graph ` | Validate hash + schema + referenced rule packs + store providers. | +| `/stargraph:new-graph ` | Scaffold an `IRDocument` graph YAML (`ir_version`, `id`, nodes, rules, governance, state). | +| `/stargraph:new-node ` | Add a `NodeSpec` (builtin `kind` — dspy/ml/interrupt/retrieval/subgraph/… — or `module:Class`). | +| `/stargraph:new-tool ` | `@tool`-decorated callable + `ToolSpec` (namespace, side-effects, replay policy, capabilities). | +| `/stargraph:new-skill ` | Python `stargraph.skills.Skill` plugin — tools, optional subgraph, prompt, `state_schema`. | +| `/stargraph:new-md-skill ` | Skill **bundle** dir (`manifest.yaml` + `stargraph.yaml` + `state.py` + `nodes/`), Shipwright-style. | +| `/stargraph:new-dir-plugin ` | Entry-point plugin — pip package + `pyproject` entry points + `stargraph_plugin` manifest factory. | +| `/stargraph:new-pack [--flavor routing\|governance]` | Bosun rule pack (group `stargraph.packs`, Ed25519/JWS signed). | +| `/stargraph:new-trigger --type manual\|cron\|webhook` | Add a trigger to `triggers.yaml`; verify `stargraph serve` pickup. | +| `/stargraph:store add ` | Wire a Store (vector/graph/doc/memory/fact) as an IR `StoreRef`. | +| `/stargraph:verify-graph ` | Validate the IR via `run --inspect` / `simulate` + `stargraph.ir.validate`. | ### Operations | Command | Purpose | |---|---| -| `/stargraph:serve [--profile dev\|prod\|cleared]` | Start `stargraph serve` (FastAPI HTTP+WS). | -| `/stargraph:run [--input-file ]` | Execute a graph; stream events; return run_id. | -| `/stargraph:simulate [--seed ]` | Dry-run with deterministic stubs for side-effecting tools. | -| `/stargraph:inspect [--events] [--facts]` | Run header, checkpoints, events, facts, provenance breakdown. | -| `/stargraph:checkpoints ` | List checkpoints with state-diff summaries. | -| `/stargraph:replay [--from ]` | Deterministic replay from a checkpoint. | -| `/stargraph:counterfactual --from --mutate ` | Fork a run with mutated facts/state; diff vs original. | -| `/stargraph:respond --decision approve\|deny\|input` | Resume a paused (HITL) run. | +| `/stargraph:serve [--profile oss-default\|cleared]` | Start `stargraph serve` (FastAPI HTTP+WS on :8000). | +| `/stargraph:run [-i K=V] [--inspect]` | Execute a graph (or print its rule trace); seed state; return run_id. | +| `/stargraph:simulate --fixtures ` | Offline rule-firing trace against synthetic node outputs (no tools/LLM/checkpoint). | +| `/stargraph:inspect --db [--step N] [--diff N M]` | Timeline / state-at-step / CLIPS fact-delta + provenance. | +| `/stargraph:checkpoints ` | Checkpoint/state/fact views over the SQLite checkpointer (via `inspect`). | +| `/stargraph:replay --db [--mutation ] [--from-step N]` | Fork a counterfactual run from a checkpoint; diff vs parent. | +| `/stargraph:counterfactual --step N --mutate ` | Compute the cf-derived `graph_hash` for a mutation (no fork). | +| `/stargraph:respond --response --actor ` | Resume an `awaiting-input` (HITL) run. | ## Agents | Agent | Purpose | |---|---| -| `graph-builder` | Scaffolds + verifies graphs (state.py, nodes/, rules/, stargraph.yaml, tests). | -| `node-builder` | Adds typed nodes; manages state-fact boundary; updates graph hash. | -| `pack-builder` | Scaffolds Bosun rule packs (routing or governance flavor); validates as Fathom-compatible. | -| `skill-builder` | Builds skill bundles with tools, sub-graphs, prompt fragments (pip-package format). | -| `md-skill-builder` | Authors a `SKILL.md` and validates via `stargraph skills compile`. | -| `dir-plugin-builder` | Scaffolds a `~/.stargraph/plugins//` dir-plugin; verifies via `stargraph plugins verify` + live reload. | -| `runner` | Drives run / replay / checkpoints; polls run state and parses events. | -| `hitl-driver` | Lists paused runs, validates payloads against schema, drives `/respond`. | +| `graph-builder` | Builds + validates `IRDocument` graphs; verifies via `simulate` / `run --inspect`. | +| `node-builder` | Adds typed nodes; manages the state-fact boundary; tracks `graph_hash` + `migrate:`. | +| `pack-builder` | Scaffolds Bosun rule packs (routing or governance flavor); signs + mounts via `governance:`. | +| `skill-builder` | Builds Python `Skill` plugins (tools, subgraph, prompt) registered via `register_skills`. | +| `md-skill-builder` | Authors a skill **bundle** dir (Shipwright layout); verifies via `simulate` / `run --inspect`. | +| `dir-plugin-builder` | Scaffolds an entry-point plugin (pii_guard archetype); verifies discovery via `STARGRAPH_TRACE_PLUGINS=1`. | +| `runner` | Drives run / inspect / replay / respond; polls run state and parses events. | +| `hitl-driver` | Finds `awaiting-input` runs, builds the response payload, drives `/respond`. | ## Skills - `smart-stargraph` — vocabulary, state-fact boundary, provenance, Stores, Bosun - mounting, project detection. Loaded by every command. + mounting, CLI/API surface, project detection. Loaded by every command. - `smart-kraken` — shared cross-project conventions for the Kraken stack (monorepo detection, verify-before-call patterns). @@ -63,13 +63,13 @@ skills, triggers, Bosun rule packs) and store wiring. Loaded on demand by the commands and agents above: - `stargraph-concepts.md` — full glossary + disambiguations. -- `graph-yaml-schema.md` — `stargraph.yaml` shape. -- `state-schema.md` — Pydantic + annotated-state mirroring rules. -- `provenance-facts.md` — `(origin, source, run_id, step, confidence, ts)`. -- `store-protocols.md` — Vector / Graph / Doc / Memory / Fact contracts. -- `triggers.md` — manual / cron / webhook config patterns + scheduler semantics. -- `bosun-packs.md` — routing vs governance flavor; in-tree pack catalog. -- `hitl-patterns.md` — pause shapes, schema validation, audit facts, cleared-profile rules. +- `graph-yaml-schema.md` — the `IRDocument` graph shape. +- `state-schema.md` — Pydantic state + annotated-state mirroring rules. +- `provenance-facts.md` — the `__stargraph_provenance__` envelope `(origin, source, external_id)`. +- `store-protocols.md` — vector / graph / doc / memory / fact contracts + providers. +- `triggers.md` — `triggers.yaml` schema (manual / cron / webhook) + scheduler semantics. +- `bosun-packs.md` — routing vs governance flavor; PackMount + signing. +- `hitl-patterns.md` — interrupt node/action, `/respond`, capability gate, timeout/on_timeout. ## Install @@ -89,22 +89,27 @@ claude plugins install stargraph | Setting | Default | Override | |---|---|---| -| Serve URL | `http://localhost:9000` | `STARGRAPH_URL` | -| Auth token | none | `STARGRAPH_TOKEN` | -| Graphs dir | `./graphs` | `STARGRAPH_GRAPHS_DIR` | -| Profile | `dev` | `STARGRAPH_PROFILE` (`dev`/`prod`/`cleared`) | +| Local install | `uv add stargraph` | — | +| Serve URL | `http://localhost:8000` | `stargraph serve --host/--port`; `stargraph respond --server` | +| Profile | `oss-default` | `--profile` / `STARGRAPH_PROFILE` (`oss-default`/`cleared`) | +| Project config | `stargraph.toml` (CWD) | `STARGRAPH_TOML_FILENAME` | +| Runtime config dir | `~/.config/stargraph/` | `STARGRAPH_CONFIG_DIR` | ## Version -0.3.0 — adds lightweight authoring formats: `SKILL.md` (compiled by -`stargraph-md-skills` to a typed `Skill`) and directory-based plugins under -`~/.stargraph/plugins/` (discovered by `stargraph-dir-plugins`). New commands -(`/stargraph:new-md-skill`, `/stargraph:new-dir-plugin`) and agents -(`md-skill-builder`, `dir-plugin-builder`). +0.4.0 — reconciled the whole plugin against the real Stargraph CLI/HTTP/plugin +surface: the 8-subcommand CLI (`run`/`serve`/`inspect`/`replay`/`respond`/ +`simulate`/`counterfactual`/`verify-audit`), the `IRDocument` graph schema, the +entry-point plugin model (no directory plugins), Python `Skill` bundles (no +markdown SKILL.md compiler), `stargraph.toml` project config, and the real +`/v1/*` API. Replaced the prior fictional `graph verify` / `plugins *` / +`skills compile` / `facts` surfaces with their real equivalents. + +0.3.0 — earlier authoring-format work (superseded by 0.4.0). 0.2.0 — full ops surface (`serve`, `inspect`, `simulate`, `counterfactual`, -`respond`), trigger authoring (`new-trigger`), HITL driver agent, and three -references (`triggers.md`, `bosun-packs.md`, `hitl-patterns.md`). +`respond`), trigger authoring, HITL driver agent, and three references +(`triggers.md`, `bosun-packs.md`, `hitl-patterns.md`). See the project [CHANGELOG](https://github.com/KrakenNet/stargraph/blob/main/CHANGELOG.md) for engine-side changes. diff --git a/plugins/stargraph/agents/dir-plugin-builder.md b/plugins/stargraph/agents/dir-plugin-builder.md index c245c8d..f65d7be 100644 --- a/plugins/stargraph/agents/dir-plugin-builder.md +++ b/plugins/stargraph/agents/dir-plugin-builder.md @@ -1,112 +1,109 @@ --- -description: Scaffold a directory-based Stargraph plugin under ~/.stargraph/plugins/ or $STARGRAPH_PLUGINS_DIR; verify via `stargraph plugins verify` + live reload. +description: Scaffold an entry-point Stargraph plugin — a pip package whose pyproject declares a `stargraph_plugin` manifest factory plus capability groups (tools/skills/stores/packs); verify discovery via STARGRAPH_TRACE_PLUGINS=1. tools: [Bash, Read, Write, Edit] --- -# Dir-Plugin Builder +# Plugin Builder -Builds a drop-in Stargraph plugin discovered by `stargraph-dir-plugins`. No pip -install; same typed contract, namespace-conflict detection, signing, and -audit chain as a pip-installed plugin once registered. +Stargraph plugins are **entry-point** plugins: a normal pip package discovered +via `importlib.metadata` entry points + `pluggy` hooks (two-stage loader). There +are no directory/drop-in plugins. The `pii_guard` plugin +(`stargraph.plugins.pii_guard`) is the reference archetype — a `@tool`-decorated +coroutine plus governance hooks. ## Inputs -- `plugin_name` — directory name and `plugin.toml` `name` field. -- `namespaces` — list of dotted namespace prefixes claimed by the plugin. -- `ships` — combination of `{md_skills, tools, packs, stores}`. -- `trust_keys` — Ed25519 pubkeys for pack signing; if empty and `ships` - includes packs, generate a dev keypair via `fathom keygen` and persist - the public half in `plugin.toml`. -- `capabilities` — list of new capability strings to declare (with - description + sensitivity). -- `target_dir` — default `~/.stargraph/plugins/`; honor `$STARGRAPH_PLUGINS_DIR` - if set. +- `plugin_name` — distribution + package name. +- `namespaces` — dotted namespace prefixes the plugin claims (conflicts abort load). +- `provides` — subset of `{tool, skill, store, pack}`. +- `order` — load priority `0..10000` (default `5000`; collisions raise `PluginLoadError`). ## Steps -1. **Scaffold layout:** +1. **Scaffold a pip package** (pii_guard layout): ``` - // - plugin.toml - skills/ (if ships includes md_skills) - tools/ (if ships includes tools) - __init__.py - packs/ (if ships includes packs) - stores/ (if ships includes stores) - capabilities.toml (if capabilities declared) - README.md + / + pyproject.toml + src// + __init__.py (docstring) + _plugin.py (manifest factory + register_* hookimpl) + hooks.py (authorize_action / before|after_tool_call — optional) + redact.py (the @tool-decorated coroutine — example tool) ``` -2. **Write `plugin.toml`:** +2. **Declare entry points in `pyproject.toml`** — the manifest factory under + group `stargraph`, plus one or more capability groups: ```toml - name = "" - version = "0.1.0" - api_version = "1.x" - order = 100 - namespaces = [] + [project.entry-points."stargraph"] + stargraph_plugin = "._plugin:manifest" # returns a PluginManifest - [author] - name = "" - email = "" - - [trust] - keys = [] - - [runtime] - python_path = ["tools"] + [project.entry-points."stargraph.tools"] + = "._plugin:register_tools" + # also available: stargraph.skills, stargraph.stores, stargraph.packs, + # stargraph.triggers, stargraph.mcp_adapters ``` -3. **Per artifact, delegate:** - - For each requested md-skill: invoke `md-skill-builder` with - `host_path=//skills//`. - - For each tool: invoke the tool-builder agent against - `tools/.py`. - - For each pack: invoke `pack-builder`; sign with the dev keypair if - no trust key was supplied. +3. **Write `_plugin.py`** — the manifest factory + a `register_*` collect-all + hookimpl per capability: + ```python + from stargraph.plugin import hookimpl + from stargraph.ir import PluginManifest, ToolSpec + from .redact import redact_pii + + def manifest() -> PluginManifest: + return PluginManifest( + name="", + version="0.1.0", + api_version="1", + namespaces=[], + provides=["tool"], + order=5000, + ) + + @hookimpl + def register_tools() -> list[ToolSpec]: + return [redact_pii.spec] # .spec off the @tool wrapper + ``` -4. **Capabilities:** - If `capabilities.toml` was requested, emit one entry per declared cap: - ```toml - [capabilities.""] - description = "" - sensitivity = "low" | "medium" | "high" +4. **Per artifact, delegate:** + - tools → the tool-builder (`@tool`, `ToolSpec`, registry key `ns.name@ver`). + - skills → `skill-builder` (Python `Skill` + `register_skills`). + - packs → `pack-builder` (group `stargraph.packs`, `register_packs`, signing). + - stores → wire a `StoreSpec` + `register_stores`. + +5. **Optional governance hooks** (`hooks.py`, pii_guard pattern): + `authorize_action(action) -> bool|None` (first-deny; first non-`None` wins), + `before_tool_call(call)` / `after_tool_call(call, result)` for audit. Register + these on the same plugin object. + +6. **Install + verify discovery** (there are no plugin verify/inspect/reload CLI subcommands): + ```bash + uv pip install -e . + STARGRAPH_TRACE_PLUGINS=1 stargraph run --inspect ``` + The trace logs every discovery, manifest validation, and registration step + (with the `order` it registered at). Failure cases: + - `api_version` not `"1"` → manifest validation fails; pin it. + - Namespace conflict with another plugin → load aborts; rename the namespace. + - `order` collision → `PluginLoadError`; pick a distinct order. + - Import failure in a capability module → trace shows the discovery/registration gap. -5. **Offline validation:** run - `stargraph plugins verify /`. Failure cases to - diagnose: - - `api_version` mismatch → bump or pin. - - Namespace conflict with existing plugin → rename namespace. - - Unsigned pack with `allow_unsigned=false` → sign or move to - dev-only path. - - Tool import failure → check `runtime.python_path`. - -6. **Live reload (optional, if `stargraph serve` is running):** - `stargraph plugins reload && stargraph plugins inspect `. - Confirm: tool count, skill count, pack count, signing status, and that - the audit chain emitted a registration record with the plugin's - manifest hash. + At runtime with `stargraph serve` up, confirm registered kinds via + `GET /v1/registry/{kind}` (`kind` ∈ `tools`, `skills`, `stores`). ## Build-Test-Fix -5 iterations across verify + inspect. On signing failures, regenerate -keypair only with explicit user confirmation — never silently rotate keys. +5 iterations across the trace + `GET /v1/registry/{kind}`. On signing failures +for packs, regenerate keys only with explicit user confirmation. ## Output -- Tree of the created dir-plugin. -- `stargraph plugins verify` output. -- `stargraph plugins inspect ` output (or "skipped — serve not - running" if applicable). -- Any unsigned-pack warnings + remediation steps. +- Tree of the created package. +- `STARGRAPH_TRACE_PLUGINS=1` discovery/validation/registration trace. +- `GET /v1/registry/{kind}` result (or "skipped — serve not running"). ## Constraints -- The plugin **must** declare at least one namespace; bare-name plugins - are rejected by the loader. -- If shipping unsigned packs in a path that production policy treats as - untrusted, surface the warning prominently; do NOT auto-sign with a - user-pubkey-claiming key. -- Stage-1 manifest validation is import-cold (NFR-7) — do not import - `tools/` modules during scaffolding; rely on `stargraph plugins verify` - to exercise stage-2 import safely. +- The manifest **must** declare at least one namespace; conflicts abort load. +- Stage-1 manifest validation is import-cold — do not import capability modules + during scaffolding; rely on the stage-2 trace to exercise imports. diff --git a/plugins/stargraph/agents/graph-builder.md b/plugins/stargraph/agents/graph-builder.md index 460b14d..e6360e3 100644 --- a/plugins/stargraph/agents/graph-builder.md +++ b/plugins/stargraph/agents/graph-builder.md @@ -1,36 +1,65 @@ --- -description: Scaffold + verify Stargraph graphs. Writes state.py, nodes/, rules/, stargraph.yaml, tests; runs stargraph graph verify; iterates. +description: Scaffold + verify Stargraph IRDocument graphs. Writes state, nodes, rules, stargraph.yaml, fixtures; validates via `stargraph simulate` / `run --inspect` and `stargraph.ir.validate`; iterates. tools: [Bash, Read, Write, Edit] --- # Graph Builder +Builds a validated **IRDocument** graph (the runnable shape: `ir_version`, +`id`, `nodes`, `rules`, plus optional `tools`/`skills`/`stores`/`governance`). +All IR models pin `extra='forbid'`, so unknown keys are rejected at load. + ## Inputs -- `graph_name`, `purpose`, `nodes` (list), `state_fields` (list of (name, type, annotated)), `rule_packs`, `stores`. +- `graph_name`, `purpose`, `nodes` (list of `(id, kind)`), `state_fields` + (name → type), `rule_packs` (governance PackMounts), `stores`. ## Steps 1. Create dir `graphs//`. -2. Write `state.py`: - -```python -from pydantic import BaseModel, Field -from typing import Annotated -from stargraph.annotations import Mirror +2. Define state — pick ONE: + - Flat `state_schema: {field: "type", ...}` inline in the IR, OR + - `state_class: "module.path:ClassName"` pointing at a Pydantic model in + `state.py`. The two are mutually exclusive. Mirrored fields (projected to + CLIPS at node boundaries) use `Annotated[T, Mirror()]`; replay-safe state + forbids `set` — use `frozenset`. +3. Write `nodes/__init__.py` for any custom node classes (referenced as + `module.path:ClassName`); builtin kinds (`echo`, `halt`, `dspy`, `ml`, + `interrupt`, `passthrough`, `write_artifact`, `retrieval`, `subgraph`, + `human_input`) need no code. +4. Write `stargraph.yaml` per `references/graph-yaml-schema.md` (IRDocument + shape; node/rule/pack ids must match `^[a-z0-9][a-z0-9_\-.]{0,127}$`): -class State(BaseModel): - # for each field: if annotated, wrap with `Annotated[, Mirror()]` - pass -``` + ```yaml + ir_version: "1.0.0" + id: "graph:" + state_schema: { message: "str", severity: "int" } + nodes: + - { id: "classify", kind: "dspy" } + - { id: "halt", kind: "halt" } + rules: + - id: "r-escalate" + when: "(severity ?s&:(>= ?s 4))" + then: [{ kind: "goto", target: "classify" }] + governance: + - id: "stargraph.bosun.routing" + version: "1.0.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } + ``` -3. Write `nodes/__init__.py` exporting node functions/classes. -4. Write `rules/_starter.yaml` (one default routing rule). -5. Write `stargraph.yaml` per the schema in `references/graph-yaml-schema.md`. -6. Write `tests/test_graph.py` — runs the graph against fixture state, asserts at least one transition. -7. Run `uv run stargraph graph verify graphs//stargraph.yaml`. -8. On error, parse + fix, retry. + Routing has no explicit edges: static fall-through (declaration order) plus + rule `goto`/`halt`/`parallel`/`interrupt` actions. +5. Write a fixtures file (`fixtures/.yaml`): a `node_id → synthetic + output` map for offline rule-trace verification. +6. **Validate** (there is no graph-verify CLI subcommand): + - Offline rule trace: `uv run stargraph simulate graphs//stargraph.yaml --fixtures fixtures/.yaml` (no tools/LLM/checkpoint). + - Rule-firing trace without node execution: `uv run stargraph run graphs//stargraph.yaml --inspect`. + - Structural validation in Python: `stargraph.ir.validate(ir)` returns + `list[ValidationError]` (empty = valid; never raises). IRDocument + validation is also automatic on load via `extra='forbid'`. +7. On error, parse the validation/simulate output (JSON-pointer `path` + `hint`), + fix the offending field, retry. ## Output -Tree, verify status. +Tree, `simulate` / `run --inspect` output, any `stargraph.ir.validate` errors. diff --git a/plugins/stargraph/agents/hitl-driver.md b/plugins/stargraph/agents/hitl-driver.md index 7ed6c24..cb2a51f 100644 --- a/plugins/stargraph/agents/hitl-driver.md +++ b/plugins/stargraph/agents/hitl-driver.md @@ -1,36 +1,52 @@ --- -description: Driver for human-in-the-loop pauses — fetches paused runs, presents context, validates payload against schema, calls /respond. +description: Driver for human-in-the-loop pauses — finds awaiting-input runs, presents the interrupt prompt + requested capability, gathers a response, and resumes via /respond. tools: [Bash, Read, AskUserQuestion] --- # HITL Driver +A graph pauses when an `interrupt` node fires (or a rule emits an `interrupt` +action). The pause emits a `WaitingForInputEvent` carrying the operator +`prompt`, the free-form `interrupt_payload`, and the `requested_capability` +the responder must hold. The run sits at status `awaiting-input` until a human +POSTs to `/v1/runs/{run_id}/respond`; the response is injected as a `respond` +fact (`origin="user", source=`). + ## Inputs -- `run_id` (optional). If omitted, list all `PAUSED_HITL` runs and pick interactively. +- `run_id` (optional). If omitted, list awaiting-input runs and pick interactively. ## Steps -1. `GET /v1/runs?status=PAUSED_HITL` — list candidates with `paused_at`, - `pause_reason`, and the prompting node. +1. List candidates: `GET /v1/runs?status=awaiting-input` (filterable list; + default base URL `http://localhost:8000`). Note `run_id` + prompting node. 2. For the chosen `run_id`: - - `GET /v1/runs/${run_id}` — pull `prompt`, `policy_reason`, - `expected_input_schema`, plus the last 3 facts emitted before the pause. - - Render a compact summary (no more than 30 lines) so the user can decide. -3. Ask the user: - - decision (`approve` / `deny` / `input`) - - payload (only for `input` — must match `expected_input_schema`) - - reason (required for `deny`) -4. Validate payload locally with `jsonschema` before submitting. -5. `POST /v1/runs/${run_id}/respond` with the response body. -6. Stream `/v1/runs/${run_id}/stream` until status leaves `RUNNING` or 30s elapse. + - `GET /v1/runs/${run_id}` — pull the `RunSummary` and the pending + `WaitingForInputEvent` (`prompt`, `interrupt_payload`, + `requested_capability`, optional `timeout` / `on_timeout`). + - Inspect the facts that led to the pause if a checkpoint DB is available: + `stargraph inspect ${run_id} --db ./.stargraph/run.sqlite --diff N M`. + - Render a compact summary (≤30 lines) so the user can decide. +3. Ask the user (AskUserQuestion) for the response payload. Confirm the actor + id and that the actor holds `requested_capability` (the `/respond` route is + gated on `runs:respond`; the gate denies with a `capability_denied` audit + event + 403 if the actor lacks the capability). +4. Submit the response. Either: + - CLI: `stargraph respond ${run_id} --response decision.json --actor alice + --server http://localhost:8000` (writes `Authorization: Bypass `). + - Direct: `POST /v1/runs/${run_id}/respond` with body `{response, actor}`. +5. Confirm resume: stream `WS /v1/runs/${run_id}/stream`, or poll + `GET /v1/runs/${run_id}` until status leaves `awaiting-input`. ## Build-Test-Fix -3 iters. On schema validation failure, surface the offending JSON-pointer path -to the user and re-prompt — do not retry blindly. +3 iters. HTTP errors map to operator-friendly cases: `401` auth failed for +actor; `404` run not found or not awaiting input; `409` run not awaiting input +(already responded / conflicting state). On `409`/`404`, re-list awaiting-input +runs rather than retrying blindly. If `on_timeout` already fired (`halt` or +`goto:`), report that the gate is no longer answerable. ## Report -- run_id, decision, audit fact -- New status; node about to run; or terminal `result` if the run completed +- run_id, actor, response payload. +- New status; node about to run; or terminal `result` if the run completed. diff --git a/plugins/stargraph/agents/md-skill-builder.md b/plugins/stargraph/agents/md-skill-builder.md index c21e178..f7ed728 100644 --- a/plugins/stargraph/agents/md-skill-builder.md +++ b/plugins/stargraph/agents/md-skill-builder.md @@ -1,83 +1,92 @@ --- -description: Author a SKILL.md (YAML frontmatter + markdown body) for the stargraph-md-skills compiler; validate via `stargraph skills compile`. +description: Author a Stargraph skill bundle — the Shipwright multi-file layout (manifest.yaml + stargraph.yaml + state.py + nodes/) that packages a Python `stargraph.skills.Skill`; verify via simulate / run --inspect. tools: [Bash, Read, Write, Edit] --- -# MD Skill Builder +# Skill Bundle Builder -Builds a single `SKILL.md` that compiles into a typed `stargraph.skills.Skill`. -No Python package; the file lives inside a directory plugin -(`~/.stargraph/plugins//skills//SKILL.md`) or a project's -local `skills/` tree. +Builds a **skill bundle**: the canonical multi-file shape for a Python +`stargraph.skills.Skill`, modeled on the in-tree Shipwright bundle +(`src/stargraph/skills/shipwright/`). There is no markdown `SKILL.md` format — +a skill is always a Python `Skill` (single file or this bundle layout). ## Inputs -- `skill_name` (slug) — directory and frontmatter `name`. -- `purpose` — one-sentence description. -- `kind` — `agent` | `workflow` | `utility`. -- `tools` — list of `.@` ids (must already - be registered, or contributed by tools elsewhere in the same dir-plugin). -- `state_schema_fields` — `[(field_name, json_schema_type)]`; becomes the - declared output write-whitelist enforced by `SubGraphNode`. -- `requires` — list of capability strings. -- `subgraph_path` (optional) — if provided, embedded as `subgraph:` in - frontmatter; absent = single-step React subgraph default. -- `examples` (optional) — list of `(inputs, expected_output)` pairs. -- `host_path` — directory to create the skill under. Default - `~/.stargraph/plugins//skills//` if invoked from a - dir-plugin context, else `./skills//`. +- `skill_name` (slug), `purpose`, `kind` (`agent` | `workflow` | `utility`). +- `state_fields` — `(name, type, mirrored?)`; the state model's FIELD NAMES are + the declared output channels / write whitelist. `set`/`set[X]` is rejected — + use `frozenset`. +- `nodes` — the subgraph nodes (builtin kind or `module.path:ClassName`). +- `rule_packs` / `governance` — Bosun packs the subgraph mounts. +- `host_path` — directory to create the bundle under. ## Steps -1. Create `/SKILL.md`. -2. Emit frontmatter: +1. Create the bundle at `//` (Shipwright layout): + ``` + / + manifest.yaml skill identity + state_schema reference + stargraph.yaml graph: state ref, nodes, rules, stores, checkpoints + state.py the State Pydantic model + nodes/ per-node modules + templates/ prompt fragments (optional) + _pack.py Bosun sub-pack loader (optional) + ``` + +2. `manifest.yaml`: ```yaml - --- - name: - version: 0.1.0 + id: + version: "0.1.0" kind: description: | - requires: [] - tools: [] - state_schema: - : { type: , ... } - examples: - - inputs: { ... } - expected_output: { ... } - --- + state_schema: .state:State ``` -3. Emit body skeleton with sections: - - `# ` — derived from `purpose`. - - `## When to <verb>` — preconditions for activation. - - `## Procedure` — numbered steps using `{{ tool_descriptions }}` template - references. - - `## Failure modes` — when to refuse / escalate. -4. If `subgraph_path` provided, write a stub IR sub-graph at that path via - `/stargraph:new-graph` (or scaffold inline JSON for trivial cases). -5. If `examples` provided, drop one fixture per example into - `<host_path>/examples/NN_<slug>.json`. -6. Validate: `stargraph skills compile <host_path>/SKILL.md`. On error, parse - the JSON envelope and surface the failure section to fix. + +3. `stargraph.yaml` — the bundle graph (authoring shape, richer than the + validated IRDocument): + ```yaml + name: <skill_name> + state: ./state.py:State + nodes: + - name: <node> + type: <module.path:ClassName | stargraph.nodes.human_input> + rules: + - pack: <bosun.pack.id> + stores: + doc: sqlite:./.<skill_name>/docs.db + fact: sqlite:./.<skill_name>/facts.db + checkpoints: + every: node-exit + store: sqlite:./.<skill_name>/checkpoints.db + ``` + +4. `state.py` — the `State` model referenced from `manifest.yaml#state_schema` + and `stargraph.yaml#state`. Mirror fields the rule packs route on with + `Annotated[T, Mirror()]`; no `set` (use `frozenset`). +5. Write `nodes/` modules for each custom node `type`. +6. Register the bundle as a Skill plugin so the loader picks it up: a + `register_skills()` hookimpl under entry-point group `stargraph.skills` + returning the `Skill` whose `subgraph` points at the bundle graph (delegate + to `skill-builder` for the pyproject + `_plugin.py` wiring). +7. **Verify** (there is no skill compiler or skill-compile CLI subcommand). Validate the + subgraph's IRDocument form and discovery: + - `uv run stargraph simulate <graph.yaml> --fixtures <fixtures.yaml>` or `run --inspect`. + - `STARGRAPH_TRACE_PLUGINS=1 stargraph run <any-graph.yaml> --inspect` to + confirm discovery + registration; or `GET /v1/registry/skills` with serve up. ## Build-Test-Fix -5 iterations. On each failure, edit only the offending field; never -overwrite the whole file blindly. +5 iterations. On each failure edit only the offending field/file; surface the +validation `path`/`hint` rather than overwriting blindly. ## Output -- Path to `SKILL.md`. -- Validation result (success or specific compile error). -- If hosted inside a dir-plugin, the plugin's `plugin.toml` did NOT need to - change — confirm by re-running `stargraph plugins inspect <plugin>` and - checking the skill appears in the registered set. +- Tree of the created bundle. +- simulate / run --inspect output and the discovery trace / `GET /v1/registry/skills`. ## Constraints -- Markdown body MUST NOT contain `<script>` tags or raw HTML — the prompt is - data, not code (`stargraph-md-skills` rejects these). -- Tool refs MUST be `<namespace>.<name>@<semver-range>` format; bare names - fail compile with an unambiguous error. -- `state_schema` fields are the **only** allowed boundary writes; document - which sub-graph step writes which field. +- The state model's field names are the **only** allowed boundary writes; + document which subgraph node writes which field. +- Tool refs are registry-key ids `<namespace>.<name>@<version>` — never bare names. diff --git a/plugins/stargraph/agents/node-builder.md b/plugins/stargraph/agents/node-builder.md index 45d7446..f4116e3 100644 --- a/plugins/stargraph/agents/node-builder.md +++ b/plugins/stargraph/agents/node-builder.md @@ -1,31 +1,57 @@ --- -description: Add a typed node to a Stargraph graph; manage state-sync boundary; update graph hash. +description: Add a NodeSpec to a Stargraph IRDocument graph (builtin kind or module:Class); update state at the node boundary; verify via simulate / run --inspect. tools: [Bash, Read, Write, Edit] --- # Node Builder +Adds a **NodeSpec** (`{id, kind}`, plus an optional `config:` block for +builtins) to a graph's `stargraph.yaml` and wires the state it reads/writes. + ## Inputs -- `graph_name`, `node_name`, `node_type` (dspy:* | model:* | tool:* | retrieval:* | subgraph:*). -- `inputs` (list of State fields), `outputs` (list with annotated flag). +- `graph_name`, `node_id` (slug `^[a-z0-9][a-z0-9_\-.]{0,127}$`). +- `kind` — either a builtin factory key (`echo`, `halt`, `dspy`, `ml`, + `interrupt`, `passthrough`, `write_artifact`, `retrieval`, `subgraph`, + `human_input`) or `module.path:ClassName` for a custom node. +- `reads` / `writes` — State fields the node consumes and produces. ## Steps -1. Add a node spec to `graphs/<graph>/stargraph.yaml`. -2. Write/extend `nodes/__init__.py` with a function: - -```python -def <node_name>(state: State) -> State: - # read inputs, call underlying ... - return state.copy(update={...}) -``` - -3. If outputs are annotated, update `state.py` to wrap them in `Annotated[<type>, Mirror()]`. -4. Recompute graph hash; warn on hash change. -5. `uv run stargraph graph verify graphs/<graph>/stargraph.yaml`. -6. Add a fixture-based test in `tests/test_<node_name>.py`. +1. Add the node to `graphs/<graph>/stargraph.yaml` under `nodes:`: + + ```yaml + nodes: + - id: <node_id> + kind: <builtin-kind | module.path:ClassName> + # builtins accept a config: block, e.g. ml/interrupt: + config: + input_field: features + output_field: risk + ``` + +2. For a custom node, write/extend the class referenced by `module.path:ClassName` + in `nodes/__init__.py` (or the module the `kind` names). +3. Wire routing: nodes have no explicit edges. Add a `goto` rule (or rely on + static fall-through in declaration order) so the new node is reachable, e.g. + + ```yaml + rules: + - id: r-<prev>-to-<node_id> + when: "?n <- (node-id (id <prev>))" + then: [{ kind: goto, target: <node_id> }] + ``` + +4. Manage the state boundary: add any new `writes` fields to the graph's + `state_schema` (flat map) or to the `state_class` Pydantic model. Fields the + downstream rules must route on are mirrored to CLIPS with + `Annotated[T, Mirror()]`. Replay-safe state forbids `set` — use `frozenset`. +5. Verify (no graph-hash CLI; the canonical `graph_hash` is recomputed + automatically from the IR on load): + - `uv run stargraph run graphs/<graph>/stargraph.yaml --inspect` (rule-firing trace, no execution). + - `uv run stargraph simulate graphs/<graph>/stargraph.yaml --fixtures fixtures/<graph>.yaml` (add a synthetic output for the new node id). +6. Add a fixture entry for the new node and confirm the rule trace reaches it. ## Output -YAML diff, state.py diff, hash before/after, test status. +YAML diff, state diff, simulate / run --inspect trace showing the node reached. diff --git a/plugins/stargraph/agents/pack-builder.md b/plugins/stargraph/agents/pack-builder.md index b3b9a35..98638b8 100644 --- a/plugins/stargraph/agents/pack-builder.md +++ b/plugins/stargraph/agents/pack-builder.md @@ -1,24 +1,59 @@ --- -description: Scaffold a Bosun rule pack (routing or governance flavor); validate as a Fathom-compatible pack. +description: Scaffold a Bosun rule pack (routing or governance flavor) carrying Fathom/CLIPS rules; distribute it as a plugin under the stargraph.packs entry-point group and mount it via IR governance. tools: [Bash, Read, Write, Edit] --- # Pack Builder +Builds a **Bosun** governance pack (Fathom/CLIPS rules) that a graph mounts via +its IR `governance:` list. A pack is identified by a slug `id` plus a separate +`version` field (e.g. `stargraph.bosun.budgets` / `soc-policy`) — a slug id with +its own separate `version`, never a combined `name@version`-style string. + ## Inputs -- `pack_name`, `flavor` (routing | governance), `description`, `initial_rules`. +- `pack_id` (slug `^[a-z0-9][a-z0-9_\-.]{0,127}$`), `version`, + `flavor` (routing | governance), `description`, `initial_rules`. ## Steps -1. Create `bosun-packs/<pack_name>/` with the Fathom rule-pack layout (`pack.yaml`, `templates/`, `rules/`, `modules/`, `functions/`, `tests/`). -2. For governance flavor, include starter rules for budget tripping, audit emit, retry-with-backoff. -3. Validate via `fathom validate` and `pytest`. +1. Create `bosun-packs/<pack_id>/` with the pack layout: `rules.clp` (the CLIPS + rules), a `manifest_path`-referenced manifest, `templates/`, and `tests/`. +2. For the governance flavor, seed CLIPS rules for budget tripping, audit emit, + and retry-with-backoff; for routing, seed disposition/route rules. The + `_pack.py`-style loader splits `rules.clp` into top-level constructs and + feeds each to `fathom.Engine._env.build` for precise compile-error attribution. +3. Distribute the pack as a plugin: a pip package whose `pyproject.toml` + declares the manifest factory under group `stargraph` and the pack under + group `stargraph.packs`. The `register_packs()` hookimpl returns + `list[PackSpec]`, where `PackSpec = (id, version, manifest_path)`. +4. Mount it from a graph's IR `governance:` (a `PackMount`): + + ```yaml + governance: + - id: <pack_id> + version: "<version>" + requires: { stargraph_facts_version: "1.0", api_version: "1" } + ``` + + `check_pack_compat` enforces `requires` at pack-load time and raises + `PackCompatError` on mismatch (force-loud; comparison is pinned-string + equality in the POC). +5. **Signing:** production requires Ed25519/JWS-signed packs (releases are + signed with Ed25519, detached `.sig`). Generate/manage the signing key per + `references/bosun-packs.md`; never silently rotate a key. +6. Validate by mounting the pack on a graph and running the offline rule trace: + `uv run stargraph simulate <graph.yaml> --fixtures <fixtures.yaml>` (or + `run --inspect`). Confirm discovery/registration with + `STARGRAPH_TRACE_PLUGINS=1 stargraph run <graph.yaml> --inspect`, and at + runtime via `GET /v1/registry/{kind}` when `stargraph serve` is up. ## Build-Test-Fix -5 iters. +5 iters. On a `PackCompatError`, reconcile `requires` against the graph's +declared versions. On a CLIPS compile error, the per-construct loader names the +offending construct — fix only that. ## Output -Tree + validate + test status. +Tree, simulate/inspect output, pack registration status, signing status. diff --git a/plugins/stargraph/agents/runner.md b/plugins/stargraph/agents/runner.md index 0c4df8d..ad344cf 100644 --- a/plugins/stargraph/agents/runner.md +++ b/plugins/stargraph/agents/runner.md @@ -1,5 +1,5 @@ --- -description: Driver for Stargraph run/replay/checkpoints. Polls run state, parses events, summarizes. +description: Driver for Stargraph run/inspect/replay/respond. Polls run state, parses events, summarizes; reads fact deltas via `stargraph inspect --diff`. tools: [Bash, Read] --- @@ -7,15 +7,50 @@ tools: [Bash, Read] ## Inputs -- Action: run | replay | checkpoints -- run_id (for replay/checkpoints), graph + input (for run). +- Action: run | inspect | replay | respond +- `graph` + `--inputs` (for run); `run_id` + `--db` (for inspect/replay); + `run_id` + response file + actor (for respond). ## Steps -1. POST/GET appropriate endpoint. -2. Poll `/v1/runs/<id>` until status ∈ {COMPLETED, FAILED, HALTED} or timeout. -3. Tail last N events from `/v1/runs/<id>/events`. +1. **run** — execute a graph end-to-end against the checkpointer: + ```bash + stargraph run <graph.yaml> -i k=v [-i k2=v2] \ + --checkpoint ./.stargraph/run.sqlite --log-file run.jsonl + ``` + Each `-i K=V` key must match the IR `state_schema`. Use `--inspect` for a + rule-firing trace with no node execution. The run drives to terminal + `done` / `failed` (exit 0 on done, non-zero on failed). + +2. **inspect** — read-only views over the checkpointer DB: + - Timeline: `stargraph inspect <run_id> --db ./.stargraph/run.sqlite --log-file run.jsonl` + - State at step N: `stargraph inspect <run_id> --db ./.stargraph/run.sqlite --step N` + - CLIPS fact delta (facts asserted/retracted between steps): `stargraph inspect <run_id> --db ./.stargraph/run.sqlite --diff N M` + - Mode selector is `--diff` > `--step` > timeline; both `--diff`/`--step` require `--db`. + +3. **replay** — fork a counterfactual run from a checkpoint: + ```bash + stargraph replay <run_id> --db ./.stargraph/run.sqlite \ + [--mutation cf/override.json] [--from-step N] --diff + ``` + Mints a `cf-<uuid>` child; the parent's checkpoint rows stay byte-identical. + +4. **respond** — deliver HITL input to an `awaiting-input` run (thin wrapper + over `POST /v1/runs/{run_id}/respond`): + ```bash + stargraph respond <run_id> --response decision.json --actor alice \ + --server http://localhost:8000 + ``` + +### Driving runs via the serve API (when `stargraph serve` is up) + +Default base URL `http://localhost:8000`. Verify-before-call: +`GET /v1/graphs` (confirm the `graph_id` is registered) → `POST /v1/runs` +(`{graph_id, inputs?, trigger_source?}`, returns `202` `{run_id, status:"pending"}`) +→ poll `GET /v1/runs/<run_id>` until status leaves pending/running, or attach +`WS /v1/runs/<run_id>/stream` for the live event stream. ## Output -Status, last node, last decision, total duration, link to checkpoints. +Status, last node, last decision, total duration. For inspect, the timeline / +state-at-step / fact-diff view requested. diff --git a/plugins/stargraph/agents/skill-builder.md b/plugins/stargraph/agents/skill-builder.md index 7c9f152..66eb7d8 100644 --- a/plugins/stargraph/agents/skill-builder.md +++ b/plugins/stargraph/agents/skill-builder.md @@ -1,24 +1,72 @@ --- -description: Scaffold a pip-installable Stargraph skill package with entry-point registration. +description: Scaffold a pip-installable Stargraph Skill plugin — a Python `stargraph.skills.Skill` registered via `register_skills` under the stargraph.skills entry-point group; verify discovery and registration. tools: [Bash, Read, Write, Edit] --- # Skill Builder +Builds a pip package that contributes a Python `stargraph.skills.Skill`. A Skill +is registered through the `register_skills()` hookimpl under the entry-point +group `stargraph.skills` (there is no markdown skill format). + ## Inputs -- `skill_name`, `purpose`, `tools`, `subgraph`, `prompt_fragment`. +- `skill_name`, `purpose`, `kind` (`agent` | `workflow` | `utility`), + `tools` (registry-key ids `<ns>.<name>@<ver>`), `subgraph` (optional IR ref), + `system_prompt` (optional). ## Steps 1. Create `stargraph-skill-<skill_name>/` package. -2. `pyproject.toml` with `[project.entry-points."stargraph.skills"]`. -3. `src/stargraph_skill_<skill_name>/__init__.py` exporting `Skill` instance. -4. For each declared tool, scaffold under `tools/`. -5. If subgraph, create under `graphs/<skill_name>/`. -6. If prompt_fragment, write `prompts/<skill_name>.md`. -7. Run `uv pip install -e .`; verify `stargraph skills list` shows it. +2. `pyproject.toml` declaring BOTH entry-point groups: + ```toml + [project.entry-points."stargraph"] + stargraph_plugin = "stargraph_skill_<skill_name>._plugin:manifest" + + [project.entry-points."stargraph.skills"] + <skill_name> = "stargraph_skill_<skill_name>._plugin:register_skills" + ``` +3. `src/stargraph_skill_<skill_name>/_plugin.py` — define the `Skill` and the + `register_skills` hookimpl: + ```python + from stargraph.plugin import hookimpl + from stargraph.skills import Skill, SkillKind + from pydantic import BaseModel + + class State(BaseModel): + answer: str = "" # FIELD NAMES are the declared output channels / write whitelist; no `set` — use `frozenset` + + MY_SKILL = Skill( + name="<skill_name>", + version="0.1.0", + kind=SkillKind.<kind>, + description="<purpose>", + tools=[<"ns.name@ver">], # registry-key ids + subgraph=None, # or path/ref to an IR doc + system_prompt=None, + state_schema=State, + requires=[], # capability strings + bubble_events=True, + ) + + @hookimpl + def register_skills() -> list[Skill]: + return [MY_SKILL] + ``` + Also expose a `manifest` factory returning a `PluginManifest` + (`api_version="1"`, `provides=["skill"]`). +4. If `subgraph` is set, author its IRDocument (delegate to `graph-builder`). + For richer multi-file skills, use the **bundle** layout (Shipwright archetype): + `manifest.yaml` (`id`, `version`, `kind`, `description`, `state_schema: + module:Class`), `stargraph.yaml`, `state.py`, `nodes/`, optional `templates/`. +5. Install: `uv pip install -e .`. +6. **Verify discovery + registration** (there is no skill-listing CLI subcommand): + - Run any subcommand with `STARGRAPH_TRACE_PLUGINS=1` and confirm the + distribution is discovered, its manifest validates, and the skill registers: + `STARGRAPH_TRACE_PLUGINS=1 stargraph run <any-graph.yaml> --inspect`. + - At runtime with `stargraph serve` up: `GET /v1/registry/skills`. ## Output -Tree + skills list. +Tree, `STARGRAPH_TRACE_PLUGINS=1` discovery trace, `GET /v1/registry/skills` +result (or trace-only if serve not running). diff --git a/plugins/stargraph/commands/checkpoints.md b/plugins/stargraph/commands/checkpoints.md index b333d73..2afe222 100644 --- a/plugins/stargraph/commands/checkpoints.md +++ b/plugins/stargraph/commands/checkpoints.md @@ -1,11 +1,15 @@ --- -description: List checkpoints for a Stargraph run with state + facts diff -argument-hint: <run_id> +description: Inspect a Stargraph run's checkpoints — timeline, state at step, CLIPS fact diff +argument-hint: <run_id> --db <path> [--step <n>] [--diff <n> <m>] allowed-tools: [Bash, Read] --- # Stargraph Checkpoints +There is no `checkpoints` subcommand or endpoint. Checkpoint, state, and fact +views come from `stargraph inspect` over the SQLite checkpointer DB (default +`./.stargraph/run.sqlite`). + ## Load Foundation Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md`. @@ -13,9 +17,20 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md`. ## Run ```bash -curl -s "${STARGRAPH_URL}/v1/runs/<run_id>/checkpoints" -H "Authorization: Bearer ${STARGRAPH_TOKEN}" | jq '.data[] | {id, node, state_diff_summary, fact_count}' +RID="$1" +DB="${DB:-./.stargraph/run.sqlite}" + +# Timeline — per-step checkpoint sequence for the run +uv run stargraph inspect "${RID}" --db "${DB}" + +# State captured at a specific checkpoint step +uv run stargraph inspect "${RID}" --db "${DB}" --step "${STEP}" + +# CLIPS fact delta between two checkpoint steps +uv run stargraph inspect "${RID}" --db "${DB}" --diff "${N}" "${M}" ``` ## Report -Table of checkpoints with timestamps + state-diff summary. +Table of checkpoint steps from the timeline, with the state snapshot at a given +`--step` and the CLIPS fact delta (added/removed) from `--diff N M`. diff --git a/plugins/stargraph/commands/counterfactual.md b/plugins/stargraph/commands/counterfactual.md index b96c8f3..32f6eb2 100644 --- a/plugins/stargraph/commands/counterfactual.md +++ b/plugins/stargraph/commands/counterfactual.md @@ -1,14 +1,16 @@ --- -description: Counterfactual replay — fork a run from a checkpoint with mutated facts/state, diff against original -argument-hint: <run_id> --from <step> --mutate <json> [--compare] +description: Compute a counterfactual-derived graph_hash for a graph + mutation, without forking a run +argument-hint: <graph> --step <n> --mutate <file.yaml> allowed-tools: [Bash, Read, AskUserQuestion] --- # Stargraph Counterfactual -Like `/stargraph:replay` but you mutate something on the way through and Stargraph -diffs the alternate run against the original. Use when answering questions -like *"would the routing have changed if `intent` had been `research`?"* +Compute the counterfactual-derived `graph_hash` for a parent IR plus a mutation +YAML, **without** forking a run. Use this to verify a mutation file round-trips +and to pin the `graph_hash` you should see in the resulting cf-checkpoint — +e.g. answering *"would the routing have changed if `intent` had been +`research`?"* before you actually fork with `/stargraph:replay`. ## Load Foundation @@ -17,41 +19,40 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md` and ## Parse Arguments -- `<run_id>` — required. Source run to fork. -- `--from <step>` — required. Checkpoint step to branch from. -- `--mutate <json>` — required. JSON Patch (RFC 6902) or shorthand - `{"facts.<template>.<slot>": <value>}` / `{"state.<field>": <value>}`. -- `--compare` — also fetch a diff vs the original (default true; pass `--no-compare` to skip). +- `<graph>` — required. Path to the parent run's IR YAML graph definition. +- `--step <n>` — required. Checkpoint step index at which to fork (recorded in output). +- `--mutate <file.yaml>` — required. A YAML file describing a + `CounterfactualMutation` (state overrides, fact asserts/retracts, etc.). If any required arg is missing, prompt with AskUserQuestion before continuing. -## Verify Graph Hash +## Run -A counterfactual is meaningless across graph versions. Refuse if the original -run's `graph_hash` doesn't match the currently-registered graph. +The mutation YAML is validated through `CounterfactualMutation` (`extra='forbid'`, +so typos surface here): ```bash -ORIG=$(curl -fsS "${STARGRAPH_URL}/v1/runs/${RID}" -H "Authorization: Bearer ${STARGRAPH_TOKEN}" | jq -r .graph_hash) -CUR=$(curl -fsS "${STARGRAPH_URL}/v1/graphs/${GRAPH}" -H "Authorization: Bearer ${STARGRAPH_TOKEN}" | jq -r .hash) -[[ "$ORIG" == "$CUR" ]] || { echo "graph_hash mismatch — refuse"; exit 2; } +uv run stargraph counterfactual "${GRAPH}" \ + --step "${STEP}" \ + --mutate cf/swap-tool.yaml ``` -## Run +Output: -```bash -NEW_RUN=$(curl -fsS -X POST "${STARGRAPH_URL}/v1/runs/${RID}/counterfactual" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" \ - -H "Content-Type: application/json" \ - -d "{\"from_step\": ${STEP}, \"mutate\": ${MUTATE}}" | jq -r .run_id) - -curl -fsS "${STARGRAPH_URL}/v1/runs/${NEW_RUN}/compare?against=${RID}" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" | jq . +```text +original_graph_hash=... +cf_step=4 +derived_graph_hash=... ``` +To actually fork a run from a checkpoint and diff the alternate against the +original, use `/stargraph:replay` (CLI) or +`POST /v1/runs/{run_id}/counterfactual` on a running `stargraph serve`. + ## Report -- New `run_id` -- Divergence point: first step where node, transition, or facts differ from original -- Side-by-side: nodes visited, terminal facts, final state -- Cost delta (LLM tokens, tool calls) if budgets pack mounted -- Whether the terminal `result` changed +- `original_graph_hash`, `cf_step`, and `derived_graph_hash` +- Whether the mutation file validated (a `CounterfactualMutation` parse error + means a bad slot/field — fix the YAML) +- Next step: fork with `/stargraph:replay --from-step <step> --mutation <json>` + to see the alternate run and its `RunDiff` diff --git a/plugins/stargraph/commands/inspect.md b/plugins/stargraph/commands/inspect.md index 99d8fd3..4131e16 100644 --- a/plugins/stargraph/commands/inspect.md +++ b/plugins/stargraph/commands/inspect.md @@ -1,6 +1,6 @@ --- -description: Inspect a Stargraph run — events, state diff per checkpoint, fact stream, graph hash -argument-hint: <run_id> [--events] [--facts] [--diff] +description: Inspect a Stargraph run — timeline, state at step, CLIPS fact diff +argument-hint: <run_id> --db <path> [--step <n>] [--diff <n> <m>] allowed-tools: [Bash, Read] --- @@ -10,43 +10,37 @@ allowed-tools: [Bash, Read] Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md`. -## Verify Server - -`GET ${STARGRAPH_URL}/health`. - ## Run +`stargraph inspect` is a read-only inspector over a SQLite checkpointer DB +(default `./.stargraph/run.sqlite`). The mode selector is `--diff` > `--step` +> timeline; both `--diff` and `--step` require `--db`. + ```bash RID="$1" +DB="${DB:-./.stargraph/run.sqlite}" -# Run header — status, graph_hash, started/finished, trigger -curl -fsS "${STARGRAPH_URL}/v1/runs/${RID}" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" | \ - jq '{run_id, status, graph_hash, started_at, finished_at, trigger}' +# Timeline view — per-step node lifecycle, enriched with the audit log +uv run stargraph inspect "${RID}" --db "${DB}" --log-file ./.stargraph/run.jsonl -# Checkpoints — node, step, state diff summary -curl -fsS "${STARGRAPH_URL}/v1/runs/${RID}/checkpoints" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" | \ - jq '.data[] | {step, node, next, fact_count, state_diff_summary}' - -# Events — full timeline (if --events) -if [[ "$*" == *--events* ]]; then - curl -fsS "${STARGRAPH_URL}/v1/runs/${RID}/events" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" | jq . +# State-at-step view — IR-canonical state dict at step N (if --step) +if [[ "$*" == *--step* ]]; then + uv run stargraph inspect "${RID}" --db "${DB}" --step "${STEP}" fi -# Facts at terminal step (if --facts) -if [[ "$*" == *--facts* ]]; then - curl -fsS "${STARGRAPH_URL}/v1/runs/${RID}/facts" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" | \ - jq '.data[] | {template, slots, origin, source, confidence}' +# Fact-diff view — CLIPS facts asserted/retracted between step N and M (if --diff) +if [[ "$*" == *--diff* ]]; then + uv run stargraph inspect "${RID}" --db "${DB}" --diff "${N}" "${M}" fi ``` +Without `--db`, passing only `--log-file PATH` streams raw JSONL events +(legacy mode); an empty filter result there exits non-zero (force-loud). + ## Report -- Status, graph_hash (full + short), wall time, total steps, total facts -- Node-by-node table: step → node → outcome → key state changes -- Any `disagreement` facts (dual-truth divergences) -- Provenance breakdown: counts by `origin` (llm/tool/rule/model/external) -- If `status == FAILED` or `HALTED`: error/halt rule + offending facts +- From the timeline: status, graph_hash (full + short), total steps +- Node-by-node table: step → node → outcome → key state changes (`--step` per step) +- Fact delta from `--diff N M`: CLIPS facts added/removed, with their `origin`/`source` +- Provenance breakdown: counts by documented `origin` (`tool`/`llm`/`rule`/`system`) +- If terminal status is `failed`: the halt/error rule + offending facts diff --git a/plugins/stargraph/commands/new-dir-plugin.md b/plugins/stargraph/commands/new-dir-plugin.md index af29807..aed59bc 100644 --- a/plugins/stargraph/commands/new-dir-plugin.md +++ b/plugins/stargraph/commands/new-dir-plugin.md @@ -1,19 +1,17 @@ --- -description: Scaffold a directory-based Stargraph plugin (drop-in folder under ~/.stargraph/plugins/) — skills, tools, packs, stores, plugin.toml +description: Scaffold an entry-point Stargraph plugin — pip package + pyproject entry-points + stargraph_plugin manifest factory + capability hooks argument-hint: <plugin-name> allowed-tools: [Bash, Read, Write, AskUserQuestion, Task] --- -# New Directory Plugin +# New Plugin (entry-point) -Drop-in plugin layout that `stargraph-dir-plugins` discovers from -`$STARGRAPH_PLUGINS_DIR` (default `~/.stargraph/plugins/`). No pip install -required for prototyping; same typed contract, namespace conflict -detection, and audit chain as a pip-installed plugin once registered. - -Use this command for personal config, prototyping, or sharing a plugin as -a tarball. Use `/stargraph:new-skill` (or the legacy plugin-package -generator) when you want to publish to PyPI. +Stargraph plugins are normal pip packages discovered via `importlib.metadata` +entry points, then wired through `pluggy` hooks. There are no directory plugins: +a distribution declares a `stargraph_plugin` manifest factory plus one or more +capability groups in `pyproject.toml`. The reference archetype is +`stargraph.plugins.pii_guard` (a `@tool`-decorated redaction coroutine plus +governance hooks). ## Load Foundation @@ -21,29 +19,66 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md`. ## Interview -1. **Plugin name?** (becomes the directory name and `name` field) -2. **Namespaces it claims?** (one or more dotted prefixes, conflict-checked) -3. **What does it ship?** Combine any of: - - **Markdown skills** (`skills/<name>/SKILL.md`) — will call `/stargraph:new-md-skill` per skill - - **Tools** (`tools/*.py`) — will call `/stargraph:new-tool` per tool - - **Bosun packs** (`packs/<name>/`) — will call `/stargraph:new-pack` per pack - - **Stores** (declarative TOML store specs) -4. **Pack signing?** (if shipping packs: provide `[trust].keys` Ed25519 pubkeys, or generate a fresh keypair for development) -5. **Capability declarations?** (any new capability strings the tools/skills require) +1. **Distribution + package name?** (`name` field on the manifest; the import package, e.g. `my_pkg`) +2. **Namespaces it claims?** (`namespaces: list[str]`; conflicts abort load) +3. **What does it provide?** (`provides: list["tool"|"skill"|"store"|"pack"]`) — combine any of: + - **Tools** (`stargraph.tools` group, `register_tools`) — scaffold each via `/stargraph:new-tool` + - **Skills** (`stargraph.skills` group, `register_skills`) — `/stargraph:new-skill` + - **Bosun packs** (`stargraph.packs` group, `register_packs`) — `/stargraph:new-pack` + - **Stores** (`stargraph.stores` group, `register_stores`) — `/stargraph:store` +4. **Cross-cutting hooks?** (optional `authorize_action` first-deny gate; + `before_tool_call` / `after_tool_call` audit; `stargraph_startup` / + `stargraph_shutdown`) + +## Layout (pii_guard archetype) + +``` +my_pkg/ + __init__.py # docstring + _plugin.py # @hookimpl register_tools() -> [my_tool.spec]; manifest factory + hooks.py # authorize_action default-deny + before/after_tool_call audit + redact.py # the @tool-decorated coroutine (or your capability module) +pyproject.toml +``` + +`_plugin.py` exposes a `manifest` factory returning a `PluginManifest` +(`name`, `version`, `api_version="1"`, `namespaces`, `provides`, `order` 0..10000 +default 5000), and the `register_*` hookimpls. Import hookimpls from +`stargraph.plugin import hookimpl` (or `stargraph.plugin._markers`). + +```toml +# pyproject.toml +[project.entry-points."stargraph"] +stargraph_plugin = "my_pkg._plugin:manifest" + +[project.entry-points."stargraph.tools"] +my_tool = "my_pkg.redact:redact_pii" +``` ## Delegate Task tool → `dir-plugin-builder`. The agent: -1. Creates `~/.stargraph/plugins/<name>/` (or `$STARGRAPH_PLUGINS_DIR/<name>/`). -2. Writes `plugin.toml` with name, version, api_version, namespaces, trust keys. -3. Calls sibling agents per requested artifact (md-skill-builder, tool-builder, pack-builder). -4. Runs `stargraph plugins verify <path>` for offline validation. -5. Runs `stargraph plugins reload && stargraph plugins inspect <name>` to confirm - the live `stargraph serve` registers it. +1. Creates the package + `pyproject.toml` with the `stargraph` manifest factory + entry-point and the relevant capability-group entries. +2. Calls sibling agents per requested artifact (tool-builder, skill-builder, pack-builder). +3. Installs the package (`uv add .` / `pip install -e .`). + +## Verify + +Plugins are discovered at process start. Run any subcommand with +`STARGRAPH_TRACE_PLUGINS=1` to trace discovery → manifest validation → +registration: + +```bash +STARGRAPH_TRACE_PLUGINS=1 uv run stargraph run <graph.yaml> --inspect +``` + +List the registered kinds against a running server: `GET /v1/registry/{kind}` +(`tool` / `skill` / `store` / `pack`). ## Report -- Tree of the new dir-plugin. -- Output of `stargraph plugins verify` and `stargraph plugins inspect`. +- Tree of the package + the `pyproject.toml` entry-point tables. +- Plugin-trace output confirming discovery + registration. - Any namespace conflicts or unsigned-pack warnings, with remediation hints. diff --git a/plugins/stargraph/commands/new-graph.md b/plugins/stargraph/commands/new-graph.md index 3089cc5..b149578 100644 --- a/plugins/stargraph/commands/new-graph.md +++ b/plugins/stargraph/commands/new-graph.md @@ -1,5 +1,5 @@ --- -description: Scaffold a Stargraph graph (state.py, nodes/, rules/, stargraph.yaml, tests) +description: Scaffold a Stargraph IRDocument graph (stargraph.yaml + optional state class, nodes, rules, governance) argument-hint: <graph-name> allowed-tools: [Bash, Read, Write, AskUserQuestion, Task] --- @@ -17,10 +17,49 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-kraken/SKILL.md` and `${CLAUDE_PLUGIN_R ## Interview 1. **Purpose?** (one line) -2. **Initial nodes?** (think/act/observe pattern? custom?) -3. **State fields?** (list of (name, type, annotated?)) -4. **Rule packs to mount?** (bosun:budgets, bosun:audit, custom) -5. **Stores?** (vector / graph / doc / memory / fact — pick providers) +2. **Initial nodes?** For each: a builtin `kind` (`echo`, `halt`, `dspy`, `ml`, + `interrupt`, `passthrough`, `write_artifact`, `retrieval`, `subgraph`, + `human_input`) or a custom `module.path:ClassName`. +3. **State?** Either a flat `state_schema` (name→type-string map) OR a + `state_class: "module.path:ClassName"` Pydantic model (mutually exclusive). +4. **Rule packs to mount?** (governance PackMounts — `stargraph.bosun.budgets`, + `stargraph.bosun.audit`, custom slug + version) +5. **Stores?** (vector / graph / doc / memory / fact — pick providers; see `/stargraph:store`) + +## IRDocument shape + +The runnable IR is an `IRDocument` (all IR models pin `extra='forbid'`, so +unknown keys are rejected at load). Top-level keys: `ir_version`, `id`, `nodes` +(required); `rules`, `tools`, `skills`, `stores`, `governance`, `migrate`, +`parallel`, and `state_schema` / `state_class` (optional). Routing is implicit: +static fall-through in declaration order plus rule `goto` actions — there are no +explicit edges. + +```yaml +ir_version: "1.0.0" +id: "graph:<graph-name>" +state_class: "graph.state:RunState" # OR a flat state_schema map +nodes: + - id: classify + kind: dspy + - id: decide + kind: passthrough + - id: halt + kind: echo +governance: + - id: stargraph.bosun.audit + version: "1.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } +rules: + - id: r-classify-to-decide + when: "?n <- (node-id (id classify))" + then: [{ kind: goto, target: decide }] + - id: r-halt + when: "?n <- (node-id (id halt))" + then: [{ kind: halt, reason: "done" }] +``` + +Node/rule/pack ids must match `^[a-z0-9][a-z0-9_\-.]{0,127}$`. ## Delegate @@ -28,4 +67,7 @@ Task tool → `graph-builder`. ## Report -File tree, `stargraph graph verify` result. +File tree, plus validation result from `stargraph simulate <graph.yaml> --fixtures <f>` +or `stargraph run <graph.yaml> --inspect` (rule trace, no execution). IRDocument +validation runs automatically on load; `stargraph.ir.validate(ir)` gives the +structured error list in Python. diff --git a/plugins/stargraph/commands/new-md-skill.md b/plugins/stargraph/commands/new-md-skill.md index d8613b6..936a3fb 100644 --- a/plugins/stargraph/commands/new-md-skill.md +++ b/plugins/stargraph/commands/new-md-skill.md @@ -1,19 +1,18 @@ --- -description: Scaffold a Stargraph SKILL.md (lightweight markdown skill format that compiles to a typed stargraph.skills.Skill) +description: Author a Stargraph skill bundle (manifest.yaml + stargraph.yaml + state.py + nodes/) per the Shipwright archetype argument-hint: <skill-name> allowed-tools: [Bash, Read, Write, AskUserQuestion, Task] --- -# New Markdown Skill +# New Skill Bundle -Lightweight authoring path: a single `SKILL.md` (YAML frontmatter + markdown -body) compiles into a typed `stargraph.skills.Skill` instance. Same runtime -guarantees (state-schema enforcement, capability gates, deterministic -replay) — no Python boilerplate. +There is exactly one skill format: the Python `stargraph.skills.Skill`. A skill +bundle is its richer multi-file packaging — a directory with a `manifest.yaml`, +a `stargraph.yaml` graph, a typed `state.py`, and per-node modules. The +reference archetype is `src/stargraph/skills/shipwright/`. -Use this command when you want a Claude-Code-style authoring experience -instead of a full pip-installable package. Use `/stargraph:new-skill` for the -heavier Python-package path. +For a single-file `Skill` instance, use `/stargraph:new-skill`. Use this command +when the skill is graph-shaped (multiple nodes + rules + stores). ## Load Foundation @@ -21,13 +20,59 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md`. ## Interview -1. **Purpose?** (one sentence — becomes the description) -2. **Kind?** (`agent` | `workflow` | `utility`) -3. **Tools to reference?** (list of `<namespace>.<name>@<semver-range>` ids; must already be registered) -4. **State schema fields?** (name + type per declared output channel) -5. **Required capabilities?** (e.g. `billing.read`, `net.fetch`) -6. **Has a typed sub-graph?** (y/n; if yes, will call `/stargraph:new-graph`) -7. **Examples?** (optional — at least one input/expected_output pair recommended) +1. **Purpose?** (one sentence — becomes the `description`) +2. **Kind?** `agent` | `workflow` | `utility`. +3. **State model?** A Pydantic `State` in `state.py`; its field names are the + declared output channels / write-whitelist. Annotated (`Mirror`) fields are + projected to CLIPS facts at node boundaries. `set` / `set[X]` rejected — use + `frozenset`. +4. **Nodes?** Each maps `name` → a builtin (`stargraph.nodes.human_input`, ...) + or a custom `module.path:Class`. +5. **Bosun rule packs / governance?** (`rules: - pack: ...`, `governance: ...`) +6. **Stores + checkpoints?** (`stores: {doc:, fact:}`, `checkpoints:`) +7. **Tools to reference?** A `list[str]` of registry-key ids (`<ns>.<name>@<ver>`). + +## Layout (Shipwright archetype) + +``` +<skill-name>/ + manifest.yaml # id, version, kind, description, state_schema: module:Class + stargraph.yaml # graph: name, state: ./state.py:State, nodes, rules, governance, stores, checkpoints + state.py # the State Pydantic model + nodes/ # per-node modules + templates/ # prompt fragments (optional) + _pack.py # CLIPS pack loader (optional) +``` + +`manifest.yaml`: + +```yaml +id: my.skills.<skill-name> +version: "0.1.0" +kind: workflow +description: | + ... +state_schema: my.skills.<skill-name>.state:State +``` + +`stargraph.yaml`: + +```yaml +name: <skill-name> +state: ./state.py:State +nodes: + - name: human_input + type: stargraph.nodes.human_input + expected_input_schema_from: open_questions +rules: + - pack: my.bosun.<skill-name>.gaps +stores: + doc: sqlite:./.<skill-name>/docs.db + fact: sqlite:./.<skill-name>/facts.db +checkpoints: + every: node-exit + store: sqlite:./.<skill-name>/checkpoints.db +``` ## Delegate @@ -35,6 +80,8 @@ Task tool → `md-skill-builder`. ## Report -- Path to the new `SKILL.md`. -- Result of `stargraph skills compile <path>` validation. -- If hosted inside an existing dir-plugin, the plugin name it was added to. +- Tree of the bundle dir. +- The `register_skills` entry-point registration (group `stargraph.skills`) if + the bundle is shipped as a plugin. +- Validation: run the bundle graph via `stargraph run <stargraph.yaml> --inspect` + or `stargraph simulate <stargraph.yaml> --fixtures <f>`. diff --git a/plugins/stargraph/commands/new-node.md b/plugins/stargraph/commands/new-node.md index ee86903..ce8964c 100644 --- a/plugins/stargraph/commands/new-node.md +++ b/plugins/stargraph/commands/new-node.md @@ -1,5 +1,5 @@ --- -description: Add a node to a Stargraph graph (DSPy module / ML model / tool call / retrieval / sub-graph) +description: Add a NodeSpec to a Stargraph graph (builtin kind or custom module:Class) argument-hint: <graph> <node-name> allowed-tools: [Bash, Read, Write, AskUserQuestion, Task] --- @@ -12,10 +12,38 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md`. ## Interview -1. **Type?** (dspy:Predict | dspy:ChainOfThought | dspy:ReAct | model:onnx | tool:<name> | retrieval:<store> | subgraph:<graph>) -2. **Inputs?** (state fields read) -3. **Outputs?** (state fields written; mark which are annotated) -4. **Annotated outputs to mirror?** +1. **Kind?** A builtin factory key (`echo`, `halt`, `dspy`, `ml`, `interrupt`, + `passthrough`, `write_artifact`, `retrieval`, `subgraph`, `human_input`) or a + custom `module.path:ClassName`. +2. **Config?** Builtins read a `config:` block (e.g. an `ml` node takes + `model_id`, `version`, `runtime`, `file_uri`, `expected_sha256`, + `input_field`, `output_field`; an `interrupt` node takes `prompt`, + `requested_capability`, `timeout`, `on_timeout`). +3. **State fields read/written?** Custom nodes return a dict of state-field + updates; annotated (mirrored) fields are projected to CLIPS facts at the node + boundary so rules can route on them. +4. **Routing in?** A `RuleSpec` `goto`/`parallel`/`interrupt` action, or static + fall-through (declaration order). There are no explicit edges. + +## NodeSpec shape + +A node is `{id, kind}` plus an optional `config:` block: + +```yaml +nodes: + - id: <node-name> + kind: passthrough + - id: risk_score + kind: ml + config: + model_id: severity + version: "1.0.0" + runtime: onnx + input_field: features + output_field: risk +``` + +`id` must match `^[a-z0-9][a-z0-9_\-.]{0,127}$`. ## Delegate @@ -23,4 +51,7 @@ Task tool → `node-builder`. ## Report -Updated graph hash; if changed, warn that existing checkpoints need migrate block. +Updated `graph_hash` (canonical IR hash from `dumps_canonical`). If it changed, +warn that existing checkpoints will be rejected on resume unless a `migrate:` +block maps `from_hash`→`to_hash`. Re-validate with `stargraph run <graph.yaml> +--inspect` or `stargraph simulate <graph.yaml> --fixtures <f>`. diff --git a/plugins/stargraph/commands/new-pack.md b/plugins/stargraph/commands/new-pack.md index 1274d85..3f68da2 100644 --- a/plugins/stargraph/commands/new-pack.md +++ b/plugins/stargraph/commands/new-pack.md @@ -1,5 +1,5 @@ --- -description: Scaffold a Bosun rule pack (routing or governance flavor) using Fathom rule patterns +description: Scaffold a Bosun rule pack (CLIPS rules + PackSpec) and mount it via IR governance argument-hint: <pack-name> [--flavor routing|governance] allowed-tools: [Bash, Read, Write, AskUserQuestion, Task] --- @@ -13,13 +13,43 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md` and the Fathom plug ## Interview 1. **Flavor?** (routing | governance) -2. **What does it govern/route?** -3. **Initial rule sketches?** +2. **Pack id?** Slug form like `stargraph.bosun.budgets`, `stargraph.bosun.audit`, + or `soc-policy`. The version is a SEPARATE field. +3. **What does it govern/route?** +4. **Initial CLIPS rule sketches?** (packs carry CLIPS rules, e.g. `rules.clp`) + +## Mount into a graph + +A pack is mounted in the IRDocument `governance:` list as a `PackMount` +(`{id, version, requires}`), where `requires` is a `PackRequires` +(`stargraph_facts_version`, `api_version`): + +```yaml +governance: + - id: <pack-name> + version: "1.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } +``` + +`check_pack_compat` enforces `requires` at pack-load time and raises +`PackCompatError` on mismatch (force-loud — no silent drift). + +## Distribute as a plugin + +Ship the pack as a plugin under entry-point group `stargraph.packs`, with +`register_packs() -> list[PackSpec]`. `PackSpec` is `(id, version, manifest_path)`. + +## Signing + +Production requires Ed25519/JWS-signed packs (see signing.md). For development +you may generate a fresh keypair; unsigned packs are dev-only. ## Delegate -Task tool → `pack-builder`. If flavor=governance, scaffold a budget/audit/retry pack template. +Task tool → `pack-builder`. If flavor=governance, scaffold a budget/audit/retry +pack template. ## Report -Tree + validate result. +Tree + validation: mount the pack in a graph and run `stargraph run <graph.yaml> +--inspect` (or `simulate`) so the pack loads and `check_pack_compat` passes. diff --git a/plugins/stargraph/commands/new-skill.md b/plugins/stargraph/commands/new-skill.md index 5ff1e69..a7bc41e 100644 --- a/plugins/stargraph/commands/new-skill.md +++ b/plugins/stargraph/commands/new-skill.md @@ -1,5 +1,5 @@ --- -description: Scaffold a Stargraph skill bundle — tools, optional sub-graph, prompt fragment, entry-point registration +description: Scaffold a Python stargraph.skills.Skill registered via register_skills (optionally a skill bundle) argument-hint: <skill-name> allowed-tools: [Bash, Read, Write, AskUserQuestion, Task] --- @@ -12,10 +12,58 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md`. ## Interview -1. **Purpose?** -2. **Tools to ship?** (list of names; will scaffold each via `/stargraph:new-tool` later) -3. **Has a sub-graph?** (y/n; if yes, will call `/stargraph:new-graph`) -4. **Prompt fragment?** (optional) +1. **Purpose?** (becomes `description`) +2. **Kind?** `SkillKind`: `agent` | `workflow` | `utility`. +3. **Tools to call?** A `list[str]` of registry-key ids (`<ns>.<name>@<ver>`); + scaffold each via `/stargraph:new-tool`. +4. **State schema?** A Pydantic `BaseModel` whose **field names** are the + declared output channels / write-whitelist. `set` / `set[X]` fields are + rejected at construction — use `frozenset` (replay-safe). +5. **Has a sub-graph?** (optional `subgraph: str | None` — path/ref to an IR doc; + if yes, scaffold via `/stargraph:new-graph`) +6. **System prompt fragment?** (optional `system_prompt: str | None`) +7. **Required capabilities?** (optional `requires: list[str]`) + +## Write + +A skill is a `stargraph.skills.Skill` instance, registered through the pluggy +`register_skills` hookimpl in a plugin under entry-point group `stargraph.skills`: + +```python +from pydantic import BaseModel +from stargraph.plugin import hookimpl +from stargraph.skills import Skill, SkillKind + +class MyState(BaseModel): + answer: str = "" + +MY_SKILL = Skill( + name="<skill-name>", + version="0.1.0", + kind=SkillKind.utility, + description="...", + tools=["<ns>.<name>@1"], + state_schema=MyState, +) + +@hookimpl +def register_skills() -> list[Skill]: + return [MY_SKILL] +``` + +`declared_output_keys` and `site_id = f"{name}@{version}"` are computed from the +above. The loader pre-validates each instance and pre-checks namespace conflicts +before any hookimpl body runs. + +## Skill bundle (multi-file layout) + +For a richer multi-file skill (the Shipwright archetype at +`src/stargraph/skills/shipwright/`), scaffold a bundle dir: `manifest.yaml` +(`id`, `version`, `kind`, `description`, `state_schema: module:Class`), +`stargraph.yaml` (graph: `name`, `state: ./state.py:State`, `nodes`, `rules: +- pack: ...`, `stores`, `checkpoints`), `state.py`, `nodes/`, and optional +`templates/` + `_pack.py`. This authoring shape is richer than the validated +IRDocument shape (see `/stargraph:new-graph`). ## Delegate @@ -23,4 +71,5 @@ Task tool → `skill-builder`. ## Report -Tree, entry-point registration in pyproject.toml. +Tree, the `register_skills` entry-point registration in `pyproject.toml` under +group `stargraph.skills`, and the `site_id`. diff --git a/plugins/stargraph/commands/new-tool.md b/plugins/stargraph/commands/new-tool.md index da42ecf..bba4e99 100644 --- a/plugins/stargraph/commands/new-tool.md +++ b/plugins/stargraph/commands/new-tool.md @@ -1,5 +1,5 @@ --- -description: Create a Stargraph tool definition with JSON Schema + namespace + permissions + side-effect flags +description: Create a Stargraph @tool — ToolSpec with namespace/version, side-effects, replay policy, capability gate argument-hint: <tool-name> allowed-tools: [Bash, Read, Write, AskUserQuestion] --- @@ -12,26 +12,56 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md`. ## Interview -1. **Namespace?** (e.g. `browser`, `db`, `internal`) -2. **Args schema?** (JSON Schema) -3. **Returns schema?** -4. **Permissions?** (read | write | network | filesystem | executes-code) -5. **Side effects?** (none | external-api | local-fs | sends-email | ...) +1. **Namespace + version?** (e.g. `namespace="nautilus"`, `version="1"`). The + registry key is `f"{namespace}.{name}@{version}"` (e.g. `nautilus.broker_request@1`). +2. **Signature?** Type hints drive auto-derived `input_schema` / `output_schema` + (via `pydantic.create_model` + `TypeAdapter`). Pass `input_schema=` / + `output_schema=` explicitly only for `Annotated`/positional-only/`BaseModel` + cases. +3. **Side effects?** `SideEffects`: `none` | `read` | `write` | `external`. The + `cleared` serve profile refuses `write`/`external`. +4. **Replay policy?** `ReplayPolicy` (kebab): `must-stub` | `fail-loud` | + `recorded-result`. Default is derived from `side_effects` (none/read → + `recorded-result`; write/external → `must-stub`) — override only when needed. +5. **Required capability?** `requires_capability="..."` → stored on + `ToolSpec.permissions`; the gate raises `CapabilityError` before the call. ## Write ```python -# tools/<namespace>/<name>.py -from stargraph.tools import tool +# tools/<name>.py +from stargraph.tools import SideEffects, tool -@tool(namespace="<ns>", side_effects=[...], permissions=[...]) -def <name>(arg1: ..., arg2: ...) -> ...: - """...""" +@tool( + name="<name>", + namespace="<ns>", + version="1", + side_effects=SideEffects.read, + requires_capability="tools:<ns>:read", +) +async def <name>(*, arg1: str) -> dict: + """First line becomes the description if none is passed.""" ... ``` -JSON Schema is auto-derived from type hints; allow override via decorator args. +The decorator wraps the callable (sync or async) and exposes `wrapper.spec` +(a `ToolSpec`). Schemas are auto-derived from type hints when omitted. + +## Wire into a graph + +Tools enter the registry via the plugin entry-point group `stargraph.tools` +(`register_tools() -> list[ToolSpec]`, returning `[<name>.spec]`). A graph node +references the tool by its registry key: + +```yaml +nodes: + - id: ask + kind: tool + tool: <ns>.<name>@1 +``` ## Report -Tool path + adapter notes for DSPy + MCP if applicable. +Tool module path, the registry key `<ns>.<name>@<version>`, `side_effects` / +`replay_policy`, required capability, and the `register_tools` entry-point line +needed in the owning plugin's `pyproject.toml`. diff --git a/plugins/stargraph/commands/new-trigger.md b/plugins/stargraph/commands/new-trigger.md index c533ef1..214329a 100644 --- a/plugins/stargraph/commands/new-trigger.md +++ b/plugins/stargraph/commands/new-trigger.md @@ -1,6 +1,6 @@ --- -description: Wire a trigger (manual / cron / webhook) onto a Stargraph graph and verify scheduler pickup -argument-hint: <graph> --type manual|cron|webhook [--cron <expr>] [--path <url>] [--name <id>] +description: Author a trigger in triggers.yaml (manual / cron / webhook) and verify scheduler pickup at serve startup +argument-hint: <graph-id> --type manual|cron|webhook [--cron <expr>] [--path <url>] [--id <id>] allowed-tools: [Bash, Read, Write, Edit, AskUserQuestion] --- @@ -13,57 +13,81 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md` and ## Parse Arguments -- `<graph>` (required) — graph the trigger fires on. -- `--type` ∈ {`manual`, `cron`, `webhook`}. Required. -- `--cron <expr>` — cronsim expression, required when type=cron. Use 5-field cron; - Stargraph scheduler is DST-safe. -- `--path <url-suffix>` — public path for webhook. Required when type=webhook. -- `--name <id>` — trigger ID. Auto-generated from `(graph, type, idx)` if omitted. +- `<graph-id>` (required) — the `graph_id` the trigger enqueues. +- `--type` ∈ {`manual`, `cron`, `webhook`}. These are the ONLY three built-ins. +- `--cron <expr>` — 5-field cron expression, required when type=cron. Parsed by + `cronsim` (DST-safe); invalid syntax fails at serve startup. +- `--path <url-suffix>` — HTTP path the webhook route mounts (must start with + `/`). Required when type=webhook. +- `--id <id>` — trigger id (e.g. `cron:nightly-cve-feed`). Goes into the + idempotency key, so it must be unique across the deployment. If a required arg is missing for the chosen type, prompt with AskUserQuestion. -## Edit stargraph.yaml +## Edit triggers.yaml -Add a `triggers:` block on the graph: +Triggers are authored in `~/.config/stargraph/triggers.yaml` (override the dir +with env `STARGRAPH_CONFIG_DIR`), loaded at `stargraph serve` startup. Top-level +`version: "1.0"`, then per-kind lists. + +**manual** — no on-disk behavior; equals `stargraph run` + `POST /v1/runs`: + +```yaml +version: "1.0" +manual: + - id: digest-now + graph_id: <graph-id> + description: "Kick a digest run on demand." +``` + +**cron** — `cronsim`-driven background loop, one task per spec: ```yaml -triggers: - - name: nightly_research - type: cron - cron: "0 3 * * *" - timezone: UTC - dedup_key: research:nightly - input: +cron: + - id: cron:nightly-research + graph_id: <graph-id> + expr: "0 3 * * *" + tz: UTC + missed_fire_policy: fire_once_catchup + params: query: "weekly digest" ``` -For webhooks, also generate an HMAC secret and store in `.env.example`: +**webhook** — HMAC-SHA256-verified POST route. Secrets live in env vars named +per-spec (set `STARGRAPH_WEBHOOK_SECRET_CURRENT` / `STARGRAPH_WEBHOOK_SECRET_PREVIOUS` +or your own var names) — never in the file: ```yaml - - name: github_pr - type: webhook - path: /hooks/github-pr - secret_env: STARGRAPH_HOOK_GITHUB_PR - dedup_key: gh:${headers.x-github-delivery} +webhook: + - id: webhook:github-pr + graph_id: <graph-id> + path: /triggers/github-pr + timestamp_window_seconds: 300 + nonce_lru_size: 10000 + current_secret_env: STARGRAPH_WEBHOOK_SECRET_CURRENT + previous_secret_env: STARGRAPH_WEBHOOK_SECRET_PREVIOUS ``` -For manual triggers (the default), no scheduler entry — runs are kicked off via -`POST /v1/runs` or `/stargraph:run`. +To build a *custom* trigger kind, ship a plugin under entry-point group +`stargraph.triggers` (name → `Trigger` class with `init/start/stop/routes`), +emitting `TriggerEvent{trigger_id, scheduled_fire, idempotency_key, payload}` +into the scheduler queue (deduped by `idempotency_key`). ## Verify ```bash -uv run stargraph graph verify "${GRAPH}" -curl -fsS "${STARGRAPH_URL}/v1/triggers" -H "Authorization: Bearer ${STARGRAPH_TOKEN}" | \ - jq --arg g "${GRAPH}" '.data[] | select(.graph == $g)' +uv run stargraph serve --graph "<graph.yaml>" ``` -For cron: ensure the scheduler picked it up — `next_fire_at` should be populated. -For webhook: hit it with a sample payload and confirm a 202 + `run_id`. +On startup the scheduler picks the trigger up: cron spawns one `asyncio.Task` +per spec; webhook mounts its `POST` route. Confirm with `GET /v1/graphs` (graph +registered) and, for webhooks, that the mounted `path` answers. Default serve +URL is `http://localhost:8000`. ## Report -- Trigger ID, graph, type, schedule/path -- Dedup key -- For webhook: signed-cURL example for testing -- For cron: next 3 fire times (use cronsim if available locally) +- Trigger id, graph_id, type, schedule/path. +- For webhook: the env vars holding the current/previous secrets and a signed + sample (HMAC of `"{ts}.{body}"`, headers `X-Stargraph-Timestamp` / + `X-Stargraph-Signature`). +- For cron: the 5-field expr and tz. diff --git a/plugins/stargraph/commands/replay.md b/plugins/stargraph/commands/replay.md index 4c9cc48..f9be810 100644 --- a/plugins/stargraph/commands/replay.md +++ b/plugins/stargraph/commands/replay.md @@ -1,6 +1,6 @@ --- description: Counterfactual replay of a Stargraph run from any checkpoint -argument-hint: <run_id> [--from <checkpoint>] [--patch <json>] +argument-hint: <run_id> --db <path> [--from-step <n>] [--mutation <json>] allowed-tools: [Bash, Read, AskUserQuestion] --- @@ -10,19 +10,26 @@ allowed-tools: [Bash, Read, AskUserQuestion] Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md`. -## Verify Server - -Standard. Confirm graph_hash of the run matches current graph; warn on mismatch. - ## Run +Fork a counterfactual run from a checkpoint in the parent run's SQLite DB. With +no `--mutation`, an empty no-op mutation is used (still produces a cf-derived +`graph_hash`). The cf-run id is minted as `cf-<uuid>`; the parent's checkpoint +rows stay byte-identical post-fork. + ```bash -curl -s -X POST "${STARGRAPH_URL}/v1/runs/<run_id>/replay" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" \ - -H "Content-Type: application/json" \ - -d '{"from_checkpoint":"<cp>","patch": <json>}' +uv run stargraph replay "<run_id>" \ + --db ./.stargraph/run.sqlite \ + --mutation cf/override.json \ + --from-step "${FROM_STEP:-0}" \ + --diff ``` +`--mutation FILE.json` loads a `CounterfactualMutation` (state overrides, fact +asserts/retracts, etc.). `--diff` renders the parent-vs-cf `RunDiff` as +canonical IR JSON after forking (omit, or pass `--no-diff`, to print just the +cf-run id). + ## Report -new run_id, divergence point, comparison summary vs original. +new cf-run id (`cf-<uuid>`), fork step, and the parent-vs-cf `RunDiff` summary. diff --git a/plugins/stargraph/commands/respond.md b/plugins/stargraph/commands/respond.md index 78ff56a..311b01a 100644 --- a/plugins/stargraph/commands/respond.md +++ b/plugins/stargraph/commands/respond.md @@ -1,12 +1,14 @@ --- -description: Respond to a paused (HITL) Stargraph run — supply human input, approve/deny, resume -argument-hint: <run_id> --decision <approve|deny|input> [--payload <json>] [--reason <str>] +description: Respond to a paused (HITL) Stargraph run — supply human input, resume +argument-hint: <run_id> --response <json> --actor <name> [--server <url>] allowed-tools: [Bash, Read, AskUserQuestion] --- # Stargraph Respond -Resume a run that's paused at a `human_input` node or a Bosun governance hold. +Deliver a HITL response to a run that is `awaiting-input` (paused at an +`interrupt` / `human_input` node). `stargraph respond` is a thin wrapper over +`POST /v1/runs/{run_id}/respond` on a running `stargraph serve`. ## Load Foundation @@ -15,38 +17,43 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md` and ## Verify Pause -Refuse unless the run is in `PAUSED_HITL` status: +The run must be awaiting input. `GET /v1/runs/{id}` folds that lifecycle state +onto `paused`; refuse unless the run is paused: ```bash -STATUS=$(curl -fsS "${STARGRAPH_URL}/v1/runs/${RID}" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" | jq -r .status) -[[ "$STATUS" == "PAUSED_HITL" ]] || { echo "not paused (status=$STATUS)"; exit 2; } +SERVER="${SERVER:-http://localhost:8000}" +STATUS=$(curl -fsS "${SERVER}/v1/runs/${RID}" \ + -H "Authorization: Bypass ${ACTOR}" | jq -r .status) +[[ "$STATUS" == "paused" ]] || { echo "not awaiting input (status=$STATUS)"; exit 2; } ``` -Show the user the pending question/context (`prompt`, `policy_reason`, expected -schema for `payload`) before asking for their response. +Show the user the pending prompt and the requested capability before asking for +their response. ## Parse Arguments -- `--decision`: `approve` | `deny` | `input` -- `--payload`: JSON matching the run's `expected_input_schema` (required for `input`, optional for `approve`) -- `--reason`: free-form audit string (required for `deny`, recommended for `approve`) +- `--response <json>`: path to a JSON file with the analyst response payload (required). +- `--actor <name>`: principal id; sent as `Authorization: Bypass <actor>` (required). +- `--server <url>`: base URL of the running `stargraph serve` (default `http://localhost:8000`). -If args are missing, prompt with AskUserQuestion. Validate `--payload` against -the schema with `jsonschema` before submitting. +If args are missing, prompt with AskUserQuestion. Build the response JSON file, +then submit: ## Run ```bash -curl -fsS -X POST "${STARGRAPH_URL}/v1/runs/${RID}/respond" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" \ - -H "Content-Type: application/json" \ - -d "{\"decision\": \"${DECISION}\", \"payload\": ${PAYLOAD:-null}, \"reason\": ${REASON_JSON:-null}}" +uv run stargraph respond "${RID}" \ + --response analyst-decision.json \ + --actor "${ACTOR}" \ + --server "${SERVER:-http://localhost:8000}" ``` +The CLI maps HTTP status to operator-friendly messages: `200` prints the +`RunSummary` JSON; `401` = auth failed; `404` = run not found or not awaiting +input; `409` = not awaiting input (already responded / conflicting state). + ## Report -- New status (`RUNNING` / `COMPLETED` / `HALTED`) -- Audit fact emitted: `(human_response decision=… reason=… by=… ts=…)` -- Next node about to run, if any -- Stream URL to follow: `${STARGRAPH_URL}/v1/runs/${RID}/stream` +- New status from the returned `RunSummary` (`running` / `done` / `paused`) +- The response payload delivered, the actor, and the run id +- Stream URL to follow: `http://localhost:8000/v1/runs/${RID}/stream` diff --git a/plugins/stargraph/commands/run.md b/plugins/stargraph/commands/run.md index c48ee3f..f872e23 100644 --- a/plugins/stargraph/commands/run.md +++ b/plugins/stargraph/commands/run.md @@ -1,6 +1,6 @@ --- -description: Execute a Stargraph graph against `stargraph serve`; stream events; return run_id -argument-hint: <graph> [--input-file <json>] +description: Execute a Stargraph graph; stream events; return run_id +argument-hint: <graph> [-i K=V]... [--inspect] allowed-tools: [Bash, Read, AskUserQuestion, Task] --- @@ -10,25 +10,51 @@ allowed-tools: [Bash, Read, AskUserQuestion, Task] Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md`. -## Verify Server +## Run -`GET ${STARGRAPH_URL}/health`. `GET /v1/graphs` — confirm graph is registered. +`<graph>` is a path to an IR YAML file (commonly `<graphdir>/stargraph.yaml`). +Seed initial state with `-i K=V` (repeatable; each key must match the IR +`state_schema`). The graph runs against a SQLite checkpointer (default +`./.stargraph/run.sqlite`). -## Run +```bash +uv run stargraph run "<graph>" \ + -i message="check pack drift" -i severity=3 \ + --checkpoint ./.stargraph/run.sqlite \ + --log-file ./.stargraph/run.jsonl \ + --summary-json +``` + +To preview routing without executing any node (rule-firing trace only), add +`--inspect`: ```bash -RUN=$(curl -s -X POST "${STARGRAPH_URL}/v1/runs" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" \ - -H "Content-Type: application/json" \ - -d "{\"graph\":\"<graph>\", \"input\": $(cat <input-file>)}" | jq -r '.data.run_id') +uv run stargraph run "<graph>" --inspect ``` -Optionally tail events via WS: +Exit code is `0` on a terminal `done`, non-zero on `failed`. Bind a local LLM +for `dspy` nodes with `--lm-url URL --lm-model NAME` (supplied together) plus +optional `--lm-key`/`--lm-timeout`. Use `--quiet`/`--verbose` for output +volume and `--non-interactive` to fail rather than prompt on a HITL pause. + +## Driving a running server + +If a `stargraph serve` process is up, start the run over HTTP instead — the +graph's `id` (e.g. `graph:triage`) is the key: ```bash -websocat -H "Authorization: Bearer ${STARGRAPH_TOKEN}" "${STARGRAPH_URL/http/ws}/v1/runs/$RUN/events" +RUN=$(curl -fsS -X POST "http://localhost:8000/v1/runs" \ + -H "Authorization: Bypass operator" \ + -H "Content-Type: application/json" \ + -d '{"graph_id":"graph:triage","params":{}}' | jq -r '.run_id') + +# Tail events over the WebSocket stream +websocat "ws://localhost:8000/v1/runs/$RUN/stream" ``` +`POST /v1/runs` returns `202` with `{run_id, status:"pending"}`; poll +`GET /v1/runs/$RUN` for terminal state. + ## Delegate Task tool → `runner` for deeper polling/parsing. diff --git a/plugins/stargraph/commands/serve.md b/plugins/stargraph/commands/serve.md index d3d8ef9..a93e2c2 100644 --- a/plugins/stargraph/commands/serve.md +++ b/plugins/stargraph/commands/serve.md @@ -1,6 +1,6 @@ --- description: Start `stargraph serve` (FastAPI HTTP+WebSocket daemon) with a chosen profile -argument-hint: [--profile dev|prod|cleared] [--port <n>] [--host <addr>] +argument-hint: [--profile oss-default|cleared] [--port <n>] [--host <addr>] allowed-tools: [Bash, Read, AskUserQuestion] --- @@ -12,31 +12,42 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md`. ## Parse Arguments -- `--profile` ∈ {`dev`, `prod`, `cleared`} — default `dev`. Cleared profile assumes air-gapped, signed-only artifacts, no telemetry. -- `--port` — default `9000`. +- `--profile` ∈ {`oss-default`, `cleared`} — default `oss-default`. Under + `cleared`, the startup gate refuses `--allow-pack-mutation` and + `--allow-side-effects` and exits non-zero with a `ProfileViolationError`. +- `--port` — default `8000`. - `--host` — default `127.0.0.1`. Use `0.0.0.0` only when the user explicitly asks for it. -If a `stargraph` process already listens on `${STARGRAPH_URL}/health`, abort and report. +If a `stargraph` process already listens on the chosen `--host:--port`, abort and report. ## Run ```bash -STARGRAPH_PROFILE="${PROFILE:-dev}" \ - uv run stargraph serve \ - --host "${HOST:-127.0.0.1}" \ - --port "${PORT:-9000}" +uv run stargraph serve \ + --profile "${PROFILE:-oss-default}" \ + --host "${HOST:-127.0.0.1}" \ + --port "${PORT:-8000}" \ + --db ./stargraph.sqlite \ + --audit-log ./audit.jsonl \ + --graph "<graphdir>/stargraph.yaml" ``` -Run in background only if the user asked for it (`run_in_background: true`). Otherwise stream the boot output for ~3 seconds, then check: +`--graph` is repeatable and loads + registers an IR YAML at boot; the graph's +`id` is the key `POST /v1/runs` uses. Bind a local LLM with +`--lm-url URL --lm-model NAME` (plus optional `--lm-key`/`--lm-timeout`). + +Run in background only if the user asked for it (`run_in_background: true`). +Otherwise stream the boot output for ~3 seconds, then confirm the API is up by +listing graphs: ```bash -curl -fsS "${STARGRAPH_URL}/health" | jq . -curl -fsS "${STARGRAPH_URL}/v1/graphs" | jq '.data | length' +curl -fsS "http://localhost:8000/v1/graphs" \ + -H "Authorization: Bypass operator" | jq 'length' ``` ## Report - Profile, host:port, PID - Number of registered graphs -- WebSocket endpoint: `${STARGRAPH_URL}/v1/runs/<id>/stream` +- WebSocket endpoint: `http://localhost:8000/v1/runs/<id>/stream` - Stop hint: `kill <PID>` or Ctrl-C in the foreground shell diff --git a/plugins/stargraph/commands/simulate.md b/plugins/stargraph/commands/simulate.md index 58f6983..0de0095 100644 --- a/plugins/stargraph/commands/simulate.md +++ b/plugins/stargraph/commands/simulate.md @@ -1,14 +1,14 @@ --- -description: Dry-run a Stargraph graph in a sandboxed engine — no side-effecting tools, deterministic stubs -argument-hint: <graph> [--input-file <json>] [--seed <int>] [--max-steps <n>] +description: Offline rule-firing trace for a Stargraph graph against synthetic fixtures — no tools/LLM/checkpoint +argument-hint: <graph> --fixtures <file> allowed-tools: [Bash, Read, AskUserQuestion] --- # Stargraph Simulate -Execute a graph end-to-end with all `side-effects ≠ none` tool calls swapped -for deterministic stubs. Use to smoke-test routing rules and Bosun packs -without touching real systems. +Validate a graph's rule logic against caller-supplied synthetic node outputs +without invoking any tool, LLM, or checkpoint. Use to smoke-test routing rules +and Bosun packs without touching real systems. ## Load Foundation @@ -17,25 +17,25 @@ Read `${CLAUDE_PLUGIN_ROOT}/skills/smart-stargraph/SKILL.md` and ## Parse Arguments -- `<graph>` — graph name registered with `stargraph serve`, or a path to `stargraph.yaml`. -- `--input-file <json>` — initial state. Defaults to the graph's `examples/smoke.json` if present. -- `--seed <int>` — RNG seed for any DSPy nodes flagged `must_stub`. Default `0`. -- `--max-steps <n>` — abort if the graph exceeds this. Default `50`. +- `<graph>` — path to an IR YAML graph definition (commonly `<graphdir>/stargraph.yaml`). +- `--fixtures <file>` — required. A YAML file mapping `node_id` → synthetic + output dict (one entry per IR node). Defaults to the graph's + `fixtures/*.yaml` if the user points at one. ## Run ```bash -uv run stargraph simulate "${GRAPH}" \ - ${INPUT_FILE:+--input-file "$INPUT_FILE"} \ - --seed "${SEED:-0}" \ - --max-steps "${MAX_STEPS:-50}" +uv run stargraph simulate "${GRAPH}" --fixtures "${FIXTURES}" ``` +Output mirrors `stargraph run --inspect`: a leading +`graph_hash=<hex>` and `rule_firings=<count>` line, followed by one row per +rule firing. + ## Report -- Whether simulation completed, halted, or hit max-steps -- Node trace: `step → node → outcome → next-rule-fired` -- Tools that would have side-effected (with the args they were called with) -- Any rule pack assertions that fired -- Suggested next step: real `/stargraph:run` if clean, otherwise inspect the - failing rule or stub +- The `graph_hash` and total `rule_firings` +- The rule-firing trace: which rules fired, in declaration/`goto` order +- Any `assert`/`retract`/`goto`/`halt` actions the fired rules produced +- Suggested next step: real `/stargraph:run` if the trace looks right, + otherwise fix the failing rule `when` pattern or the fixture output it matches diff --git a/plugins/stargraph/commands/store.md b/plugins/stargraph/commands/store.md index 86e411e..b7b0605 100644 --- a/plugins/stargraph/commands/store.md +++ b/plugins/stargraph/commands/store.md @@ -1,6 +1,6 @@ --- description: Wire a Store provider for a Stargraph graph (vector/graph/doc/memory/fact) -argument-hint: add <type> <provider> [--config <json>] +argument-hint: add <protocol> <provider> [--config <json>] allowed-tools: [Bash, Read, Write, AskUserQuestion] --- @@ -8,14 +8,47 @@ allowed-tools: [Bash, Read, Write, AskUserQuestion] ## Subcommand: add -`<type>` ∈ {vector, graph, doc, memory, fact}. `<provider>` ∈ {lancedb, kuzu, sqlite, ...}. +`<protocol>` ∈ {`vector`, `graph`, `doc`, `memory`, `fact`} — the five `Store` +protocols. `<provider>` is a provider id; the default embedded tier: + +| Protocol | Default provider id | Compact scheme | Backed by | +|----------|------------------------------|----------------|------------------------| +| vector | `stargraph.stores.lancedb` | `lancedb:` | LanceDB | +| graph | `stargraph.stores.ryugraph` (or `stargraph.stores.cypher`) | `ryugraph:` | RyuGraph (Kuzu fork) | +| doc | `stargraph.stores.sqlite_doc` | `sqlite:` | SQLite (WAL) | +| memory | `stargraph.stores.sqlite_memory` | `sqlite:` | SQLite (WAL) | +| fact | `stargraph.stores.sqlite_fact` | `sqlite:` | SQLite (WAL) + Fathom | ## Steps -1. Read `stargraph.yaml`. -2. Insert `stores: <type>: <provider>:<config-uri>`. -3. Validate via `stargraph graph verify`. +1. Read the graph IR `stargraph.yaml`. +2. Insert/extend the top-level `stores:` block. Compact form keys by protocol; + value is `<scheme>:<path>`: + + ```yaml + stores: + vector: lancedb:./.lance + graph: ryugraph:./.ryu + doc: sqlite:./.docs + memory: sqlite:./.memory + fact: sqlite:./.facts + ``` + + The runtime parses each entry into a `StoreRef(name, provider)`, whose + `to_capabilities()` derives `db.{name}:read` / `db.{name}:write` for the + Bosun policy gates. +3. Validate by loading: `stargraph run <graph.yaml> --inspect` (rule trace, no + exec) or `stargraph simulate <graph.yaml> --fixtures <f>`. IRDocument + validation runs automatically on load (`extra='forbid'`). + +## Custom providers + +A third-party provider ships as a plugin under entry-point group +`stargraph.stores`, with `register_stores() -> list[StoreSpec]`. A `StoreSpec` +is the canonical registration record: `{name, provider, protocol, +config_schema, capabilities}`. List registered stores against a running server +with `GET /v1/registry/stores`. ## Report -YAML diff + verify status. +YAML diff + the validation result (`run --inspect` / `simulate`). diff --git a/plugins/stargraph/commands/verify-graph.md b/plugins/stargraph/commands/verify-graph.md index 4cbe24d..6cb0457 100644 --- a/plugins/stargraph/commands/verify-graph.md +++ b/plugins/stargraph/commands/verify-graph.md @@ -1,17 +1,46 @@ --- -description: Validate a Stargraph graph (hash, schema, referenced rule packs, store providers) -argument-hint: <graph-path-or-name> +description: Validate a Stargraph graph by loading it (IRDocument schema) and tracing its rules +argument-hint: <graph-path> [--fixtures <file>] allowed-tools: [Bash, Read] --- # Stargraph Verify Graph +There is no dedicated verify subcommand. A graph is validated by **loading** +it: the IR YAML is parsed and validated against `IRDocument`, and every IR +model pins `extra='forbid'`, so unknown keys, bad node `kind`s, malformed +rule `when/then` actions, or out-of-spec ids fail loudly at load. + ## Run +Validate + print the rule-firing trace without executing any node: + ```bash -uv run stargraph graph verify <graph-path-or-name> +uv run stargraph run "<graph-path>" --inspect +``` + +This prints a leading `graph_hash=<hex>` and `rule_firings=<count>` line. A +non-zero exit or a parse/validation error means the graph is invalid. + +For a deeper check of rule logic against synthetic node outputs (still no +tools/LLM/checkpoint): + +```bash +uv run stargraph simulate "<graph-path>" --fixtures "<fixtures.yaml>" +``` + +To validate in Python (e.g. in a test): + +```python +import yaml +from stargraph import ir + +doc = ir.validate(yaml.safe_load(open("<graph-path>"))) # raises on invalid IR +print(doc.id) ``` ## Report -✓ valid, hash=<hex>, summary of nodes/rules/stores. Or ✗ with error block. +✓ valid, `graph_hash=<hex>`, summary of nodes/rules/stores (and `rule_firings` +from the `--inspect`/`simulate` trace). Or ✗ with the IRDocument validation +error block. diff --git a/plugins/stargraph/references/bosun-packs.md b/plugins/stargraph/references/bosun-packs.md index 879cabc..2d51af9 100644 --- a/plugins/stargraph/references/bosun-packs.md +++ b/plugins/stargraph/references/bosun-packs.md @@ -1,15 +1,15 @@ # Bosun Rule Packs Bosun is the in-tree set of governance rule packs that ship with Stargraph -(`stargraph.bosun.*`). They're regular Fathom rule packs, mounted on graphs -declaratively. +(`stargraph.bosun.*`). They're regular Fathom/CLIPS rule packs, mounted on +graphs declaratively through the IR `governance:` section. ## Two flavors | Flavor | Job | Examples | |---|---|---| -| **Routing** | Decide what node runs next. Match on state/facts; emit `goto` / `parallel` / `halt`. | `bosun:routing/research`, `bosun:routing/triage` | -| **Governance** | Constrain, observe, or modify execution. Budgets, retries, audit, safety guards. | `bosun:budgets`, `bosun:retries`, `bosun:audit`, `bosun:safety` | +| **Routing** | Decide what node runs next. Match on state/facts; emit `goto` / `parallel` / `halt`. | project routing packs (e.g. `soc-policy`) | +| **Governance** | Constrain, observe, or modify execution. Budgets, retries, audit, safety guards. | `stargraph.bosun.budgets`, `stargraph.bosun.audit` | The engine treats them identically — `flavor` is convention, not a code path. A pack should pick one flavor and stick to it; mixing routing and governance @@ -17,20 +17,30 @@ in the same pack makes inspection harder. ## Mounting +Packs are mounted as `PackMount` entries under the IR `governance:` key. A +`PackMount` is `{id, version, requires}`, where `requires` is a `PackRequires` +compat block: + ```yaml -# in stargraph.yaml -rules: - - pack: bosun:routing/research@1.4 +# in the graph IR (stargraph.yaml) governance: - - bosun:budgets@2.0 - - bosun:retries - - bosun:audit + - id: stargraph.bosun.budgets + version: "1.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } + - id: stargraph.bosun.audit + version: "1.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } + - id: soc-policy + version: "1.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } ``` -- `pack: vendor:name@version` — version is required for packs that move state - between major versions; optional otherwise (resolves to latest). -- `governance:` is a *list* — packs fire in declaration order on every - matching transition. Order matters for budgets-before-retries etc. +- Pack `id` is a **slug** (`^[a-z0-9][a-z0-9_\-.]{0,127}$`), with `version` as a + **separate field** — not `vendor:pack@version`. +- `requires.stargraph_facts_version` and `requires.api_version` are enforced at + pack-load by `stargraph.ir._versioning.check_pack_compat`, which raises + `PackCompatError` on mismatch (force-loud — silent runtime drift is impossible). + Comparison is pinned-string equality. ## Reference: Bosun packs shipped with Stargraph @@ -38,39 +48,26 @@ These live in `src/stargraph/bosun/` and are governance-flavor: | Pack | Purpose | Key facts emitted | |---|---|---| -| `bosun:budgets` | Enforce token / time / dollar caps per run. | `budget.exceeded`, `budget.warn` | -| `bosun:retries` | Exponential backoff on tool failures. | `retry.attempt`, `retry.giveup` | -| `bosun:audit` | Emit a structured, signed audit fact per transition (see `signing.py`). | `audit.transition` | -| `bosun:safety_pii` | Halt on PII/secret leak markers in tool outputs. | `safety.violation` | +| `stargraph.bosun.budgets` | Enforce token / time / dollar caps per run. | `budget.exceeded`, `budget.warn` | +| `stargraph.bosun.audit` | Emit a structured, signed audit fact per transition. | `audit.transition` | -Routing-flavor packs are not shipped in-tree — author them per project with -`/stargraph:new-pack --flavor routing` and mount via `rules:`. +Routing-flavor packs are typically authored per project (e.g. the demos' +`soc-policy`) and mounted the same way. ## Authoring a new pack -Use `/stargraph:new-pack <name> --flavor routing|governance`. It scaffolds: - -``` -bosun-packs/<name>/ - pack.yaml - templates/ # deftemplate YAML — fact shapes - rules/ # defrule YAML — productions - modules/ # CLIPS modules (if you need scoping) - functions/ # deffunction YAML - tests/ # pytest cases — pack-level unit tests - README.md -``` - -`pack.yaml` declares the public fact vocabulary (templates) and which Stargraph -runtime fact templates the pack reads. Anything else is private. +A pack's CLIPS rules live in `rules.clp` (split into top-level constructs at +load). It declares a public fact vocabulary (deftemplates) and which runtime +fact templates it reads; anything else is private. ### Provenance discipline -Every fact a pack asserts MUST carry `(origin=rule, source=<pack>:<rule>, -run_id, step, ts)`. This is enforced by the Fathom adapter — packs that -forget will fail validation. +Every fact a pack asserts MUST carry its provenance — `origin`, `source` +(`<pack>:<rule>`), `run_id`, `step`, `confidence`, `timestamp`. The Fathom +adapter enforces this; packs that forget will fail validation. See +`references/provenance-facts.md`. -### Routing rules — emit shape +### Routing rule — emit shape ```yaml - name: when-confident-act @@ -80,7 +77,7 @@ forget will fail validation. goto: act ``` -### Governance rules — emit shape +### Governance rule — emit shape ```yaml - name: budget-trip @@ -93,29 +90,30 @@ forget will fail validation. - { template: budget.exceeded, slots: { kind: tokens, cap: ${cap} } } ``` -## Validation +## Distribution -```bash -fathom validate bosun-packs/<name> -fathom test bosun-packs/<name> -fathom bench bosun-packs/<name> # µs/eval; regressions vs published targets +In-tree Bosun packs ship under `stargraph.bosun.*`. Third-party packs ship as +separate pip distributions registering under the `stargraph.packs` entry-point +group via a `register_packs()` hook returning `list[PackSpec]` (`PackSpec` = +`{id, version, manifest_path}`). + +```toml +# pyproject.toml of a pack-providing distribution +[project.entry-points."stargraph.packs"] +my_pack = "my_pkg._plugin:manifest" ``` -Mount a candidate pack on a real graph in a sandbox: +## Signing -```bash -stargraph simulate <graph> --mount bosun-packs/<name> -``` +Production requires Ed25519 / JWS-signed packs. Stargraph release artifacts are +themselves signed with Ed25519 (detached `.sig`); the signing key is rotated on +a published schedule (fingerprint recorded in `SECURITY.md`). At pack load, +`check_pack_compat` enforces the `requires` block before the pack's rules are +admitted. ## Versioning Packs use semver. Breaking = changed fact templates, removed rules, changed -emit shapes. Re-version on any of those. Stargraph's graph hash captures the -mounted pack version, so checkpoints from `bosun:budgets@1.x` won't resume -under `@2.x` unless you declare a `migrate` block. - -## Distribution - -Bosun packs ship in-tree under `stargraph.bosun.*`. Third-party packs ship as -separate pip distributions registering under the `stargraph.packs` entry point -(see `design-docs/stargraph-plugin-api.md`). +emit shapes. Re-version on any of those. The graph hash captures the mounted +pack version, so checkpoints from a `1.x` pack won't resume under `2.x` unless +the graph declares a `migrate` block mapping `from_hash → to_hash`. diff --git a/plugins/stargraph/references/graph-yaml-schema.md b/plugins/stargraph/references/graph-yaml-schema.md index 1c9a3f4..c839de1 100644 --- a/plugins/stargraph/references/graph-yaml-schema.md +++ b/plugins/stargraph/references/graph-yaml-schema.md @@ -1,231 +1,253 @@ -# Stargraph stargraph.yaml Reference +# Stargraph IRDocument (graph YAML) Reference -`stargraph.yaml` is the declarative manifest for a Stargraph graph. It binds a Pydantic state class, a list of nodes, mounted Bosun rule and governance packs, store providers, checkpoint policy, triggers, and plugin dependencies. +A Stargraph graph is an **IRDocument** — a portable, JSON-Schema-typed description of an executable graph plus its rules, tools, skills, stores, and governance packs. It is authored as YAML (commonly `<graphdir>/stargraph.yaml`) and loaded by `stargraph run`, `stargraph serve --graph`, and `stargraph simulate`. + +Every IR model subclasses `IRBase`, which pins `extra='forbid'` — unknown keys are rejected at load time. Validation is automatic on load; you can also call `stargraph.ir.validate(ir)` in Python (it returns a `list[ValidationError]`, never raises). ## Top-level Keys | Key | Type | Required | Purpose | |---|---|---|---| -| `name` | string | yes | Graph identifier; unique within a registry. | -| `state` | string | yes | Path to Pydantic State class. | -| `nodes` | list | yes | Ordered (or DAG) list of node definitions. | -| `rules` | list | no | Bosun rule packs to mount. | -| `governance` | list | no | Governance packs (budgets, audit, policy). | -| `stores` | map | no | Store-tier provider mounts. | -| `checkpoints` | map | no | Checkpoint policy. | -| `triggers` | list | no | External run initiators. | -| `plugins` | list | no | Stargraph plugins this graph depends on. | +| `ir_version` | string | yes | `MAJOR.MINOR.PATCH` (e.g. `"1.0.0"`). Major divergence from the build's IR version is rejected. | +| `id` | string | yes | Document identifier (e.g. `"graph:triage"`). | +| `nodes` | list | yes | Graph nodes (`NodeSpec`). | +| `rules` | list | no | Top-level rule definitions (`RuleSpec`). | +| `tools` | list | no | Tool references (`ToolRef`). | +| `skills` | list | no | Skill references (`SkillRef`). | +| `stores` | list | no | Store bindings (`StoreRef`). | +| `state_schema` | map | no | Flat `name -> type-string` map. Mutually exclusive with `state_class`. | +| `state_class` | string | no | `module.path:ClassName` of an existing Pydantic model. Mutually exclusive with `state_schema`. | +| `parallel` | list | no | Top-level parallel/join declarations (`ParallelBlock`). | +| `governance` | list | no | Mounted Bosun packs (`PackMount`). | +| `migrate` | list | no | Hash-to-hash migration descriptors for resume (`MigrateBlock`). | + +A minimal valid document needs only `ir_version`, `id`, and `nodes`; every other section defaults to an empty list/dict. -## name +## id -A graph identifier. Must be unique in the registry that hosts it. +A free-form document identifier. Convention is a `graph:<slug>` form. ```yaml -name: research +id: "graph:research" ``` -Conventions: lowercase, kebab-case, no version suffix (versions are tracked by graph hash and registry tags). +## state_schema / state_class -## state +State is Pydantic-typed. Declare it one of two mutually-exclusive ways: -Module path with class name, separated by `:`. +```yaml +# flat primitive map: field name -> type string +state_schema: + message: "str" + severity: "int" +``` ```yaml -state: ./state.py:State +# OR reference an existing Pydantic model +state_class: "graph.state:RunState" ``` -The path is resolved relative to `stargraph.yaml`. The class must subclass `pydantic.BaseModel`. See `references/state-schema.md`. +The two are mutually exclusive (resolved at `Graph` construction). See `references/state-schema.md`. ## nodes -List of node definitions. Each entry has `name` and `type`; additional keys depend on the type. +List of `NodeSpec`. Each entry has `id` and `kind`; builtins may carry a `config` block. ```yaml nodes: - - name: plan - type: dspy:ChainOfThought - signature: ./signatures.py:PlanSignature - - - name: search - type: tool:browser.search - args: - max_results: 10 + - id: ingest + kind: "graph.nodes:IngestAlert" # custom node: module.path:ClassName + + - id: risk_score + kind: ml # builtin factory key + config: + model_id: soc-severity + version: "1.0.0" + runtime: onnx + expected_sha256: "c314b6f6…" + input_field: features + output_field: risk + + - id: triage_decide + kind: dspy + + - id: halt + kind: echo +``` - - name: classify - type: model:onnx:./models/intent.onnx +`kind` is either a **builtin factory key** or a `module.path:ClassName` reference to a custom node. - - name: retrieve - type: retrieval:vector - k: 8 +| Builtin `kind` | Purpose | +|---|---| +| `echo` | Pass state through unchanged. | +| `halt` | Terminal node. | +| `passthrough` | Pure dispatch point (no side effect); governance rules fire on it. | +| `dspy` | LLM-backed DSPy node. | +| `ml` | Direct ML model node (e.g. ONNX, sha256-pinned). | +| `interrupt` | HITL pause node (see `references/hitl-patterns.md`). | +| `human_input` | Human-input node. | +| `retrieval` | Retrieval over a mounted store. | +| `subgraph` | Embed another IR document as a node. | +| `write_artifact` | Write a run artifact. | + +Tool-call nodes use `kind: tool` and reference the tool by its registry key: - - name: answer - type: subgraph:synthesize +```yaml + - id: ask_broker + kind: tool + tool: nautilus.broker_request@1 + inputs: + agent_id: "agent-42" + intent: "{{state.user_intent}}" + out: broker_reply ``` -Type formats: - -| Format | Meaning | -|---|---| -| `dspy:<Module>` | DSPy module class (e.g. `Predict`, `ChainOfThought`, `ReAct`). | -| `model:<format>:<id>` | Direct ML model: `onnx`, `transformers`, `gguf`, etc. `<id>` is a path or hub identifier. | -| `tool:<namespace>.<name>` | Registered tool. Namespace is the plugin; name is the tool. | -| `retrieval:<store>` | Retrieval over a mounted store tier (`vector`, `graph`, `doc`). | -| `subgraph:<name>` | Embed another graph by name. | +Node ids must match the slug regex `^[a-z0-9][a-z0-9_\-.]{0,127}$`. ## rules -List of Bosun rule pack mounts. Each entry is a `{pack: <spec>}` mapping. +A `RuleSpec` is `{id, when, then}`. `when` is a CLIPS-pattern condition string; `then` is a list of discriminated-union **actions** (no nesting). ```yaml rules: - - pack: bosun:routing/research - - pack: bosun:safety/pii@1.2.0 - - pack: ./packs/local-rules + - id: r-ingest-to-retrieval + when: "?n <- (node-id (id ingest))" + then: [{ kind: goto, target: retrieval }] + + - id: rule.escalate + when: "(severity ?s&:(>= ?s 4))" + then: + - { kind: goto, target: triage_decide } ``` -Pack specs: +Action kinds (discriminated on `kind`): + +| `kind` | Notable fields | +|---|---| +| `goto` | `target` | +| `halt` | `reason` (default `""`) | +| `parallel` | `targets`, `join`, `strategy` (`all`/`any`/`race`/`quorum`) | +| `retry` | `target`, `backoff_ms` | +| `assert` | `fact`, `slots` (JSON-encoded slot dict) | +| `retract` | `pattern` | +| `interrupt` | `prompt`, `interrupt_payload`, `requested_capability`, `timeout`, `on_timeout` | -- `bosun:<group>/<name>` — registry-resolved, latest. -- `bosun:<group>/<name>@<version>` — pinned version. -- `./relative/path` — local pack. +There are **no explicit edges**. Routing is static fall-through (the engine walks `nodes` in declaration order when no rule fires) plus rule-driven `goto`. ## governance -Same shape as `rules`, but mounted into the governance phase (runs before/after every node, not as routing). +Mount Bosun packs as `PackMount` entries: `{id, version, requires}`. `requires` is a `PackRequires` compat block checked at load (`check_pack_compat` raises `PackCompatError` on mismatch). ```yaml governance: - - pack: bosun:budgets - - pack: bosun:audit - - pack: bosun:policy/no-pii-egress + - id: stargraph.bosun.budgets + version: "1.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } + - id: stargraph.bosun.audit + version: "1.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } + - id: soc-policy + version: "1.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } ``` +Pack ids are slug form with a separate `version` field. See `references/bosun-packs.md`. + ## stores -Map of tier name to `<provider>:<config>`. +`StoreRef` is `{name, provider}`. `to_capabilities()` derives `["db.{name}:read", "db.{name}:write"]`. ```yaml stores: - vector: lancedb:./.lance - graph: kuzu:./.kuzu - doc: sqlite:./.docs - memory: sqlite:./.memory - fact: sqlite:./.facts -``` - -Tier names: `vector`, `graph`, `doc`, `memory`, `fact`. Any omitted tier uses its embedded default. See `references/store-protocols.md`. - -## checkpoints - -```yaml -checkpoints: - every: node-exit # node-exit | rule-fire | manual - store: sqlite:./.checkpoints - retain: 100 # optional: keep last N + - { name: "kb", provider: "stargraph.stores.lancedb" } + - { name: "facts", provider: "stargraph.stores.sqlite_fact" } ``` -`every` controls cadence: +See `references/store-protocols.md` for the five protocols and real provider ids. -- `node-exit` — snapshot after every node returns (default). -- `rule-fire` — also snapshot after each Bosun rule activation. -- `manual` — only when a node calls `stargraph.checkpoint()` explicitly. +## migrate -`store` is a checkpoint provider URI (typically SQLite). - -## triggers +`graph_hash` is the canonical IR hash (`dumps_canonical`, sorted keys). Resume rejects on hash mismatch unless a `migrate` block maps `from_hash -> to_hash`. ```yaml -triggers: - - type: cron - schedule: "0 */6 * * *" - payload: - query: "weekly digest" - - - type: webhook - path: /research - auth: token - - - type: file_watch - glob: "./inbox/*.json" - - - type: mcp - method: research/start - - - type: manual +migrate: + - { from_hash: "<old>", to_hash: "<new>" } ``` -Each trigger initiates a run with provided or constructed initial state. `manual` is implicit if no triggers are declared. +## Validating a graph -## plugins +There is no `graph verify` subcommand. Validate by loading: -Entry-point names of installed Stargraph plugins this graph depends on. Used at load time to verify the environment. +```bash +# rule-firing trace, no node execution +stargraph run graphs/triage.yaml --inspect -```yaml -plugins: - - stargraph-browser - - stargraph-bosun - - stargraph-pinecone +# offline trace against synthetic node outputs +stargraph simulate graphs/triage.yaml --fixtures fixtures/triage.yaml ``` -Missing a declared plugin is a load-time error. +In Python: `stargraph.ir.validate(ir)` returns structured `ValidationError`s. ## Full Example -A complete Research graph: +A SOC-triage graph (modeled on `demos/soc-triage/graph/stargraph.yaml`): ```yaml -name: research -state: ./state.py:ResearchState +ir_version: "1.0.0" +id: "graph:soc-triage" -nodes: - - name: plan - type: dspy:ChainOfThought - signature: ./signatures.py:PlanSignature - - - name: search - type: tool:browser.search - args: - max_results: 10 - - - name: rerank - type: model:onnx:./models/reranker.onnx - - - name: synthesize - type: dspy:ChainOfThought - signature: ./signatures.py:SynthesizeSignature +state_class: "graph.state:RunState" -rules: - - pack: bosun:routing/research - - pack: bosun:quality/citation-required +nodes: + - id: ingest + kind: "graph.nodes:IngestAlert" + - id: retrieval + kind: "graph.nodes:RetrievalPriors" + - id: risk_score + kind: ml + config: + model_id: soc-severity + version: "1.0.0" + runtime: onnx + expected_sha256: "c314b6f6…" + input_field: features + output_field: risk + - id: triage_decide + kind: dspy + - id: soc_policy + kind: passthrough + - id: analyst_gate + kind: interrupt + config: + prompt: "Approve disposition {disposition} for alert {alert_id}?" + requested_capability: "runs:respond" + timeout: "PT900S" + on_timeout: "halt" + - id: write_artifact + kind: "graph.nodes:SocWriteArtifact" + - id: audit + kind: "graph.nodes:AuditChain" + - id: halt + kind: echo governance: - - pack: bosun:budgets - - pack: bosun:audit - - pack: bosun:policy/no-pii-egress + - id: stargraph.bosun.budgets + version: "1.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } + - id: soc-policy + version: "1.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } -stores: - vector: lancedb:./.lance - graph: kuzu:./.kuzu - doc: sqlite:./.docs - -checkpoints: - every: node-exit - store: sqlite:./.checkpoints - retain: 50 - -triggers: - - type: cron - schedule: "0 9 * * MON" - payload: - query: "weekly research digest" - - type: webhook - path: /research - auth: token - - type: manual - -plugins: - - stargraph-browser - - stargraph-bosun +rules: + - id: r-ingest-to-retrieval + when: "?n <- (node-id (id ingest))" + then: [{ kind: goto, target: retrieval }] + - id: r-policy-escalate-hitl + when: "?n <- (node-id (id soc_policy)) (state (disposition escalate))" + then: [{ kind: goto, target: analyst_gate }] + - id: r-halt + when: "?n <- (node-id (id halt))" + then: [{ kind: halt, reason: "run complete" }] ``` -This graph plans, searches, reranks, and synthesizes; mounts routing and quality rule packs; enforces budgets, audit, and a no-PII-egress policy; persists to embedded stores; checkpoints at every node-exit retaining the last 50; and accepts cron, webhook, and manual triggers. +This graph ingests an alert, retrieves priors, scores risk with a pinned ONNX model, decides a disposition with a DSPy node, applies the `soc-policy` Bosun pack at a passthrough dispatch point, routes escalations through a HITL interrupt gate, writes a case-note artifact, and seals a hash-chained audit record before halting. diff --git a/plugins/stargraph/references/hitl-patterns.md b/plugins/stargraph/references/hitl-patterns.md index 3ddefb8..a64397f 100644 --- a/plugins/stargraph/references/hitl-patterns.md +++ b/plugins/stargraph/references/hitl-patterns.md @@ -4,116 +4,122 @@ How a Stargraph run pauses for human input or approval, and how to resume it. ## When a run pauses -A run enters `PAUSED_HITL` when one of these happens: +A run enters `awaiting-input` when an **interrupt** fires. Dispatch happens on +`Action.kind == "interrupt"` (or an `interrupt` node) **before** routing is +translated — it is a control-flow primitive, not a routing decision. The runtime +checkpoints, marks the run awaiting input, and emits a `WaitingForInputEvent` +carrying the prompt, the `interrupt_payload`, and the `requested_capability`. -1. **`human_input` node** — a node whose `type: stargraph.nodes.human_input` - produces no value on its own; it just declares an `expected_input_schema` - and a `prompt`. The runtime checkpoints, marks the run paused, and emits - a `(hitl.required ...)` fact. -2. **Governance halt** — a Bosun rule emits `halt: true, reason: hitl.<…>`. - Common causes: budget cap, safety violation, policy gate. -3. **Approval gate** — a transition rule fires `pause_for_approval: <node>` - instead of `goto`. The next node won't execute until a `respond` arrives. - -In all three cases the run stays resumable from its last checkpoint until the -operator either responds or cancels. +The run stays resumable from its last checkpoint until the operator either +responds or the wait times out. ## Authoring a pause point -### `human_input` node (preferred) +### `interrupt` node ```yaml nodes: - - name: confirm_action - type: stargraph.nodes.human_input - prompt: "About to delete {{state.target}}. Confirm?" - expected_input_schema: - type: object - required: [confirmed] - properties: - confirmed: {type: boolean} - reason: {type: string} - timeout_seconds: 3600 - on_timeout: halt # or: continue (with default), goto:<node> + - id: analyst_gate + kind: interrupt + config: + prompt: "Approve disposition {disposition} for alert {alert_id}?" + requested_capability: "runs:respond" + interrupt_payload: + requested_capability: "runs:respond" + timeout: "PT900S" # ISO-8601 duration; null = no timeout + on_timeout: "halt" # "halt" (terminal) or "goto:<node_id>" ``` -The `prompt` is interpolated against state at pause time. The schema is -served to the responder UI / CLI so payloads can be validated before submit. +The `prompt` is interpolated against state at pause time and surfaced on the +`WaitingForInputEvent`. + +### `interrupt` rule action -### Governance pause +The same primitive is available as a `RuleSpec.then` action, so a routing rule +can pause the run when a condition matches: ```yaml -# in a Bosun pack -- name: require-approval-on-prod-write - when: - - { template: tool.target.env, op: eq, value: prod } - - { template: tool.permissions, op: contains, value: write } - then: - pause_for_approval: ${last_node} - reason: prod-write-needs-approval - audience: ops +rules: + - id: r-analyst-gate + when: "?n <- (node-id (id analyst_gate))" + then: + - kind: interrupt + prompt: "Approve disposition {disposition} for alert {alert_id}?" + interrupt_payload: + requested_capability: "runs:respond" + requested_capability: "runs:respond" + timeout: null + on_timeout: "halt" ``` -`audience` is a free-form tag the responder UI can filter on (`ops`, `legal`, -`compliance`, …). It's also written into the `(hitl.required …)` fact so -downstream queries can find what's pending and for whom. +| Field | Type | Default | Purpose | +|---|---|---|---| +| `prompt` | `str` | required | Operator-facing prompt on the wait event. | +| `interrupt_payload` | `dict` | `{}` | Free-form payload echoed on the wait event. | +| `requested_capability` | `str \| None` | `None` | Capability gate for `POST /v1/runs/{id}/respond`. | +| `timeout` | `timedelta \| None` | `None` | Wait bound; `None` means no timeout. | +| `on_timeout` | `"halt" \| "goto:<node_id>"` | `"halt"` | Terminal halt, or resume at a node. | ## Responding -Three decision shapes: +Resume a paused run by delivering a response to its `respond` endpoint. The +caller must hold the interrupt's `requested_capability` (e.g. `runs:respond`). -| Decision | When to use | Resume behavior | -|---|---|---| -| `approve` | Approval-gate / `human_input` boolean confirms. | Run continues from the next transition. | -| `deny` | Reject the action. | Halt with reason; emits `(hitl.denied …)`. | -| `input` | `human_input` node expects structured data. | Payload becomes the node's output state slice. | +### `stargraph respond` (CLI) -Submit via `/stargraph:respond` or directly: +A thin wrapper over `POST /v1/runs/{run_id}/respond` on a running +`stargraph serve` process: ```bash -curl -fsS -X POST "${STARGRAPH_URL}/v1/runs/${RID}/respond" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" \ - -H "Content-Type: application/json" \ - -d '{"decision":"input","payload":{"confirmed":true,"reason":"verified ticket #123"}}' +stargraph respond <RUN_ID> --response analyst-decision.json --actor alice ``` -`payload` is validated against `expected_input_schema` server-side; bad -payloads return `422` with the failing JSON-pointer. +| Flag | Required | Description | +|---|---|---| +| `RUN_ID` | yes | Run id that is `awaiting-input`. | +| `--response FILE` | yes | JSON file with the analyst response payload. | +| `--actor NAME` | yes | Principal id; sent as `Authorization: Bypass <actor>`. | +| `--server URL` | no | Base URL of the serve process (default `http://localhost:8000`). | -## Audit fact +The CLI maps HTTP errors to operator-friendly messages: `200` prints the +`RunSummary` JSON; `401` = auth failed for actor; `404` = run not found or not +awaiting input; `409` = run not awaiting input (already responded or in a +conflicting state). -Every response emits a `(human_response …)` fact carrying: +### Direct HTTP +```bash +curl -fsS -X POST "http://localhost:8000/v1/runs/${RID}/respond" \ + -H "Authorization: Bypass alice" \ + -H "Content-Type: application/json" \ + -d @analyst-decision.json ``` -(human_response - decision approve|deny|input - payload <jsonb> ; null for approve/deny without data - reason <string> ; required for deny - by <responder_id> ; from auth token's `sub` claim - ts <iso8601> - origin user ; provenance - source hitl-driver ; or whichever responder client - run_id <run_id> - step <step>) -``` -Bosun `audit` pack signs these facts so the resume action is non-repudiable. +## Audit trail + +The response is sealed into the run's provenance trail. Provenance carries the +documented origin values (`tool`, `llm`, `rule`, `system`); the responding +actor is recorded from the `Authorization: Bypass <actor>` principal. The +`stargraph.bosun.audit` pack signs transition facts so the resume action is +non-repudiable. See `references/provenance-facts.md`. ## Cleared / air-gapped deployments -- `expected_input_schema` MUST be present — payloads without a schema are - rejected outright in the `cleared` profile. -- Every responder authenticates via JWT; `sub` claim becomes the `by` slot. -- `respond` calls write to the JSONL audit log even before the run resumes, - so the operator action is recorded even if the resume fails downstream. +- Every responder is identified by the `--actor` principal, which becomes the + recorded actor on the response. +- The serve process writes to the JSONL audit log (see `stargraph serve + --audit-log`), so the operator action is recorded. +- Inspect what a paused run was waiting on, and the state at the pause, with + `stargraph inspect RUN_ID --db DB --step N`. ## Common patterns -- **Two-key approval**: emit two `pause_for_approval` rules requiring - distinct `audience` tags. The second `respond` call resumes; the first - flips a `(hitl.first_key …)` fact. -- **Deferred input**: `human_input` with `timeout_seconds: 86400` for slow - human review (overnight ops). Pair with `bosun:budgets` so wall-time - doesn't blow the run budget. -- **Inline rationale**: require `reason` in the schema for any `deny` — - audit captures *why*, not just *that*. +- **Finite wait for hot-resume**: set a finite `timeout` (e.g. `PT900S`) so the + serve loop takes its hot-resume path — `POST /v1/runs/{id}/respond` wakes the + same live loop, which advances past the gate. `on_timeout: "halt"` keeps an + unanswered gate terminal, not hung. +- **Resume-to-node**: `on_timeout: "goto:<node_id>"` resumes at a fallback node + instead of halting when the operator never responds. +- **Capability-scoped approvals**: set `requested_capability` so only principals + holding that capability can respond — the gate runs on the `respond` endpoint + before the run resumes. diff --git a/plugins/stargraph/references/provenance-facts.md b/plugins/stargraph/references/provenance-facts.md index 0538945..0c2ed6f 100644 --- a/plugins/stargraph/references/provenance-facts.md +++ b/plugins/stargraph/references/provenance-facts.md @@ -1,36 +1,45 @@ # Stargraph Provenance-Typed Facts -Every fact asserted in a Stargraph run carries provenance metadata. Rules can pattern-match on it, audit can reconstruct it, and replay can filter on it. - -## The Tuple - -Every fact carries: - -``` -(origin, source, run_id, step, confidence, timestamp) +Every fact asserted in a Stargraph run carries provenance metadata. Rules can +pattern-match on it, audit can reconstruct it, and replay can filter on it. +Provenance is non-negotiable — a run without a complete trace is treated as an +error, not a warning. + +## The Envelope + +Tools and nodes attach a `__stargraph_provenance__` envelope to their outputs: + +```json +{ + "...": "output fields", + "__stargraph_provenance__": { + "origin": "tool", + "source": "nautilus", + "external_id": "<broker request_id>" + } +} ``` | Field | Type | Meaning | |---|---|---| -| `origin` | symbol | Where the fact came from (see Origin Values). | -| `source` | string | Specific emitter — node name, tool name, rule name, user id. | -| `run_id` | string | The run that produced the fact. | -| `step` | int | Monotonic step counter within the run. | -| `confidence` | float | In [0, 1]. See Confidence Convention. | -| `timestamp` | datetime | UTC, ISO-8601. | +| `origin` | string | Where the value came from (see Origin Values). | +| `source` | string | Specific emitter — node name, tool name, rule name, subsystem. | +| `external_id` | string | Optional id correlating to the external system that produced it. | -The runtime stamps these automatically; nodes and rules cannot forge them. +When the envelope is folded into the fact store, the fact additionally carries +run-scoped metadata — `run_id`, `step`, `confidence`, and a `timestamp` — which +the runtime stamps automatically. Nodes and rules cannot forge them. ## Origin Values +The documented origin values are: + | `origin` | Emitted when | Meaning | |---|---|---| -| `llm` | An LLM-backed node (DSPy module, raw chat) emits a fact derived from model output. | Treat as plausibly-true; verify before acting. | -| `tool` | A tool call returns structured data. | High trust if the tool itself is trusted; carries the tool's own confidence if available. | -| `user` | A human input arrives via API or UI. | Authoritative for the user's domain — typically pinned at confidence 1.0. | -| `rule` | A Bosun/CLIPS rule asserts a derived fact in its RHS. | Symbolic derivation; confidence inherited from premises (min by default). | -| `model` | A non-LLM ML model (classifier, regressor, embedder) emits a prediction. | Carry the model's calibrated probability as confidence. | -| `external` | A trigger or webhook injects facts at run start. | Trust depends on the trigger source. | +| `tool` | A tool call returns structured data. | High trust if the tool itself is trusted. | +| `llm` | An LLM-backed node (DSPy module, raw chat) emits a value derived from model output. | Treat as plausibly-true; verify before acting. | +| `rule` | A Fathom/CLIPS rule asserts a derived fact in its RHS. | Symbolic derivation from premises. | +| `system` | The runtime or a subsystem (e.g. trigger / scheduler / audit sink) emits the fact. | Runtime-attested. | ## Pattern Matching on Provenance @@ -47,67 +56,64 @@ Only act on tool-origin facts with confidence at least 0.8: (assert (next-node synthesize))) ``` -Halt if a user-origin fact contradicts a rule-origin fact: +Prefer rule-derived facts over raw LLM output for the same key: ```clips -(defrule user-overrides-rule - (claim (subject ?s) (value ?v1) (origin user)) - (claim (subject ?s) (value ?v2&~?v1) (origin rule)) +(defrule prefer-rule-over-llm + ?bad <- (label (key ?k) (origin llm)) + (label (key ?k) (origin rule)) => - (assert (control halt)) - (assert (audit (reason "user contradicts rule") (subject ?s)))) + (retract ?bad)) ``` -Prefer model-origin facts over llm-origin for the same key: +Halt if a system-origin fact contradicts an LLM-origin one: ```clips -(defrule prefer-model-over-llm - ?bad <- (label (key ?k) (origin llm)) - (label (key ?k) (origin model)) +(defrule system-overrides-llm + (claim (subject ?s) (value ?v1) (origin system)) + (claim (subject ?s) (value ?v2&~?v1) (origin llm)) => - (retract ?bad)) + (assert (control halt)) + (assert (audit (reason "system contradicts llm") (subject ?s)))) ``` ## Confidence Convention -`confidence` is a float in `[0, 1]`. Each origin documents its calibration: +`confidence` is a float in `[0, 1]`. Calibration varies by origin: -- `user` — typically 1.0 unless the UI captured uncertainty. -- `tool` — the tool's own confidence if it returns one; else 1.0 for deterministic tools, 0.9 default for retrieval scores normalized to [0, 1]. -- `llm` — DSPy modules emit logprob-derived confidence when available; otherwise the convention is 0.7 default unless the prompt asks the model for a self-reported score. -- `model` — the calibrated probability (must be calibrated, not raw softmax, for downstream rules to mean what they say). -- `rule` — `min` of premise confidences by default; rules may override with explicit aggregation. -- `external` — set by the trigger; webhooks default to 1.0, file watchers to 1.0, MCP-injected facts inherit from the source. +- `tool` — the tool's own confidence if it returns one; else 1.0 for + deterministic tools, normalized retrieval scores for lookups. +- `llm` — logprob-derived confidence when available; otherwise a documented + default unless the prompt asks the model for a self-reported score. +- `rule` — `min` of premise confidences by default; rules may override with an + explicit aggregation. +- `system` — runtime-attested events are typically pinned at 1.0. Sources should document their calibration in their skill or pack README. -## Querying History - -Facts are indexed by `run_id` and `step`. Access: +## What replay captures -CLI: +Every run emits a trace sufficient to replay it: the IR hash, the plugin set +(distribution name, version, `api_version` per plugin), the Fathom decision log +(rule firings in order with fact snapshots), and content-addressed I/O envelopes +for all node inputs and outputs. Facts are indexed by `run_id` and `step`; the +fact log is persisted by the configured `FactStore` provider. -```bash -stargraph facts list --run <run_id> -stargraph facts list --run <run_id> --origin tool --since-step 12 -stargraph facts get --run <run_id> --step 7 -``` +## Inspecting facts -Python: +There is no `facts` subcommand. Inspect a run's facts and state over a SQLite +checkpointer DB with `stargraph inspect`: -```python -from stargraph.client import StargraphClient - -c = StargraphClient() -facts = c.facts.list(run_id="r-abc123", origin="tool", min_confidence=0.8) -for f in facts: - print(f.step, f.source, f.body) -``` +```bash +# CLIPS facts asserted/retracted between step 5 and step 9 +stargraph inspect <RUN_ID> --db .stargraph/run.sqlite --diff 5 9 -REST: +# state snapshot at step 7 +stargraph inspect <RUN_ID> --db .stargraph/run.sqlite --step 7 -``` -GET /v1/runs/{run_id}/facts?origin=tool&min_confidence=0.8&since_step=12 +# timeline (enriched with an audit log) +stargraph inspect <RUN_ID> --db .stargraph/run.sqlite --log-file run.jsonl ``` -The fact log is append-only and persisted by the configured FactStore provider. Replays consume this log to reconstruct any prior CLIPS state. +The fact-diff view (`--diff N M`) prints the CLIPS fact delta between two steps; +the state view (`--step N`) prints the IR-canonical state dict at that step. diff --git a/plugins/stargraph/references/stargraph-concepts.md b/plugins/stargraph/references/stargraph-concepts.md index 5a6756b..1f87ebf 100644 --- a/plugins/stargraph/references/stargraph-concepts.md +++ b/plugins/stargraph/references/stargraph-concepts.md @@ -25,7 +25,7 @@ | **Provider** | A concrete implementation of a Store Protocol (e.g., LanceDB is a vector provider). | | **Checkpoint** | A persisted snapshot at a transition: state, facts, last node, next action, graph hash. | | **Graph hash** | Structural fingerprint: topology + node signatures + state schema. Used for resume safety. | -| **Trigger** | An external event source that initiates a run: `cron`, `webhook`, `file_watch`, `mcp`, `manual`. | +| **Trigger** | An external event source that initiates a run: `manual`, `cron`, `webhook`. | | **Run history** | The ordered list of checkpoints + events for a run. The basis for replay and counterfactuals. | --- @@ -37,7 +37,7 @@ These pairs are routinely confused. The distinction matters. ### Node vs Tool - A **node** is a step in a graph. Has state in/out semantics. - A **tool** is a callable a node may invoke. May not appear in the graph at all. -- A node *can be* a single tool call (`stargraph.nodes.tool_call`), but most nodes do more. +- A node *can be* a single tool call (a node of `kind: tool`), but most nodes do more. ### Skill vs Plugin - A **skill** is a logical bundle of capability (e.g., "research"). @@ -82,11 +82,11 @@ These pairs are routinely confused. The distinction matters. ## Naming conventions -- **Tool names:** `namespace.name` (e.g., `web.search`, `kraken.servicenow.create_incident`) -- **Skill names:** lowercase, hyphenated, plugin-namespaced (`research-agent`, `kraken/triage`) -- **Pack names:** `vendor:pack@version` (e.g., `bosun:budgets@1.2`) +- **Tool registry key:** `namespace.name@version` (e.g., `nautilus.broker_request@1`). This is the canonical form a graph references — not bare `namespace.name`. +- **Skill names:** lowercase, hyphenated; the computed `site_id` is `name@version`. +- **Pack ids:** slug form with a separate `version` field (e.g., `stargraph.bosun.budgets` at version `1.0`, `soc-policy`). Not `vendor:pack@version`. - **Fact templates:** `stargraph.*` reserved for runtime; `bosun.*` for governance; `user.*` for application-defined -- **Run IDs:** `r-` prefix + 6-char base32 +- **Run IDs:** UUIDv7 strings (sortable by creation time); counterfactual forks are `cf-<uuid>` --- diff --git a/plugins/stargraph/references/state-schema.md b/plugins/stargraph/references/state-schema.md index 0103dc7..e1db639 100644 --- a/plugins/stargraph/references/state-schema.md +++ b/plugins/stargraph/references/state-schema.md @@ -2,6 +2,27 @@ State is the typed bundle that flows through every node of a Stargraph graph. It is the source of truth — facts in CLIPS are merely a projection of annotated state at node boundaries. +## Declaring state in the IR + +An `IRDocument` declares state one of two mutually-exclusive ways: + +- `state_class: "module.path:ClassName"` — reference an existing Pydantic model. +- `state_schema:` — a flat `name -> type-string` map for simple primitive state. + +```yaml +# stargraph.yaml — reference a Pydantic model +state_class: "graph.state:RunState" +``` + +```yaml +# OR a flat primitive map +state_schema: + message: "str" + severity: "int" +``` + +The two are mutually exclusive (resolved at `Graph` construction, not in IR validation). + ## Pydantic Foundation State is a Pydantic `BaseModel`. Nodes receive State as input and return a (possibly mutated) State as output. Inside the node body, mutate freely in plain Python — Stargraph only cares about the boundary. @@ -26,7 +47,7 @@ Fields wrapped in `Annotated[<type>, Mirror()]` are mirrored to CLIPS at the nod ```python from typing import Annotated from pydantic import BaseModel, Field -from stargraph.annotations import Mirror +from stargraph.ir import Mirror class State(BaseModel): query: Annotated[str, Mirror()] @@ -35,7 +56,7 @@ class State(BaseModel): raw_embeddings: list[float] = Field(default_factory=list) # NOT mirrored ``` -`Mirror()` accepts options: `Mirror(name="...")` to override the CLIPS slot name, `Mirror(template="...")` to target a specific deftemplate. +`Mirror` is a frozen marker appended to a field's `Annotated[...]` chain. `Mirror(template="...")` overrides the target CLIPS deftemplate (default: the field name); `Mirror(lifecycle=...)` tags the sync boundary as `"run"`, `"step"`, or `"pinned"`. ## Type Compatibility @@ -97,7 +118,11 @@ graph_hash = sha256(topology + node_signatures + state_schema_hash) `state_schema_hash` is computed from the JSON Schema of the State model (stable field ordering). Adding a non-mirrored field with a default does not invalidate the hash if the JSON Schema is unchanged. Renaming a mirrored field, changing a type, or removing a field changes the hash. -Checkpoints record the graph hash. `stargraph replay` and `stargraph run --resume` reject mismatches unless the graph declares a `migrate:` block mapping old → new fields. +Checkpoints record the graph hash. Resume rejects a hash mismatch unless the graph declares a `migrate:` block mapping `from_hash → to_hash`. + +## Replay-Safe Collections + +State fields must use hashable, immutable collections so replay is deterministic. A field typed as `set` or `set[X]` (including a nested `set`) is **rejected** at construction time — use `frozenset` instead. When state is a skill's `state_schema`, its field names double as the declared output channels (the write-whitelist the engine enforces at the subgraph boundary); see `references/store-protocols.md` and the skill model for detail. ## Worked Example @@ -107,7 +132,7 @@ A small Research graph state: from typing import Annotated, Literal from datetime import datetime from pydantic import BaseModel, Field -from stargraph.annotations import Mirror +from stargraph.ir import Mirror class Citation(BaseModel): url: Annotated[str, Mirror()] diff --git a/plugins/stargraph/references/store-protocols.md b/plugins/stargraph/references/store-protocols.md index 6c862d6..ac6cc05 100644 --- a/plugins/stargraph/references/store-protocols.md +++ b/plugins/stargraph/references/store-protocols.md @@ -1,6 +1,8 @@ # Stargraph Store Protocols -Stargraph abstracts data tiers behind five Protocols. Plugins register concrete providers via entry points; graphs declare which provider to mount per tier in `stargraph.yaml`. Embedded providers are the default so a fresh `stargraph run` works air-gapped without external services. +Stargraph abstracts data tiers behind five Protocols: `vector`, `graph`, `doc`, `memory`, `fact`. Plugins register concrete providers via entry points; graphs declare which provider to mount per tier in the graph IR (`stargraph.yaml`). Embedded providers are the default so a fresh `stargraph run` works air-gapped without external services. + +Every Protocol exposes a uniform lifecycle: `bootstrap()` (idempotent schema install), `health() -> StoreHealth`, and `migrate(plan)` (v1 supports `add_column` only). Embedded providers serialize writes through a single-writer-per-path lock. ## VectorStore @@ -15,7 +17,7 @@ class VectorStore(Protocol): `Hit` is `(id, score, metadata)`. `filter` is a provider-specific predicate language; portable filters use a small subset (`{"key": "value"}` exact match, `{"key": {"$gt": n}}` range). -Default provider: **LanceDB** (`lancedb:./.lance`). External: Pinecone, Weaviate, Qdrant. +Default provider: **`LanceDBVectorStore`** (provider id `stargraph.stores.lancedb`, wired as `lancedb:./.lance`). External: Pinecone, Weaviate, Qdrant. ## GraphStore @@ -29,7 +31,7 @@ class GraphStore(Protocol): def cypher(self, query: str, params: dict | None = None) -> list[dict]: ... ``` -Default provider: **Kuzu** (`kuzu:./.kuzu`). External: Neo4j, Memgraph. +Default provider: **`RyuGraphStore`** (provider id `stargraph.stores.ryugraph` / `stargraph.stores.cypher`, wired as `ryugraph:./.ryu`). RyuGraph is a Kuzu fork; queries use a portable Cypher subset (checked by `Linter`). External: Neo4j, Memgraph. ## DocStore @@ -44,7 +46,7 @@ class DocStore(Protocol): `doc` is any JSON-serializable mapping. `search` runs FTS on text fields the provider has indexed. -Default provider: **SQLite + FTS5** (`sqlite:./.docs`). External: Elasticsearch, OpenSearch, Postgres + tsvector. +Default provider: **`SQLiteDocStore`** (provider id `stargraph.stores.sqlite_doc`, wired as `sqlite:./.docs`, SQLite WAL). External: Elasticsearch, OpenSearch, Postgres + tsvector. ## MemoryStore @@ -59,51 +61,56 @@ class MemoryStore(Protocol): `ttl` in seconds. Values are JSON-serializable. Keys are scoped per run by default; cross-run keys require an explicit prefix. -Default provider: **SQLite** (`sqlite:./.memory`). External: Redis, Memcached. +Default provider: **`SQLiteMemoryStore`** (provider id `stargraph.stores.sqlite_memory`, wired as `sqlite:./.memory`, SQLite WAL). External: Redis, Memcached. ## FactStore -Provenance-typed fact log. Methods: +Semantic-fact storage, keyed at `(user, agent)` and session-independent. Methods: ```python class FactStore(Protocol): - def assert_fact(self, fact: Fact, provenance: Provenance) -> None: ... - def query(self, pattern: dict) -> list[Fact]: ... - def retract(self, fact: Fact) -> None: ... + async def pin(self, fact: Fact) -> None: ... + async def query(self, pattern: FactPattern) -> list[Fact]: ... + async def unpin(self, fact_id: str) -> None: ... ``` -Note: `assert` is a Python keyword, so the method is `assert_fact`. `pattern` is a structured matcher: `{"template": "citation", "origin": "tool", "min_confidence": 0.8}`. +`pin` is insert-or-replace by `fact.id`. A `Fact` carries `id`, `user`, `agent`, `payload`, a **mandatory** `lineage` (each entry traces back to originating episode/triple ids or rule firings), `confidence`, `pinned_at`, and `metadata`. `FactPattern` matches on `subject`/`predicate`/`object` slots plus the `user`/`agent` columns; a `None` slot is a wildcard. -Default provider: **in-memory + SQLite persist** (`sqlite:./.facts`). The in-memory tier is the live CLIPS environment; the SQLite tier is the append-only audit log used for replay. +Default provider: **`SQLiteFactStore`** (provider id `stargraph.stores.sqlite_fact`, wired as `sqlite:./.facts`, SQLite WAL + the `FathomAdapter`). The `apply_delta` provider extension is the lineage seam used to promote consolidated memory deltas into pinned facts. -## Provider Registration +## Registration: StoreSpec and StoreRef -Providers register via Python entry points under the `stargraph.stores.<type>` group: +A provider plugin registers via the `stargraph.stores` entry-point group plus a `register_stores()` hook returning `list[StoreSpec]`: ```toml -# pyproject.toml of a stargraph-pinecone plugin -[project.entry-points."stargraph.stores.vector"] -pinecone = "stargraph_pinecone:PineconeProvider" +# pyproject.toml of a store-providing plugin +[project.entry-points."stargraph.stores"] +my_vector = "my_pkg._plugin:manifest" ``` -A graph then mounts it: +`StoreSpec` is the canonical registration record: `{name, provider, protocol, config_schema, capabilities}`. `protocol` is one of `vector`/`graph`/`doc`/`memory`/`fact`; `config_schema` is the JSON Schema for the provider's config; an empty `capabilities` list defaults to the `db.{name}:read` / `db.{name}:write` pair. + +Inside an `IRDocument`, a graph mounts a store as a lightweight **`StoreRef`** — `{name, provider}`: ```yaml stores: - vector: pinecone:my-index?env=production + - { name: "kb", provider: "stargraph.stores.lancedb" } + - { name: "facts", provider: "stargraph.stores.sqlite_fact" } ``` -The string after the provider name is provider-specific config (URI, query string, or JSON depending on the provider). +`StoreRef.to_capabilities()` returns `["db.{name}:read", "db.{name}:write"]` — the capability strings Bosun's policy gates check. + +The compact `<kind>: <provider>:<path>` form (e.g. `vector: lancedb:./.lance`) is the YAML shorthand the runtime parses into a `StoreRef`. ## Embedded vs External | Tier | Embedded default | External options | |---|---|---| -| Vector | LanceDB | Pinecone, Weaviate, Qdrant | -| Graph | Kuzu | Neo4j, Memgraph | -| Doc | SQLite + FTS5 | Elasticsearch, Postgres + tsvector | -| Memory | SQLite | Redis, Memcached | -| Fact | SQLite + in-memory CLIPS | Postgres | +| Vector | LanceDB (`stargraph.stores.lancedb`) | Pinecone, Weaviate, Qdrant | +| Graph | RyuGraph (`stargraph.stores.ryugraph` / `…cypher`) | Neo4j, Memgraph | +| Doc | SQLite WAL (`stargraph.stores.sqlite_doc`) | Elasticsearch, Postgres + tsvector | +| Memory | SQLite WAL (`stargraph.stores.sqlite_memory`) | Redis, Memcached | +| Fact | SQLite WAL + FathomAdapter (`stargraph.stores.sqlite_fact`) | Postgres | Embedded providers are the default because: diff --git a/plugins/stargraph/references/triggers.md b/plugins/stargraph/references/triggers.md index 6061325..1c9bdc5 100644 --- a/plugins/stargraph/references/triggers.md +++ b/plugins/stargraph/references/triggers.md @@ -1,94 +1,148 @@ # Stargraph Triggers -Reference for trigger plugins: how runs get initiated by external events. +Reference for triggers: how runs get initiated by external events. + +Triggers are pluggy plugins that emit `TriggerEvent` objects into the scheduler +queue. They are authored in `triggers.yaml` under the runtime config dir +(`~/.config/stargraph/triggers.yaml`, overridable via `STARGRAPH_CONFIG_DIR`) and +loaded at `stargraph serve` startup. ## Built-in trigger types -| Type | Use when | Plugin entry-point | +There are exactly three built-ins: + +| Type | Use when | Module | |---|---|---| -| `manual` | Default. Runs kicked off via `POST /v1/runs` (or `stargraph run`). | `stargraph.triggers.manual:ManualTrigger` | -| `cron` | Scheduled, recurring runs (digests, sweeps, polls). | `stargraph.triggers.cron:CronTrigger` | -| `webhook` | External system push (GitHub, Linear, Slack, …). | `stargraph.triggers.webhook:WebhookTrigger` | +| `manual` | Default. Runs kicked off via `stargraph run` or `POST /v1/runs`. | `stargraph.triggers.manual` | +| `cron` | Scheduled, recurring runs (digests, sweeps, polls). | `stargraph.triggers.cron` | +| `webhook` | External system push (GitHub, Linear, Slack, …). | `stargraph.triggers.webhook` | -Other trigger types (`mcp`, `file_watch`) are reserved for future plugins — -ship them as separate distributions when needed. +(`mcp_adapters` is a plugin group, not a trigger kind. There is no `file_watch` +built-in.) -## Cron triggers +## triggers.yaml schema -Stargraph's scheduler uses `cronsim` for DST-safe expressions. 5-field cron only. +A top-level `version: "1.0"` followed by per-kind lists: ```yaml -triggers: - - name: nightly_research - type: cron - cron: "0 3 * * *" # 03:00 every day - timezone: UTC # default; or America/Los_Angeles - dedup_key: research:nightly - jitter_seconds: 60 # optional; spreads cluster-wide load - input: +version: "1.0" + +manual: + - id: research-adhoc + graph_id: research + description: "Ad-hoc research run" + +cron: + - id: cron:nightly-research + graph_id: research + expr: "0 3 * * *" # standard 5-field cron + tz: UTC # IANA name; resolved at init + missed_fire_policy: fire_once_catchup # or: skip + params: query: "weekly digest" + +webhook: + - id: webhook:github-pr + graph_id: pr_triage + path: /triggers/github # must start with /; mounted by the webhook trigger + timestamp_window_seconds: 300 + nonce_lru_size: 10000 + current_secret_env: STARGRAPH_WEBHOOK_SECRET_CURRENT + previous_secret_env: STARGRAPH_WEBHOOK_SECRET_PREVIOUS ``` -**Per-graph capacity** is enforced by `anyio.CapacityLimiter` honoring the -graph IR's `concurrency`. Two firings in the same minute won't trample each -other; the second waits or is dropped depending on `concurrency` policy. +## Cron triggers + +The cron trigger uses `cronsim.CronSim` for DST-safe expressions (5-field cron +only). On `start` it spawns one background `asyncio.Task` per spec that computes +`next_fire`, sleeps until then, derives the idempotency key, and enqueues. -**Idempotency**: `dedup_key` is BLAKE3-keyed by the scheduler. Re-firings -with the same key inside a graph's `dedup_window` are coalesced. +- `tz` is an IANA timezone name (e.g. `UTC`, `America/New_York`), resolved at + init — bad config fails fast. +- `missed_fire_policy`: `fire_once_catchup` (default) fires once for the most + recent missed scheduled time so the idempotency key matches a never-down + system; `skip` jumps straight to the next future fire. +- **Idempotency key**: `sha256(trigger_id || scheduled_fire.isoformat())`. The + ISO format includes the tz offset, so the same wall-clock instant in different + zones produces distinct keys. ## Webhook triggers -```yaml -triggers: - - name: github_pr - type: webhook - path: /hooks/github-pr - secret_env: STARGRAPH_HOOK_GITHUB_PR # HMAC verify on inbound POST - method: POST # default - dedup_key: gh:${headers.x-github-delivery} - input_template: # JSON-pointer mapping - pr_number: /pull_request/number - repo: /repository/full_name - action: /action -``` - -- `${STARGRAPH_URL}/hooks/github-pr` returns `202 {run_id}` on accept, - `401` on bad HMAC, `409` on dedup hit. -- `secret_env` MUST be set in the deployment env. Cleared profile refuses - to register a webhook trigger missing the secret. -- `input_template` resolves JSON pointers against the request body and a - small `${headers.*}` namespace. Missing pointers are nullable; type - coercion follows the graph's State schema. +The webhook trigger mounts a FastAPI `POST` route per spec and verifies inbound +bodies with a Stripe-style HMAC-SHA256 signature before enqueueing a run. + +- The route is mounted at `path` on the running `stargraph serve` app. +- `current_secret_env` / `previous_secret_env` name the environment variables + holding the HMAC keys (e.g. `STARGRAPH_WEBHOOK_SECRET_CURRENT` / + `STARGRAPH_WEBHOOK_SECRET_PREVIOUS`). `current_secret` is used for both signing + and verification; `previous_secret` is valid for verification only (rotation + grace). +- Verification gauntlet (in order): read `X-Stargraph-Timestamp` / + `X-Stargraph-Signature` headers → timestamp within `timestamp_window_seconds` + → constant-time HMAC compare against current then previous → nonce LRU replay + check → enqueue. +- **Idempotency key**: `sha256(trigger_id || sha256(raw_body))`. +- Status codes: `401` on missing headers / out-of-window timestamp / bad HMAC; + `409` on a duplicate nonce; `400` on malformed JSON body. ## Manual triggers -No YAML entry. The `manual` plugin is always registered. `POST /v1/runs` -with `{graph, input}` is the canonical entry point. +The `manual` trigger is the convergence point for operator-initiated runs: both +`stargraph run` and `POST /v1/runs` resolve to the same `enqueue` call. List an +entry in `triggers.yaml` to document an intended manual entry point; it has no +on-disk polling behavior. ```bash -curl -fsS -X POST "${STARGRAPH_URL}/v1/runs" \ - -H "Authorization: Bearer ${STARGRAPH_TOKEN}" \ +curl -fsS -X POST "http://localhost:8000/v1/runs" \ -H "Content-Type: application/json" \ - -d '{"graph":"research","input":{"query":"…"}}' + -d '{"graph_id":"research","params":{"query":"…"}}' ``` -## Listing & introspection +Retrieve the run handle via `GET /v1/runs/{run_id}`. -```bash -GET /v1/triggers # all triggers, all graphs -GET /v1/triggers?graph=research # per graph -GET /v1/triggers/<name> # next_fire_at, last_fire_at, fire_count +## TriggerEvent + +Every trigger emits a `stargraph.triggers.TriggerEvent` into the scheduler queue: + +| Field | Type | Description | +|---|---|---| +| `trigger_id` | `str` | Emitting trigger instance (e.g. `"cron:nightly-research"`). | +| `scheduled_fire` | `datetime` | Canonical fire time (cron-tick instant; receipt time for webhook/manual). | +| `idempotency_key` | `str` | Pre-computed dedup key; the scheduler dedupes against pending-run state before enqueueing. | +| `payload` | `dict` | JSON-serializable parameters forwarded to the run as `params`. | + +`TriggerEvent` carries `extra='forbid'`, so `payload` is the escape hatch for +trigger-specific data. + +## Trigger plugins + +To add a new trigger family, build a plugin that registers under the +`stargraph.triggers` entry-point group (name → `TriggerPlugin` class): + +```toml +[project.entry-points."stargraph.triggers"] +my_trigger = "my_pkg.triggers:MyTriggerPlugin" ``` -`stargraph.yaml` is the source of truth; the scheduler reconciles on graph -register/update. There is no separate trigger CRUD API — change the YAML, -re-register the graph. +A trigger implements the `Trigger` protocol — `init(deps)` / `start()` / +`stop()` / `routes()`. The serve lifespan dispatches these per-plugin with +exception isolation so one bad trigger cannot block the others. + +## Verifying a trigger + +Add the entry to `triggers.yaml`, start `stargraph serve`, and confirm the +scheduler picks it up: + +- Cron: a background task is spawned per spec. +- Webhook: its `POST` route is mounted — confirm the mounted path and that the + target graph appears in `GET /v1/graphs`. ## Authoring checklist -- [ ] Every trigger has a `dedup_key` — never rely on accidental uniqueness. -- [ ] Webhook secrets live in env vars, not in YAML. -- [ ] Cron triggers declare `timezone` explicitly (don't trust the host). -- [ ] `input` / `input_template` is shaped to the graph's State schema — - run `stargraph graph verify` after editing. -- [ ] Cleared deployments: webhook triggers only with a registered HMAC secret. +- [ ] Cron triggers declare `tz` explicitly (don't trust the host); a UTC server + is recommended for air-gapped deployments. +- [ ] Webhook secrets live in env vars named by `current_secret_env` / + `previous_secret_env`, never in YAML. +- [ ] `params` / webhook body is shaped to the target graph's state schema — + validate by loading the graph with `stargraph run GRAPH --inspect`. +- [ ] Webhook `path` starts with `/` and is unique across triggers. diff --git a/plugins/stargraph/skills/smart-stargraph/SKILL.md b/plugins/stargraph/skills/smart-stargraph/SKILL.md index 7182312..89e19fd 100644 --- a/plugins/stargraph/skills/smart-stargraph/SKILL.md +++ b/plugins/stargraph/skills/smart-stargraph/SKILL.md @@ -11,99 +11,120 @@ user-invocable: false | Setting | Default | Override | |---|---|---| -| Local install | `uv add stargraph` | — | -| Serve URL | `http://localhost:9000` | `STARGRAPH_URL` | -| Auth token | none | `STARGRAPH_TOKEN` | -| Graphs dir | `./graphs` | `STARGRAPH_GRAPHS_DIR` | +| Local install | `uv add stargraph` (or `pip install stargraph`) | — | +| Serve URL | `http://localhost:8000` | `--server` on `stargraph respond`; `--host`/`--port` on `stargraph serve` | +| Profile | `oss-default` | `--profile` flag / `STARGRAPH_PROFILE` (`oss-default`/`cleared`) | +| Project config | `stargraph.toml` (CWD) | `STARGRAPH_TOML_FILENAME` | +| Runtime config dir | `~/.config/stargraph/` (`triggers.yaml`, `nautilus.yaml`) | `STARGRAPH_CONFIG_DIR` | +| Local run artifacts | `./.stargraph/run.sqlite`, `./.stargraph/runs/<run_id>/` | `--checkpoint` / `--db` | +| Plugin-discovery trace | off | `STARGRAPH_TRACE_PLUGINS=1` | ## Project Detection -`pyproject.toml` `name = "stargraph"` → contributor mode (scan `src/stargraph/`, `design-docs/`, `specs/`). +`pyproject.toml` `name = "stargraph"` → contributor mode (scan `src/stargraph/`, `design-docs/`, `specs/`, `docs/`). ## Vocabulary | Term | Definition | |---|---| -| Graph | Definition: nodes, state schema, rules, governance. Blueprint, not running thing. | -| Run | Single execution of a graph (`run_id`). | -| Node | Unit of work — DSPy module, ML model, tool call, retrieval, sub-graph. | -| State | Pydantic-typed bundle flowing through a run. | -| Annotated state | Subset mirrored into CLIPS at node boundaries. | +| Graph | IR definition (`IRDocument`): nodes, state schema, rules, governance. Blueprint, not running thing. Authored as an `*.yaml` IR file (often `stargraph.yaml`). | +| Run | Single execution of a graph. `run_id` is a UUIDv7; counterfactual forks are `cf-<uuid>`. | +| Node | Unit of work. Builtin `kind` (`echo`, `halt`, `dspy`, `ml`, `interrupt`, `passthrough`, `write_artifact`, `retrieval`, `subgraph`, `human_input`) or `module.path:ClassName`. | +| State | Pydantic-typed bundle flowing through a run (`state_class: module:Class` or flat `state_schema`). | +| Annotated state | Subset mirrored into CLIPS facts at node boundaries. Replay-safe: use `frozenset`, never `set`. | | Fact | CLIPS tuple — mirrored from annotated state, emitted by runtime, or asserted by rules. | -| Rule | Fathom/CLIPS production matching facts; emits `goto`/`parallel`/`halt`. | -| Pack | Versioned named rule collection (Bosun-style). | -| Tool | Typed callable (JSON Schema + namespace + permissions + side-effect flags). | -| Skill | Bundle of tools, optional sub-graph, optional prompt fragment. Authorable as a typed Pydantic `Skill` instance OR a `SKILL.md` (compiled by `stargraph-md-skills`). | -| Plugin | Pip-installable Python package, OR a drop-in directory under `~/.stargraph/plugins/` (discovered by `stargraph-dir-plugins`). Ships skills/tools/nodes/stores/packs. | -| Store | Data tier abstraction — Vector/Graph/Doc/Memory/Fact. | -| Provider | Concrete Store implementation (LanceDB, Kuzu, SQLite, …). | -| Checkpoint | Persisted snapshot at a transition. | -| Graph hash | Topology + node signatures + state schema. | -| Trigger | External run initiator — cron / webhook / file_watch / mcp / manual. | +| Rule | Fathom/CLIPS production matching facts; `then` emits actions (`goto`/`parallel`/`halt`/`interrupt`/…). | +| Pack | Versioned named rule collection (Bosun). Mounted via IR `governance:` (`{id, version, requires}`). | +| Tool | `@tool`-decorated typed callable. Registry key `namespace.name@version`. SideEffects + ReplayPolicy. | +| Skill | Python `stargraph.skills.Skill` (Pydantic): tools, optional subgraph, prompt, `state_schema` write-whitelist. | +| Plugin | Pip-installable Python package registered via `pyproject.toml` entry points. Ships tools/skills/stores/packs/triggers. | +| Store | Data tier abstraction — `vector` / `graph` / `doc` / `memory` / `fact`. | +| Provider | Concrete Store impl (LanceDB, RyuGraph, SQLite, …). | +| Checkpoint | Persisted snapshot at a node transition (SQLite checkpointer). | +| Graph hash | Canonical IR hash (topology + node signatures + state schema). | +| Trigger | Run initiator — `manual` / `cron` / `webhook` (authored in `triggers.yaml`). | ## State-Fact Boundary - Mutate State freely inside a node (Python). - On node exit, mirror annotated fields into CLIPS, fire rules, persist checkpoint. -- Source of truth: State. Facts are projection. +- Source of truth: State. Facts are projection. `set`/`set[X]` on `state_schema` is rejected — use `frozenset`. ## Provenance-typed Facts -Every fact carries `(origin, source, run_id, step, confidence, timestamp)`. `origin ∈ {llm, tool, user, rule, model, external}`. +Tool/node outputs carry a `__stargraph_provenance__` envelope: `{origin, source, external_id}`. +Documented `origin` values: `tool`, `llm`, `rule`, `system`. Facts carry provenance + run_id/step. ## Stores -Protocols: `VectorStore`, `GraphStore`, `DocStore`, `MemoryStore`, `FactStore`. Default providers: LanceDB, Kuzu, SQLite (embedded). +Protocols: `vector`, `graph`, `doc`, `memory`, `fact`. Default providers (embedded): LanceDB +(`stargraph.stores.lancedb`), RyuGraph/cypher (`stargraph.stores.ryugraph` / `stargraph.stores.cypher`), +SQLite (`stargraph.stores.sqlite_doc` / `sqlite_memory` / `sqlite_fact`). Capabilities: `db.{name}:read|write`. -## stargraph.yaml (sketch) +## Graph IR (sketch) + +The validated runnable shape is `IRDocument` (`extra='forbid'` — unknown keys rejected on load): ```yaml -name: research -state: ./state.py:State +ir_version: "1.0.0" +id: "graph:research" +state_class: "graph.state:State" # or flat state_schema: {message: "str", severity: "int"} nodes: - - name: think - type: dspy:ChainOfThought - - name: act - type: tool:browser.search + - { id: think, kind: dspy } + - { id: act, kind: "graph.nodes:SearchNode" } + - { id: halt, kind: halt } rules: - - pack: bosun:routing/research + - id: r-think-to-act + when: "?n <- (node-id (id think))" + then: [{ kind: goto, target: act }] governance: - - bosun:budgets - - bosun:audit + - id: stargraph.bosun.budgets + version: "1.0" + requires: { stargraph_facts_version: "1.0", api_version: "1" } stores: - vector: lancedb:./.lance - graph: kuzu:./.kuzu - doc: sqlite:./.docs -checkpoints: - every: node-exit - store: sqlite:./.checkpoints + - { name: kb, provider: stargraph.stores.lancedb } ``` +Skill **bundles** (e.g. Shipwright) use a richer authoring `stargraph.yaml` (`state:`, `nodes: [{name,type}]`, +`rules: [{pack:}]`, `stores: {doc:,fact:}`, `checkpoints:`) alongside `manifest.yaml` + `state.py` + `nodes/`. + ## Graph Hash -`sha256(topology + node_signatures + state_schema)`. Checkpoints carry the hash. Resume rejects on mismatch unless `migrate:` declared. +Canonical IR hash (`dumps_canonical`, sorted keys). Checkpoints carry the hash. Resume rejects on +mismatch unless a `migrate:` block maps `from_hash`→`to_hash`. -## REST + WS Endpoints (stargraph serve) +## REST + WS Endpoints (stargraph serve, v1) | Verb | Path | Purpose | |---|---|---| -| POST | /v1/runs | start a run | -| GET | /v1/runs/{id} | run status | -| GET | /v1/runs/{id}/checkpoints | list checkpoints | -| POST | /v1/runs/{id}/replay | replay from checkpoint | +| POST | /v1/runs | start a run (`graph_id` + `params`) | +| GET | /v1/runs | list runs (paged) | +| GET | /v1/runs/{id} | run status / RunSummary | +| POST | /v1/runs/{id}/cancel | cancel | +| POST | /v1/runs/{id}/pause | pause | +| WS | /v1/runs/{id}/stream | stream run events | +| POST | /v1/runs/{id}/respond | deliver HITL response | +| POST | /v1/runs/{id}/counterfactual | fork counterfactual | +| GET | /v1/runs/{id}/artifacts | list run artifacts | +| GET | /v1/artifacts/{id} | fetch an artifact | | GET | /v1/graphs | list registered graphs | -| WS | /v1/runs/{id}/events | stream run events | -| GET | /health, /ready | probes | +| GET | /v1/registry/{kind} | list registered tools/skills/stores | ## CLI -`stargraph serve / run / replay / checkpoints / graphs list / graph verify`. +Eight subcommands (typer app `stargraph.cli:main`): +`stargraph run / serve / inspect / replay / respond / simulate / counterfactual / verify-audit`. + +- Validate a graph: `stargraph run GRAPH --inspect` (rule trace, no exec) or `stargraph simulate GRAPH --fixtures FILE`. +- Inspect a run: `stargraph inspect RUN_ID --db DB` (timeline), `--step N` (state), `--diff N M` (CLIPS fact delta). ## Verify-Before-Call -1. `GET /health`. -2. `GET /v1/graphs` — confirm graph registered before run. -3. After run start, GET `/v1/runs/<id>` — confirm transitioned out of PENDING. +1. `GET /v1/graphs` — confirm graph registered before starting a run. +2. After `POST /v1/runs`, `GET /v1/runs/<id>` — confirm it left pending. +3. Parse error envelopes; map 401/404/409 (don't retry blindly). + +(There is no `/health` or `/ready` probe — use `GET /v1/graphs`.) ## Build-Test-Fix @@ -111,31 +132,14 @@ Same 5-iter pattern. ## Authoring Formats -Two ways to author skills and plugins. Same runtime contract; different DX. - -### Skills - -| Format | When to use | Command | +| Artifact | How | Command | |---|---|---| -| Typed Python `Skill` (Pydantic) | Pip-distributed, versioned via PyPI | `/stargraph:new-skill` | -| `SKILL.md` (YAML frontmatter + body) | Personal config, prototyping, dir-plugins | `/stargraph:new-md-skill` | - -`SKILL.md` compiles to a typed `stargraph.skills.Skill` via `stargraph-md-skills`. -The `state_schema` declared in frontmatter is enforced as the engine's -boundary write-whitelist exactly as in the typed path. - -### Plugins - -| Format | When to use | Command | -|---|---|---| -| Pip-installable package + entry point | Production, CI-built artifacts | (write `pyproject.toml` directly) | -| Directory under `~/.stargraph/plugins/` | Personal config, prototyping, tarballs | `/stargraph:new-dir-plugin` | - -Dir-plugins are scanned by `stargraph-dir-plugins` at startup (or on -`stargraph plugins reload`). Manifest is `plugin.toml` mapping 1:1 to -`PluginManifest`. Bosun packs inside dir-plugins MUST be Ed25519-signed in -production (`allow_unsigned=false`); dev keys can be auto-generated by -`dir-plugin-builder`. - -Both formats register through the same pluggy hookspecs: -`register_tools`, `register_skills`, `register_stores`, `register_packs`. +| Tool | `@tool`-decorated callable → entry-point group `stargraph.tools` | `/stargraph:new-tool` | +| Skill | Python `Skill` (single file or bundle dir) → `register_skills`, group `stargraph.skills` | `/stargraph:new-skill`, `/stargraph:new-md-skill` (bundle) | +| Plugin | Pip package + `pyproject.toml` entry points + `stargraph_plugin` manifest factory | `/stargraph:new-dir-plugin` | +| Pack | Bosun rule pack → group `stargraph.packs`, Ed25519/JWS signed | `/stargraph:new-pack` | +| Trigger | Entry in `~/.config/stargraph/triggers.yaml`, or trigger plugin under `stargraph.triggers` | `/stargraph:new-trigger` | + +All capability kinds register through pluggy hookspecs: `register_tools`, `register_skills`, +`register_stores`, `register_packs`, plus the `trigger_*` family. Discovery is via entry points only +(no directory scanning); trace it with `STARGRAPH_TRACE_PLUGINS=1`.