Skip to content

refactor(stargraph)!: reconcile plugin with real stargraph tool surface #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
se-jo-ma wants to merge 1 commit into
base: main
Choose a base branch
from
Open

refactor(stargraph)!: reconcile plugin with real stargraph tool surface #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions plugins/stargraph/.claude-plugin/plugin.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
{
"name": "stargraph",
"version": "0.3.0",
"description": "Author + operate Stargraph — graphs, skills, nodes, tools, triggers, Bosun rule packs, markdown SKILL.md skills, directory-based plugins; run/replay/counterfactual against stargraph serve, drive HITL, inspect checkpoints.",
"version": "0.4.0",
"description": "Author + operate Stargraph — IRDocument graphs, nodes, @tool callables, Python Skill bundles, entry-point plugins, triggers, Bosun rule packs; run/inspect/replay/counterfactual against stargraph serve, drive HITL, verify audit logs.",
"author": {"name": "kraken-networks"},
"repository": "https://github.com/KrakenNet/stargraph",
"license": "Apache-2.0",
"keywords": ["stargraph","graphs","skills","tools","triggers","provenance","authoring","orchestration","hitl","counterfactual","bosun","skill-md","dir-plugin"]
"keywords": ["stargraph","graphs","skills","tools","triggers","provenance","authoring","orchestration","hitl","counterfactual","bosun","ir","plugins"]
}
99 changes: 52 additions & 47 deletions plugins/stargraph/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,56 +5,56 @@ Slash commands, agents, and skills for authoring **and operating**
framework with deterministic governance via Fathom.

Covers the full Stargraph CLI surface (`run`, `serve`, `inspect`, `replay`,
`counterfactual`, `respond`, `simulate`) plus authoring (graphs, nodes, tools,
skills, triggers, Bosun rule packs) and store wiring.
`counterfactual`, `respond`, `simulate`, `verify-audit`) plus authoring (graphs,
nodes, tools, skills, triggers, Bosun rule packs) and store wiring.

## Commands

### Authoring

| Command | Purpose |
|---|---|
| `/stargraph:new-graph <name>` | Scaffold a graph (state.py, nodes/, rules/, stargraph.yaml, tests). |
| `/stargraph:new-node <graph> <name>` | Add a typed node (DSPy / ML / tool / retrieval / sub-graph). |
| `/stargraph:new-tool <name>` | Tool definition with JSON Schema + namespace + permissions + side-effects. |
| `/stargraph:new-skill <name>` | Skill bundle — tools, optional sub-graph, prompt fragment (pip-package format). |
| `/stargraph:new-md-skill <name>` | Lightweight `SKILL.md` (YAML + markdown) compiled by `stargraph-md-skills` to a typed `Skill`. |
| `/stargraph:new-dir-plugin <name>` | Drop-in directory plugin under `~/.stargraph/plugins/` discovered by `stargraph-dir-plugins`. |
| `/stargraph:new-pack <name> [--flavor routing\|governance]` | New Bosun-compatible rule pack. |
| `/stargraph:new-trigger <graph> --type manual\|cron\|webhook` | Wire a trigger and verify scheduler pickup. |
| `/stargraph:store add <type> <provider>` | Wire a Store provider (vector/graph/doc/memory/fact). |
| `/stargraph:verify-graph <graph>` | Validate hash + schema + referenced rule packs + store providers. |
| `/stargraph:new-graph <name>` | Scaffold an `IRDocument` graph YAML (`ir_version`, `id`, nodes, rules, governance, state). |
| `/stargraph:new-node <graph> <name>` | Add a `NodeSpec` (builtin `kind` — dspy/ml/interrupt/retrieval/subgraph/… — or `module:Class`). |
| `/stargraph:new-tool <name>` | `@tool`-decorated callable + `ToolSpec` (namespace, side-effects, replay policy, capabilities). |
| `/stargraph:new-skill <name>` | Python `stargraph.skills.Skill` plugin — tools, optional subgraph, prompt, `state_schema`. |
| `/stargraph:new-md-skill <name>` | Skill **bundle** dir (`manifest.yaml` + `stargraph.yaml` + `state.py` + `nodes/`), Shipwright-style. |
| `/stargraph:new-dir-plugin <name>` | Entry-point plugin — pip package + `pyproject` entry points + `stargraph_plugin` manifest factory. |
| `/stargraph:new-pack <name> [--flavor routing\|governance]` | Bosun rule pack (group `stargraph.packs`, Ed25519/JWS signed). |
| `/stargraph:new-trigger <graph> --type manual\|cron\|webhook` | Add a trigger to `triggers.yaml`; verify `stargraph serve` pickup. |
| `/stargraph:store add <protocol> <provider>` | Wire a Store (vector/graph/doc/memory/fact) as an IR `StoreRef`. |
| `/stargraph:verify-graph <graph>` | Validate the IR via `run --inspect` / `simulate` + `stargraph.ir.validate`. |

### Operations

| Command | Purpose |
|---|---|
| `/stargraph:serve [--profile dev\|prod\|cleared]` | Start `stargraph serve` (FastAPI HTTP+WS). |
| `/stargraph:run <graph> [--input-file <json>]` | Execute a graph; stream events; return run_id. |
| `/stargraph:simulate <graph> [--seed <n>]` | Dry-run with deterministic stubs for side-effecting tools. |
| `/stargraph:inspect <run_id> [--events] [--facts]` | Run header, checkpoints, events, facts, provenance breakdown. |
| `/stargraph:checkpoints <run_id>` | List checkpoints with state-diff summaries. |
| `/stargraph:replay <run_id> [--from <cp>]` | Deterministic replay from a checkpoint. |
| `/stargraph:counterfactual <run_id> --from <step> --mutate <json>` | Fork a run with mutated facts/state; diff vs original. |
| `/stargraph:respond <run_id> --decision approve\|deny\|input` | Resume a paused (HITL) run. |
| `/stargraph:serve [--profile oss-default\|cleared]` | Start `stargraph serve` (FastAPI HTTP+WS on :8000). |
| `/stargraph:run <graph> [-i K=V] [--inspect]` | Execute a graph (or print its rule trace); seed state; return run_id. |
| `/stargraph:simulate <graph> --fixtures <yaml>` | Offline rule-firing trace against synthetic node outputs (no tools/LLM/checkpoint). |
| `/stargraph:inspect <run_id> --db <path> [--step N] [--diff N M]` | Timeline / state-at-step / CLIPS fact-delta + provenance. |
| `/stargraph:checkpoints <run_id>` | Checkpoint/state/fact views over the SQLite checkpointer (via `inspect`). |
| `/stargraph:replay <run_id> --db <path> [--mutation <json>] [--from-step N]` | Fork a counterfactual run from a checkpoint; diff vs parent. |
| `/stargraph:counterfactual <graph> --step N --mutate <yaml>` | Compute the cf-derived `graph_hash` for a mutation (no fork). |
| `/stargraph:respond <run_id> --response <json> --actor <name>` | Resume an `awaiting-input` (HITL) run. |

## Agents

| Agent | Purpose |
|---|---|
| `graph-builder` | Scaffolds + verifies graphs (state.py, nodes/, rules/, stargraph.yaml, tests). |
| `node-builder` | Adds typed nodes; manages state-fact boundary; updates graph hash. |
| `pack-builder` | Scaffolds Bosun rule packs (routing or governance flavor); validates as Fathom-compatible. |
| `skill-builder` | Builds skill bundles with tools, sub-graphs, prompt fragments (pip-package format). |
| `md-skill-builder` | Authors a `SKILL.md` and validates via `stargraph skills compile`. |
| `dir-plugin-builder` | Scaffolds a `~/.stargraph/plugins/<name>/` dir-plugin; verifies via `stargraph plugins verify` + live reload. |
| `runner` | Drives run / replay / checkpoints; polls run state and parses events. |
| `hitl-driver` | Lists paused runs, validates payloads against schema, drives `/respond`. |
| `graph-builder` | Builds + validates `IRDocument` graphs; verifies via `simulate` / `run --inspect`. |
| `node-builder` | Adds typed nodes; manages the state-fact boundary; tracks `graph_hash` + `migrate:`. |
| `pack-builder` | Scaffolds Bosun rule packs (routing or governance flavor); signs + mounts via `governance:`. |
| `skill-builder` | Builds Python `Skill` plugins (tools, subgraph, prompt) registered via `register_skills`. |
| `md-skill-builder` | Authors a skill **bundle** dir (Shipwright layout); verifies via `simulate` / `run --inspect`. |
| `dir-plugin-builder` | Scaffolds an entry-point plugin (pii_guard archetype); verifies discovery via `STARGRAPH_TRACE_PLUGINS=1`. |
| `runner` | Drives run / inspect / replay / respond; polls run state and parses events. |
| `hitl-driver` | Finds `awaiting-input` runs, builds the response payload, drives `/respond`. |

## Skills

- `smart-stargraph` — vocabulary, state-fact boundary, provenance, Stores, Bosun
mounting, project detection. Loaded by every command.
mounting, CLI/API surface, project detection. Loaded by every command.
- `smart-kraken` — shared cross-project conventions for the Kraken stack
(monorepo detection, verify-before-call patterns).

Expand All @@ -63,13 +63,13 @@ skills, triggers, Bosun rule packs) and store wiring.
Loaded on demand by the commands and agents above:

- `stargraph-concepts.md` — full glossary + disambiguations.
- `graph-yaml-schema.md` — `stargraph.yaml` shape.
- `state-schema.md` — Pydantic + annotated-state mirroring rules.
- `provenance-facts.md` — `(origin, source, run_id, step, confidence, ts)`.
- `store-protocols.md` — Vector / Graph / Doc / Memory / Fact contracts.
- `triggers.md` — manual / cron / webhook config patterns + scheduler semantics.
- `bosun-packs.md` — routing vs governance flavor; in-tree pack catalog.
- `hitl-patterns.md` — pause shapes, schema validation, audit facts, cleared-profile rules.
- `graph-yaml-schema.md` — the `IRDocument` graph shape.
- `state-schema.md` — Pydantic state + annotated-state mirroring rules.
- `provenance-facts.md` — the `__stargraph_provenance__` envelope `(origin, source, external_id)`.
- `store-protocols.md` — vector / graph / doc / memory / fact contracts + providers.
- `triggers.md` — `triggers.yaml` schema (manual / cron / webhook) + scheduler semantics.
- `bosun-packs.md` — routing vs governance flavor; PackMount + signing.
- `hitl-patterns.md` — interrupt node/action, `/respond`, capability gate, timeout/on_timeout.

## Install

Expand All @@ -89,22 +89,27 @@ claude plugins install stargraph

| Setting | Default | Override |
|---|---|---|
| Serve URL | `http://localhost:9000` | `STARGRAPH_URL` |
| Auth token | none | `STARGRAPH_TOKEN` |
| Graphs dir | `./graphs` | `STARGRAPH_GRAPHS_DIR` |
| Profile | `dev` | `STARGRAPH_PROFILE` (`dev`/`prod`/`cleared`) |
| Local install | `uv add stargraph` | — |
| Serve URL | `http://localhost:8000` | `stargraph serve --host/--port`; `stargraph respond --server` |
| Profile | `oss-default` | `--profile` / `STARGRAPH_PROFILE` (`oss-default`/`cleared`) |
| Project config | `stargraph.toml` (CWD) | `STARGRAPH_TOML_FILENAME` |
| Runtime config dir | `~/.config/stargraph/` | `STARGRAPH_CONFIG_DIR` |

## Version

0.3.0 — adds lightweight authoring formats: `SKILL.md` (compiled by
`stargraph-md-skills` to a typed `Skill`) and directory-based plugins under
`~/.stargraph/plugins/` (discovered by `stargraph-dir-plugins`). New commands
(`/stargraph:new-md-skill`, `/stargraph:new-dir-plugin`) and agents
(`md-skill-builder`, `dir-plugin-builder`).
0.4.0 — reconciled the whole plugin against the real Stargraph CLI/HTTP/plugin
surface: the 8-subcommand CLI (`run`/`serve`/`inspect`/`replay`/`respond`/
`simulate`/`counterfactual`/`verify-audit`), the `IRDocument` graph schema, the
entry-point plugin model (no directory plugins), Python `Skill` bundles (no
markdown SKILL.md compiler), `stargraph.toml` project config, and the real
`/v1/*` API. Replaced the prior fictional `graph verify` / `plugins *` /
`skills compile` / `facts` surfaces with their real equivalents.

0.3.0 — earlier authoring-format work (superseded by 0.4.0).

0.2.0 — full ops surface (`serve`, `inspect`, `simulate`, `counterfactual`,
`respond`), trigger authoring (`new-trigger`), HITL driver agent, and three
references (`triggers.md`, `bosun-packs.md`, `hitl-patterns.md`).
`respond`), trigger authoring, HITL driver agent, and three references
(`triggers.md`, `bosun-packs.md`, `hitl-patterns.md`).

See the project [CHANGELOG](https://github.com/KrakenNet/stargraph/blob/main/CHANGELOG.md)
for engine-side changes.
161 changes: 79 additions & 82 deletions plugins/stargraph/agents/dir-plugin-builder.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,112 +1,109 @@
---
description: Scaffold a directory-based Stargraph plugin under ~/.stargraph/plugins/ or $STARGRAPH_PLUGINS_DIR; verify via `stargraph plugins verify` + live reload.
description: Scaffold an entry-point Stargraph plugin — a pip package whose pyproject declares a `stargraph_plugin` manifest factory plus capability groups (tools/skills/stores/packs); verify discovery via STARGRAPH_TRACE_PLUGINS=1.
tools: [Bash, Read, Write, Edit]
---

# Dir-Plugin Builder
# Plugin Builder

Builds a drop-in Stargraph plugin discovered by `stargraph-dir-plugins`. No pip
install; same typed contract, namespace-conflict detection, signing, and
audit chain as a pip-installed plugin once registered.
Stargraph plugins are **entry-point** plugins: a normal pip package discovered
via `importlib.metadata` entry points + `pluggy` hooks (two-stage loader). There
are no directory/drop-in plugins. The `pii_guard` plugin
(`stargraph.plugins.pii_guard`) is the reference archetype — a `@tool`-decorated
coroutine plus governance hooks.

## Inputs

- `plugin_name` — directory name and `plugin.toml` `name` field.
- `namespaces` — list of dotted namespace prefixes claimed by the plugin.
- `ships` — combination of `{md_skills, tools, packs, stores}`.
- `trust_keys` — Ed25519 pubkeys for pack signing; if empty and `ships`
includes packs, generate a dev keypair via `fathom keygen` and persist
the public half in `plugin.toml`.
- `capabilities` — list of new capability strings to declare (with
description + sensitivity).
- `target_dir` — default `~/.stargraph/plugins/`; honor `$STARGRAPH_PLUGINS_DIR`
if set.
- `plugin_name` — distribution + package name.
- `namespaces` — dotted namespace prefixes the plugin claims (conflicts abort load).
- `provides` — subset of `{tool, skill, store, pack}`.
- `order` — load priority `0..10000` (default `5000`; collisions raise `PluginLoadError`).

## Steps

1. **Scaffold layout:**
1. **Scaffold a pip package** (pii_guard layout):
```
<target_dir>/<plugin_name>/
plugin.toml
skills/ (if ships includes md_skills)
tools/ (if ships includes tools)
__init__.py
packs/ (if ships includes packs)
stores/ (if ships includes stores)
capabilities.toml (if capabilities declared)
README.md
<plugin_name>/
pyproject.toml
src/<pkg>/
__init__.py (docstring)
_plugin.py (manifest factory + register_* hookimpl)
hooks.py (authorize_action / before|after_tool_call — optional)
redact.py (the @tool-decorated coroutine — example tool)
```

2. **Write `plugin.toml`:**
2. **Declare entry points in `pyproject.toml`** — the manifest factory under
group `stargraph`, plus one or more capability groups:
```toml
name = "<plugin_name>"
version = "0.1.0"
api_version = "1.x"
order = 100
namespaces = [<namespaces>]
[project.entry-points."stargraph"]
stargraph_plugin = "<pkg>._plugin:manifest" # returns a PluginManifest

[author]
name = ""
email = ""

[trust]
keys = [<trust_keys>]

[runtime]
python_path = ["tools"]
[project.entry-points."stargraph.tools"]
<pkg> = "<pkg>._plugin:register_tools"
# also available: stargraph.skills, stargraph.stores, stargraph.packs,
# stargraph.triggers, stargraph.mcp_adapters
```

3. **Per artifact, delegate:**
- For each requested md-skill: invoke `md-skill-builder` with
`host_path=<target_dir>/<plugin_name>/skills/<skill>/`.
- For each tool: invoke the tool-builder agent against
`tools/<name>.py`.
- For each pack: invoke `pack-builder`; sign with the dev keypair if
no trust key was supplied.
3. **Write `_plugin.py`** — the manifest factory + a `register_*` collect-all
hookimpl per capability:
```python
from stargraph.plugin import hookimpl
from stargraph.ir import PluginManifest, ToolSpec
from <pkg>.redact import redact_pii

def manifest() -> PluginManifest:
return PluginManifest(
name="<plugin_name>",
version="0.1.0",
api_version="1",
namespaces=[<namespaces>],
provides=["tool"],
order=5000,
)

@hookimpl
def register_tools() -> list[ToolSpec]:
return [redact_pii.spec] # .spec off the @tool wrapper
```

4. **Capabilities:**
If `capabilities.toml` was requested, emit one entry per declared cap:
```toml
[capabilities."<name>"]
description = "<description>"
sensitivity = "low" | "medium" | "high"
4. **Per artifact, delegate:**
- tools → the tool-builder (`@tool`, `ToolSpec`, registry key `ns.name@ver`).
- skills → `skill-builder` (Python `Skill` + `register_skills`).
- packs → `pack-builder` (group `stargraph.packs`, `register_packs`, signing).
- stores → wire a `StoreSpec` + `register_stores`.

5. **Optional governance hooks** (`hooks.py`, pii_guard pattern):
`authorize_action(action) -> bool|None` (first-deny; first non-`None` wins),
`before_tool_call(call)` / `after_tool_call(call, result)` for audit. Register
these on the same plugin object.

6. **Install + verify discovery** (there are no plugin verify/inspect/reload CLI subcommands):
```bash
uv pip install -e .
STARGRAPH_TRACE_PLUGINS=1 stargraph run <any-graph.yaml> --inspect
```
The trace logs every discovery, manifest validation, and registration step
(with the `order` it registered at). Failure cases:
- `api_version` not `"1"` → manifest validation fails; pin it.
- Namespace conflict with another plugin → load aborts; rename the namespace.
- `order` collision → `PluginLoadError`; pick a distinct order.
- Import failure in a capability module → trace shows the discovery/registration gap.

5. **Offline validation:** run
`stargraph plugins verify <target_dir>/<plugin_name>`. Failure cases to
diagnose:
- `api_version` mismatch → bump or pin.
- Namespace conflict with existing plugin → rename namespace.
- Unsigned pack with `allow_unsigned=false` → sign or move to
dev-only path.
- Tool import failure → check `runtime.python_path`.

6. **Live reload (optional, if `stargraph serve` is running):**
`stargraph plugins reload && stargraph plugins inspect <plugin_name>`.
Confirm: tool count, skill count, pack count, signing status, and that
the audit chain emitted a registration record with the plugin's
manifest hash.
At runtime with `stargraph serve` up, confirm registered kinds via
`GET /v1/registry/{kind}` (`kind` ∈ `tools`, `skills`, `stores`).

## Build-Test-Fix

5 iterations across verify + inspect. On signing failures, regenerate
keypair only with explicit user confirmation — never silently rotate keys.
5 iterations across the trace + `GET /v1/registry/{kind}`. On signing failures
for packs, regenerate keys only with explicit user confirmation.

## Output

- Tree of the created dir-plugin.
- `stargraph plugins verify` output.
- `stargraph plugins inspect <plugin_name>` output (or "skipped — serve not
running" if applicable).
- Any unsigned-pack warnings + remediation steps.
- Tree of the created package.
- `STARGRAPH_TRACE_PLUGINS=1` discovery/validation/registration trace.
- `GET /v1/registry/{kind}` result (or "skipped — serve not running").

## Constraints

- The plugin **must** declare at least one namespace; bare-name plugins
are rejected by the loader.
- If shipping unsigned packs in a path that production policy treats as
untrusted, surface the warning prominently; do NOT auto-sign with a
user-pubkey-claiming key.
- Stage-1 manifest validation is import-cold (NFR-7) — do not import
`tools/` modules during scaffolding; rely on `stargraph plugins verify`
to exercise stage-2 import safely.
- The manifest **must** declare at least one namespace; conflicts abort load.
- Stage-1 manifest validation is import-cold — do not import capability modules
during scaffolding; rely on the stage-2 trace to exercise imports.
Loading