Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci-gate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,9 @@ jobs:
bash tests/ci_graph_layout_tests.sh
bash scripts/prepare_graph_fixture_layout.sh

- name: Install pinned experimental operator host dependencies
run: bash scripts/install_operator_dependencies.sh

- name: Run Dispatch release gate
# Full static graph, real-process crash and resource proofs run serially.
timeout-minutes: 300
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -198,3 +198,5 @@ tests/tmp/*

# Reviewed, hash-inventoried proof evidence; ordinary runtime logs stay ignored.
!docs/evidence/resource-accounting/*.log

.operator-deps/
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ Version source: `kennel.toml`; CLI and `kujo.toml` must agree. Historical releas
entries are retained below. Experimental contract versions are independent of
Dispatch's version.

## Unreleased — experimental operator rehearsal

- Add a bounded installed batch-summary operator host with separate inspect,
plan, apply, review and run commands over existing Dispatch control APIs.
- Provide human/JSON output and exact Workcell/Eval dependency setup for the
experimental graph path; retain source-runtime and trusted-host boundaries.
- Add installed-package failure, review, budget and real controller-crash proofs.

## Unreleased — sourced resources and Eval budgets

- Add opt-in graph/node Eval ceilings with locked reservations, durable dispatch
Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -765,3 +765,6 @@ Finalization does not schedule work, replay effects or change participant author
Experimental sourced resource accounting and bounded Eval reservation planning are
documented in [the resource contract](docs/contracts/resources/protocol.md). Existing
runtime/provider artifacts remain observations; Dispatch owns budget authority.

For the experimental typed-output, review, Eval and budget path, see the
[operator rehearsal](docs/operator-rehearsal.md).
10 changes: 10 additions & 0 deletions dispatch.kujo
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ func print_help() {
"Usage:",
" kujo run dispatch.kujo run \"Research topic\" [--workflow-file workflow.json] [--input-json '{...}'] [--output-root outputs]",
" kujo run dispatch.kujo init [--dir dispatch-workflow] [--force]",
" kujo run dispatch.kujo operator --help (experimental local batch host)",
" kujo run dispatch.kujo catalog <generate|sync> <config.json> --output model-catalog.json",
" kujo run dispatch.kujo webhooks <status|replay> [--output-root outputs] [--url https-url] [--destination lifecycle] [--json]",
" kujo run dispatch.kujo demo \"Research topic\" [--workflow research-report] [--policy-profile staging] [--tags prod,project-a] [--yes] [--non-interactive] [--output-root outputs]",
Expand Down Expand Up @@ -2162,6 +2163,15 @@ while i < len(raw_args) {
i := i + 1
}

if command == "operator" {
argv := ["bash", dispatch_root_path("scripts/operator.sh")]
for value in command_args {argv = push(argv, value)}
result := spawn_process(argv, {"inherit_env":true, "timeout_ms":300000, "max_output_bytes":4194304})
if result.stdout != "" {print(trim(result.stdout))}
if result.stderr != "" {print(trim(result.stderr))}
if result.success == false {exit(1)}
return
}
if command == "run"||command == "demo" {
run_demo(command_args)
return
Expand Down
78 changes: 78 additions & 0 deletions docs/OPERATOR_FIRST_HOUR.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Operator rehearsal: first-hour findings

Starting points: Dispatch `de66a668a6bceb0b27dd2d4ae97797c969c595c2` and Kujo
`fbaaf244708b9384fcb321e8e6772c6764e21425`. The unrelated primary Kujo checkout was
left untouched; work used isolated branches. This is a factual installation and
operator exercise, not a subjective usability score or a new architecture phase.

| Category | Observed friction | Classification | Bounded response |
| --- | --- | --- | --- |
| Installation | Default Dispatch manifest names a Kujo source SHA; the installer attempted a nonexistent release for that SHA, then required Cargo in the clean environment. | Required prerequisite insufficiently explicit for binary-only use | Document exact binary manifest, release checksum verification and source-build distinction. Preserve canonical source pin. |
| Setup | The Dispatch package closure installs AI/Agents SDK but not the Workcell/Eval host used by graph proofs. | Required but undocumented | Install two exact experimental dependency archives; no fixture symlinks or inherited module path. |
| Public control | Stable installed `dispatch --help` has no graph operator path. The documented routed-review demo succeeds, but does not exercise graph terminal/resource authority. | Internal-only integration assumption | Add one closed experimental batch host behind `dispatch operator`; retain existing core APIs. |
| Status | An operator would need fixture functions/raw state to join output, review, branch, attempt and resource facts. | Missing explanation surface | Human and versioned JSON report with exact provenance, unknown usage and recovery assessment. |
| State terminology | Newly allocated child runs have a persisted `running` control status before any dispatch. Showing that as process liveness would be misleading. | Confusing terminology | Report control status separately from durable dispatch/result/finalization facts; make no process-liveness claim. |
| Continuation output | The host API returned a 21 KB raw state object for one paused `run`. | Implementation-centric output | Return a small versioned action summary; use `inspect` for the bounded facts/provenance view. |
| Review | Internal examples constructed subject/revision-bound decision JSON. | Command requiring source knowledge | `review` opens an exact request; `plan approve/deny` constructs the candidate; `apply` checks the current OS user and core authority. |
| Recovery | Rehearsal must identify surviving reservations before continuing after a killed controller. | Recovery ambiguity | Expose the existing retained-host plan/apply API; no implicit resume or refund. |
| Error text | Capacity/path/stale-candidate codes lacked a corrective operator action. | Implementation-centric diagnostic | Preserve the exact code and add a bounded next action. Never recommend raw state editing. |

## Workflow and interfaces

A bounded integer batch produces a typed summary and one SQLite publication
receipt. H reviews its exact output; E checks the configured expected total. The
existing Eval branch selects B (publication) or R (repair report). Each executable
node retains its own Workcell, effect admission, schema evaluation and parent
finalization. The graph terminal decision is separate. There is no live model.
Graph Eval budgets count explicit E dispatches; the local schema check within a
program's parent contract is not another graph evaluator node.

Public entrypoints: `dispatch operator init`, `inspect [--json]`, `plan`, `apply`,
`review`, and `run`. Plans name one operation and preserve exact existing revision
and evidence preconditions. Planning a receipt operation is a requested command,
not a guarantee that admission will pass final revalidation. The wrapper's explicit
absolute workspace argument opts into the existing non-default output-root policy.

The executable black-box proof is `tests/operator_rehearsal.mjs`. It copies only
package files into a new installation, supplies pinned real dependencies, clears
the inherited environment except documented prerequisites, uses a fresh HOME,
and invokes the public launcher from outside the installation. It imports no
internal fixture helpers and never edits control state. Its only low-level access
is read-only hashing and watching immutable record publication to time SIGKILL.
A separate archive installation rehearses the real GitHub installer path.

## Proof boundaries

- Happy path: independent program/receipt, human decision, Eval, selected consumer
and graph terminal authority. Exact A digest remains B's input; source input,
review and branch facts remain inspectable.
- Failure: an actual wrong expected total yields failed Eval and the declared R
branch. The failed verdict remains historical; B stays `not_selected`.
- Review: wrong subject and stale revision reject. Approval starts no evaluator
and grants no consumer effect admission.
- Budget: no Eval capacity blocks reservation; a graph already using its last
program/Eval units may still record evidence and finalize.
- Recovery: SIGKILL occurs after immutable reservation publication. A fresh
controller inventories/reconciles surviving facts, retains the reservation and
observes no program admission or output. Ordinary continuation follows. This
operator scenario does not newly claim every lost-reply crash case; the existing
canonical recovery/consumed-attempt suites cover those boundaries.
- Resources: real Eval runtime observation is retained. Process-time partial sums
are not graph elapsed time or CPU. Provider tokens, estimates and money remain
unknown where no source exists; unknown totals are null, not zero.

## Remaining scope

This is a supported experimental walkthrough for one installed host, not arbitrary
workflow authoring or a public graph SDK. Trusted local OS/filesystem authority,
static topology, exact installed implementation and surviving local storage are
required. Evidence/preservation expire; this walkthrough does not expose renewal,
unsafe retries, mutable ceilings, remote trust, machine migration or force recovery.
No pricing, dashboard, scheduler, participant wire changes or new language syntax
were added. Workcell and Eval sources were inspected/installed but not modified.

Freeze the local Wave F architecture lane once canonical verification passes.
Return to product/release work and real user feedback. No additional graph or
resource architecture follows automatically from this rehearsal. Validation counts,
source pins, failures and artifact hashes belong in the accompanying evidence
record; a green happy path alone is not a release approval.
3 changes: 3 additions & 0 deletions docs/first-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,6 @@ Next: [author a workflow](../examples/quickstart-walkthrough.md), inspect
[architecture and extension points](architecture-and-extension-diagrams.md),
and read [enterprise deployment boundaries](enterprise-deployment.md) before
enabling live providers.

For the experimental typed-output, review, Eval and budget path, see the
[operator rehearsal](operator-rehearsal.md).
172 changes: 172 additions & 0 deletions docs/operator-rehearsal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# Operate a local batch-summary graph (experimental)

This walkthrough uses a trusted local host, a static five-node graph and retained
local storage. It is not a stable graph SDK or a claim of general production
readiness. The installed host runs this one workflow; it does not accept arbitrary
participant code or topology.

## Install and configure

Prerequisites: macOS or Linux, Bash, curl, tar, Git, and Kujo 1.6.0 with the
`sync_directory_beneath` runtime primitive. No provider account is needed. The
current release manifest pins a runtime **source commit**: its normal installer
path needs Cargo. To use a binary-only installation, use the verified 1.6.0 release
archive and its published SHA-256 file instead. Do not mistake an unavailable
release named after a source SHA for a verified binary installation.

The following binary-install manifest was rehearsed without Cargo. The installer
verifies the runtime release checksum. Use a new prefix and run the commands from
a fresh directory (the source refs below are intentionally pinned):

```bash
curl -fsSL https://raw.githubusercontent.com/kujolang/kujo/fc53093f35e4236682bbbf5ab1ccc8d2b4df990c/install.sh -o install.sh
cat > operator.refs <<'REFS'
kujo=v1.6.0
ai-sdk=5184a122590dd5698770b50d99b69ed11c7ec84a
agents-sdk=5a9d5525e511a293b743b1c44d9ef270fd491f9f
dispatch=21aac0d35d8709ea3b8cb09844315c7b14fde5bd
REFS
bash install.sh --package dispatch --release-manifest "$PWD/operator.refs" \
--prefix "$PWD/installed" --bin-dir "$PWD/bin"
export PATH="$PWD/bin:$PATH"
bash installed/sources/dispatch/scripts/install_operator_dependencies.sh
cp installed/sources/dispatch/examples/operator/batch.json batch.json
```

Source archives are the installed package format; an editable Git checkout is not
needed. This manifest installs the bounded host, not an unpublished participant
SDK or a new stable graph release. The package's default source-runtime manifest
remains unchanged for canonical source-build validation.

The script downloads Workcell and Eval into `.operator-deps` without symlinks or
test-fixture preparation. It refuses to overwrite an existing dependency directory.
Set `DISPATCH_OPERATOR_DEPS` to a fresh absolute directory to install elsewhere.
Keep the installed code and dependencies unchanged for the lifetime of a run:
implementation/configuration changes invalidate its authority.

Copy `examples/operator/batch.json` into your project. Its `amounts` are bounded
integer inputs; `expected_total` is the Eval criterion, not a claimed result.
`program_limit` and `eval_limit` are immutable ceilings for this run. Initializing
copies the exact source input, records its digest and the local OS user ID, creates
retained Git workspaces and negotiates each independent node run.

```bash
dispatch operator init /absolute/batch-run /absolute/batch.json
dispatch operator inspect /absolute/batch-run --json
```

If you installed a source archive without a `dispatch` shim, use
`kujo run /installed/dispatch/dispatch.kujo operator ...` with `DISPATCH_ROOT` set
to that installed directory, or `bash /installed/dispatch/scripts/operator.sh ...`.
The latter resolves its installed files itself and works from any directory.
Only the initializing OS user is accepted by this trusted-host example. Filesystem
ownership and local executable permissions remain the host trust boundary.

## Plan, apply, then separately continue

A plan is an exact, reviewable candidate. It grants no authority and writes no run
state. Keep plans outside the retained workspace when comparing read-only snapshots.
For each operation below, use this pattern, inspect the resulting JSON, then apply:

```bash
dispatch operator plan /absolute/batch-run reserve A > /absolute/plan.json
cat /absolute/plan.json
dispatch operator apply /absolute/batch-run /absolute/plan.json
dispatch operator run /absolute/batch-run
```

`run` uses Dispatch's existing DAG and one-use node admission. It may stop with
`node_requires_independent_finalization`; that is the expected control boundary,
not permission to replay. A program produces typed JSON and a proposed SQLite
receipt. Admit that receipt separately, and verify it by independent sink readback.
For A, plan/apply each operation in this order:

```text
refresh A
select A
admit A
refresh A
parent-finalize A
```

`admit` explicitly authorizes the one selected SQLite mutation. `refresh` reads the
sink and retains evidence. `parent-finalize` accepts the exact output, its local
schema evaluation, preservation and effect evidence. No operation runs B or R.
Then call `run` again so the DAG observes A's independent finalization.

## Human review, Eval and branch

```bash
dispatch operator review /absolute/batch-run
```

This creates H's durable review request for the exact A output. Inspect it before
planning `approve H` (or `deny H`), then apply that plan. Approval binds the subject,
input digest, request, revision and OS-user actor. A stale plan must be regenerated;
a wrong subject or actor is rejected. Approval does not execute Eval or consumers.

Plan/apply each operation:

```text
reserve E
eval-dispatch E
observe E
node-finalize E
branch graph
```

Eval runs in a fresh process against the exact bound A output. `observe E` records
its actual runtime measurement artifact; it does not invent model usage. Eval
finalization and branch activation remain separate from executing the selected
consumer. On pass, reserve B and call `run`. On fail, reserve R and call `run`.
The other branch is `not_selected`, consuming no program capacity.

Finish the selected consumer with the same five receipt/finalization operations
used for A. Call `run` again, then plan/apply `graph-finalize graph`.

## Inspect failure, budget and provenance

```bash
dispatch operator inspect /absolute/batch-run --json
```

The report references exact input bindings, output digests, node terminals,
attempt history, review evidence, branch decisions, graph outcome and retained
recovery assessment. Resource totals preserve observed/provider-reported/estimated
classes and explicit unknowns. A known partial sum is not a complete total.
No model was invoked in this workflow; monetary cost and model usage have no
sourced measurement and are not reported as measured zero.

To exercise a real failed quality check, initialize a **new** workspace from a
config with `expected_total: 99`. Eval fails and the explicitly configured repair
branch R publishes the summary receipt. A failed quality check remains in history;
routing it does not rewrite the verdict as pass.

To exercise exhaustion, initialize another workspace with `eval_limit: 0`.
After A and H complete, `plan ... reserve E` rejects for lack of Eval capacity.
Do not edit the anchored ceiling or restore old state. Consumed work can still
record evidence and finalize when the remaining available capacity is zero.

## Controller failure and retained recovery

After a controller dies, begin with `inspect`, then `plan ... recover graph`.
The plan comes from the existing retained-host recovery engine. Apply only a
permitted mechanical repair, then inspect again. A consistent run needs no repair;
continue with ordinary plans/admission. For a child, use its node ID instead of
`graph`. Recovery never runs a node or chooses a branch.

A consumed attempt remains consumed even if its reply was lost. An unresolved
Eval may be resolved only from the installed adapter's exact retained response
(`eval-resolve E`); absence of that response remains uncertainty. Never refund or
rerun it based on budget availability. This example deliberately provides no
force-recover command, ceiling extension, evidence renewal, or unsafe retry.

The preservation/effect evidence window is bounded to roughly one hour. Expiry
may prevent further execution/finalization even though history remains readable.
Do not reset the workspace to evade expiry. The deeper renewal APIs are outside
this first-hour workflow.

See [resource accounting](contracts/resources/protocol.md),
[graph attempts](contracts/graph-attempts/operator.md), and
[retained recovery](contracts/recovery/retained-host.md) for the authority
contracts. The rehearsal evidence and remaining friction are recorded separately.
1 change: 1 addition & 0 deletions examples/operator/batch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"schema":"dispatch.batch-summary/v1alpha1","amounts":[12,18,20],"expected_total":50,"program_limit":2,"eval_limit":1}
19 changes: 19 additions & 0 deletions scripts/install_operator_dependencies.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/usr/bin/env bash
set -euo pipefail
operator_root="$(cd "$(dirname "$0")/.." && pwd)"
operator_deps="${DISPATCH_OPERATOR_DEPS:-$operator_root/.operator-deps}"
mkdir -p "$operator_deps"
while read -r repository revision; do
destination="$operator_deps/$repository"
[[ ! -e "$destination" ]] || { echo "Refusing to replace $destination; use a fresh DISPATCH_OPERATOR_DEPS directory." >&2; exit 1; }
archive="$(mktemp)"
curl -fsSL --retry 3 "https://api.github.com/repos/kujolang/$repository/tarball/$revision" -o "$archive"
mkdir "$destination"
tar -xzf "$archive" --strip-components=1 -C "$destination"
rm "$archive"
printf '%s\n' "$revision" > "$destination/.operator-source-ref"
done <<'PINS'
workcell 0f9595b79872d2500f0bb35a8cf6227171fd7eeb
eval 6a5ab09e5c24cc1a842b374afccb64da40db2874
PINS
printf 'Installed pinned operator dependencies in %s\n' "$operator_deps"
Loading
Loading