diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0c9a808..66b699f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,8 +1,8 @@ # Contributing -Conductor `0.3.0` is attended Stage 2 software for exactly Herdr `0.7.5`. -Changes must stay fail-closed, preserve one runtime authority, and avoid claims -beyond implemented and independently checked behavior. +Conductor `0.4.0` is attended Stage 2 and Stage 3 software for exactly Herdr +`0.7.5`. Changes must stay fail-closed, preserve one runtime authority, and +avoid claims beyond implemented and independently checked behavior. ## Development requirements @@ -30,7 +30,8 @@ manifest, historical evidence, and pinned Actionlint. ## Change boundaries - `scripts/stage1-runtime.mjs` remains the one implementation behind assemble, - board, status, harvest, and stand-down. Do not add a competing runtime. + board, status, harvest, preview, apply, and stand-down. Do not add a + competing runtime. - Task/outbox authority must be durable before pane/agent creation. Reports use only bounded stdin and one exact task-bound destination. - Preserve complete producer selection, collector-computed paths, deterministic @@ -44,7 +45,13 @@ manifest, historical evidence, and pinned Actionlint. - Product runtime never removes worktrees, branches, tasks, outboxes, reports, gate sources, artifacts, recordings, logs, or state. Only the explicit live harness may delete its exact identity-bound disposable inventory. -- Do not add Stage 3 preview/approval/apply/recovery, suite adapters, unattended +- Stage 3 stays the documented attended single-ref apply: journal-only + fixed-statement approval receipts, durable consumption before any effect, + exactly zero or one fast-forward apply CAS per attempt, and uncertainty + resolution only for the apply publication by exact re-observation. Do not add + approval delegation, multi-ref or non-fast-forward apply, or resolution for + any other operation. +- Do not add suite adapters, unattended launch, Browser/Guard/Swarm promotion, site changes, push/release automation, cleanup/prune/migration/expiry, or newer-Herdr claims. @@ -91,8 +98,9 @@ A pull request must state: no-recovery limitations; 5. whether package, manifest, README, SECURITY, private-state docs, roles, source definition, evidence inventory, and CI agree; and -6. explicit confirmation that no Stage 3, adapter, unattended, product cleanup, - Browser, site, push, tag, or release change entered scope. +6. explicit confirmation that no adapter, unattended, product cleanup, + Browser, site, push, tag, or release change entered scope, and that Stage 3 + changes stay inside the single-ref attended apply boundary. Never include credentials, tokens, private paths/state, raw action output, prompts, transcripts, recordings, customer data, or unsanitized evidence. diff --git a/README.md b/README.md index 9d7393a..0f81198 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,14 @@ # Herdr Conductor > [!IMPORTANT] -> Conductor `0.3.0` implements **Stage 2 attended strict task/report contracts** -> on exactly Herdr `0.7.5`, protocol `17`, API schema `1`. An operator explicitly -> invokes every transition. This is cooperative same-UID coordination, not -> authentication, sandboxing, unattended orchestration, or Stage 3 -> approval/apply. - -Conductor coordinates task-bound producer and gate roles through five installed +> Conductor `0.4.0` implements **Stage 2 attended strict task/report contracts** +> and the **Stage 3 attended single-ref apply** on exactly Herdr `0.7.5`, +> protocol `17`, API schema `1`. An operator explicitly invokes every +> transition, records every approval receipt, and resolves every apply +> uncertainty. This is cooperative same-UID coordination, not authentication, +> sandboxing, or unattended orchestration. + +Conductor coordinates task-bound producer and gate roles through seven installed Herdr actions and one passive board pane. One runtime authority in `scripts/stage1-runtime.mjs` owns the complete lifecycle. It preserves the Stage 1 physical repository/workspace/run identity, hash-chained journal, @@ -63,10 +64,53 @@ writable report outboxes remain outside source. This is an ordinary-write and review boundary, not malicious same-UID enforcement. Gate reports require empty changed paths and artifacts and exhaustive worker-asserted requirement results. +## Stage 3 apply + +With configuration v3 declaring a non-null `apply.target_ref`, a run whose gate +reports are collected (or a gateless harvested run) may be applied through an +attended four-operation attempt: + +- `preview` journals one zero-effect document binding the exact run, + integration entry, byte-ordered completed gate assertions, and the observed + apply target. The target ref must exist, must differ from the integration + branch, must not be checked out in any worktree, and must sit exactly at the + run's integration base, so the proposed move is a pure fast-forward with a + non-empty rename-free change summary. Any drift fails closed with zero + recorded authority and zero Git mutation. +- The operator records one approval receipt through `npm run apply:approve`, + which accepts at most 16384 canonical bytes on stdin, requires the fixed + approve/reject statement token bound to the exact preview journal entry + digest, and stores the receipt only in the hash-chained journal. An approve + receipt requires the preview to still be live; a reject receipt records on a + drifted target too, so an attempt can always close. Approval receipts remain + unauthenticated same-UID operator records, never signatures or authorization + proof. +- `apply` first journals a durable consumption marker spending the receipt + before any Git effect, then publishes the attempt's single outcome: a live + target moves with exactly one double-preflighted compare-and-swap from the + previewed SHA to the integrated final SHA, and a drifted target records a + durable `unapplied` outcome with zero target CAS, closing the attempt so the + run can always progress. A spent receipt never authorizes a second + compare-and-swap. The attended `apply` action may reclaim the repository + mutation lock only from a dead process that held exactly its own apply + operation id; every other retained lock keeps refusing. +- A crash between consumption and publication observation leaves the run + apply-uncertain, and every other surface keeps refusing with + `recovery_required`. The attended `apply` action alone resolves it by exact + re-observation of the target ref: exactly the final SHA is applied, exactly + the previewed SHA voids the attempt, and any other observation fails closed + permanently. Resolution exists only for the apply publication; no other + operation gains it. + +A closed attempt (reject receipt or voided publication) permits one fresh +preview; an applied attempt is terminal; at most 8 attempts may exist. +Configuration v2 runs never enter Stage 3 states. + Lifecycle scanning derives one disjoint state or fails with `bookkeeping_unknown`/`recovery_required`. Stable states cover provisioning, waiting reports, terminal rejection, nonprogressable delivery, ready/integrated -results, gate provisioning/waiting/refusal/collection, every stand-down close +results, gate provisioning/waiting/refusal/collection, apply +preview/approval/consumption/void/applied, every stand-down close prefix, and archive. Attended stand-down is available from every stable state, binds a deterministic exact pane close set, closes only the next full live tuple, and archives only after every close is observed. It never removes product @@ -75,11 +119,16 @@ artifacts. ## Configuration -Commit `.herdr-conductor.json` in the invoking repository: +Commit `.herdr-conductor.json` in the invoking repository. Configuration v3 is +exactly v2 plus one required `apply` member — `null` to declare Stage 3 +disabled, or an exact `{ "target_ref": "refs/heads/…" }` object naming the +single apply target. Configuration v2 remains accepted with unchanged meaning +and no Stage 3 operations: ```json { - "version": 2, + "version": 3, + "apply": { "target_ref": "refs/heads/release" }, "state_root": { "kind": "default" }, "worktree_root": ".conductor-worktrees", "roles": [ @@ -140,13 +189,18 @@ herdr plugin action invoke assemble --plugin structupath.conductor herdr plugin action invoke board --plugin structupath.conductor herdr plugin action invoke status --plugin structupath.conductor herdr plugin action invoke harvest --plugin structupath.conductor +herdr plugin action invoke preview --plugin structupath.conductor +herdr plugin action invoke apply --plugin structupath.conductor herdr plugin action invoke stand-down --plugin structupath.conductor ``` `assemble` returns exact task paths, source roots, outbox slots, and publisher commands. A worker sends canonical report bytes to that publisher through stdin. `harvest` is explicitly invoked and attended; producer report collection and -gate report collection may require separate invocations. Board/status are passive +gate report collection may require separate invocations. `preview` prints the +journaled preview document, its entry digest, and the exact approval command; +`apply` consumes the recorded approve receipt and performs or resolves the +single target compare-and-swap. Board/status are passive and infer no missing input. `stand-down` closes only panes that were observed; each exact workspace/pane/cwd/generation identity must still match, and the full attached-agent tuple is also required when an agent was observed. @@ -187,7 +241,8 @@ bash -n scripts/*.sh shellcheck --shell=bash scripts/*.sh python3 -m py_compile scripts/harness-fs-helper.py go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -node --test tests/stage1-runtime-*.test.mjs tests/stage2-*.test.mjs +node --test tests/stage1-runtime-*.test.mjs tests/stage2-*.test.mjs \ + tests/stage3-*.test.mjs ``` The live harness requires an explicit candidate, review record, empty canonical @@ -219,15 +274,19 @@ npm run evidence:stage2:finalize -- \ TOCTOU windows. - Worker results are unauthenticated assertions. The retained external-review record uses only fixed independent-human/independence/GO tokens and requires - zero findings; those closed assertions remain unauthenticated. + zero findings; those closed assertions remain unauthenticated. Stage 3 + approval receipts remain unauthenticated same-UID operator records. - Product resources are retained indefinitely and consume cumulative disk. -- A crash after a possible external effect remains uncertain; Stage 2 has no - ambiguous-operation recovery or replay. +- A crash after a possible external pane/agent effect remains uncertain; Stage + 2 has no ambiguous-operation recovery or replay. The only resolvable + uncertainty is the Stage 3 apply publication, whose single-ref outcome is + exactly observable; resolution exists only for the apply publication. - Guard is observational and cannot prove prevention. Conductor does not invoke Swarm and makes no Browser, suite-adapter, site, sandbox, promotion, or unattended-readiness claim. -- Stage 3 preview, approval, approval consumption, apply, and recovery are not - implemented. +- Stage 3 applies exactly one configured local ref by fast-forward. It does not + push, publish remotely, deploy, tag, release, delegate approval, or apply + multiple refs. See [SECURITY.md](SECURITY.md), [CONTRIBUTING.md](CONTRIBUTING.md), and [docs/private-state-v1.md](docs/private-state-v1.md) for the authoritative diff --git a/SECURITY.md b/SECURITY.md index 684e078..d650261 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -2,18 +2,21 @@ ## Supported scope -Conductor `0.3.0` provides Stage 2 **attended-operational** strict task/report -contracts on exactly Herdr `0.7.5`, protocol `17`, API schema `1`, under one -cooperative OS user. The five actions preserve physical repository/workspace/run -identity, strict private state, one repository mutation lock, journaled crash -truth, deterministic zero-or-one target compare-and-swap, exact-SHA gate sources, +Conductor `0.4.0` provides Stage 2 **attended-operational** strict task/report +contracts and the Stage 3 attended single-ref apply on exactly Herdr `0.7.5`, +protocol `17`, API schema `1`, under one cooperative OS user. The seven actions +preserve physical repository/workspace/run identity, strict private state, one +repository mutation lock, journaled crash truth, deterministic zero-or-one +target compare-and-swap for integration and for apply, exact-SHA gate sources, and exact-identity stand-down/archive. These controls prevent accidental authority confusion; they are not authentication, sandboxing, signatures, or remote attestation. Worker-reported `delivered`, `approve`, `pass`, command/criterion results, identity, evidence, and external reviewer identity/independence/findings are unauthenticated assertions. -They do not authorize apply, release, or any other effect. +They do not by themselves authorize apply, release, or any other effect. Stage 3 +approval receipts are unauthenticated same-UID operator records: fixed-statement +journal entries, not signatures, credentials, or delegated authority. ## Preserved boundaries @@ -33,10 +36,20 @@ They do not authorize apply, release, or any other effect. - Stand-down binds one deterministic close set and closes only an exact live pane/agent/cwd tuple. It retains every product worktree, branch, task, outbox, report, gate source, artifact, recording, log, and operation record. -- No product cleanup, prune, migration, expiry, adoption, ambiguous recovery, - preview, approval receipt, approval consumption, apply, suite adapter, - unattended launch, Browser promotion, site repin, push, or deployment is - implemented. +- Stage 3 preview performs zero Git mutation; an approve receipt is durably + consumed before any apply effect and never authorizes a second + compare-and-swap; the apply moves exactly one configured local ref by + fast-forward from its previewed SHA, or durably records an unapplied outcome + with zero CAS when the target drifted. An uncertain apply publication is + resolved only by attended exact re-observation of that ref, and any foreign + observation fails closed permanently. The attended apply may reclaim a + repository lock only from a dead process holding exactly its own apply + operation id; the pid-liveness check remains cooperative, not + authentication. +- No product cleanup, prune, migration, expiry, adoption, suite adapter, + unattended launch, Browser promotion, site repin, push, remote publication, + or deployment is implemented. Ambiguous-operation resolution exists only for + the Stage 3 apply publication. ## Residual risks @@ -45,8 +58,10 @@ They do not authorize apply, release, or any other effect. authenticate it. - Final filesystem/Git checks and Herdr pane-ID close retain same-user TOCTOU windows. Herdr `0.7.5` offers no conditional close tuple. -- A crash after possible Git CAS, pane close, archive publication, or evidence - unlink remains uncertain. Stage 2 does not infer success or replay it. +- A crash after possible pane close, archive publication, or evidence unlink + remains uncertain; Stage 2 does not infer success or replay it. A crash + around the Stage 3 apply CAS is resolved only by attended exact + re-observation, never inferred or replayed. - Retained product state consumes cumulative disk; no product expiry or cleanup API exists. - Only 40-hex SHA-1-width Git repositories are supported. @@ -100,5 +115,6 @@ npm run test:stage2 shellcheck --shell=bash scripts/*.sh python3 -m py_compile scripts/harness-fs-helper.py go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -node --test tests/stage1-runtime-*.test.mjs tests/stage2-*.test.mjs +node --test tests/stage1-runtime-*.test.mjs tests/stage2-*.test.mjs \ + tests/stage3-*.test.mjs ``` diff --git a/docs/herdr-plugins-cheatsheet.md b/docs/herdr-plugins-cheatsheet.md index 479ad3e..773c854 100644 --- a/docs/herdr-plugins-cheatsheet.md +++ b/docs/herdr-plugins-cheatsheet.md @@ -4,13 +4,14 @@ The canonical cross-plugin guide lives in the [`herdr-suite-site` documentation](https://github.com/StructuPath/herdr-suite-site/tree/main/docs-src). That separate site is not changed or promoted by Conductor Stage 2. -For Conductor `0.3.0`, use this repository's [README](../README.md) as the +For Conductor `0.4.0`, use this repository's [README](../README.md) as the operational authority. Its bounded support is exactly Herdr `0.7.5`, protocol -`17`, API schema `1`, five attended actions, and one passive board pane. +`17`, API schema `1`, seven attended actions including `preview` and `apply`, +and one passive board pane. Key boundaries: -- configuration v2 has no launch arguments; +- configuration v2 and v3 have no launch arguments; - immutable task and empty private outbox authority precede pane/agent creation; - reports use only the exact task-bound bounded-stdin publisher; - every worker/reviewer result is an unauthenticated assertion; diff --git a/docs/private-state-v1.md b/docs/private-state-v1.md index 93cd212..80eae9c 100644 --- a/docs/private-state-v1.md +++ b/docs/private-state-v1.md @@ -1,6 +1,6 @@ # Conductor private state v1 -**Status:** Stage 2 in Conductor `0.3.0`. All five actions use this state +**Status:** Stage 2 and Stage 3 in Conductor `0.4.0`. All seven actions use this state exclusively through `scripts/stage1-runtime.mjs`. Tasks/outboxes precede agents; report collection, deterministic integration, exact-SHA gates, stand-down close prefixes, and archive share the same journal authority. @@ -254,16 +254,18 @@ Herdr `0.7.5`, protocol `17`, schema `1`. Private state remains cooperative same-user coordination. The repository lock does not stop unrelated Git or same-UID mutation after final checks, and Herdr's -pane close accepts only pane ID, leaving documented TOCTOU. Stage 2 has no -ambiguous-operation recovery, product cleanup/prune/migration/expiry, Stage 3 -preview/approval/apply, suite adapter, unattended trigger, Browser promotion, -push, or deployment. +pane close accepts only pane ID, leaving documented TOCTOU. Stage 3 preview, +approval, consumption, and apply authority live only in the hash-chained +journal; the sole ambiguous-operation resolution is the attended apply +publication, resolved by exact target-ref re-observation. There is no product +cleanup/prune/migration/expiry, suite adapter, unattended trigger, Browser +promotion, push, or deployment. ## Verification ```bash node --test tests/private-state-schema.test.mjs tests/state-kernel.test.mjs \ - tests/stage1-runtime-*.test.mjs tests/stage2-*.test.mjs + tests/stage1-runtime-*.test.mjs tests/stage2-*.test.mjs tests/stage3-*.test.mjs npm run check ``` diff --git a/docs/stage3-apply-v1.md b/docs/stage3-apply-v1.md new file mode 100644 index 0000000..00314a8 --- /dev/null +++ b/docs/stage3-apply-v1.md @@ -0,0 +1,270 @@ +# Stage 3 Apply v1 — preview, approval, consumption, apply, recovery + +Stage 3 v1 adds one attended, strict, fail-closed path from a finished Stage 2 +run to one explicit Git ref move. It is cooperative same-UID coordination on +exactly Herdr `0.7.5`, not authentication, sandboxing, remote publication, or +unattended orchestration. Every Stage 3 transition is explicitly invoked by an +attended operator. + +## What Stage 3 v1 is + +After Stage 2 collects every configured gate report, the operator may: + +1. **preview** — derive and durably journal one zero-effect preview document + binding the exact run, integration, gate-report, and apply-target state; +2. **approve** — record one attended approval receipt bound to the exact + preview journal entry through bounded stdin; +3. **consume** — durably spend that receipt for exactly one apply attempt + before any Git effect; +4. **apply** — move the configured apply ref with exactly zero or one + compare-and-swap from its precommitted SHA to the integrated final SHA; and +5. **recover** — resolve a crash between consumption and observation by exact + Git re-observation, because the apply effect is a single atomic ref update + whose outcome is exactly observable. + +Approval receipts, like every other worker or operator field, are +unauthenticated same-UID records — not signatures, authorization proof, or +remote attestation. Recovery exists for the apply publication only; Stage 2 +pane/agent/external effects retain their existing no-recovery boundary. + +## Configuration v3 + +Stage 3 requires configuration `version: 3`, which is exactly configuration v2 +plus one required top-level `apply` member: + +```json +{ + "version": 3, + "state_root": { "kind": "default" }, + "worktree_root": ".conductor-worktrees", + "apply": { "target_ref": "refs/heads/release" }, + "roles": [ ... ] +} +``` + +- `"apply": null` declares Stage 3 disabled; the run behaves exactly as + configuration v2. +- `apply.target_ref` is a full `refs/` ref name. It must differ from the + integration target branch ref observed at `integration.bind`; preview fails + closed otherwise. +- Configuration v2 remains accepted and byte-for-byte unchanged in meaning; + v2 runs have no Stage 3 operations. + +The configuration digest binding is unchanged: every action rereads the +configuration and refuses a changed run-bound digest. + +## Attempt model + +Stage 3 operations are grouped into **apply attempts**. An attempt is minted at +preview with one fresh attempt generation `g` and owns exactly four operation +ids: + +| Operation id | Operation type | Subject kind | Effect | +| --- | --- | --- | --- | +| `apply-preview-` | `apply.preview` | `apply` | none | +| `apply-approve-` | `approval.record` | `approval` | none | +| `apply-consume-` | `approval.consume` | `approval` | none | +| `apply-publish-` | `apply.publish` | `apply` | zero or one ref CAS | + +A new attempt is legal only when configuration v3 declares an apply target, +the lifecycle is `gate_reports_collected` or `integration_harvested_no_gates`, +and either no attempt exists or every prior attempt is closed. An attempt is +closed when its `apply.publish` is observed with outcome `unapplied`, or its +`approval.record` is observed with decision `reject`. An attempt whose +`apply.publish` is observed with outcome `applied` terminates Stage 3 for the +run: no further attempt is ever legal. At most 8 attempts may exist. Any +non-observed Stage 3 journal entry must be resolved before any new Stage 3 +operation. + +## Preview contract + +`preview` is an attended action. It requires complete strict Stage 2 +authority: one active version-2 run, validated run configuration binding, one +observed `integration.reconcile` and `integration.harvest`, and one accepted +completed report from every configured gate role (or a gateless +`integration_harvested_no_gates` run). It then observes, all under the +repository mutation lock: + +- the live integration target: its ref must still resolve to the integrated + `final_sha` with clean tracked state; +- the configured apply ref: it must exist, must not be the integration branch + ref, must not be checked out in any worktree of the repository, and must + resolve to exactly the integration `starting_sha` (the run's fork SHA), so + the proposed move is a pure fast-forward that cannot discard commits; +- the deterministic change summary between `starting_sha` and `final_sha` + (`git diff --name-status` byte digest and path count). + +The preview document is the operation's observed identity: + +- `document_type: "herdr-conductor-stage3-preview"`, `schema_version: 1` +- run scope (repository key, workspace, run id, run generation) +- integration binding (target ref, starting SHA, final SHA, integration + entry digest) +- gate binding: for every configured gate role in byte order, its task digest, + report digest, and asserted verdict fields — recorded as unauthenticated + worker assertions +- apply binding: target ref, observed SHA (= starting SHA), proposed final + SHA, `diff_name_status_sha256`, `changed_path_count` +- `attempt_generation` + +Preview performs zero Git mutations. A preview whose observation does not +match a re-observation inside the journaled effect fails closed. Any drift of +the apply ref or integration ref fails closed with zero recorded authority. +Gate reports asserting non-positive verdicts do not block preview; the +verdicts are recorded in the document and the decision remains the attended +operator's. The action prints the preview document, its journal entry digest, +and the exact approval command. + +## Approval receipt contract + +Approval is recorded by the attended CLI (`npm run apply:approve`), not by a +Herdr action, because it requires bounded stdin. The recorder accepts at most +16384 bytes, reads only stdin through actual EOF, and requires canonical +bytes. The receipt is: + +- `document_type: "herdr-conductor-stage3-approval"`, `schema_version: 1` +- run scope +- `preview`: the attempt generation and the exact observed preview journal + entry digest +- `decision`: `"approve"` or `"reject"` +- `statement`: exactly + `I_ATTENDED_THIS_EXACT_PREVIEW_AND_APPROVE` for approve or + `I_ATTENDED_THIS_EXACT_PREVIEW_AND_REJECT` for reject + +The recorder validates the receipt against the live journal under the +repository lock and requires the referenced preview to be the newest, still +open attempt. An `approve` receipt additionally requires the preview to still +be live (integration and apply refs unchanged); a `reject` receipt records on +a drifted target too, so an attempt can always close. The recorder journals +`approval.record` with the receipt as observed identity and the receipt digest +as request digest. The journal is the only durable approval store; there is no +approval file slot, payload path, destination, alternate FD, environment +payload, or replacement receipt. A second receipt for the same attempt is +refused (`replay_refused` when byte-identical, `operation_conflict` +otherwise). A `reject` receipt closes the attempt durably. + +The receipt remains an unauthenticated operator record. Fixed statement +tokens exist so no free-text claim can masquerade as broader authority. + +## Consumption and apply contract + +`apply` is an attended action. With an approve receipt observed for the +newest attempt and no uncertain Stage 3 entries, it: + +1. revalidates complete strict journal, configuration, and receipt-binding + authority; +2. journals `approval.consume`, whose observed identity binds the approval + journal entry digest and the attempt generation — the receipt is spent + before any Git effect and is never reusable, whatever follows; +3. journals `apply.publish`, whose effect decides the attempt's single + durable outcome: a live target (integration ref still `final_sha`, apply + ref still `starting_sha`, not checked out anywhere) moves through the + single compare-and-swap + `git update-ref ` after one final + in-effect preflight and is revalidated at exactly `final_sha`; a drifted + target records outcome `unapplied` with zero CAS, closing the attempt so + the run always progresses. + +The apply outcome document is the observed identity: + +- `document_type: "herdr-conductor-stage3-apply"`, `schema_version: 1` +- run scope, attempt generation, consumed approval entry digest +- `target_ref`, `expected_sha` (= starting SHA), `final_sha` +- `cas_count`: `1` for `applied`, `0` for `unapplied` +- `outcome`: `"applied"` or `"unapplied"` + +A drift observed before the CAS performs zero target CAS and durably closes +the attempt; its receipt is already consumed and a fresh attempt (new +preview, new receipt) is required. The action never retries the CAS, never +force-updates, never touches any other ref, never pushes, and never deletes +or rewrites anything. + +## Recovery contract + +A crash after `approval.consume` is observed but while `apply-publish-` is +non-observed (phase `intent` or `needs_attention`) leaves the run +`apply_uncertain`. Every other Conductor surface keeps refusing with +`recovery_required`. The attended `apply` action alone may resolve it: it +first reclaims the repository mutation lock when — and only when — the +retained owner process is dead and held exactly the apply action's own +operation id (any other retained lock keeps refusing), then resolves under +the lock by exact re-observation of the target ref: + +- ref resolves to exactly `final_sha` → the effect completed; the entry is + transitioned to `observed` with outcome `applied`, `cas_count: 1`; +- ref resolves to exactly `starting_sha` → the effect did not complete; the + entry is transitioned to `observed` with outcome `unapplied`, + `cas_count: 0`, closing the attempt; +- any other SHA, a missing ref, or an unreadable repository → resolution is + refused (`foreign_or_stale`); the run stays `apply_uncertain` and no Stage 3 + operation is ever again legal for the run. + +Resolution transitions the exact retained journal entry through the same +durable rewrite used by intent→observed, preserving the hash chain, and +removes the operation guard. Recovery never mutates Git, never replays the +CAS, and exists only for `apply.publish` — no other Conductor operation gains +recovery in Stage 3 v1. + +## Lifecycle states + +Stage 3 adds these disjoint stable states after `gate_reports_collected` / +`integration_harvested_no_gates` for configuration v3 runs with a non-null +apply target: + +| State | Meaning | Legal next operations | +| --- | --- | --- | +| `apply_previewed` | newest attempt has an observed preview, no receipt | approval record, stand-down | +| `apply_approved` | approve receipt observed, not consumed | apply, stand-down | +| `apply_rejected` | newest attempt closed by reject receipt | new preview, stand-down | +| `apply_consumed` | receipt consumed, publish not yet journaled | apply, stand-down refused | +| `applied` | publish observed with outcome applied | stand-down | +| `apply_voided` | newest attempt publish observed unapplied | new preview, stand-down | +| `apply_uncertain` | publish non-observed | attended apply resolution only | + +Stand-down from `applied`, `apply_rejected`, and `apply_voided` uses +`normal_completion`, `operator_abandoned`, or the states' existing reasons; +stand-down is refused in `apply_consumed` and `apply_uncertain` until the +publish is journaled or resolved. Configuration v2 runs and v3 runs with +`"apply": null` never enter these states. + +## Operation policy additions + +| Operation type | Subject kind | Prerequisite | Observed identity | +| --- | --- | --- | --- | +| `apply.preview` | `apply` | `integration.harvest` | `stage3-preview` | +| `approval.record` | `approval` | `apply.preview` | `stage3-approval` | +| `approval.consume` | `approval` | `approval.record` | `stage3-consumption` | +| `apply.publish` | `apply` | `approval.consume` | `stage3-apply` | + +All four are journaled with the existing intent/observed/needs_attention +machinery, hash chain, request digests, and durable guards. + +## Verification obligations + +Every Stage 3 change lands with negative tests proving zero forbidden +effects, in the existing suites' style: + +- preview drift (apply ref moved, integration ref moved, checked-out apply + ref, apply ref equal to integration ref, missing ref, non-fast-forward) + refuses with zero journal authority and zero Git mutation; +- foreign, stale, duplicate, replayed, malformed, oversized, and + non-canonical receipts are refused with exact codes; +- consumption is durable before the CAS under fault injection at every + checkpoint; +- crash inside the publish effect yields `apply_uncertain`; resolution + recovers `applied` and `unapplied` exactly and refuses any third + observation; +- a consumed receipt can never authorize a second CAS; +- the CAS count across any run history is exactly zero or one per attempt and + at most one `applied` attempt per run; +- configuration v2 byte-compatibility: the full existing Stage 2 suite passes + unchanged. + +## Non-goals + +Stage 3 v1 does not add: push, remote publication, deployment, release/tag +automation, suite adapters, unattended launch or triggers, Browser/Guard/ +Swarm promotion, product cleanup/prune/migration/expiry, recovery for any +operation other than `apply.publish`, approval delegation, or multi-ref +apply. Worker and operator fields remain unauthenticated same-UID +assertions; Git final checks retain the documented same-user TOCTOU windows. diff --git a/herdr-plugin.toml b/herdr-plugin.toml index 2d96059..65b0f13 100644 --- a/herdr-plugin.toml +++ b/herdr-plugin.toml @@ -1,13 +1,14 @@ id = "structupath.conductor" name = "Conductor" -version = "0.3.0" +version = "0.4.0" min_herdr_version = "0.7.5" -description = "Stage 2 attended strict task/report contracts and exact-SHA review gates for Herdr 0.7.5" +description = "Attended strict task/report contracts, exact-SHA review gates, and single-ref Stage 3 apply for Herdr 0.7.5" platforms = ["macos", "linux"] -# Exactly five attended actions share one strict context-bound runtime authority. -# Herdr actions receive no arguments. Configuration v2 contains no launch -# arguments; task authority is durable before any pane or agent starts. +# Exactly seven attended actions share one strict context-bound runtime +# authority. Herdr actions receive no arguments. Configuration contains no +# launch arguments; task authority is durable before any pane or agent starts, +# and approval authority is durable and consumed before any apply effect. [[actions]] id = "assemble" @@ -33,6 +34,18 @@ title = "Harvest the run" description = "Attended report collection, complete deterministic one-CAS integration, and exact-SHA gate dispatch" command = ["bash", "scripts/harvest.sh"] +[[actions]] +id = "preview" +title = "Preview the apply" +description = "Journal one zero-effect Stage 3 apply preview binding the exact integration, gate reports, and configured apply target ref" +command = ["bash", "scripts/preview.sh"] + +[[actions]] +id = "apply" +title = "Apply the approved run" +description = "Consume the recorded approve receipt exactly once and move the configured apply ref with exactly zero or one compare-and-swap" +command = ["bash", "scripts/apply.sh"] + [[actions]] id = "stand-down" title = "Stand down the team" diff --git a/package.json b/package.json index abc67de..6092026 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "herdr-conductor", - "version": "0.3.0", + "version": "0.4.0", "private": true, "type": "module", "engines": { @@ -8,7 +8,7 @@ }, "scripts": { "test": "node --test", - "test:stage2": "node --test tests/task-report-schema.test.mjs tests/report-publisher.test.mjs tests/stage2-*.test.mjs tests/state-kernel.test.mjs tests/stage1-runtime-*.test.mjs", + "test:stage2": "node --test tests/task-report-schema.test.mjs tests/report-publisher.test.mjs tests/stage2-*.test.mjs tests/stage3-*.test.mjs tests/state-kernel.test.mjs tests/stage1-runtime-*.test.mjs", "check:shell": "bash -n scripts/*.sh", "check:shellcheck": "shellcheck --shell=bash scripts/*.sh", "check:python": "python3 -m py_compile scripts/harness-fs-helper.py", @@ -18,6 +18,7 @@ "check:evidence": "node scripts/check-evidence-inventory.mjs && node scripts/check-b0-identity-evidence.mjs && node scripts/check-stage1-live-smoke-evidence.mjs", "check:release": "node scripts/check-stage2-live-evidence.mjs", "report:publish": "node scripts/report-publisher.mjs publish", + "apply:approve": "node scripts/approval-recorder.mjs record", "evidence:stage2:live": "node scripts/run-stage2-live-evidence.mjs", "evidence:stage2:finalize": "node scripts/finalize-stage2-live-evidence.mjs", "check": "npm test && npm run check:shell && npm run check:python && npm run check:manifest && npm run check:docs && npm run check:evidence" diff --git a/scripts/apply.sh b/scripts/apply.sh new file mode 100755 index 0000000..9c59d06 --- /dev/null +++ b/scripts/apply.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +# Stage 3: consume the approve receipt and move the apply ref with one CAS. +set -uo pipefail +cd "${HERDR_PLUGIN_ROOT:-$(dirname "$0")/..}" || { + echo "herdr-conductor: cannot resolve plugin root" + exit 1 +} +exec node scripts/stage1-runtime.mjs apply diff --git a/scripts/approval-recorder.mjs b/scripts/approval-recorder.mjs new file mode 100755 index 0000000..9514881 --- /dev/null +++ b/scripts/approval-recorder.mjs @@ -0,0 +1,299 @@ +#!/usr/bin/env node +import { createHash } from "node:crypto"; +import { execFileSync } from "node:child_process"; +import { readFileSync } from "node:fs"; +import { dirname, join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { stateRootForConfig } from "./state-root.mjs"; +import { + StateKernelError, + canonicalJson, + parseStrictJsonBytes, + validateStage3ApprovalIdentity, +} from "./private-state-schema.mjs"; +import { parseStage2ConfigBytes } from "./task-report-schema.mjs"; +import { + observeStage3ApplyTarget, + stage3ApplyTargetRef, +} from "./stage3-apply.mjs"; +import { + acquireRepositoryLock, + inspectRepositoryLock, + loadActiveRun, + openRepositoryStore, + performJournaledOperation, + releaseRepositoryLock, +} from "./state-kernel.mjs"; +import { readBoundedReportInput } from "./report-publisher.mjs"; + +export const APPROVAL_MAX_BYTES = 16_384; + +function fail(code, message, cause) { + throw new StateKernelError(code, message, cause ? { cause } : undefined); +} + +function sha256(value) { + return createHash("sha256") + .update(typeof value === "string" ? value : canonicalJson(value)) + .digest("hex"); +} + +function recorderConfiguration(configPath) { + const canonicalPath = resolve(configPath); + const config = parseStage2ConfigBytes(readFileSync(canonicalPath)); + const targetRef = stage3ApplyTargetRef(config); + if (!targetRef) + fail( + "capability_unavailable", + "configuration does not declare a Stage 3 apply target", + ); + return Object.freeze({ + configPath: canonicalPath, + config, + targetRef, + configurationDigest: sha256(config), + stateRoot: stateRootForConfig(config), + repositoryRoot: dirname(canonicalPath), + }); +} + +function refuseDeadRepositoryLock(store) { + const lock = inspectRepositoryLock(store); + if (lock.status !== "locked") return; + try { + process.kill(lock.owner.pid, 0); + } catch (error) { + if (error?.code !== "ESRCH") throw error; + fail( + "recovery_required", + "operation_uncertain: retained dead-process lock requires recovery", + ); + } +} + +function validateRunConfigurationBinding(active, configurationDigest) { + const bindings = active.journal.filter( + (entry) => + entry.operation_type === "integration.bind" && + entry.phase === "observed", + ); + if (bindings.length !== 1) + fail("bookkeeping_unknown", "run configuration binding is missing"); + const expected = sha256({ + configuration_digest: configurationDigest, + integration_target: bindings[0].observed_identity, + }); + if (bindings[0].request_digest !== expected) + fail("stale_task", "active run configuration digest changed"); +} + +function stage3AttemptEntries(active) { + const observed = (type) => + active.journal.filter( + (entry) => entry.operation_type === type && entry.phase === "observed", + ); + const previews = observed("apply.preview"); + if (previews.length === 0) + fail("state_unknown", "run has no previewed apply attempt"); + const previewEntry = previews.at(-1); + const generation = previewEntry.subject.generation; + const matching = (type) => + observed(type).filter((entry) => entry.subject.generation === generation); + return { + generation, + previewEntry, + approvalEntries: matching("approval.record"), + consumptionEntries: matching("approval.consume"), + publicationEntries: matching("apply.publish"), + }; +} + +function requireApprovableAttempt(active, receipt) { + if ( + active.journal.some( + (entry) => + entry.operation_type === "run.stand-down.begin" && + entry.phase === "observed", + ) + ) + fail("operation_conflict", "run has begun stand-down"); + const attempt = stage3AttemptEntries(active); + if (receipt.attempt_generation !== attempt.generation) + fail( + "stale_task", + "approval receipt does not name the newest apply attempt", + ); + if (receipt.preview_entry_digest !== attempt.previewEntry.entry_digest) + fail( + "digest_mismatch", + "approval receipt does not bind the exact preview journal entry", + ); + if ( + attempt.consumptionEntries.length > 0 || + attempt.publicationEntries.length > 0 + ) + fail("operation_conflict", "apply attempt is already consumed"); + if (attempt.approvalEntries.length > 0) + fail( + attempt.approvalEntries[0].request_digest === sha256(receipt) + ? "replay_refused" + : "operation_conflict", + "apply attempt already has a recorded receipt", + ); + return attempt; +} + +function validateReceiptScope(receipt, store, active) { + if ( + receipt.repository_key !== store.repository.key || + receipt.workspace_id !== active.state.workspace_id || + receipt.run_id !== active.state.run_id || + receipt.run_generation !== active.state.generation + ) + fail("foreign_or_stale", "approval receipt names a foreign scope"); +} + +function requireLivePreview(configuration, previewIdentity, exec) { + const observed = observeStage3ApplyTarget({ + repository: configuration.repositoryRoot, + targetRef: previewIdentity.apply.target_ref, + integration: { + target_ref: previewIdentity.integration.target_ref, + starting_sha: previewIdentity.integration.starting_sha, + final_sha: previewIdentity.integration.final_sha, + }, + exec, + }); + if (canonicalJson({ ...observed }) !== canonicalJson(previewIdentity.apply)) + fail( + "stale_source", + "previewed apply authority drifted and cannot be approved", + ); +} + +function defaultExec(command, args, options = {}) { + try { + return execFileSync(command, args, { + encoding: "utf8", + stdio: ["ignore", "pipe", "pipe"], + timeout: 60_000, + killSignal: "SIGKILL", + ...options, + }).trim(); + } catch (error) { + const stderr = + typeof error?.stderr === "string" ? error.stderr.trim().slice(0, 4096) : ""; + fail( + "external_operation_failed", + `${command} ${args.join(" ")} failed${stderr ? `: ${stderr}` : ""}`, + error, + ); + } +} + +export async function recordApprovalFromStdin({ + configPath, + input = process.stdin, + exec = defaultExec, + fault, +} = {}) { + const configuration = recorderConfiguration(configPath); + const bytes = await readBoundedReportInput(input, { + maxBytes: APPROVAL_MAX_BYTES, + }); + let receipt; + try { + receipt = validateStage3ApprovalIdentity( + parseStrictJsonBytes(bytes, { maxBytes: APPROVAL_MAX_BYTES }), + "approval receipt", + ); + } catch (error) { + if (error instanceof StateKernelError) throw error; + fail("invalid_json", "approval receipt is not strict JSON", error); + } + if (!bytes.equals(Buffer.from(canonicalJson(receipt)))) + fail("digest_mismatch", "approval receipt must equal its canonical bytes"); + const store = openRepositoryStore({ + stateRoot: configuration.stateRoot, + repoPath: configuration.repositoryRoot, + }); + if (join(store.repositoryRoot, ".herdr-conductor.json") !== configuration.configPath) + fail("path_mismatch", "recorder config path is not the repository config"); + refuseDeadRepositoryLock(store); + const lock = acquireRepositoryLock(store, { + operationId: `apply-approve-${receipt.attempt_generation}`, + fault, + }); + try { + const active = loadActiveRun(store, { + workspaceId: receipt.workspace_id, + expectedRunId: receipt.run_id, + expectedGeneration: receipt.run_generation, + }); + validateReceiptScope(receipt, store, active); + validateRunConfigurationBinding(active, configuration.configurationDigest); + const attempt = requireApprovableAttempt(active, receipt); + const previewIdentity = attempt.previewEntry.observed_identity; + if (receipt.decision === "approve") + requireLivePreview(configuration, previewIdentity, exec); + await performJournaledOperation(lock, { + workspaceId: receipt.workspace_id, + runId: active.state.run_id, + runGeneration: active.state.generation, + operationId: `apply-approve-${attempt.generation}`, + operationType: "approval.record", + subject: { + kind: "approval", + id: "apply", + generation: attempt.generation, + }, + requestDigest: sha256(receipt), + fault, + effect: async () => { + if (receipt.decision === "approve") + requireLivePreview(configuration, previewIdentity, exec); + return { resultDigest: sha256(receipt), observedIdentity: receipt }; + }, + }); + return Object.freeze({ + decision: receipt.decision, + attemptGeneration: attempt.generation, + receiptDigest: sha256(receipt), + }); + } finally { + releaseRepositoryLock(lock); + } +} + +async function main() { + const argv = process.argv.slice(2); + const command = argv[0]; + if ( + command !== "record" || + argv[1] !== "--config" || + !argv[2] || + argv.length !== 3 + ) { + process.stderr.write( + "usage: approval-recorder.mjs record --config \n", + ); + process.exitCode = 64; + return; + } + const result = await recordApprovalFromStdin({ + configPath: argv[2], + input: process.stdin, + }); + process.stdout.write(`${result.decision} ${result.receiptDigest}\n`); +} + +if ( + process.argv[1] && + resolve(process.argv[1]) === fileURLToPath(import.meta.url) +) + main().catch((error) => { + process.stderr.write( + `herdr-conductor: ${error.code ?? "internal_error"}: ${error.message}\n`, + ); + process.exitCode = 1; + }); diff --git a/scripts/check-docs.mjs b/scripts/check-docs.mjs index 2d33a9c..abbe7d0 100755 --- a/scripts/check-docs.mjs +++ b/scripts/check-docs.mjs @@ -9,6 +9,8 @@ const EXPECTED_ACTIONS = [ "board", "status", "harvest", + "preview", + "apply", "stand-down", ]; const EXPECTED_PANES = ["board-pane"]; @@ -30,7 +32,7 @@ const CANONICAL_VERIFICATION_FILES = [ "SECURITY.md", "docs/private-state-v1.md", ]; -const STAGE_RUNTIME_TEST = /^(?:stage1-runtime-.*|stage2-.*)\.test\.mjs$/; +const STAGE_RUNTIME_TEST = /^(?:stage1-runtime-.*|stage2-.*|stage3-.*)\.test\.mjs$/; const CURRENT_CLAIM_FILES = [ "README.md", "SECURITY.md", @@ -296,6 +298,20 @@ export function validateDocs(root) { "README.md", "there is no `CONDUCTOR_REPO`, ambient-cwd, process-ID, or newest-global fallback", ], + [ + "README.md", + "Approval receipts remain unauthenticated same-UID operator records", + ], + ["README.md", "A spent receipt never authorizes a second compare-and-swap"], + ["README.md", "Resolution exists only for the apply publication"], + [ + "SECURITY.md", + "an approve receipt is durably consumed before any apply effect", + ], + [ + "SECURITY.md", + "Ambiguous-operation resolution exists only for the Stage 3 apply publication", + ], ["README.md", "`harvest` is explicitly invoked and attended"], ["README.md", "`stand-down` closes only panes"], ["README.md", "B4 live smoke report"], @@ -349,7 +365,7 @@ if (process.argv[1] && path.resolve(process.argv[1]) === sourcePath) { process.exitCode = 1; } else { process.stdout.write( - `Docs valid: ${result.actionCount} actions agree and ${result.currentDocumentCount} current documents retain Stage 2 boundaries.\n`, + `Docs valid: ${result.actionCount} actions agree and ${result.currentDocumentCount} current documents retain release boundaries.\n`, ); } } diff --git a/scripts/check-manifest.mjs b/scripts/check-manifest.mjs index c0a1e6a..c02f590 100755 --- a/scripts/check-manifest.mjs +++ b/scripts/check-manifest.mjs @@ -113,6 +113,8 @@ export function validateRepository(root) { "board", "status", "harvest", + "preview", + "apply", "stand-down", ]; if ( @@ -121,7 +123,7 @@ export function validateRepository(root) { JSON.stringify(expectedActions) ) errors.push( - "manifest must declare exactly the five ordered Stage 2 actions", + "manifest must declare exactly the seven ordered Conductor actions", ); for (const action of manifest.actions ?? []) { const allowed = ["id", "title", "description", "command"]; diff --git a/scripts/operation-journal.mjs b/scripts/operation-journal.mjs index 50c2e0b..d601704 100755 --- a/scripts/operation-journal.mjs +++ b/scripts/operation-journal.mjs @@ -790,3 +790,128 @@ export async function performJournaledOperation( throw error; } } + +export function loadUncertainApplyRun(store, { workspaceId } = {}) { + const workspace = workspacePaths(store, workspaceId); + const pointers = scanActivePointers(store, workspace); + if (pointers.length !== 1) + throw kernelError( + "bookkeeping_unknown", + "apply resolution requires exactly one active pointer", + ); + const runStates = scanRunStates(store, workspace); + if (runStates.some(({ activationGuard }) => activationGuard !== null)) + throw kernelError( + "recovery_required", + "workspace has an unresolved activation guard", + ); + const activeStates = runStates.filter(({ state }) => state.status === "active"); + const nonArchived = runStates.filter(({ state }) => state.status !== "archived"); + if (activeStates.length !== 1 || nonArchived.length !== 1) + throw kernelError( + "bookkeeping_unknown", + "active pointer does not have one exclusive non-archived run state", + ); + const pointer = pointers[0].value; + const selected = activeStates[0]; + compareRunIdentity(store, workspace, pointer, selected.state); + const active = { + store, + workspace, + pointer, + pointerPath: pointers[0].path, + state: selected.state, + paths: selected.paths, + journal: [], + }; + active.journal = scanJournal(store, active, { allowGuards: true }); + const unresolved = active.journal.filter((entry) => entry.phase !== "observed"); + if (unresolved.length !== 1 || active.journal.at(-1) !== unresolved[0]) + throw kernelError( + "recovery_required", + "apply resolution requires exactly one terminal unresolved operation", + ); + const entry = unresolved[0]; + if (entry.operation_type !== "apply.publish") + throw kernelError( + "recovery_required", + "the unresolved operation is not an apply publication", + ); + const guards = active.journal.guards; + if ( + guards.length > 1 || + (guards.length === 1 && + (guards[0].operation_id !== entry.operation_id || + guards[0].sequence !== entry.sequence || + guards[0].phase !== "intent")) + ) + throw kernelError( + "recovery_required", + "apply resolution has foreign uncertainty guards", + ); + if (active.state.journal_head !== entry.entry_digest) + throw kernelError( + "bookkeeping_unknown", + "the uncertain apply publication is not the journal head", + ); + active.uncertain = entry; + active.uncertainGuard = guards[0] ?? null; + return active; +} + +export async function resolveUncertainApplyPublication( + handle, + { workspaceId, operationId, resolve, fault } = {}, +) { + assertLock(handle); + if (typeof resolve !== "function") + throw kernelError("invalid_state", "apply resolution requires a resolver"); + const active = loadUncertainApplyRun(handle.store, { workspaceId }); + const entry = active.uncertain; + if (operationId !== undefined && entry.operation_id !== operationId) + throw kernelError( + "operation_conflict", + "apply resolution does not match the uncertain operation", + ); + const observation = await resolve(entry, active); + if ( + observation === null || + typeof observation !== "object" || + Array.isArray(observation) || + Object.keys(observation).sort().join(",") !== "observedIdentity,resultDigest" + ) + throw kernelError( + "invalid_state", + "apply resolution returned an invalid observation", + ); + validateKey(observation.resultDigest, "result digest"); + checkpoint(fault, "resolution.after_observation"); + assertLock(handle); + const observed = transitionJournalEntry( + active, + journalPath(active, entry.sequence, entry.operation_id), + entry, + { + phase: "observed", + resultDigest: observation.resultDigest, + observedIdentity: observation.observedIdentity, + errorCode: null, + fault, + scope: "journal_resolution", + }, + ); + if (active.uncertainGuard) + removePrivateGuard( + journalGuardPath(active, entry.sequence, entry.operation_id), + { + parent: active.paths.operationGuardsDir, + fault, + scope: "journal_resolution_guard_remove", + }, + ); + return { + resolved: true, + resultDigest: observed.result_digest, + sequence: entry.sequence, + }; +} diff --git a/scripts/operation-policy.mjs b/scripts/operation-policy.mjs index 60e43fa..04bcc0b 100755 --- a/scripts/operation-policy.mjs +++ b/scripts/operation-policy.mjs @@ -70,6 +70,26 @@ export const OPERATION_POLICIES = Object.freeze({ prerequisite: null, observedIdentity: null, }), + "apply.preview": Object.freeze({ + subjectKind: "apply", + prerequisite: "integration.harvest", + observedIdentity: "stage3-preview", + }), + "approval.record": Object.freeze({ + subjectKind: "approval", + prerequisite: "apply.preview", + observedIdentity: "stage3-approval", + }), + "approval.consume": Object.freeze({ + subjectKind: "approval", + prerequisite: "approval.record", + observedIdentity: "stage3-consumption", + }), + "apply.publish": Object.freeze({ + subjectKind: "apply", + prerequisite: "approval.consume", + observedIdentity: "stage3-apply", + }), }); export const OPERATION_TYPES = Object.freeze(Object.keys(OPERATION_POLICIES)); diff --git a/scripts/preview.sh b/scripts/preview.sh new file mode 100755 index 0000000..2a42eac --- /dev/null +++ b/scripts/preview.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +# Stage 3: journal one zero-effect apply preview for the configured target ref. +set -uo pipefail +cd "${HERDR_PLUGIN_ROOT:-$(dirname "$0")/..}" || { + echo "herdr-conductor: cannot resolve plugin root" + exit 1 +} +exec node scripts/stage1-runtime.mjs preview diff --git a/scripts/private-state-schema.mjs b/scripts/private-state-schema.mjs index bb564a6..7d6d2be 100755 --- a/scripts/private-state-schema.mjs +++ b/scripts/private-state-schema.mjs @@ -527,6 +527,246 @@ function validateMergeIdentity(value, label) { return value; } +export const STAGE3_APPROVAL_STATEMENTS = Object.freeze({ + approve: "I_ATTENDED_THIS_EXACT_PREVIEW_AND_APPROVE", + reject: "I_ATTENDED_THIS_EXACT_PREVIEW_AND_REJECT", +}); + +function validateStage3RunScope(value, label) { + validateKey(value.repository_key, `${label}.repository_key`); + validateId(value.workspace_id, `${label}.workspace_id`); + validateId(value.run_id, `${label}.run_id`); + validateGeneration(value.run_generation, `${label}.run_generation`); + validateGeneration(value.attempt_generation, `${label}.attempt_generation`); +} + +export function validateStage3PreviewIdentity(value, label) { + exactKeys( + value, + [ + "document_type", + "schema_version", + "repository_key", + "workspace_id", + "run_id", + "run_generation", + "attempt_generation", + "integration", + "gates", + "apply", + ], + label, + ); + if ( + value.document_type !== "herdr-conductor-stage3-preview" || + value.schema_version !== 1 + ) + fail("invalid_state", `${label} has the wrong document type or version`); + validateStage3RunScope(value, label); + exactKeys( + value.integration, + ["target_ref", "starting_sha", "final_sha", "integration_entry_digest"], + `${label}.integration`, + ); + validateFullRef(value.integration.target_ref, `${label}.integration.target_ref`); + validateGitObjectId( + value.integration.starting_sha, + `${label}.integration.starting_sha`, + ); + validateGitObjectId( + value.integration.final_sha, + `${label}.integration.final_sha`, + ); + validateKey( + value.integration.integration_entry_digest, + `${label}.integration.integration_entry_digest`, + ); + if (!Array.isArray(value.gates) || value.gates.length > 64) + fail("invalid_state", `${label}.gates is invalid`); + for (const [index, gate] of value.gates.entries()) { + const gateLabel = `${label}.gates[${index}]`; + exactKeys( + gate, + [ + "role_name", + "contract_role", + "task_digest", + "report_digest", + "status", + "result_kind", + "verdict", + ], + gateLabel, + ); + stringMatching( + gate.role_name, + /^[a-z][a-z0-9_-]{0,31}$/, + `${gateLabel}.role_name`, + ); + enumValue( + gate.contract_role, + new Set(["reviewer", "validator"]), + `${gateLabel}.contract_role`, + ); + validateKey(gate.task_digest, `${gateLabel}.task_digest`); + validateKey(gate.report_digest, `${gateLabel}.report_digest`); + enumValue(gate.status, new Set(["completed"]), `${gateLabel}.status`); + const expectedKind = + gate.contract_role === "reviewer" ? "review" : "validation"; + if (gate.result_kind !== expectedKind) + fail("invalid_state", `${gateLabel}.result_kind is invalid`); + enumValue( + gate.verdict, + gate.contract_role === "reviewer" + ? new Set(["approve", "request_changes"]) + : new Set(["pass", "fail"]), + `${gateLabel}.verdict`, + ); + if ( + index > 0 && + Buffer.compare( + Buffer.from(value.gates[index - 1].role_name), + Buffer.from(gate.role_name), + ) >= 0 + ) + fail("invalid_state", `${label}.gates is not byte-ordered and unique`); + } + exactKeys( + value.apply, + [ + "target_ref", + "observed_sha", + "final_sha", + "diff_name_status_sha256", + "changed_path_count", + ], + `${label}.apply`, + ); + validateFullRef(value.apply.target_ref, `${label}.apply.target_ref`); + validateGitObjectId(value.apply.observed_sha, `${label}.apply.observed_sha`); + validateGitObjectId(value.apply.final_sha, `${label}.apply.final_sha`); + validateKey( + value.apply.diff_name_status_sha256, + `${label}.apply.diff_name_status_sha256`, + ); + safeInteger(value.apply.changed_path_count, `${label}.apply.changed_path_count`, 1); + if ( + value.apply.target_ref === value.integration.target_ref || + value.apply.observed_sha !== value.integration.starting_sha || + value.apply.final_sha !== value.integration.final_sha || + value.apply.observed_sha === value.apply.final_sha + ) + fail("invalid_state", `${label}.apply does not bind its integration`); + return value; +} + +export function validateStage3ApprovalIdentity(value, label) { + exactKeys( + value, + [ + "document_type", + "schema_version", + "repository_key", + "workspace_id", + "run_id", + "run_generation", + "attempt_generation", + "preview_entry_digest", + "decision", + "statement", + ], + label, + ); + if ( + value.document_type !== "herdr-conductor-stage3-approval" || + value.schema_version !== 1 + ) + fail("invalid_state", `${label} has the wrong document type or version`); + validateStage3RunScope(value, label); + validateKey(value.preview_entry_digest, `${label}.preview_entry_digest`); + enumValue( + value.decision, + new Set(["approve", "reject"]), + `${label}.decision`, + ); + if (value.statement !== STAGE3_APPROVAL_STATEMENTS[value.decision]) + fail("invalid_state", `${label}.statement is invalid`); + return value; +} + +export function validateStage3ConsumptionIdentity(value, label) { + exactKeys( + value, + [ + "document_type", + "schema_version", + "repository_key", + "workspace_id", + "run_id", + "run_generation", + "attempt_generation", + "approval_entry_digest", + "preview_entry_digest", + ], + label, + ); + if ( + value.document_type !== "herdr-conductor-stage3-consumption" || + value.schema_version !== 1 + ) + fail("invalid_state", `${label} has the wrong document type or version`); + validateStage3RunScope(value, label); + validateKey(value.approval_entry_digest, `${label}.approval_entry_digest`); + validateKey(value.preview_entry_digest, `${label}.preview_entry_digest`); + return value; +} + +export function validateStage3ApplyIdentity(value, label) { + exactKeys( + value, + [ + "document_type", + "schema_version", + "repository_key", + "workspace_id", + "run_id", + "run_generation", + "attempt_generation", + "consumption_entry_digest", + "target_ref", + "expected_sha", + "final_sha", + "cas_count", + "outcome", + ], + label, + ); + if ( + value.document_type !== "herdr-conductor-stage3-apply" || + value.schema_version !== 1 + ) + fail("invalid_state", `${label} has the wrong document type or version`); + validateStage3RunScope(value, label); + validateKey( + value.consumption_entry_digest, + `${label}.consumption_entry_digest`, + ); + validateFullRef(value.target_ref, `${label}.target_ref`); + validateGitObjectId(value.expected_sha, `${label}.expected_sha`); + validateGitObjectId(value.final_sha, `${label}.final_sha`); + enumValue( + value.outcome, + new Set(["applied", "unapplied"]), + `${label}.outcome`, + ); + if ( + value.expected_sha === value.final_sha || + value.cas_count !== (value.outcome === "applied" ? 1 : 0) + ) + fail("invalid_state", `${label} outcome does not bind its effect`); + return value; +} + export function validateRepositoryDocument(value) { exactKeys( value, @@ -773,8 +1013,47 @@ export function validateJournalEntry(value) { value.observed_identity, "journal observed_identity", ); + else if (policy.observedIdentity === "stage3-preview") + validateStage3PreviewIdentity( + value.observed_identity, + "journal observed_identity", + ); + else if (policy.observedIdentity === "stage3-approval") + validateStage3ApprovalIdentity( + value.observed_identity, + "journal observed_identity", + ); + else if (policy.observedIdentity === "stage3-consumption") + validateStage3ConsumptionIdentity( + value.observed_identity, + "journal observed_identity", + ); + else if (policy.observedIdentity === "stage3-apply") + validateStage3ApplyIdentity( + value.observed_identity, + "journal observed_identity", + ); else fail("invalid_state", "operation does not permit observed identity"); } + if (policy.observedIdentity?.startsWith("stage3-")) { + const identity = value.observed_identity; + if (identity === null) + fail( + "invalid_state", + "stage3 journal entry requires its observed identity", + ); + if ( + identity.repository_key !== value.repository_key || + identity.workspace_id !== value.workspace_id || + identity.run_id !== value.run_id || + identity.run_generation !== value.run_generation || + identity.attempt_generation !== value.subject.generation + ) + fail( + "invalid_state", + "stage3 observed identity does not bind its journal entry", + ); + } if (value.error_code !== null) fail("invalid_state", "observed journal has an error code"); } else { diff --git a/scripts/run-stage2-live-evidence.mjs b/scripts/run-stage2-live-evidence.mjs index dea5f50..735965d 100755 --- a/scripts/run-stage2-live-evidence.mjs +++ b/scripts/run-stage2-live-evidence.mjs @@ -951,11 +951,11 @@ export async function runLiveStage2Evidence({ ); assert.equal( JSON.parse(readFileSync(join(pluginCheckout, "package.json"))).version, - "0.3.0", + "0.4.0", ); assert.match( readFileSync(join(pluginCheckout, "herdr-plugin.toml"), "utf8"), - /^version = "0\.3\.0"$/m, + /^version = "0\.4\.0"$/m, ); assert.equal(command("herdr", ["--version"]), "herdr 0.7.5"); const server = command("herdr", ["status", "server"]); diff --git a/scripts/stage1-runtime.mjs b/scripts/stage1-runtime.mjs index 621ad2b..12a2cdd 100755 --- a/scripts/stage1-runtime.mjs +++ b/scripts/stage1-runtime.mjs @@ -18,6 +18,9 @@ import { validateGeneration, validateGitObjectId, validateId, + validateStage3ApplyIdentity, + validateStage3ConsumptionIdentity, + validateStage3PreviewIdentity, } from "./private-state-schema.mjs"; import { parseReportBytes, @@ -47,9 +50,17 @@ import { openRepositoryStore, performJournaledOperation, readStablePrivateBytes, + reclaimDeadRepositoryLock, releaseRepositoryLock, resolveGitCommonDirectory, + resolveUncertainApplyPublication, } from "./state-kernel.mjs"; +import { + observeStage3ApplyTarget, + publishStage3ApplyCas, + resolveStage3ApplyOutcome, + stage3ApplyTargetRef, +} from "./stage3-apply.mjs"; import { createGateSource, validateGateSource } from "./gate-source.mjs"; import { bindIntegrationTarget, @@ -2184,6 +2195,571 @@ export async function standDown({ } } +function stage3IntegrationAuthority(active) { + const entries = exactObservedEntries(active, "integration.harvest"); + if (entries.length !== 1) + fail("bookkeeping_unknown", "stage3 requires one observed integration harvest"); + return { + integration: entries[0].observed_identity, + integrationEntryDigest: entries[0].entry_digest, + }; +} + +function stage3GateBinding(active, config, stateRoot) { + const acceptedReports = deriveRetainedReportAuthority(active, stateRoot); + const gateRoles = config.roles + .filter( + (role) => + role.contract_role === "reviewer" || role.contract_role === "validator", + ) + .sort((left, right) => + Buffer.compare(Buffer.from(left.name), Buffer.from(right.name)), + ); + return gateRoles.map((role) => { + const report = acceptedReports[role.name]; + if (!report) + fail( + "operation_conflict", + `gate ${role.name} has no accepted report for preview`, + ); + if (report.status !== "completed" || report.result === null) + fail( + "operation_conflict", + `gate ${role.name} report is not completed and cannot be previewed`, + ); + return { + role_name: role.name, + contract_role: role.contract_role, + task_digest: report.task.digest, + report_digest: report.report_digest, + status: report.status, + result_kind: report.result.kind, + verdict: report.result.verdict, + }; + }); +} + +function stage3AttemptAuthority(active) { + const observed = (type) => + active.journal.filter( + (entry) => entry.operation_type === type && entry.phase === "observed", + ); + const previews = observed("apply.preview"); + if (previews.length === 0) return null; + const newest = previews.at(-1); + const generation = newest.subject.generation; + const one = (entries, label) => { + const matching = entries.filter( + (entry) => entry.subject.generation === generation, + ); + if (matching.length > 1) + fail("bookkeeping_unknown", `stage3 ${label} authority is duplicated`); + return matching[0] ?? null; + }; + return { + generation, + attemptCount: previews.length, + previewEntry: newest, + approvalEntry: one(observed("approval.record"), "approval"), + consumptionEntry: one(observed("approval.consume"), "consumption"), + publicationEntry: one(observed("apply.publish"), "publication"), + }; +} + +function buildStage3PreviewIdentity({ + active, + store, + config, + targetRef, + attemptGeneration, + stateRoot, + exec, +}) { + const { integration, integrationEntryDigest } = stage3IntegrationAuthority(active); + const gates = stage3GateBinding(active, config, stateRoot); + const applyObservation = observeStage3ApplyTarget({ + repository: store.repositoryRoot, + targetRef, + integration: { + target_ref: integration.target_ref, + starting_sha: integration.starting_sha, + final_sha: integration.final_sha, + }, + exec, + }); + const identity = { + document_type: "herdr-conductor-stage3-preview", + schema_version: 1, + repository_key: store.repository.key, + workspace_id: active.state.workspace_id, + run_id: active.state.run_id, + run_generation: active.state.generation, + attempt_generation: attemptGeneration, + integration: { + target_ref: integration.target_ref, + starting_sha: integration.starting_sha, + final_sha: integration.final_sha, + integration_entry_digest: integrationEntryDigest, + }, + gates, + apply: { + target_ref: applyObservation.target_ref, + observed_sha: applyObservation.observed_sha, + final_sha: applyObservation.final_sha, + diff_name_status_sha256: applyObservation.diff_name_status_sha256, + changed_path_count: applyObservation.changed_path_count, + }, + }; + return validateStage3PreviewIdentity(identity, "stage3 preview"); +} + +function stage3LiveApplyObservation(store, previewIdentity, exec) { + const observed = observeStage3ApplyTarget({ + repository: store.repositoryRoot, + targetRef: previewIdentity.apply.target_ref, + integration: { + target_ref: previewIdentity.integration.target_ref, + starting_sha: previewIdentity.integration.starting_sha, + final_sha: previewIdentity.integration.final_sha, + }, + exec, + }); + if (canonicalJson({ ...observed }) !== canonicalJson(previewIdentity.apply)) + fail( + "stale_source", + "apply target observation no longer matches the previewed authority", + ); + return observed; +} + +function stage3ApprovalCommand(config) { + return `node ${JSON.stringify(fileURLToPath(new URL("./approval-recorder.mjs", import.meta.url)))} record --config ${JSON.stringify(config.path)}`; +} + +function requireStage3Target(config) { + const targetRef = stage3ApplyTargetRef(config.raw); + if (!targetRef) + fail( + "capability_unavailable", + "configuration does not declare a Stage 3 apply target", + ); + return targetRef; +} + +export async function preview({ + contextJson = process.env.HERDR_PLUGIN_CONTEXT_JSON, + configPath, + herdrBin = process.env.HERDR_BIN_PATH ?? "herdr", + exec = defaultExec, + random = randomBytes, + fault, +} = {}) { + const context = parsePluginContext(contextJson); + const config = parseConfig( + configPath ?? join(context.workspaceCwd, ".herdr-conductor.json"), + ); + const stateRoot = config.stateRoot; + requireHerdrRuntime(exec, herdrBin); + const store = openExistingStore({ stateRoot, repoPath: context.workspaceCwd }); + if (store.repositoryRoot !== context.workspaceCwd) + fail( + "foreign_repository", + "workspace_cwd must be the canonical repository root", + ); + const targetRef = requireStage3Target(config); + refuseDeadRepositoryLock(store); + validateRunConfiguration( + loadActiveRun(store, { workspaceId: context.workspaceId }), + config, + ); + const lock = acquireRepositoryLock(store, { + operationId: `preview-${sha256(context.workspaceId).slice(0, 24)}`, + fault, + }); + try { + let active = loadActiveRun(store, { workspaceId: context.workspaceId }); + validateRunConfiguration(active, config); + const lifecycle = scanStage2Authority(active, config.raw, { + acceptedReports: deriveRetainedReportAuthority(active, stateRoot), + sourceRefused: gateSourceRefusals(active, config, exec), + }); + const attempt = stage3AttemptAuthority(active); + if ( + ["apply_previewed", "apply_approved", "apply_consumed"].includes( + lifecycle.state, + ) + ) + return { + run_id: active.state.run_id, + workspace_id: context.workspaceId, + lifecycle: lifecycle.state, + preview: attempt.previewEntry.observed_identity, + preview_entry_digest: attempt.previewEntry.entry_digest, + approval_command: stage3ApprovalCommand(config), + replayed: true, + }; + if (lifecycle.state === "applied") + return { + run_id: active.state.run_id, + workspace_id: context.workspaceId, + lifecycle: "applied", + apply: attempt.publicationEntry.observed_identity, + replayed: true, + }; + if ( + !["gate_reports_collected", "integration_harvested_no_gates"].includes( + lifecycle.state, + ) && + !["apply_rejected", "apply_voided"].includes(lifecycle.state) + ) + fail( + "operation_conflict", + `preview is not legal in lifecycle ${lifecycle.state}`, + ); + if ((attempt?.attemptCount ?? 0) >= 8) + fail("operation_conflict", "stage3 attempt limit is exhausted"); + const attemptGeneration = randomToken(16, random); + const identity = buildStage3PreviewIdentity({ + active, + store, + config, + targetRef, + attemptGeneration, + stateRoot, + exec, + }); + await performJournaledOperation(lock, { + workspaceId: context.workspaceId, + runId: active.state.run_id, + runGeneration: active.state.generation, + operationId: `apply-preview-${attemptGeneration}`, + operationType: "apply.preview", + subject: { kind: "apply", id: "apply", generation: attemptGeneration }, + requestDigest: sha256({ + configuration_digest: config.digest, + identity, + }), + fault: operationFault(fault, "apply.preview"), + effect: async () => { + const reobserved = buildStage3PreviewIdentity({ + active, + store, + config, + targetRef, + attemptGeneration, + stateRoot, + exec, + }); + if (canonicalJson(reobserved) !== canonicalJson(identity)) + fail("foreign_or_stale", "apply preview changed during binding"); + return { resultDigest: sha256(identity), observedIdentity: identity }; + }, + }); + active = loadActiveRun(store, { workspaceId: context.workspaceId }); + const recorded = stage3AttemptAuthority(active); + if (recorded?.generation !== attemptGeneration) + fail("bookkeeping_unknown", "recorded preview authority is missing"); + return { + run_id: active.state.run_id, + workspace_id: context.workspaceId, + lifecycle: "apply_previewed", + preview: recorded.previewEntry.observed_identity, + preview_entry_digest: recorded.previewEntry.entry_digest, + approval_command: stage3ApprovalCommand(config), + replayed: false, + }; + } finally { + releaseRepositoryLock(lock); + } +} + +function stage3PublishRequest(previewIdentity, consumptionEntryDigest) { + return { + consumption_entry_digest: consumptionEntryDigest, + target_ref: previewIdentity.apply.target_ref, + expected_sha: previewIdentity.apply.observed_sha, + final_sha: previewIdentity.apply.final_sha, + attempt_generation: previewIdentity.attempt_generation, + }; +} + +async function resolveStage3Uncertainty({ lock, store, context, exec, fault }) { + return resolveUncertainApplyPublication(lock, { + workspaceId: context.workspaceId, + fault, + resolve: (entry, uncertainActive) => { + const previewEntry = uncertainActive.journal.find( + (candidate) => + candidate.operation_type === "apply.preview" && + candidate.phase === "observed" && + candidate.subject.generation === entry.subject.generation, + ); + const consumptionEntry = uncertainActive.journal.find( + (candidate) => + candidate.operation_type === "approval.consume" && + candidate.phase === "observed" && + candidate.subject.generation === entry.subject.generation, + ); + if (!previewEntry || !consumptionEntry) + fail( + "bookkeeping_unknown", + "uncertain apply publication is missing its attempt authority", + ); + const previewIdentity = previewEntry.observed_identity; + if ( + entry.request_digest !== + sha256(stage3PublishRequest(previewIdentity, consumptionEntry.entry_digest)) + ) + fail( + "bookkeeping_unknown", + "uncertain apply publication does not bind its attempt", + ); + const outcome = resolveStage3ApplyOutcome({ + repository: store.repositoryRoot, + targetRef: previewIdentity.apply.target_ref, + expectedSha: previewIdentity.apply.observed_sha, + finalSha: previewIdentity.apply.final_sha, + exec, + }); + const identity = validateStage3ApplyIdentity( + { + document_type: "herdr-conductor-stage3-apply", + schema_version: 1, + repository_key: store.repository.key, + workspace_id: uncertainActive.state.workspace_id, + run_id: uncertainActive.state.run_id, + run_generation: uncertainActive.state.generation, + attempt_generation: entry.subject.generation, + consumption_entry_digest: consumptionEntry.entry_digest, + target_ref: previewIdentity.apply.target_ref, + expected_sha: previewIdentity.apply.observed_sha, + final_sha: previewIdentity.apply.final_sha, + cas_count: outcome.cas_count, + outcome: outcome.outcome, + }, + "stage3 apply resolution", + ); + return { resultDigest: sha256(identity), observedIdentity: identity }; + }, + }); +} + +export async function applyStage3({ + contextJson = process.env.HERDR_PLUGIN_CONTEXT_JSON, + configPath, + herdrBin = process.env.HERDR_BIN_PATH ?? "herdr", + exec = defaultExec, + fault, +} = {}) { + const context = parsePluginContext(contextJson); + const config = parseConfig( + configPath ?? join(context.workspaceCwd, ".herdr-conductor.json"), + ); + const stateRoot = config.stateRoot; + requireHerdrRuntime(exec, herdrBin); + const store = openExistingStore({ stateRoot, repoPath: context.workspaceCwd }); + if (store.repositoryRoot !== context.workspaceCwd) + fail( + "foreign_repository", + "workspace_cwd must be the canonical repository root", + ); + requireStage3Target(config); + const lock = reclaimDeadRepositoryLock(store, { + operationId: `apply-${sha256(context.workspaceId).slice(0, 24)}`, + fault, + }); + try { + let resolution = null; + try { + loadActiveRun(store, { workspaceId: context.workspaceId }); + } catch (error) { + if ( + !(error instanceof StateKernelError) || + error.code !== "recovery_required" + ) + throw error; + resolution = await resolveStage3Uncertainty({ + lock, + store, + context, + exec, + fault, + }); + } + let active = loadActiveRun(store, { workspaceId: context.workspaceId }); + validateRunConfiguration(active, config); + const lifecycle = scanStage2Authority(active, config.raw, { + acceptedReports: deriveRetainedReportAuthority(active, stateRoot), + sourceRefused: gateSourceRefusals(active, config, exec), + }); + const attempt = stage3AttemptAuthority(active); + if (lifecycle.state === "applied") + return { + run_id: active.state.run_id, + workspace_id: context.workspaceId, + lifecycle: "applied", + apply: attempt.publicationEntry.observed_identity, + replayed: resolution === null, + resolved: resolution !== null, + }; + if (lifecycle.state === "apply_voided") + return { + run_id: active.state.run_id, + workspace_id: context.workspaceId, + lifecycle: "apply_voided", + apply: attempt.publicationEntry.observed_identity, + resolved: resolution !== null, + }; + if (!["apply_approved", "apply_consumed"].includes(lifecycle.state)) + fail( + "operation_conflict", + `apply is not legal in lifecycle ${lifecycle.state}`, + ); + const previewIdentity = attempt.previewEntry.observed_identity; + if (attempt.approvalEntry.observed_identity.decision !== "approve") + fail("operation_conflict", "apply requires an approve receipt"); + if ( + attempt.approvalEntry.observed_identity.preview_entry_digest !== + attempt.previewEntry.entry_digest + ) + fail( + "bookkeeping_unknown", + "approval receipt does not bind the previewed authority", + ); + const consumptionIdentity = validateStage3ConsumptionIdentity( + { + document_type: "herdr-conductor-stage3-consumption", + schema_version: 1, + repository_key: store.repository.key, + workspace_id: active.state.workspace_id, + run_id: active.state.run_id, + run_generation: active.state.generation, + attempt_generation: attempt.generation, + approval_entry_digest: attempt.approvalEntry.entry_digest, + preview_entry_digest: attempt.previewEntry.entry_digest, + }, + "stage3 consumption", + ); + await performJournaledOperation(lock, { + workspaceId: context.workspaceId, + runId: active.state.run_id, + runGeneration: active.state.generation, + operationId: `apply-consume-${attempt.generation}`, + operationType: "approval.consume", + subject: { + kind: "approval", + id: "apply", + generation: attempt.generation, + }, + requestDigest: sha256(consumptionIdentity), + fault: operationFault(fault, "approval.consume"), + effect: async () => ({ + resultDigest: sha256(consumptionIdentity), + observedIdentity: consumptionIdentity, + }), + }); + active = loadActiveRun(store, { workspaceId: context.workspaceId }); + const consumed = stage3AttemptAuthority(active); + if ( + consumed?.generation !== attempt.generation || + !consumed.consumptionEntry + ) + fail("bookkeeping_unknown", "consumption authority is missing"); + const publishRequest = stage3PublishRequest( + previewIdentity, + consumed.consumptionEntry.entry_digest, + ); + const applyIdentityFor = (outcome) => + validateStage3ApplyIdentity( + { + document_type: "herdr-conductor-stage3-apply", + schema_version: 1, + repository_key: store.repository.key, + workspace_id: active.state.workspace_id, + run_id: active.state.run_id, + run_generation: active.state.generation, + attempt_generation: attempt.generation, + consumption_entry_digest: consumed.consumptionEntry.entry_digest, + target_ref: previewIdentity.apply.target_ref, + expected_sha: previewIdentity.apply.observed_sha, + final_sha: previewIdentity.apply.final_sha, + cas_count: outcome === "applied" ? 1 : 0, + outcome, + }, + "stage3 apply", + ); + let publishedOutcome = null; + await performJournaledOperation(lock, { + workspaceId: context.workspaceId, + runId: active.state.run_id, + runGeneration: active.state.generation, + operationId: `apply-publish-${attempt.generation}`, + operationType: "apply.publish", + subject: { kind: "apply", id: "apply", generation: attempt.generation }, + requestDigest: sha256(publishRequest), + fault: operationFault(fault, "apply.publish"), + effect: async () => { + let live = true; + try { + stage3LiveApplyObservation(store, previewIdentity, exec); + } catch (error) { + if ( + !(error instanceof StateKernelError) || + !["stale_source", "foreign_or_stale"].includes(error.code) + ) + throw error; + live = false; + } + if (live) + publishStage3ApplyCas({ + repository: store.repositoryRoot, + targetRef: previewIdentity.apply.target_ref, + expectedSha: previewIdentity.apply.observed_sha, + finalSha: previewIdentity.apply.final_sha, + preflight: () => + stage3LiveApplyObservation(store, previewIdentity, exec), + exec, + fault: operationFault(fault, "apply.publish"), + }); + publishedOutcome = live ? "applied" : "unapplied"; + const identity = applyIdentityFor(publishedOutcome); + return { + resultDigest: sha256(identity), + observedIdentity: identity, + }; + }, + validateBeforeResultPublication: async () => { + if ( + publishedOutcome === "applied" && + gitLine(exec, store.repositoryRoot, [ + "rev-parse", + previewIdentity.apply.target_ref, + ]) !== previewIdentity.apply.final_sha + ) + fail( + "durability_unknown", + "published apply ref changed before result publication", + ); + }, + }); + active = loadActiveRun(store, { workspaceId: context.workspaceId }); + const published = stage3AttemptAuthority(active); + const outcome = published?.publicationEntry?.observed_identity?.outcome; + if (outcome !== "applied" && outcome !== "unapplied") + fail("bookkeeping_unknown", "published apply authority is missing"); + return { + run_id: active.state.run_id, + workspace_id: context.workspaceId, + lifecycle: outcome === "applied" ? "applied" : "apply_voided", + apply: published.publicationEntry.observed_identity, + replayed: false, + resolved: resolution !== null, + }; + } finally { + releaseRepositoryLock(lock); + } +} + export function readStatus({ contextJson = process.env.HERDR_PLUGIN_CONTEXT_JSON, configPath, @@ -2320,11 +2896,15 @@ async function main() { process.stdout.write(canonicalJson(readStatus())); else if (command === "harvest") process.stdout.write(canonicalJson(await reconcile())); + else if (command === "preview") + process.stdout.write(canonicalJson(await preview())); + else if (command === "apply") + process.stdout.write(canonicalJson(await applyStage3())); else if (command === "stand-down") process.stdout.write(canonicalJson(await standDown())); else { process.stderr.write( - "usage: stage1-runtime.mjs assemble|board|status|harvest|stand-down\n", + "usage: stage1-runtime.mjs assemble|board|status|harvest|preview|apply|stand-down\n", ); process.exitCode = 64; } diff --git a/scripts/stage2-checkpoint-catalog.mjs b/scripts/stage2-checkpoint-catalog.mjs index f2e6325..34ba8cb 100755 --- a/scripts/stage2-checkpoint-catalog.mjs +++ b/scripts/stage2-checkpoint-catalog.mjs @@ -107,6 +107,19 @@ const GATE_PANE_CHECKPOINTS = unique(JOURNALED_OPERATION_CHECKPOINTS, [ const GATE_AGENT_CHECKPOINTS = JOURNALED_OPERATION_CHECKPOINTS; const PANE_CLOSE_CHECKPOINTS = JOURNALED_OPERATION_CHECKPOINTS; const STAND_DOWN_BEGIN_CHECKPOINTS = JOURNALED_OPERATION_CHECKPOINTS; +const APPLY_PREVIEW_CHECKPOINTS = JOURNALED_OPERATION_CHECKPOINTS; +const APPROVAL_RECORD_CHECKPOINTS = JOURNALED_OPERATION_CHECKPOINTS; +const APPROVAL_CONSUME_CHECKPOINTS = JOURNALED_OPERATION_CHECKPOINTS; +export const APPLY_PUBLISH_CHECKPOINTS = unique( + JOURNALED_OPERATION_CHECKPOINTS, + ["apply.before_cas", "apply.after_cas"], +); +export const APPLY_RESOLUTION_CHECKPOINTS = unique( + ["resolution.after_observation"], + jsonWrite("journal_resolution"), + jsonWrite("journal_resolution_head"), + guardRemoval("journal_resolution_guard_remove"), +); const LOCK_ACQUISITION_CHECKPOINTS = Object.freeze([ "lock.after_mkdir", "lock.after_owner_publish", @@ -127,6 +140,11 @@ export const STAGE2_CHECKPOINT_CATALOG = Object.freeze({ gate_agent: GATE_AGENT_CHECKPOINTS, pane_close: PANE_CLOSE_CHECKPOINTS, stand_down_begin: STAND_DOWN_BEGIN_CHECKPOINTS, + apply_preview: APPLY_PREVIEW_CHECKPOINTS, + approval_record: APPROVAL_RECORD_CHECKPOINTS, + approval_consume: APPROVAL_CONSUME_CHECKPOINTS, + apply_publish: APPLY_PUBLISH_CHECKPOINTS, + apply_resolution: APPLY_RESOLUTION_CHECKPOINTS, archive: ARCHIVE_CHECKPOINTS, lock_acquisition: LOCK_ACQUISITION_CHECKPOINTS, run_activation: RUN_ACTIVATION_CHECKPOINTS, diff --git a/scripts/stage2-lifecycle.mjs b/scripts/stage2-lifecycle.mjs index 6c4c1f5..5a89754 100755 --- a/scripts/stage2-lifecycle.mjs +++ b/scripts/stage2-lifecycle.mjs @@ -111,6 +111,57 @@ export function classifyCleanGates(gateFacts) { return "gate_reports_collected"; } +export const STAGE3_BASE_STATES = new Set([ + "gate_reports_collected", + "integration_harvested_no_gates", +]); +export function classifyStage3Apply(baseState, attempts) { + if (!Array.isArray(attempts) || attempts.length > 8) + fail("stage3 attempt cardinality is invalid"); + if (attempts.length === 0) return baseState; + if (!STAGE3_BASE_STATES.has(baseState)) + fail("stage3 apply authority precedes gate collection"); + const generations = attempts.map((attempt) => attempt?.generation); + if ( + generations.some((generation) => typeof generation !== "string") || + new Set(generations).size !== generations.length + ) + fail("stage3 attempt generations are invalid"); + for (const attempt of attempts) { + if ( + attempt.approval !== null && + attempt.approval !== "approve" && + attempt.approval !== "reject" + ) + fail("stage3 approval decision is invalid"); + if ( + attempt.publication !== null && + attempt.publication !== "applied" && + attempt.publication !== "unapplied" + ) + fail("stage3 publication outcome is invalid"); + if (typeof attempt.consumed !== "boolean") + fail("stage3 consumption fact is invalid"); + if (attempt.publication !== null && !attempt.consumed) + fail("stage3 publication skips consumption"); + if (attempt.consumed && attempt.approval !== "approve") + fail("stage3 consumption skips an approve receipt"); + } + const closed = (attempt) => + attempt.publication === "unapplied" || attempt.approval === "reject"; + for (const attempt of attempts.slice(0, -1)) + if (!closed(attempt)) fail("a prior stage3 attempt is not closed"); + if (attempts.slice(0, -1).some((a) => a.publication === "applied")) + fail("an applied stage3 attempt was superseded"); + const newest = attempts.at(-1); + if (newest.publication === "applied") return "applied"; + if (newest.publication === "unapplied") return "apply_voided"; + if (newest.consumed) return "apply_consumed"; + if (newest.approval === "approve") return "apply_approved"; + if (newest.approval === "reject") return "apply_rejected"; + return "apply_previewed"; +} + export function classifyStandDownPrefix(standDown) { if (!standDown || typeof standDown !== "object" || Array.isArray(standDown)) fail("stand-down facts are invalid"); @@ -151,7 +202,7 @@ export function classifyStandDownPrefix(standDown) { }); } -function nextOperations(state) { +function nextOperations(state, { stage3Configured = false } = {}) { if (state === "archived") return ["status"]; if (state === "delivery_ready_reconcile") return ["integration.reconcile", "run.stand-down.begin"]; @@ -166,6 +217,16 @@ function nextOperations(state) { state === "recovery_required" ) return []; + if (state === "apply_previewed") + return ["approval.record", "run.stand-down.begin"]; + if (state === "apply_approved") + return ["approval.consume", "run.stand-down.begin"]; + if (state === "apply_consumed") return ["apply.publish"]; + if (state === "applied") return ["run.stand-down.begin"]; + if (state === "apply_rejected" || state === "apply_voided") + return ["apply.preview", "run.stand-down.begin"]; + if (stage3Configured && STAGE3_BASE_STATES.has(state)) + return ["apply.preview", "run.stand-down.begin"]; if (CLEAN_STATES.has(state)) return ["run.stand-down.begin"]; return []; } @@ -257,6 +318,56 @@ function deriveFacts(active, config, options) { contract_role === "reviewer" || contract_role === "validator", ) .map(roleFacts); + const stage3Configured = + config.version === 3 && config.apply !== null && config.apply !== undefined; + const observedStage3 = (type) => + active.journal.filter( + (entry) => entry.operation_type === type && entry.phase === "observed", + ); + const previews = observedStage3("apply.preview"); + const approvals = observedStage3("approval.record"); + const consumptions = observedStage3("approval.consume"); + const publications = observedStage3("apply.publish"); + const previewGenerations = new Set( + previews.map((entry) => entry.subject.generation), + ); + for (const orphanSource of [approvals, consumptions, publications]) + if ( + orphanSource.some( + (entry) => !previewGenerations.has(entry.subject.generation), + ) + ) + fail("stage3 authority is missing its preview"); + if ( + (previews.length > 0 || + approvals.length > 0 || + consumptions.length > 0 || + publications.length > 0) && + !stage3Configured + ) + fail("stage3 authority exists without a configured apply target"); + const stage3Attempts = previews.map((entry) => { + const generation = entry.subject.generation; + const match = (entries) => + entries.filter( + (candidate) => candidate.subject.generation === generation, + ); + const approvalEntries = match(approvals); + const consumptionEntries = match(consumptions); + const publicationEntries = match(publications); + if ( + approvalEntries.length > 1 || + consumptionEntries.length > 1 || + publicationEntries.length > 1 + ) + fail("stage3 attempt authority is duplicated"); + return { + generation, + approval: approvalEntries[0]?.observed_identity?.decision ?? null, + consumed: consumptionEntries.length === 1, + publication: publicationEntries[0]?.observed_identity?.outcome ?? null, + }; + }); const standDownEntry = active.journal.find( (entry) => entry.operation_type === "run.stand-down.begin" && @@ -295,6 +406,8 @@ function deriveFacts(active, config, options) { return { producers, gates, + stage3Configured, + stage3Attempts, reconciliation: active.journal.find( (entry) => @@ -338,6 +451,7 @@ export function scanStage2Authority(active, config, options = {}) { ) fail("integration harvest is not observed"); state = classifyCleanGates(facts.gates ?? []); + state = classifyStage3Apply(state, facts.stage3Attempts ?? []); } } else { if (facts.integrationHarvest) @@ -349,12 +463,18 @@ export function scanStage2Authority(active, config, options = {}) { ) ) fail("gate authority precedes integration harvest"); + if ((facts.stage3Attempts ?? []).length > 0) + fail("stage3 apply authority precedes integration harvest"); state = classifyCleanDelivery(facts.producers ?? []); } return Object.freeze({ state, detail, - legalNextOperations: Object.freeze(nextOperations(state)), + legalNextOperations: Object.freeze( + nextOperations(state, { + stage3Configured: facts.stage3Configured ?? false, + }), + ), }); } @@ -364,6 +484,7 @@ export function standDownReasonForState(state, requestedReason) { normal_completion: new Set([ "integration_harvested_no_gates", "gate_reports_collected", + "applied", ]), nonprogressable_delivery: new Set(["delivery_nonprogressable"]), source_policy_refusal: new Set(["gate_source_refused"]), @@ -376,7 +497,13 @@ export function standDownReasonForState(state, requestedReason) { "delivery_waiting_reports", "gate_waiting_reports", ]), - operator_abandoned: CLEAN_STATES, + operator_abandoned: new Set([ + ...CLEAN_STATES, + "apply_previewed", + "apply_approved", + "apply_rejected", + "apply_voided", + ]), }; if (!allowed[requestedReason]?.has(state)) fail("stand-down reason does not match lifecycle state"); diff --git a/scripts/stage3-apply.mjs b/scripts/stage3-apply.mjs new file mode 100755 index 0000000..8115ed0 --- /dev/null +++ b/scripts/stage3-apply.mjs @@ -0,0 +1,209 @@ +#!/usr/bin/env node +import { createHash } from "node:crypto"; +import { + StateKernelError, + canonicalJson, + validateFullRef, + validateGitObjectId, +} from "./private-state-schema.mjs"; +import { STAGE3_CONFIG_VERSION } from "./task-report-schema.mjs"; + +function fail(code, message, cause) { + throw new StateKernelError(code, message, cause ? { cause } : undefined); +} + +function gitLine(exec, repo, args) { + const output = exec("git", ["-C", repo, ...args]); + if (typeof output !== "string" || !output || output.includes("\n")) + fail( + "invalid_repository", + `git ${args.join(" ")} returned ambiguous output`, + ); + return output; +} + +export function stage3ApplyTargetRef(config) { + if (!config || config.version !== STAGE3_CONFIG_VERSION) return null; + if (config.apply === null) return null; + return validateFullRef(config.apply.target_ref, "configured apply target"); +} + +export function observeStage3ApplyTarget({ + repository, + targetRef, + integration, + exec, +} = {}) { + validateFullRef(targetRef, "apply target ref"); + validateFullRef(integration?.target_ref, "integration target ref"); + validateGitObjectId(integration.starting_sha, "integration starting SHA"); + validateGitObjectId(integration.final_sha, "integration final SHA"); + if (targetRef === integration.target_ref) + fail( + "invalid_config", + "apply target must differ from the integration target", + ); + if (integration.starting_sha === integration.final_sha) + fail("stale_source", "integration produced nothing to apply"); + const integrationSha = gitLine(exec, repository, [ + "rev-parse", + integration.target_ref, + ]); + if (integrationSha !== integration.final_sha) + fail( + "stale_source", + "integration target ref no longer holds the integrated SHA", + ); + let observedSha; + try { + observedSha = gitLine(exec, repository, [ + "show-ref", + "--verify", + "--hash", + targetRef, + ]); + } catch (error) { + if (error instanceof StateKernelError && error.code === "invalid_repository") + throw error; + fail("stale_source", "apply target ref does not exist", error); + } + validateGitObjectId(observedSha, "apply target SHA"); + if (observedSha !== integration.starting_sha) + fail( + "stale_source", + "apply target ref is not at the run's integration base", + ); + const worktrees = exec("git", [ + "-C", + repository, + "worktree", + "list", + "--porcelain", + ]); + if (typeof worktrees !== "string") + fail("invalid_repository", "git worktree inventory is ambiguous"); + if ( + worktrees + .split("\n") + .some((line) => line === `branch ${targetRef}`) + ) + fail( + "stale_source", + "apply target ref is checked out in a repository worktree", + ); + try { + exec("git", [ + "-C", + repository, + "merge-base", + "--is-ancestor", + observedSha, + integration.final_sha, + ]); + } catch (error) { + fail( + "stale_source", + "proposed apply is not a fast-forward of the target", + error, + ); + } + const diff = exec( + "git", + [ + "-C", + repository, + "diff", + "--name-status", + "--no-renames", + "-z", + observedSha, + integration.final_sha, + ], + { maxBuffer: 16 * 1024 * 1024 }, + ); + if (typeof diff !== "string") + fail("invalid_repository", "apply change summary is ambiguous"); + const changedPathCount = diff + .split("\0") + .filter((field, index) => field !== "" && index % 2 === 0).length; + if (changedPathCount < 1) + fail("stale_source", "apply change summary is empty"); + return Object.freeze({ + target_ref: targetRef, + observed_sha: observedSha, + final_sha: integration.final_sha, + diff_name_status_sha256: createHash("sha256").update(diff).digest("hex"), + changed_path_count: changedPathCount, + }); +} + +export function publishStage3ApplyCas({ + repository, + targetRef, + expectedSha, + finalSha, + preflight, + exec, + fault, +} = {}) { + validateFullRef(targetRef, "apply target ref"); + validateGitObjectId(expectedSha, "apply expected SHA"); + validateGitObjectId(finalSha, "apply final SHA"); + if (expectedSha === finalSha) + fail("stale_source", "apply has nothing to publish"); + if (typeof preflight !== "function") + fail("bookkeeping_unknown", "apply publication preflight is required"); + const before = preflight(); + if (canonicalJson(before) !== canonicalJson(preflight())) + fail("stale_source", "apply authority changed between preflights"); + fault?.("apply.before_cas"); + if (canonicalJson(before) !== canonicalJson(preflight())) + fail( + "stale_source", + "apply authority changed immediately before compare-and-swap", + ); + exec("git", ["-C", repository, "update-ref", targetRef, finalSha, expectedSha]); + fault?.("apply.after_cas"); + const published = gitLine(exec, repository, ["rev-parse", targetRef]); + if (published !== finalSha) + fail( + "durability_unknown", + "published apply ref does not equal the approved final SHA", + ); + return Object.freeze({ casCount: 1, finalSha }); +} + +export function resolveStage3ApplyOutcome({ + repository, + targetRef, + expectedSha, + finalSha, + exec, +} = {}) { + validateFullRef(targetRef, "apply target ref"); + validateGitObjectId(expectedSha, "apply expected SHA"); + validateGitObjectId(finalSha, "apply final SHA"); + let observed; + try { + observed = gitLine(exec, repository, [ + "show-ref", + "--verify", + "--hash", + targetRef, + ]); + } catch (error) { + fail( + "foreign_or_stale", + "uncertain apply target cannot be observed", + error, + ); + } + if (observed === finalSha) + return Object.freeze({ outcome: "applied", cas_count: 1 }); + if (observed === expectedSha) + return Object.freeze({ outcome: "unapplied", cas_count: 0 }); + fail( + "foreign_or_stale", + "uncertain apply target holds a foreign SHA and cannot be resolved", + ); +} diff --git a/scripts/state-internal.mjs b/scripts/state-internal.mjs index fad311e..916fe3a 100755 --- a/scripts/state-internal.mjs +++ b/scripts/state-internal.mjs @@ -1348,6 +1348,42 @@ export function inspectRepositoryLock(store) { return inspectLock(store); } +export function reclaimDeadRepositoryLock(store, { operationId, fault } = {}) { + validateId(operationId, "lock operation id"); + const existing = inspectLock(store); + if (existing.status === "locked") { + let dead = false; + try { + process.kill(existing.owner.pid, 0); + } catch (error) { + if (error?.code !== "ESRCH") throw error; + dead = true; + } + if (!dead) + throw kernelError( + "lock_busy", + "repository mutation lock is already held", + ); + if (existing.owner.operation_id !== operationId) + throw kernelError( + "recovery_required", + "operation_uncertain: retained dead-process lock requires recovery", + ); + try { + unlinkSync(join(store.lockDir, "owner.json")); + rmdirSync(store.lockDir); + fsyncDirectory(store.repositoryDir); + } catch (error) { + throw kernelError( + "lock_unknown", + "dead repository lock cannot be reclaimed", + error, + ); + } + } + return acquireRepositoryLock(store, { operationId, fault }); +} + export function acquireRepositoryLock(store, { operationId, fault } = {}) { validateId(operationId, "lock operation id"); try { diff --git a/scripts/state-kernel.mjs b/scripts/state-kernel.mjs index c46d460..4f614ea 100755 --- a/scripts/state-kernel.mjs +++ b/scripts/state-kernel.mjs @@ -13,6 +13,7 @@ export { copyExclusivePrivateBytes, readPrivateJson, readStablePrivateBytes, + reclaimDeadRepositoryLock, scanExactDirectory, releaseRepositoryLock, resolveGitCommonDirectory, @@ -21,5 +22,7 @@ export { export { archiveActiveRun, inspectArchiveUncertainty, + loadUncertainApplyRun, performJournaledOperation, + resolveUncertainApplyPublication, } from "./operation-journal.mjs"; diff --git a/scripts/task-report-schema.mjs b/scripts/task-report-schema.mjs index f4723a3..a540b84 100755 --- a/scripts/task-report-schema.mjs +++ b/scripts/task-report-schema.mjs @@ -8,6 +8,7 @@ import { } from "./private-state-schema.mjs"; export const STAGE2_CONFIG_VERSION = 2; +export const STAGE3_CONFIG_VERSION = 3; export const TASK_MAX_BYTES = 262_144; export const REPORT_MAX_BYTES = 1_048_576; export const PUBLICATION_METADATA_MAX_BYTES = 16_384; @@ -256,14 +257,7 @@ function gitSource(value, label = "source") { ); canonicalPath(value.root, `${label}.root`); commonDirectory(value.common_dir, `${label}.common_dir`); - matching(value.branch_ref, FULL_REF, `${label}.branch_ref`); - if ( - value.branch_ref.includes("\\") || - value.branch_ref - .split("/") - .some((part) => !part || part === "." || part === "..") - ) - invalid(`${label}.branch_ref`); + fullRefWithoutTraversal(value.branch_ref, `${label}.branch_ref`); for (const field of ["fork_sha", "expected_sha", "tree_sha"]) matching(value[field], SHA, `${label}.${field}`); matching( @@ -417,6 +411,22 @@ function assignment(value, label = "assignment") { invalid(`${label} path overlap`); } +function fullRefWithoutTraversal(value, label) { + matching(value, FULL_REF, label); + if ( + value.includes("\\") || + value.split("/").some((part) => !part || part === "." || part === "..") + ) + invalid(label); + return value; +} + +function applySelector(value) { + if (value === null) return; + exact(value, ["target_ref"], "configuration.apply"); + fullRefWithoutTraversal(value.target_ref, "configuration.apply.target_ref"); +} + function stateRootSelector(value) { exact( value, @@ -477,13 +487,19 @@ export function parseStage2ConfigBytes(bytes) { throw error; } plain(value, "configuration"); - if (value.version !== STAGE2_CONFIG_VERSION) - fail("wrong_version", "configuration version must be 2"); + if ( + value.version !== STAGE2_CONFIG_VERSION && + value.version !== STAGE3_CONFIG_VERSION + ) + fail("wrong_version", "configuration version must be 2 or 3"); exact( value, - ["version", "state_root", "worktree_root", "roles"], + value.version === STAGE3_CONFIG_VERSION + ? ["version", "state_root", "worktree_root", "roles", "apply"] + : ["version", "state_root", "worktree_root", "roles"], "configuration", ); + if (value.version === STAGE3_CONFIG_VERSION) applySelector(value.apply); stateRootSelector(value.state_root); validateRelativePath(value.worktree_root, "configuration.worktree_root"); if (!Array.isArray(value.roles) || value.roles.length > 64) diff --git a/tests/fixtures/stage3-crash-child.mjs b/tests/fixtures/stage3-crash-child.mjs new file mode 100644 index 0000000..9d495c9 --- /dev/null +++ b/tests/fixtures/stage3-crash-child.mjs @@ -0,0 +1,36 @@ +import { execFileSync } from "node:child_process"; +import { applyStage3 } from "../../scripts/stage1-runtime.mjs"; + +const [repository, workspace, boundary] = process.argv.slice(2); + +function exec(command, args, options = {}) { + if (command !== "git") { + if (args.length === 1 && args[0] === "--version") return "herdr 0.7.5"; + if (args.join(" ") === "api schema --json") + return JSON.stringify({ protocol: 17, schema_version: 1 }); + throw new Error(`unexpected fake Herdr call: ${args.join(" ")}`); + } + return execFileSync(command, args, { + encoding: "utf8", + stdio: ["ignore", "pipe", "pipe"], + ...options, + }).trim(); +} + +try { + await applyStage3({ + contextJson: JSON.stringify({ + workspace_id: workspace, + workspace_cwd: repository, + focused_pane_id: `${workspace}:p0`, + }), + herdrBin: "fake", + exec, + fault(point) { + if (point === boundary) process.kill(process.pid, "SIGKILL"); + }, + }); +} catch (error) { + process.stderr.write(`${error.code ?? "internal_error"}:${error.message}\n`); + process.exitCode = 1; +} diff --git a/tests/private-state-schema.test.mjs b/tests/private-state-schema.test.mjs index 52d7b2d..d4b88fd 100644 --- a/tests/private-state-schema.test.mjs +++ b/tests/private-state-schema.test.mjs @@ -1,6 +1,7 @@ import { test } from "node:test"; import assert from "node:assert/strict"; import { + STAGE3_APPROVAL_STATEMENTS, StateKernelError, canonicalJson, parseStrictJsonBytes, @@ -9,6 +10,10 @@ import { validateLockOwner, validateRepositoryDocument, validateRunState, + validateStage3ApplyIdentity, + validateStage3ApprovalIdentity, + validateStage3ConsumptionIdentity, + validateStage3PreviewIdentity, } from "../scripts/private-state-schema.mjs"; const REPO_KEY = "a".repeat(64); @@ -179,6 +184,242 @@ test("lock and journal validators enforce exact fields and phase conditionals", expectCode("invalid_state", () => validateJournalEntry(unknownOperation)); }); +function stage3Scope() { + return { + repository_key: REPO_KEY, + workspace_id: "wB0", + run_id: "run-b1", + run_generation: GENERATION, + attempt_generation: "9".repeat(32), + }; +} + +function stage3Preview() { + return { + document_type: "herdr-conductor-stage3-preview", + schema_version: 1, + ...stage3Scope(), + integration: { + target_ref: "refs/heads/main", + starting_sha: SHA, + final_sha: "e".repeat(40), + integration_entry_digest: DIGEST, + }, + gates: [ + { + role_name: "reviewer", + contract_role: "reviewer", + task_digest: DIGEST, + report_digest: DIGEST, + status: "completed", + result_kind: "review", + verdict: "approve", + }, + { + role_name: "validator", + contract_role: "validator", + task_digest: DIGEST, + report_digest: DIGEST, + status: "completed", + result_kind: "validation", + verdict: "pass", + }, + ], + apply: { + target_ref: "refs/heads/release", + observed_sha: SHA, + final_sha: "e".repeat(40), + diff_name_status_sha256: DIGEST, + changed_path_count: 3, + }, + }; +} + +function stage3Approval(decision = "approve") { + return { + document_type: "herdr-conductor-stage3-approval", + schema_version: 1, + ...stage3Scope(), + preview_entry_digest: DIGEST, + decision, + statement: STAGE3_APPROVAL_STATEMENTS[decision], + }; +} + +function stage3Apply(outcome = "applied") { + return { + document_type: "herdr-conductor-stage3-apply", + schema_version: 1, + ...stage3Scope(), + consumption_entry_digest: DIGEST, + target_ref: "refs/heads/release", + expected_sha: SHA, + final_sha: "e".repeat(40), + cas_count: outcome === "applied" ? 1 : 0, + outcome, + }; +} + +test("stage3 preview identity binds integration, gates, and the apply target", () => { + const label = "preview"; + assert.deepEqual( + validateStage3PreviewIdentity(stage3Preview(), label), + stage3Preview(), + ); + const wrongOrder = stage3Preview(); + wrongOrder.gates.reverse(); + expectCode("invalid_state", () => + validateStage3PreviewIdentity(wrongOrder, label), + ); + const sameRef = stage3Preview(); + sameRef.apply.target_ref = sameRef.integration.target_ref; + expectCode("invalid_state", () => + validateStage3PreviewIdentity(sameRef, label), + ); + const drifted = stage3Preview(); + drifted.apply.observed_sha = "f".repeat(40); + expectCode("invalid_state", () => + validateStage3PreviewIdentity(drifted, label), + ); + const emptyMove = stage3Preview(); + emptyMove.integration.final_sha = emptyMove.integration.starting_sha; + emptyMove.apply.final_sha = emptyMove.apply.observed_sha; + expectCode("invalid_state", () => + validateStage3PreviewIdentity(emptyMove, label), + ); + const zeroPaths = stage3Preview(); + zeroPaths.apply.changed_path_count = 0; + expectCode("invalid_state", () => + validateStage3PreviewIdentity(zeroPaths, label), + ); + const wrongVerdict = stage3Preview(); + wrongVerdict.gates[0].verdict = "pass"; + expectCode("invalid_state", () => + validateStage3PreviewIdentity(wrongVerdict, label), + ); + const blockedGate = stage3Preview(); + blockedGate.gates[0].status = "blocked"; + expectCode("invalid_state", () => + validateStage3PreviewIdentity(blockedGate, label), + ); +}); + +test("stage3 approval, consumption, and apply identities are closed and bound", () => { + const label = "stage3"; + assert.deepEqual( + validateStage3ApprovalIdentity(stage3Approval(), label), + stage3Approval(), + ); + assert.deepEqual( + validateStage3ApprovalIdentity(stage3Approval("reject"), label), + stage3Approval("reject"), + ); + const crossedStatement = stage3Approval(); + crossedStatement.statement = STAGE3_APPROVAL_STATEMENTS.reject; + expectCode("invalid_state", () => + validateStage3ApprovalIdentity(crossedStatement, label), + ); + const freeText = stage3Approval(); + freeText.statement = "I approve"; + expectCode("invalid_state", () => + validateStage3ApprovalIdentity(freeText, label), + ); + const consumption = { + document_type: "herdr-conductor-stage3-consumption", + schema_version: 1, + ...stage3Scope(), + approval_entry_digest: DIGEST, + preview_entry_digest: DIGEST, + }; + assert.deepEqual( + validateStage3ConsumptionIdentity(consumption, label), + consumption, + ); + expectCode("invalid_state", () => + validateStage3ConsumptionIdentity( + { ...consumption, extra: true }, + label, + ), + ); + assert.deepEqual(validateStage3ApplyIdentity(stage3Apply(), label), stage3Apply()); + assert.deepEqual( + validateStage3ApplyIdentity(stage3Apply("unapplied"), label), + stage3Apply("unapplied"), + ); + const casMismatch = stage3Apply(); + casMismatch.cas_count = 0; + expectCode("invalid_state", () => + validateStage3ApplyIdentity(casMismatch, label), + ); + const noMove = stage3Apply(); + noMove.final_sha = noMove.expected_sha; + expectCode("invalid_state", () => validateStage3ApplyIdentity(noMove, label)); +}); + +test("stage3 journal entries validate their observed identities by policy", () => { + const base = journalEntry("observed"); + const previewEntry = { + ...base, + operation_id: "apply-preview-attempt", + operation_type: "apply.preview", + subject: { kind: "apply", id: "apply", generation: "9".repeat(32) }, + observed_identity: stage3Preview(), + }; + assert.deepEqual(validateJournalEntry(previewEntry), previewEntry); + const approvalEntry = { + ...base, + operation_id: "apply-approve-attempt", + operation_type: "approval.record", + subject: { kind: "approval", id: "apply", generation: "9".repeat(32) }, + observed_identity: stage3Approval(), + }; + assert.deepEqual(validateJournalEntry(approvalEntry), approvalEntry); + const applyEntry = { + ...base, + operation_id: "apply-publish-attempt", + operation_type: "apply.publish", + subject: { kind: "apply", id: "apply", generation: "9".repeat(32) }, + observed_identity: stage3Apply(), + }; + assert.deepEqual(validateJournalEntry(applyEntry), applyEntry); + expectCode("invalid_state", () => + validateJournalEntry({ + ...applyEntry, + observed_identity: stage3Approval(), + }), + ); + expectCode("invalid_state", () => + validateJournalEntry({ + ...applyEntry, + subject: { ...applyEntry.subject, kind: "approval" }, + }), + ); + expectCode("invalid_state", () => + validateJournalEntry({ ...applyEntry, observed_identity: null }), + ); + expectCode("invalid_state", () => + validateJournalEntry({ + ...applyEntry, + observed_identity: { + ...stage3Apply(), + attempt_generation: "8".repeat(32), + }, + }), + ); + expectCode("invalid_state", () => + validateJournalEntry({ + ...applyEntry, + observed_identity: { ...stage3Apply(), run_id: "run-b2" }, + }), + ); + expectCode("invalid_state", () => + validateJournalEntry({ + ...applyEntry, + observed_identity: { ...stage3Apply(), workspace_id: "wOther" }, + }), + ); +}); + test("canonical JSON recursively sorts keys and terminates with one newline", () => { assert.equal( canonicalJson({ z: 1, a: { y: 2, b: 3 } }), diff --git a/tests/stage1-runtime-helpers.mjs b/tests/stage1-runtime-helpers.mjs index e9152ef..cafb47f 100644 --- a/tests/stage1-runtime-helpers.mjs +++ b/tests/stage1-runtime-helpers.mjs @@ -18,11 +18,14 @@ import { dirname, join, relative } from "node:path"; import { assemble, parsePluginContext, + preview, readStatus, reconcile, standDown, } from "../scripts/stage1-runtime.mjs"; +import { recordApprovalFromStdin } from "../scripts/approval-recorder.mjs"; import { + STAGE3_APPROVAL_STATEMENTS, canonicalJson, StateKernelError, parseStrictJsonBytes, @@ -335,6 +338,29 @@ class FakeHerdr { ); } else if (args[2] === "-c" && args[3] === "diff.renames=false") { assert.equal(args[4], "diff"); + } else if (args[2] === "show-ref") { + assert.deepEqual(args, [ + "-C", + args[1], + "show-ref", + "--verify", + "--hash", + args[5], + ]); + assert.match(args[5], /^refs\//); + } else if (args[2] === "diff") { + assert.deepEqual(args, [ + "-C", + args[1], + "diff", + "--name-status", + "--no-renames", + "-z", + args[6], + args[7], + ]); + assert.match(args[6], /^[a-f0-9]{40}$/); + assert.match(args[7], /^[a-f0-9]{40}$/); } else if (args[2] === "symbolic-ref") { assert.deepEqual(args, ["-C", args[1], "symbolic-ref", "-q", "HEAD"]); } else if (args[2] === "worktree") { @@ -813,6 +839,7 @@ async function assembledFixture({ seed = 1, random = deterministicRandom(seed), stateRoot = join(temp("conductor-b2-state-"), "state"), + configOverrides = {}, fault, } = {}) { const fake = new FakeHerdr(); @@ -820,7 +847,7 @@ async function assembledFixture({ stateRoot = realpathSync(stateRoot); const result = await assemble({ contextJson: context(repository, workspace), - configPath: config(repository, roles, {}, stateRoot), + configPath: config(repository, roles, configOverrides, stateRoot), exec: fake.exec, herdrBin: "fake-herdr", random, @@ -829,6 +856,81 @@ async function assembledFixture({ return { repository, workspace, stateRoot, fake, result }; } +const STAGE3_APPLY_TARGET = "refs/heads/release"; + +function stage3ProducerRoles() { + return [ + { name: "builder", contract_role: "builder", kind: "pi", mode: "write" }, + ]; +} + +function stage3ApplyOverrides() { + return { version: 3, apply: { target_ref: STAGE3_APPLY_TARGET } }; +} + +function stage3Invocation(fixture, extra = {}) { + return { + contextJson: context(fixture.repository, fixture.workspace), + exec: fixture.fake.exec, + herdrBin: "fake", + ...extra, + }; +} + +async function stage3HarvestedApplyFixture( + seed, + { roles = stage3ProducerRoles() } = {}, +) { + const fixture = await assembledFixture({ + roles, + seed, + configOverrides: stage3ApplyOverrides(), + }); + git(fixture.repository, "update-ref", STAGE3_APPLY_TARGET, fixture.result.fork_sha); + const producer = fixture.result.workers[0]; + mkdirSync(join(producer.cwd, "src")); + writeFileSync(join(producer.cwd, "src", "feature.mjs"), "export default 1;\n"); + git(producer.cwd, "add", "src/feature.mjs"); + git(producer.cwd, "commit", "-qm", "feature"); + await publishWorkerReport(fixture, producer); + const harvested = await reconcile( + stage3Invocation(fixture, { random: deterministicRandom(40 + seed) }), + ); + return { fixture, harvested }; +} + +function stage3Receipt(previewResult, decision) { + const identity = previewResult.preview; + return { + document_type: "herdr-conductor-stage3-approval", + schema_version: 1, + repository_key: identity.repository_key, + workspace_id: identity.workspace_id, + run_id: identity.run_id, + run_generation: identity.run_generation, + attempt_generation: identity.attempt_generation, + preview_entry_digest: previewResult.preview_entry_digest, + decision, + statement: STAGE3_APPROVAL_STATEMENTS[decision], + }; +} + +function recordStage3Receipt(fixture, receipt) { + return recordApprovalFromStdin({ + configPath: join(fixture.repository, ".herdr-conductor.json"), + input: Readable.from([Buffer.from(canonicalJson(receipt))]), + exec: fixture.fake.exec, + }); +} + +async function stage3ApprovedApplyFixture(seed) { + const { fixture, harvested } = await stage3HarvestedApplyFixture(seed); + assert.equal(harvested.lifecycle, "integration_harvested_no_gates"); + const previewed = await preview(stage3Invocation(fixture)); + await recordStage3Receipt(fixture, stage3Receipt(previewed, "approve")); + return { fixture, harvested, previewed }; +} + export { assert, execFileSync, @@ -868,4 +970,12 @@ export { buildWorkerReport, publishWorkerReport, assembledFixture, + STAGE3_APPLY_TARGET, + stage3ProducerRoles, + stage3ApplyOverrides, + stage3Invocation, + stage3HarvestedApplyFixture, + stage3Receipt, + recordStage3Receipt, + stage3ApprovedApplyFixture, }; diff --git a/tests/stage2-lifecycle.test.mjs b/tests/stage2-lifecycle.test.mjs index fe2bd98..7a2d67a 100644 --- a/tests/stage2-lifecycle.test.mjs +++ b/tests/stage2-lifecycle.test.mjs @@ -6,6 +6,7 @@ import { join } from "node:path"; import { classifyCleanDelivery, classifyCleanGates, + classifyStage3Apply, classifyStandDownPrefix, scanStage2Authority, standDownReasonForState, @@ -788,3 +789,185 @@ test("integration transitions are disjoint and terminal archive wins", () => { "archived", ); }); + +const attempt = (generation, overrides = {}) => ({ + generation, + approval: null, + consumed: false, + publication: null, + ...overrides, +}); + +test("stage3 apply states cover every attempt shape and barrier", () => { + assert.equal( + classifyStage3Apply("gate_reports_collected", []), + "gate_reports_collected", + ); + assert.equal( + classifyStage3Apply("integration_harvested_no_gates", [attempt("g1")]), + "apply_previewed", + ); + assert.equal( + classifyStage3Apply("gate_reports_collected", [ + attempt("g1", { approval: "approve" }), + ]), + "apply_approved", + ); + assert.equal( + classifyStage3Apply("gate_reports_collected", [ + attempt("g1", { approval: "reject" }), + ]), + "apply_rejected", + ); + assert.equal( + classifyStage3Apply("gate_reports_collected", [ + attempt("g1", { approval: "approve", consumed: true }), + ]), + "apply_consumed", + ); + assert.equal( + classifyStage3Apply("gate_reports_collected", [ + attempt("g1", { + approval: "approve", + consumed: true, + publication: "applied", + }), + ]), + "applied", + ); + assert.equal( + classifyStage3Apply("gate_reports_collected", [ + attempt("g1", { + approval: "approve", + consumed: true, + publication: "unapplied", + }), + ]), + "apply_voided", + ); + assert.equal( + classifyStage3Apply("gate_reports_collected", [ + attempt("g1", { approval: "reject" }), + attempt("g2", { + approval: "approve", + consumed: true, + publication: "unapplied", + }), + attempt("g3"), + ]), + "apply_previewed", + ); + code("bookkeeping_unknown", () => + classifyStage3Apply("gate_waiting_reports", [attempt("g1")]), + ); + code("bookkeeping_unknown", () => + classifyStage3Apply("delivery_ready_reconcile", [attempt("g1")]), + ); + code("bookkeeping_unknown", () => + classifyStage3Apply("gate_reports_collected", [ + attempt("g1"), + attempt("g2"), + ]), + ); + code("bookkeeping_unknown", () => + classifyStage3Apply("gate_reports_collected", [ + attempt("g1", { + approval: "approve", + consumed: true, + publication: "applied", + }), + attempt("g2"), + ]), + ); + code("bookkeeping_unknown", () => + classifyStage3Apply("gate_reports_collected", [ + attempt("g1", { publication: "applied", consumed: false }), + ]), + ); + code("bookkeeping_unknown", () => + classifyStage3Apply("gate_reports_collected", [ + attempt("g1", { consumed: true }), + ]), + ); + code("bookkeeping_unknown", () => + classifyStage3Apply("gate_reports_collected", [ + attempt("g1", { approval: "reject", consumed: true }), + ]), + ); + code("bookkeeping_unknown", () => + classifyStage3Apply("gate_reports_collected", [ + attempt("g1"), + attempt("g1"), + ]), + ); + code("bookkeeping_unknown", () => + classifyStage3Apply( + "gate_reports_collected", + Array.from({ length: 9 }, (_, index) => attempt(`g${index}`)), + ), + ); +}); + +test("stage3 states gate their legal operations and stand-down reasons", () => { + const ready = { + producers: [producer("a", 5)], + gates: [], + uncertainty: null, + standDown: null, + archived: false, + archiveUncertain: false, + reconciliation: { observed: true }, + integrationHarvest: { observed: true }, + }; + const scan = (extra) => + scanStage2Authority(null, null, { facts: { ...ready, ...extra } }); + assert.deepEqual( + [...scan({ stage3Configured: true }).legalNextOperations], + ["apply.preview", "run.stand-down.begin"], + ); + assert.deepEqual( + [...scan({}).legalNextOperations], + ["run.stand-down.begin"], + ); + assert.deepEqual( + [ + ...scan({ + stage3Configured: true, + stage3Attempts: [attempt("g1", { approval: "approve" })], + }).legalNextOperations, + ], + ["approval.consume", "run.stand-down.begin"], + ); + assert.deepEqual( + [ + ...scan({ + stage3Configured: true, + stage3Attempts: [ + attempt("g1", { approval: "approve", consumed: true }), + ], + }).legalNextOperations, + ], + ["apply.publish"], + ); + assert.equal( + standDownReasonForState("applied", "normal_completion"), + "normal_completion", + ); + assert.equal( + standDownReasonForState("apply_previewed", "operator_abandoned"), + "operator_abandoned", + ); + assert.equal( + standDownReasonForState("apply_voided", "operator_abandoned"), + "operator_abandoned", + ); + code("bookkeeping_unknown", () => + standDownReasonForState("apply_consumed", "operator_abandoned"), + ); + code("bookkeeping_unknown", () => + standDownReasonForState("applied", "operator_abandoned"), + ); + code("bookkeeping_unknown", () => + standDownReasonForState("apply_previewed", "normal_completion"), + ); +}); diff --git a/tests/stage2-reconcile-determinism.test.mjs b/tests/stage2-reconcile-determinism.test.mjs index f3adafa..adb3a00 100644 --- a/tests/stage2-reconcile-determinism.test.mjs +++ b/tests/stage2-reconcile-determinism.test.mjs @@ -68,10 +68,11 @@ test("synthetic integration commits ignore ambient identity, clock, and timezone repo, "show", "-s", - "--format=%an <%ae>|%aI|%cn <%ce>|%cI|%B", + "--date=raw", + "--format=%an <%ae>|%ad|%cn <%ce>|%cd|%B", first.finalSha, ), - `Herdr Conductor |2000-01-01T00:00:00Z|Herdr Conductor |2000-01-01T00:00:00Z|Conductor Stage 2 integrate builder ${source}`, + `Herdr Conductor |946684800 +0000|Herdr Conductor |946684800 +0000|Conductor Stage 2 integrate builder ${source}`, ); let preflights = 0; const published = publishIntegrationCas({ diff --git a/tests/stage3-actions.test.mjs b/tests/stage3-actions.test.mjs new file mode 100644 index 0000000..93953b0 --- /dev/null +++ b/tests/stage3-actions.test.mjs @@ -0,0 +1,386 @@ +import test from "node:test"; +import { mkdirSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +import { Readable } from "node:stream"; +import { + applyStage3, + preview, + reconcile, +} from "../scripts/stage1-runtime.mjs"; +import { recordApprovalFromStdin } from "../scripts/approval-recorder.mjs"; +import { + STAGE3_APPROVAL_STATEMENTS, + canonicalJson, +} from "../scripts/private-state-schema.mjs"; +import { + STAGE3_APPLY_TARGET as APPLY_TARGET, + assert, + assembledFixture, + context, + deterministicRandom, + expectCodeAsync, + git, + publishWorkerReport, + readStatus, + recordStage3Receipt as record, + stage3ApplyOverrides, + stage3HarvestedApplyFixture, + stage3Invocation as invocation, + stage3ProducerRoles as producerRoles, + stage3Receipt as receiptFor, + standDown, +} from "./stage1-runtime-helpers.mjs"; + +async function harvestedApplyFixture(seed = 1) { + const result = await stage3HarvestedApplyFixture(seed); + assert.equal(result.harvested.lifecycle, "integration_harvested_no_gates"); + return result; +} + +function applyCasCount(fixture) { + return fixture.fake.log.filter( + ({ command, args }) => + command === "git" && + args.includes("update-ref") && + args.includes(APPLY_TARGET), + ).length; +} + +test("preview, approval, consumption, and apply move the target exactly once", async () => { + const { fixture, harvested } = await harvestedApplyFixture(1); + const effectsBeforePreview = fixture.fake.effects; + const previewed = await preview(invocation(fixture)); + assert.equal(previewed.lifecycle, "apply_previewed"); + assert.equal(fixture.fake.effects, effectsBeforePreview); + assert.equal(previewed.replayed, false); + assert.equal( + previewed.preview.integration.final_sha, + harvested.integration.final_sha, + ); + assert.equal( + previewed.preview.apply.observed_sha, + harvested.integration.starting_sha, + ); + assert.equal(previewed.preview.apply.changed_path_count, 1); + assert.deepEqual(previewed.preview.gates, []); + assert.match(previewed.approval_command, /approval-recorder\.mjs/); + const replayedPreview = await preview(invocation(fixture)); + assert.equal(replayedPreview.replayed, true); + assert.equal( + replayedPreview.preview_entry_digest, + previewed.preview_entry_digest, + ); + const recorded = await record(fixture, receiptFor(previewed, "approve")); + assert.equal(recorded.decision, "approve"); + await expectCodeAsync("replay_refused", () => + record(fixture, receiptFor(previewed, "approve")), + ); + await expectCodeAsync("operation_conflict", () => + record(fixture, receiptFor(previewed, "reject")), + ); + const applied = await applyStage3(invocation(fixture)); + assert.equal(applied.lifecycle, "applied"); + assert.equal(applied.apply.outcome, "applied"); + assert.equal(applied.apply.cas_count, 1); + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + harvested.integration.final_sha, + ); + assert.equal(applyCasCount(fixture), 1); + const replayedApply = await applyStage3(invocation(fixture)); + assert.equal(replayedApply.lifecycle, "applied"); + assert.equal(replayedApply.replayed, true); + assert.equal(applyCasCount(fixture), 1); + const appliedPreview = await preview(invocation(fixture)); + assert.equal(appliedPreview.lifecycle, "applied"); + assert.equal(appliedPreview.replayed, true); +}); + +test("stage3 actions refuse configuration v2 and premature lifecycles", async () => { + const v2 = await assembledFixture({ roles: producerRoles(), seed: 7 }); + await expectCodeAsync("capability_unavailable", () => + preview(invocation(v2)), + ); + await expectCodeAsync("capability_unavailable", () => + applyStage3(invocation(v2)), + ); + const v3 = await assembledFixture({ + roles: producerRoles(), + seed: 8, + configOverrides: stage3ApplyOverrides(), + }); + git(v3.repository, "update-ref", APPLY_TARGET, v3.result.fork_sha); + await expectCodeAsync("operation_conflict", () => preview(invocation(v3))); + await expectCodeAsync("operation_conflict", () => + applyStage3(invocation(v3)), + ); +}); + +test("a reject receipt closes the attempt and a fresh attempt applies", async () => { + const { fixture, harvested } = await harvestedApplyFixture(2); + const previewed = await preview(invocation(fixture)); + const rejected = await record(fixture, receiptFor(previewed, "reject")); + assert.equal(rejected.decision, "reject"); + await expectCodeAsync("operation_conflict", () => + applyStage3(invocation(fixture)), + ); + const second = await preview(invocation(fixture)); + assert.equal(second.lifecycle, "apply_previewed"); + assert.notEqual( + second.preview.attempt_generation, + previewed.preview.attempt_generation, + ); + await record(fixture, receiptFor(second, "approve")); + const applied = await applyStage3(invocation(fixture)); + assert.equal(applied.lifecycle, "applied"); + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + harvested.integration.final_sha, + ); + assert.equal(applyCasCount(fixture), 1); +}); + +test("approval and apply refuse a drifted target with zero CAS", async () => { + const { fixture, harvested } = await harvestedApplyFixture(3); + const previewed = await preview(invocation(fixture)); + git(fixture.repository, "update-ref", APPLY_TARGET, harvested.integration.final_sha); + await expectCodeAsync("stale_source", () => + record(fixture, receiptFor(previewed, "approve")), + ); + const rejected = await record(fixture, receiptFor(previewed, "reject")); + assert.equal(rejected.decision, "reject"); + assert.equal(applyCasCount(fixture), 0); + const drifted = await harvestedApplyFixture(4); + const driftedPreview = await preview(invocation(drifted.fixture)); + await record(drifted.fixture, receiptFor(driftedPreview, "approve")); + git( + drifted.fixture.repository, + "update-ref", + APPLY_TARGET, + drifted.harvested.integration.final_sha, + ); + const voided = await applyStage3(invocation(drifted.fixture)); + assert.equal(voided.lifecycle, "apply_voided"); + assert.equal(voided.apply.outcome, "unapplied"); + assert.equal(voided.apply.cas_count, 0); + assert.equal(applyCasCount(drifted.fixture), 0); + git( + drifted.fixture.repository, + "update-ref", + APPLY_TARGET, + drifted.harvested.integration.starting_sha, + ); + const reopened = await preview(invocation(drifted.fixture)); + assert.equal(reopened.lifecycle, "apply_previewed"); + await record(drifted.fixture, receiptFor(reopened, "approve")); + const applied = await applyStage3(invocation(drifted.fixture)); + assert.equal(applied.lifecycle, "applied"); + assert.equal(applyCasCount(drifted.fixture), 1); +}); + +test("stale receipts and foreign digests are refused exactly", async () => { + const { fixture } = await harvestedApplyFixture(5); + const previewed = await preview(invocation(fixture)); + await expectCodeAsync("digest_mismatch", () => + record(fixture, { + ...receiptFor(previewed, "approve"), + preview_entry_digest: "9".repeat(64), + }), + ); + await expectCodeAsync("stale_task", () => + record(fixture, { + ...receiptFor(previewed, "approve"), + attempt_generation: "9".repeat(32), + }), + ); + await expectCodeAsync("invalid_state", () => + record(fixture, { + ...receiptFor(previewed, "approve"), + statement: STAGE3_APPROVAL_STATEMENTS.reject, + }), + ); + await expectCodeAsync("digest_mismatch", () => + recordApprovalFromStdin({ + configPath: join(fixture.repository, ".herdr-conductor.json"), + input: Readable.from([ + Buffer.from(`${JSON.stringify(receiptFor(previewed, "approve"))}`), + ]), + exec: fixture.fake.exec, + }), + ); +}); + +test("a crash before the CAS voids the attempt and recovery re-arms apply", async () => { + const { fixture, harvested } = await harvestedApplyFixture(6); + const previewed = await preview(invocation(fixture)); + await record(fixture, receiptFor(previewed, "approve")); + await expectCodeAsync("recovery_required", () => + applyStage3( + invocation(fixture, { + fault: (name) => { + if (name === "apply.before_cas") throw new Error("killed before CAS"); + }, + }), + ), + ); + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + harvested.integration.starting_sha, + ); + const resolved = await applyStage3(invocation(fixture)); + assert.equal(resolved.lifecycle, "apply_voided"); + assert.equal(resolved.resolved, true); + assert.equal(resolved.apply.outcome, "unapplied"); + assert.equal(resolved.apply.cas_count, 0); + assert.equal(applyCasCount(fixture), 0); + const second = await preview(invocation(fixture)); + assert.equal(second.lifecycle, "apply_previewed"); + await record(fixture, receiptFor(second, "approve")); + const applied = await applyStage3(invocation(fixture)); + assert.equal(applied.lifecycle, "applied"); + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + harvested.integration.final_sha, + ); + assert.equal(applyCasCount(fixture), 1); +}); + +test("a gated run previews byte-ordered verdicts, applies, and archives", async () => { + const fixture = await assembledFixture({ + roles: [ + ...producerRoles(), + { name: "reviewer", contract_role: "reviewer", kind: "codex", mode: "read-only" }, + { name: "validator", contract_role: "validator", kind: "codex", mode: "gated" }, + ], + seed: 11, + configOverrides: stage3ApplyOverrides(), + }); + git(fixture.repository, "update-ref", APPLY_TARGET, fixture.result.fork_sha); + const producer = fixture.result.workers[0]; + mkdirSync(join(producer.cwd, "src")); + writeFileSync(join(producer.cwd, "src", "feature.mjs"), "export default 1;\n"); + git(producer.cwd, "add", "src/feature.mjs"); + git(producer.cwd, "commit", "-qm", "feature"); + await publishWorkerReport(fixture, producer); + const provisioned = await reconcile( + invocation(fixture, { random: deterministicRandom(60) }), + ); + assert.equal(provisioned.lifecycle, "gate_waiting_reports"); + await expectCodeAsync("operation_conflict", () => + preview(invocation(fixture)), + ); + for (const worker of provisioned.gate_workers) + await publishWorkerReport(fixture, worker); + const collected = await reconcile( + invocation(fixture, { random: deterministicRandom(61) }), + ); + assert.equal(collected.lifecycle, "gate_reports_collected"); + const status = readStatus(invocation(fixture)); + assert.equal(status.lifecycle, "gate_reports_collected"); + assert.deepEqual( + [...status.legal_next_operations], + ["apply.preview", "run.stand-down.begin"], + ); + const previewed = await preview(invocation(fixture)); + assert.deepEqual( + previewed.preview.gates.map( + ({ role_name, contract_role, result_kind, verdict, status: gateStatus }) => [ + role_name, + contract_role, + result_kind, + verdict, + gateStatus, + ], + ), + [ + ["reviewer", "reviewer", "review", "approve", "completed"], + ["validator", "validator", "validation", "pass", "completed"], + ], + ); + await record(fixture, receiptFor(previewed, "approve")); + const applied = await applyStage3(invocation(fixture)); + assert.equal(applied.lifecycle, "applied"); + const appliedStatus = readStatus(invocation(fixture)); + assert.equal(appliedStatus.lifecycle, "applied"); + assert.deepEqual( + [...appliedStatus.legal_next_operations], + ["run.stand-down.begin"], + ); + const stood = await standDown( + invocation(fixture, { reason: "normal_completion" }), + ); + assert.equal(stood.archived, true); + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + collected.integration.final_sha, + ); +}); + +test("a blocked gate report and a foreign workspace refuse preview", async () => { + const fixture = await assembledFixture({ + roles: [ + ...producerRoles(), + { name: "validator", contract_role: "validator", kind: "codex", mode: "gated" }, + ], + seed: 12, + configOverrides: stage3ApplyOverrides(), + }); + git(fixture.repository, "update-ref", APPLY_TARGET, fixture.result.fork_sha); + const producer = fixture.result.workers[0]; + mkdirSync(join(producer.cwd, "src")); + writeFileSync(join(producer.cwd, "src", "feature.mjs"), "export default 1;\n"); + git(producer.cwd, "add", "src/feature.mjs"); + git(producer.cwd, "commit", "-qm", "feature"); + await publishWorkerReport(fixture, producer); + const provisioned = await reconcile( + invocation(fixture, { random: deterministicRandom(70) }), + ); + assert.equal(provisioned.lifecycle, "gate_waiting_reports"); + await publishWorkerReport(fixture, provisioned.gate_workers[0], { + status: "blocked", + result: null, + }); + const collected = await reconcile( + invocation(fixture, { random: deterministicRandom(71) }), + ); + assert.equal(collected.lifecycle, "gate_reports_collected"); + await expectCodeAsync("operation_conflict", () => + preview(invocation(fixture)), + ); + await expectCodeAsync("bookkeeping_unknown", () => + preview({ + contextJson: context(fixture.repository, "wForeign"), + exec: fixture.fake.exec, + herdrBin: "fake", + }), + ); + assert.equal(applyCasCount(fixture), 0); +}); + +test("a crash after the CAS resolves to applied without a second CAS", async () => { + const { fixture, harvested } = await harvestedApplyFixture(9); + const previewed = await preview(invocation(fixture)); + await record(fixture, receiptFor(previewed, "approve")); + await expectCodeAsync("recovery_required", () => + applyStage3( + invocation(fixture, { + fault: (name) => { + if (name === "apply.after_cas") throw new Error("killed after CAS"); + }, + }), + ), + ); + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + harvested.integration.final_sha, + ); + const resolved = await applyStage3(invocation(fixture)); + assert.equal(resolved.lifecycle, "applied"); + assert.equal(resolved.resolved, true); + assert.equal(resolved.apply.outcome, "applied"); + assert.equal(resolved.apply.cas_count, 1); + assert.equal(applyCasCount(fixture), 1); + const replayed = await applyStage3(invocation(fixture)); + assert.equal(replayed.lifecycle, "applied"); + assert.equal(applyCasCount(fixture), 1); +}); diff --git a/tests/stage3-apply.test.mjs b/tests/stage3-apply.test.mjs new file mode 100644 index 0000000..8eb7da3 --- /dev/null +++ b/tests/stage3-apply.test.mjs @@ -0,0 +1,263 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { execFileSync } from "node:child_process"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + observeStage3ApplyTarget, + publishStage3ApplyCas, + resolveStage3ApplyOutcome, + stage3ApplyTargetRef, +} from "../scripts/stage3-apply.mjs"; +import { StateKernelError } from "../scripts/private-state-schema.mjs"; + +const roots = []; +process.on("exit", () => + roots.forEach((path) => rmSync(path, { recursive: true, force: true })), +); +const exec = (command, args, options = {}) => + execFileSync(command, args, { + encoding: "utf8", + stdio: ["ignore", "pipe", "pipe"], + ...options, + }).trim(); +const git = (repo, ...args) => exec("git", ["-C", repo, ...args]); + +function fixture() { + const repo = mkdtempSync(join(tmpdir(), "conductor-stage3-apply-")); + roots.push(repo); + execFileSync("git", ["init", "-q", "-b", "main", repo]); + git(repo, "config", "user.name", "Stage3 Test"); + git(repo, "config", "user.email", "stage3@example.invalid"); + writeFileSync(join(repo, "base"), "base\n"); + git(repo, "add", "."); + git(repo, "commit", "-qm", "base"); + const starting = git(repo, "rev-parse", "HEAD"); + git(repo, "update-ref", "refs/heads/release", starting); + writeFileSync(join(repo, "feature"), "feature\n"); + git(repo, "add", "."); + git(repo, "commit", "-qm", "integrated"); + const final = git(repo, "rev-parse", "HEAD"); + return { + repo, + integration: { + target_ref: "refs/heads/main", + starting_sha: starting, + final_sha: final, + }, + }; +} + +function code(expected, fn) { + assert.throws( + fn, + (error) => error instanceof StateKernelError && error.code === expected, + ); +} + +test("configured apply targets exist only for explicit version 3 configuration", () => { + assert.equal(stage3ApplyTargetRef({ version: 2 }), null); + assert.equal(stage3ApplyTargetRef({ version: 3, apply: null }), null); + assert.equal( + stage3ApplyTargetRef({ + version: 3, + apply: { target_ref: "refs/heads/release" }, + }), + "refs/heads/release", + ); + code("invalid_state", () => + stage3ApplyTargetRef({ version: 3, apply: { target_ref: "main" } }), + ); +}); + +test("apply target observation binds the exact fast-forward proposal", () => { + const { repo, integration } = fixture(); + const observed = observeStage3ApplyTarget({ + repository: repo, + targetRef: "refs/heads/release", + integration, + exec, + }); + assert.equal(observed.observed_sha, integration.starting_sha); + assert.equal(observed.final_sha, integration.final_sha); + assert.equal(observed.changed_path_count, 1); + assert.match(observed.diff_name_status_sha256, /^[a-f0-9]{64}$/); + assert.deepEqual( + observeStage3ApplyTarget({ + repository: repo, + targetRef: "refs/heads/release", + integration, + exec, + }), + observed, + ); +}); + +test("apply target observation fails closed on every drift", () => { + const { repo, integration } = fixture(); + code("invalid_config", () => + observeStage3ApplyTarget({ + repository: repo, + targetRef: integration.target_ref, + integration, + exec, + }), + ); + code("stale_source", () => + observeStage3ApplyTarget({ + repository: repo, + targetRef: "refs/heads/missing", + integration, + exec, + }), + ); + code("stale_source", () => + observeStage3ApplyTarget({ + repository: repo, + targetRef: "refs/heads/release", + integration: { ...integration, final_sha: integration.starting_sha }, + exec, + }), + ); + git(repo, "update-ref", "refs/heads/release", integration.final_sha); + code("stale_source", () => + observeStage3ApplyTarget({ + repository: repo, + targetRef: "refs/heads/release", + integration, + exec, + }), + ); + git(repo, "update-ref", "refs/heads/release", integration.starting_sha); + const checkout = mkdtempSync(join(tmpdir(), "conductor-stage3-checkout-")); + roots.push(checkout); + git(repo, "worktree", "add", "-q", checkout, "release"); + code("stale_source", () => + observeStage3ApplyTarget({ + repository: repo, + targetRef: "refs/heads/release", + integration, + exec, + }), + ); + git(repo, "worktree", "remove", "--force", checkout); + const emptyTree = git(repo, "hash-object", "-t", "tree", "/dev/null"); + const foreignRoot = git(repo, "commit-tree", emptyTree, "-m", "foreign"); + git(repo, "update-ref", integration.target_ref, foreignRoot); + code("stale_source", () => + observeStage3ApplyTarget({ + repository: repo, + targetRef: "refs/heads/release", + integration: { ...integration, final_sha: foreignRoot }, + exec, + }), + ); + git(repo, "update-ref", integration.target_ref, integration.final_sha); + const startingTree = git(repo, "rev-parse", `${integration.starting_sha}^{tree}`); + const sameTree = git( + repo, + "commit-tree", + startingTree, + "-p", + integration.starting_sha, + "-m", + "no-op", + ); + git(repo, "update-ref", integration.target_ref, sameTree); + code("stale_source", () => + observeStage3ApplyTarget({ + repository: repo, + targetRef: "refs/heads/release", + integration: { ...integration, final_sha: sameTree }, + exec, + }), + ); +}); + +test("apply publication performs exactly one compare-and-swap", () => { + const { repo, integration } = fixture(); + let preflights = 0; + const published = publishStage3ApplyCas({ + repository: repo, + targetRef: "refs/heads/release", + expectedSha: integration.starting_sha, + finalSha: integration.final_sha, + preflight: () => { + preflights += 1; + return { stable: true }; + }, + exec, + }); + assert.deepEqual(published, { + casCount: 1, + finalSha: integration.final_sha, + }); + assert.ok(preflights >= 2); + assert.equal( + git(repo, "rev-parse", "refs/heads/release"), + integration.final_sha, + ); +}); + +test("apply publication refuses drifting preflights with zero CAS", () => { + const { repo, integration } = fixture(); + let calls = 0; + code("stale_source", () => + publishStage3ApplyCas({ + repository: repo, + targetRef: "refs/heads/release", + expectedSha: integration.starting_sha, + finalSha: integration.final_sha, + preflight: () => ({ call: ++calls }), + exec, + }), + ); + assert.equal( + git(repo, "rev-parse", "refs/heads/release"), + integration.starting_sha, + ); + code("stale_source", () => + publishStage3ApplyCas({ + repository: repo, + targetRef: "refs/heads/release", + expectedSha: integration.starting_sha, + finalSha: integration.starting_sha, + preflight: () => ({ stable: true }), + exec, + }), + ); +}); + +test("uncertain apply outcomes resolve only to exact observations", () => { + const { repo, integration } = fixture(); + const request = { + repository: repo, + targetRef: "refs/heads/release", + expectedSha: integration.starting_sha, + finalSha: integration.final_sha, + exec, + }; + assert.deepEqual(resolveStage3ApplyOutcome(request), { + outcome: "unapplied", + cas_count: 0, + }); + git(repo, "update-ref", "refs/heads/release", integration.final_sha); + assert.deepEqual(resolveStage3ApplyOutcome(request), { + outcome: "applied", + cas_count: 1, + }); + const foreign = git( + repo, + "commit-tree", + git(repo, "rev-parse", `${integration.starting_sha}^{tree}`), + "-p", + integration.starting_sha, + "-m", + "foreign", + ); + git(repo, "update-ref", "refs/heads/release", foreign); + code("foreign_or_stale", () => resolveStage3ApplyOutcome(request)); + git(repo, "update-ref", "-d", "refs/heads/release"); + code("foreign_or_stale", () => resolveStage3ApplyOutcome(request)); +}); diff --git a/tests/stage3-crash-boundary.test.mjs b/tests/stage3-crash-boundary.test.mjs new file mode 100644 index 0000000..d867870 --- /dev/null +++ b/tests/stage3-crash-boundary.test.mjs @@ -0,0 +1,180 @@ +import test from "node:test"; +import { spawnSync } from "node:child_process"; +import { fileURLToPath } from "node:url"; +import { applyStage3, preview } from "../scripts/stage1-runtime.mjs"; +import { + acquireRepositoryLock, + loadActiveRun, + openRepositoryStore, + releaseRepositoryLock, +} from "../scripts/state-kernel.mjs"; +import { + STAGE3_APPLY_TARGET as APPLY_TARGET, + assert, + expectCode, + expectCodeAsync, + git, + privateDocuments, + stage3ApprovedApplyFixture, + stage3Invocation, +} from "./stage1-runtime-helpers.mjs"; + +const fixtureChild = fileURLToPath( + new URL("./fixtures/stage3-crash-child.mjs", import.meta.url), +); + +async function approvedApplyFixture(seed) { + const { fixture, harvested } = await stage3ApprovedApplyFixture(seed); + return { fixture, harvested, invocation: stage3Invocation(fixture) }; +} + +function killedApply(fixture, boundary) { + const child = spawnSync( + process.execPath, + [fixtureChild, fixture.repository, fixture.workspace, boundary], + { encoding: "utf8", timeout: 30_000 }, + ); + assert.equal(child.signal, "SIGKILL", `${boundary}: ${child.stderr}`); + return child; +} + +function retainedApplyPublication(fixture) { + const documents = privateDocuments(fixture.stateRoot); + const entries = documents.filter( + ({ path, value }) => + path.includes("/operations/") && + value.operation_type === "apply.publish", + ); + const guards = documents.filter( + ({ path, value }) => + path.includes("/operation-guards/") && + value.operation_type === "apply.publish", + ); + return { entries, guards }; +} + +test("true SIGKILL before the apply CAS retains an exact resolvable intent", async () => { + const { fixture, harvested, invocation } = await approvedApplyFixture(21); + killedApply(fixture, "apply.before_cas"); + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + harvested.integration.starting_sha, + ); + const retained = retainedApplyPublication(fixture); + assert.equal(retained.entries.length, 1); + assert.equal(retained.entries[0].value.phase, "intent"); + assert.equal(retained.guards.length, 1); + const store = openRepositoryStore({ + stateRoot: fixture.stateRoot, + repoPath: fixture.repository, + }); + expectCode("recovery_required", () => + loadActiveRun(store, { workspaceId: fixture.workspace }), + ); + const resolved = await applyStage3(invocation); + assert.equal(resolved.lifecycle, "apply_voided"); + assert.equal(resolved.resolved, true); + assert.equal(resolved.apply.outcome, "unapplied"); + assert.equal(resolved.apply.cas_count, 0); + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + harvested.integration.starting_sha, + ); + const after = retainedApplyPublication(fixture); + assert.equal(after.entries[0].value.phase, "observed"); + assert.equal(after.guards.length, 0); +}); + +test("true SIGKILL after the apply CAS resolves to applied without replay", async () => { + const { fixture, harvested, invocation } = await approvedApplyFixture(22); + killedApply(fixture, "apply.after_cas"); + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + harvested.integration.final_sha, + ); + const retained = retainedApplyPublication(fixture); + assert.equal(retained.entries.length, 1); + assert.equal(retained.entries[0].value.phase, "intent"); + assert.equal(retained.guards.length, 1); + const resolved = await applyStage3(invocation); + assert.equal(resolved.lifecycle, "applied"); + assert.equal(resolved.resolved, true); + assert.equal(resolved.apply.outcome, "applied"); + assert.equal(resolved.apply.cas_count, 1); + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + harvested.integration.final_sha, + ); + const replayed = await applyStage3(invocation); + assert.equal(replayed.lifecycle, "applied"); + assert.equal(replayed.replayed, true); +}); + +test("true SIGKILL at the durable intent resolves without a guard", async () => { + const { fixture, harvested, invocation } = await approvedApplyFixture(23); + killedApply(fixture, "apply.publish:journal.after_intent_durable"); + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + harvested.integration.starting_sha, + ); + const retained = retainedApplyPublication(fixture); + assert.equal(retained.entries.length, 1); + assert.equal(retained.entries[0].value.phase, "intent"); + assert.equal(retained.guards.length, 0); + const resolved = await applyStage3(invocation); + assert.equal(resolved.lifecycle, "apply_voided"); + assert.equal(resolved.apply.outcome, "unapplied"); +}); + +test("a foreign SHA on the uncertain apply target refuses resolution forever", async () => { + const { fixture, harvested, invocation } = await approvedApplyFixture(24); + killedApply(fixture, "apply.before_cas"); + const startingTree = git( + fixture.repository, + "rev-parse", + `${harvested.integration.starting_sha}^{tree}`, + ); + const foreign = git( + fixture.repository, + "commit-tree", + startingTree, + "-p", + harvested.integration.starting_sha, + "-m", + "foreign", + ); + git(fixture.repository, "update-ref", APPLY_TARGET, foreign); + await expectCodeAsync("foreign_or_stale", () => applyStage3(invocation)); + const retained = retainedApplyPublication(fixture); + assert.equal(retained.entries[0].value.phase, "intent"); + assert.equal(retained.guards.length, 1); + git( + fixture.repository, + "update-ref", + APPLY_TARGET, + harvested.integration.starting_sha, + ); + const resolved = await applyStage3(invocation); + assert.equal(resolved.lifecycle, "apply_voided"); +}); + +test("a held repository lock refuses preview and apply without effects", async () => { + const { fixture, harvested, invocation } = await approvedApplyFixture(25); + const store = openRepositoryStore({ + stateRoot: fixture.stateRoot, + repoPath: fixture.repository, + }); + const lock = acquireRepositoryLock(store, { operationId: "competing-owner" }); + try { + await expectCodeAsync("lock_busy", () => applyStage3(invocation)); + await expectCodeAsync("lock_busy", () => preview(invocation)); + } finally { + releaseRepositoryLock(lock); + } + assert.equal( + git(fixture.repository, "rev-parse", APPLY_TARGET), + harvested.integration.starting_sha, + ); + const applied = await applyStage3(invocation); + assert.equal(applied.lifecycle, "applied"); +}); diff --git a/tests/state-kernel.test.mjs b/tests/state-kernel.test.mjs index 930d3eb..bb29614 100644 --- a/tests/state-kernel.test.mjs +++ b/tests/state-kernel.test.mjs @@ -24,12 +24,14 @@ import { createRun, inspectRepositoryLock, loadActiveRun, + loadUncertainApplyRun, openRepositoryStore, performJournaledOperation, publishExclusiveJson, readPrivateJson, releaseRepositoryLock, resolveGitCommonDirectory, + resolveUncertainApplyPublication, writeAtomicJson, } from "../scripts/state-kernel.mjs"; import { @@ -1020,3 +1022,114 @@ test("exclusive publication never overwrites an active pointer", () => { ); releaseRepositoryLock(lock); }); + +test("uncertain apply publication resolves only by exact observation", async () => { + const context = setup(); + const { lock } = startRun(context); + await expectCodeAsync("recovery_required", () => + performJournaledOperation( + lock, + operation({ + operationId: "apply-publish-attempt", + operationType: "apply.publish", + subject: { kind: "apply", id: "apply", generation: SUBJECT_GENERATION }, + effect: async () => { + throw new Error("crash before observation"); + }, + }), + ), + ); + expectCode("recovery_required", () => + loadActiveRun(context.store, { workspaceId: "wB1" }), + ); + const identity = { + document_type: "herdr-conductor-stage3-apply", + schema_version: 1, + repository_key: context.store.repository.key, + workspace_id: "wB1", + run_id: "run-b1", + run_generation: GENERATION, + attempt_generation: SUBJECT_GENERATION, + consumption_entry_digest: REQUEST_DIGEST, + target_ref: "refs/heads/release", + expected_sha: context.forkSha, + final_sha: "e".repeat(40), + cas_count: 0, + outcome: "unapplied", + }; + await expectCodeAsync("operation_conflict", () => + resolveUncertainApplyPublication(lock, { + workspaceId: "wB1", + operationId: "apply-publish-other", + resolve: () => ({ + resultDigest: RESULT_DIGEST, + observedIdentity: identity, + }), + }), + ); + await expectCodeAsync("invalid_state", () => + resolveUncertainApplyPublication(lock, { + workspaceId: "wB1", + operationId: "apply-publish-attempt", + resolve: () => ({ resultDigest: RESULT_DIGEST }), + }), + ); + const uncertain = loadUncertainApplyRun(context.store, { + workspaceId: "wB1", + }); + assert.equal(uncertain.uncertain.operation_id, "apply-publish-attempt"); + const resolved = await resolveUncertainApplyPublication(lock, { + workspaceId: "wB1", + operationId: "apply-publish-attempt", + resolve: (entry) => { + assert.equal(entry.operation_type, "apply.publish"); + assert.notEqual(entry.phase, "observed"); + return { resultDigest: RESULT_DIGEST, observedIdentity: identity }; + }, + }); + assert.equal(resolved.resolved, true); + assert.equal(resolved.resultDigest, RESULT_DIGEST); + const active = loadActiveRun(context.store, { workspaceId: "wB1" }); + const entry = active.journal.find( + (candidate) => candidate.operation_id === "apply-publish-attempt", + ); + assert.equal(entry.phase, "observed"); + assert.deepEqual(entry.observed_identity, identity); + assert.equal(active.state.journal_head, entry.entry_digest); + await expectCodeAsync("recovery_required", () => + resolveUncertainApplyPublication(lock, { + workspaceId: "wB1", + operationId: "apply-publish-attempt", + resolve: () => ({ + resultDigest: RESULT_DIGEST, + observedIdentity: identity, + }), + }), + ); + releaseRepositoryLock(lock); +}); + +test("apply resolution refuses foreign uncertain operations", async () => { + const context = setup(); + const { lock } = startRun(context); + await expectCodeAsync("recovery_required", () => + performJournaledOperation( + lock, + operation({ + effect: async () => { + throw new Error("pane crash"); + }, + }), + ), + ); + expectCode("recovery_required", () => + loadUncertainApplyRun(context.store, { workspaceId: "wB1" }), + ); + await expectCodeAsync("recovery_required", () => + resolveUncertainApplyPublication(lock, { + workspaceId: "wB1", + resolve: () => ({ resultDigest: RESULT_DIGEST, observedIdentity: null }), + }), + ); + releaseRepositoryLock(lock); +}); diff --git a/tests/task-report-schema.test.mjs b/tests/task-report-schema.test.mjs index 1211fdf..0f89b98 100644 --- a/tests/task-report-schema.test.mjs +++ b/tests/task-report-schema.test.mjs @@ -200,6 +200,72 @@ test("configuration v2 is closed and artifact allowlists are empty", () => { ), ), ); + code("invalid_contract", () => + parseStage2ConfigBytes( + Buffer.from(canonicalJson({ ...config, apply: null })), + ), + ); +}); + +test("configuration v3 requires one explicit apply member", () => { + const v3 = { + version: 3, + state_root: { kind: "default" }, + worktree_root: ".conductor-worktrees", + apply: { target_ref: "refs/heads/release" }, + roles: [ + { + name: "builder", + contract_role: "builder", + kind: "codex", + mode: "write", + assignment, + validator_artifacts: [], + }, + ], + }; + assert.equal( + parseStage2ConfigBytes(Buffer.from(canonicalJson(v3))).apply.target_ref, + "refs/heads/release", + ); + assert.equal( + parseStage2ConfigBytes( + Buffer.from(canonicalJson({ ...v3, apply: null })), + ).apply, + null, + ); + const { apply: _apply, ...withoutApply } = v3; + code("invalid_contract", () => + parseStage2ConfigBytes(Buffer.from(canonicalJson(withoutApply))), + ); + code("invalid_contract", () => + parseStage2ConfigBytes( + Buffer.from(canonicalJson({ ...v3, apply: {} })), + ), + ); + for (const targetRef of [ + "main", + "refs//heads/release", + "refs/heads/../release", + "refs/heads/./release", + "refs\\heads\\release", + "refs/heads/re lease", + ]) + code("invalid_contract", () => + parseStage2ConfigBytes( + Buffer.from(canonicalJson({ ...v3, apply: { target_ref: targetRef } })), + ), + ); + code("invalid_contract", () => + parseStage2ConfigBytes( + Buffer.from( + canonicalJson({ + ...v3, + apply: { target_ref: "refs/heads/release", mode: "fast-forward" }, + }), + ), + ), + ); }); test("task and report digests are domain separated and validated", () => {