design(runtime): define the runtime control-plane architecture

[Brad-Edwards]

Context

RAES has a raes_runtime control-plane abstraction, an in-memory default store, an optional local JSON store, and a reference HTTP adapter. The repository does not yet define one coherent architecture for how those surfaces relate to embedded consumers, local tools, long-lived services, targets/backends, or distributed execution.

Issue #1092 and PR #1136 exposed the gap by proposing SQLite durability, operation recovery, single-process ownership, and store compatibility rules inside what appears to be a local storage change. Those choices cannot be evaluated in isolation because expected RAES use ranges from tests and single-user single-scenario environments through RAE, env-packs, ETV, air-gapped deployments, multiprocess execution, and high-load services.

This issue owns the architectural design. It does not presuppose SQLite, another database, a queue, an event log, a service topology, or a single universal deployment model.

Questions the design must answer

• What is the runtime control-plane: an in-process facade, an embedded component, a deployable service, a shared service, a family of conforming implementations, or some combination?
• Where is its boundary relative to SDL authoring, processor compile/plan behavior, runtime execution, RuntimeTarget, reference implementations, concrete backends, RAE, env-packs, and ETV?
• What state is authoritative, derived, reconstructable, operational, auditable, evidentiary, or provenance-bearing?
• What owns a target, scenario, run, participant, tenant, and operation, and what are the required isolation boundaries?
• Is an operation a synchronous backend call with bookkeeping, durable work that can outlive its submitter, desired-state reconciliation, or profile-dependent behavior?
• Which concurrency models must be supported: threads, multiple processes, multiple hosts, multiple controllers, multiple targets, and multiple tenants?
• What consistency and concurrency-control model applies to snapshots, operations, idempotency claims, audit events, and external backend effects?
• What are the required behaviors after process loss, memory loss, host loss, storage loss, network partition, cancellation, timeout, and an indeterminate backend effect?
• Which responsibilities belong to the runtime, the control-plane implementation, the persistence/coordination provider, the backend, and the embedding application?
• Which operating contexts are supported, and what are their explicit guarantees and nonclaims: tests, development, single-user local execution, embedded RAE/env-pack/ETV use, air-gapped deployments, high-load services, and highly available deployments?
• What extension contracts and capability semantics are required for persistence, coordination, scheduling, recovery, and observation without baking one deployment topology into RuntimeControlPlane?
• How are startup, shutdown, configuration, migrations, backup/restore, upgrades, compatibility, deprecation, and operational health owned?
• Which existing control-plane, store, HTTP adapter, requirement, documentation, and test surfaces should be retained, changed, split, deprecated, or removed?
• What is the resulting disposition of issue fix(runtime): make local operation state transactional and durable #1092 and PR fix(runtime): make local operation state transactional and durable #1136?

Acceptance criteria

• The required new ADRs and amendments are merged, indexed, and citable. They record the control-plane role and boundaries, state authority, execution and concurrency model, failure/recovery model, supported operating contexts, extension seams, alternatives, consequences, risks, and explicit nonclaims.
• Architectural design documents are merged for every architecture selected by the ADRs. They show component boundaries, ownership, data and control flow, lifecycle, persistence/coordination interactions, security boundaries, and failure/recovery behavior in enough detail to guide implementation and review.
• The design includes an evidence-backed audit of the current repository surfaces and an explicit retain/change/split/deprecate/remove disposition, including issue fix(runtime): make local operation state transactional and durable #1092, PR fix(runtime): make local operation state transactional and durable #1136, RuntimeControlPlane, the store protocols and implementations, and the reference HTTP adapter.
• All implementation work required by the accepted design is represented by concrete issues in the Runtime Control-Plane milestone. Each issue has a bounded scope, dependencies and ordering, acceptance criteria, verification expectations, and requirement/ADR traceability.
• The implementation issue set covers all required code, contracts, migrations, compatibility work, tests, documentation, deployment/composition work, and operational tooling; no required work remains hidden in the design issue or deferred only in prose.
• The design issue itself does not implement a storage engine or select one by incidental code change. It closes only when the ADRs, architectural designs, and complete implementation issue set above are in place.

Related

• fix(runtime): make local operation state transactional and durable #1092
• fix(runtime): make local operation state transactional and durable #1136
• Verify API-404 completion against control-plane profiles #8 (API-404)

[Brad-Edwards]

🛠️ Picked up by /implement - driver codex, branch 1151-api-404-runtime-control-plane-architecture, 2026-09-03T01:46:29.004Z.

[Brad-Edwards]

Pre-PR base synchronization

• Source: refs/remotes/origin/dev at cba73a815d054cd7203578c0f30ec46e53bd239a
• Outcome: merged_conflicts_resolved
• Published feature head: 271982755dc52652e2e8e0bc2b699d4cb49b63b4
• Verified tree: fdf51eff28d64e0c13771b352f75085085ee4274

[Brad-Edwards]

gc workflow phase recorded: preflight (issue #1151). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

[Brad-Edwards]

gc workflow phase recorded: plan (issue #1151). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

Implementation plan

1. Retain and review the architecture-preflight changes as binding design guidance.
2. Extend the issue-1151 structural test first so it requires an FM3 assurance artifact, explicit target/run and actor isolation, and concrete issue linkage for every implementation work package; run it to record the expected red state.
3. Reclassify ADR-104 as FM3 and add a scoped abstract operation-state model with lifecycle, concurrency, atomicity, recovery, provenance, and security invariants plus explicit design-only waivers.
4. Align the ADR, composition design, disposition, machine-readable program, and prose program on one mutation authority; immutable actor propagation; target/run store pinning; actor-scoped idempotency; authorization-bound receipts; atomic terminal audit; strict codec validation; and P0–P3 guarantees/nonclaims.
5. File or reuse bounded Runtime Control-Plane milestone issues for every CP work package, with dependencies, acceptance criteria, verification, and API-404/ADR-104 traceability, then record their issue numbers in the design program.
6. Update API-404 traceability and the ADR index/policy checksum through the repository’s governed surfaces.
7. Run the focused issue-1151 test, then run RAES_REQUIREMENT_UID=API-404 implementations/python/.venv/bin/python tools/check_repo_policy.py, tools/check_requirement_governance.py, and tools/verify_all.py; complete the configured mechanical verification, pre-push code/design review, and test-quality review.
8. Publish with a neutral Conventional Commit, resynchronize with dev, render a policy-compliant body, and open the replacement PR without merging it.

[Brad-Edwards]

gc_codex_review — cycle 1 of 1 (pre-push) on issue #1151 (branch 1151-api-404-runtime-control-plane-architecture)
Diff mode: inline — the complete diff was supplied in one prompt

Core review

Verdict: ship-with-fixes

The change is broadly shaped correctly: it promotes the control-plane design to an explicit FM3 lifecycle model, preserves the incumbent control-plane, store, contract, backend, and HTTP seams, and separates P0–P2 guarantees from P3 nonclaims without prematurely selecting an implementation. The abstract model is now the load-bearing design authority, however, so its transition semantics and the dependency ordering of the implementation program must be internally complete before downstream work relies on them.

Blocking findings (2):

1. [class] Architecture-fit: Declared lifecycle transitions lack abstract-operation semantics — specs/formal/runtime-control-plane/README.md:29
   The lifecycle declares both deny and cancel, but the exhaustive-looking abstract-operations table defines neither. This leaves rejection without pinned audit semantics and leaves ACCEPTED -> CANCELLED outside commit, whose precondition requires RUNNING. Those omissions are material because rejection and cancellation are explicitly named elsewhere as mutation families governed by the common audit and atomic-commit discipline. Add corresponding abstract operations with their preconditions and atomic effects, and clarify whether REJECTED is an operation state or only an audit outcome.
2. [one-off] Code-quality: CP-9 omits dependencies required by its own conformance scope — docs/research/runtime-control-plane/implementation-program.json:152
   CP-9 depends only on CP-2, CP-3, and CP-6, while its acceptance and Markdown verification scope explicitly includes actor-scoped idempotency and authorization-boundary tests delivered by CP-7 and CP-8. The dependency-ordered program therefore permits CP-9 to complete before the capabilities it must verify exist. Add the required CP-7/CP-8 dependency relationship and keep the machine-readable and Markdown programs aligned.

Security review

Verdict: ship

This design-only change is shaped correctly around the control plane’s actual security seam: authenticated transport identity becomes immutable operation context, authorization follows that context through retries and reads, and each store is explicitly isolated to one target/run. It consistently addresses authentication, authorization, idempotency, audit redaction, strict state validation, filesystem safety, and atomic persistence without changing executable behavior or prematurely claiming implementation guarantees. The capability boundary leaves the obvious P3 coordination and multitenancy variation open under a future explicit contract. I found no concrete exploitable security issue introduced by this diff.

No blocking findings.

[Brad-Edwards]

gc_codex_review pre-push cycle 1 of 1 complete for issue #1151 on branch '1151-api-404-runtime-control-plane-architecture'. Posted by the MCP server to enforce the pre-push hard-cap-1 contract (issues #796, #804, #906). Do not edit or delete — used by the next gc_codex_review (uncommitted) invocation to count cycles.

[Brad-Edwards]

Review decision record — codex cycle 1 (issue #1151)

Reviewer: codex
Cycle: 1

Architectural read:

Core reviewer: The change is broadly shaped correctly: it promotes the control-plane design to an explicit FM3 lifecycle model, preserves the incumbent control-plane, store, contract, backend, and HTTP seams, and separates P0–P2 guarantees from P3 nonclaims without prematurely selecting an implementation. The abstract model is now the load-bearing design authority, however, so its transition semantics and the dependency ordering of the implementation program must be internally complete before downstream work relies on them.

Security reviewer: This design-only change is shaped correctly around the control plane’s actual security seam: authenticated transport identity becomes immutable operation context, authorization follows that context through retries and reads, and each store is explicitly isolated to one target/run. It consistently addresses authentication, authorization, idempotency, audit redaction, strict state validation, filesystem safety, and atomic persistence without changing executable behavior or prematurely claiming implementation guarantees. The capability boundary leaves the obvious P3 coordination and multitenancy variation open under a future explicit contract. I found no concrete exploitable security issue introduced by this diff.

Blocking findings: 2

Finding 1 — class (2 instances)

• ID: F1
• Title: [core] Architecture-fit: Declared lifecycle transitions lack abstract-operation semantics
• Location: specs/formal/runtime-control-plane/README.md:29
• Decision: fix
• Rationale: The lifecycle declares both deny and cancel, but the exhaustive-looking abstract-operations table defines neither. This leaves rejection without pinned audit semantics and leaves ACCEPTED -> CANCELLED outside commit, whose precondi…
• Instances:
  • specs/formal/runtime-control-plane/README.md:29
  • specs/formal/runtime-control-plane/README.md:30

Finding 2 — one-off

• ID: F2
• Title: [core] Code-quality: CP-9 omits dependencies required by its own conformance scope
• Location: docs/research/runtime-control-plane/implementation-program.json:152
• Decision: fix
• Rationale: CP-9 depends only on CP-2, CP-3, and CP-6, while its acceptance and Markdown verification scope explicitly includes actor-scoped idempotency and authorization-boundary tests delivered by CP-7 and CP-8. The dependency-ordered program theref…

[Brad-Edwards]

gc_test_quality_review cycle 1 of 1 — issue #1151

Reviewer: test-quality (claude-sonnet-5 via gc_test_quality_review)
Branch: 1151-api-404-runtime-control-plane-architecture
Cycle: 1 / 1
Findings: 0 (clean run)

[Brad-Edwards]

gc_test_quality_review cycle 1 of 1 complete for issue #1151 on branch '1151-api-404-runtime-control-plane-architecture'. Posted by the MCP server to enforce the gc_test_quality_review hard-cap-1 contract (issue #884 follow-up, default lowered in #906). Do not edit or delete — used by the next gc_test_quality_review invocation to count cycles.

[Brad-Edwards]

Review decision record — test-quality cycle 1 (issue #1151)

Reviewer: test-quality
Cycle: 1

Architectural read:

This is a design/documentation milestone gate, not a code change: the diff adds ADR-104, the runtime-control-plane research set, and the FM3 formal-model README, with a pytest file whose job is to pin that the design-set deliverables exist, are non-empty, cross-reference each other, and encode an acyclic work-package graph. That is a known, repeated shape in this repo — five prior issues (794, 810, 811, 812, 813) use the identical "structural acceptance gate" pattern — so this fits established vocabulary rather than inventing a new one. The test file does real, non-trivial verification within that scope: the dependency-graph test performs an actual topological-sort/cycle check against the JSON (not just membership), and the deliverable-set test enforces exact-match plus non-empty file content. Critically, the spec document itself states the limiting principle a reviewer would otherwise have to supply: "The issue-1151 structural test pins this artifact and its required terms... they do not demonstrate the profile guarantees above" — the design explicitly disclaims that these are behavioral/security tests, which forecloses the main risk category (mistaking a doc-keyword check for enforcement proof) before it can mislead a future reader. There is no production code in this diff for the false-assurance rubric (mocks, DB/HTTP/subprocess, security enforcement) to attach to.

Blocking findings: 0 (clean run)

[Brad-Edwards]

Pre-PR base synchronization

• Source: refs/remotes/origin/dev at 635825a15a4dfe78a6aac4596097f2e7adf44dde
• Outcome: merged_clean
• Published feature head: 513ea735479e61db524d86192884aa5561b11c8c
• Verified tree: df122d102e262bc3cd253edd995202984970ce99