Epic: Two-tier FS-free harness + shared durable sandbox pool

[pdettori]

Epic: Two-tier FS-free harness + shared durable sandbox pool

Origin

Started as "run the Archetype-A demo on OpenShift." Investigation surfaced a deeper architectural issue: the credentialed harness does filesystem I/O for the leaf envelope (run-leaf.ts readItem/writeVerdict via node:fs on the /work PVC), which forces cross-node RWX on OCP. The OCP RWX pain is a symptom, not the problem.

Target architecture

A two-tier fleet:

• Harness (agent brain). Holds credentials, runs the model loop, makes decisions. Performs no filesystem access and executes no untrusted operations — only network I/O (LLM, Redis, sandbox exec API). Every operation the agent issues — model tool calls and the leaf envelope — is delegated to the sandbox or transported over the network. Stateless, fresh context per leaf, scales densely, needs no Kata.
• Sandbox (execution surface / hands). The sole filesystem/syscall surface. Backed by durable storage (not emptyDir). Runs all untrusted operations. The only component that must be Kata-isolated to contain a kernel exploit. A shared pool: many harnesses exec into a smaller set of sandboxes (target ~20:1, ratio TBD empirically). Harnesses sharing a sandbox share one repo copy; each leaf gets an isolated worktree.

Threat model (corrected)

There is no agent in the sandbox — it is a passive kubectl exec target. The threat is a compromised/injected harness (malicious skill, prompt injection) and kernel exploits. Blast radius is contained by giving the harness zero FS/exec surface and Kata-isolating only the sandboxes. "Verdict integrity via the filesystem" is a non-issue (a compromised harness can forge regardless).

Key decisions (locked)

• Envelope leaves the filesystem: inputs inline in the request, sync verdict inline in the response, async verdict + done-marker in Redis (already a dependency).
• Sandbox working set on a durable PVC, not emptyDir.
• Shared sandbox pool, N:M, ratio ~20:1 TBD by perf experiments.
• RWX does not vanish — it relocates to the sandbox tier, and only if experiments show one sandbox is insufficient. The harness mounts nothing.
• Model tool ops are already fully sandboxed (7/7 Pi ops) — no change needed there.
• PoC without Kata first; Kata + ratio experiments (P3) deferred until after the architecture is proven on OCP (P0′).

Superseded

Local un-pushed branch docs/archetype-a-ocp-support (docs/specs/2026-07-01-archetype-a-ocp-support-design.md + its plan) proposed NFS RWX so the harness could co-mount /work. Superseded — after P1 the harness mounts nothing. Keep as reference only.

Phases (dependency-ordered)

• P1: FS-free harness — leaf envelope off the filesystem (inline + Redis) #45 — P1: FS-free harness (foundation; brainstorm first)
• P2: Shared sandbox pool + harness→sandbox routing #46 — P2: Shared sandbox pool + routing (depends on P1)
• P0′: OpenShift deployment of the two-tier architecture #47 — P0′: OpenShift deployment (threads through P1/P2)
• P3: Sandbox sharing-ratio experiments (CAP/N tuning) #48 — P3 (deferred): Kata isolation + ratio experiments (after P0′)

Working conventions

• Specs in docs/specs/YYYY-MM-DD-<topic>-design.md + registry (docs/specs/README.md); plans in docs/superpowers/plans/.
• Each phase: brainstorming → spec → writing-plans → implementation, its own cycle.
• DCO: git commit -s; attribution trailer Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> (never Co-authored-by).