Add the registrar-redteam Docker scenario and gate every pull request on it

[sehkone]

Add the registrar-redteam Docker scenario and gate every pull request on it

Context

The registrar surface exists to make one thing impossible: a holder of the runtime registrar's credential must be unable to create or modify any OpenBao role or policy whose effect grants registrar authority. Every other part of the design serves that guarantee, and no unit test can assert it — an escalation attempt refused by a mock is a mock asserting its own refusal logic.

The single most important fact this scenario asserts, and the one that decides how the red-team cases are written: the registrar holds no OpenBao credential of any kind. Its only credential is the mTLS client key and certificate for the daemon's unix socket. The broad role/policy-write authority lives in the daemon's bootroot-internal credential, which authenticates by OpenBao TLS certificate auth (auth/cert) under a trusted-certificate entry bound to the daemon's internal leaf alone. So "escalation denied" is not "the registrar's token was rejected by an ACL" — it is "there is no token, and the registrar's certificate cannot obtain one."

The test is deliberately credential-level: it models a credential leaked or copied off the host — a disk image, a filesystem backup restored elsewhere. It deliberately does not model an exploited request handler (that handler runs as root on this host, so exploiting it is root here), a compromised control plane (which never holds this credential and is bounded by the verb's input contract instead), or root on the bootroot host (which can read the daemon's internal credential directly and is out of scope). Writing the scenario as if it covered those would overstate what the design delivers.

This is one of three acceptance arms. It carries the security assertions and everything else that costs one stack bring-up and then runs in seconds, and it gates every pull request — a security guarantee that only re-proves itself on a manual dispatch is not a regression gate. A sibling cargo arm carries everything that needs no Docker; a sibling endurance scenario carries the two cases whose cost is waiting.

Scope

The scenario and its wiring

• Script scripts/impl/run-registrar-redteam.sh, #[ignore]d cargo wrapper tests/docker_e2e_registrar_redteam.rs, a step in scripts/preflight/ci/e2e-matrix.sh, and a test-docker-e2e-matrix matrix entry in .github/workflows/ci.yml with label registrar-redteam and artifact ci-registrar-redteam.
• Follow the repository's existing E2E harness conventions rather than inventing a parallel harness: scripts/impl/run-<scenario>.sh plus a tests/docker_e2e_<scenario>.rs wrapper plus a preflight step, in the shape of the two-instance-isolation scenario.
• Create the shared setup helper the sibling endurance scenario will source. Put the registrar-surface bring-up that both Docker scenarios need in a sourced helper under scripts/impl/lib/ rather than leaving it to be copied — scripts/impl/lib/audit-log.sh is the precedent for that placement.
• The scenario must not leave residue in a shared OpenBao instance or collide with a developer's real install on the same machine — derive run-scoped instance names and pick free host ports, exactly as run-two-instance-isolation.sh does.

Escalation denied — the core test

Holding only what the registrar actually holds — its client key and certificate, and nothing else — it is impossible to create or update any OpenBao role or policy whose effect grants registrar authority. All four of the following must be asserted:

• An auth/cert/login presenting the registrar's client leaf is refused, because the trusted-certificate entry binds only the daemon's internal leaf. This is the assertion that carries the guarantee; the rest are corroboration.
• A sweep of every file the registrar host holds for the registrar surface finds no role_id, no secret_id, and no OpenBao token — there is no credential to escalate with.
• Attack shape (a): POST auth/approle/role/<attacker-chosen-name> with token_policies naming each of the privileged runtime policies fails, unauthenticated, for want of a token.
• Attack shape (b): PUT sys/policies/acl/<attacker-chosen-name> with a body granting sys/policies/acl/* and auth/approle/role/* fails the same way.

The privileged policy set, enumerated — and kept enumerated. "Any privileged runtime policy" means, literally, every POLICY_BOOTROOT_* constant in src/commands/init/constants.rs. As of this issue that is six: POLICY_BOOTROOT_AGENT = bootroot-agent, POLICY_BOOTROOT_RESPONDER = bootroot-responder, POLICY_BOOTROOT_STEPCA = bootroot-stepca, POLICY_BOOTROOT_RUNTIME_SERVICE_ADD = bootroot-runtime-service-add, POLICY_BOOTROOT_RUNTIME_ROTATE = bootroot-runtime-rotate, POLICY_BOOTROOT_INFRA_ROTATE = bootroot-infra-rotate (:133–:144). The bootroot-internal credential's policy is a seventh whose name this issue deliberately does not spell: the internal-credential issue adds its constant to that same module, and hardcoding a guess here would drift.

The completeness guard, so the seventh policy is covered without naming it. The attack list is a checked-in newline-delimited file — tests/e2e/registrar/privileged-policies.txt, a new registrar/ subdirectory under the existing tests/e2e/, which Cargo treats as data rather than as a test target since only .rs files directly in tests/ become targets (the checked-in tests/e2e/docker_harness/scenarios/*.json files are the in-repo precedent for data living there) — that run-registrar-redteam.sh reads to drive attack shape (a), and that a cargo test validates: the test scans the constants module's source (include_str!("../src/commands/init/constants.rs")) for POLICY_BOOTROOT_ declarations, extracts each declared string literal, and asserts the file contains every one of them and nothing that is not one of them. A new privileged policy constant — the internal credential's, or any later one — then fails that test until it is added to the file, instead of silently shrinking the attack surface. This is the mechanism that makes the enumeration self-maintaining; do not replace it with a hand-kept list and a comment, and do not let the scenario script carry its own second copy of the names.

The guard is this scenario's machinery, and it stays here even though it runs in the cheap tier. The suite's tiering rule sends every case needing no Docker to the sibling cargo arm — but that rule governs cases, assertions about the registrar's behaviour, and this guard asserts something about this scenario's own data: that the attack list it drives shape (a) from still matches the constants module. The file, the script's use of it and the guard are one mechanism; splitting the guard into the other arm would leave two issues each owning a third of it and neither reviewer seeing it whole. So it ships here, the cargo arm neither owns nor duplicates it, and no second scan of the constants module is added anywhere.

The guard must run where Docker does not, and that placement is part of the deliverable. It is a source-scanning test that needs no container, no OpenBao and no root, so it goes in a non-#[ignore]d test that plain cargo test runs — not inside the #[ignore]d Docker wrapper, whose contents run only when the matrix job invokes them. The distinction is the guard's entire value: a contributor adding a POLICY_BOOTROOT_* constant in an ordinary pull request must see it fail in the fast unit job, immediately and for an obvious reason, rather than in a Docker scenario whose failure reads as an environment problem — or, if the wrapper is skipped for any reason, not at all. Note also that the enumeration lives in src/commands/init/constants.rs, which is in the binary crate, so a narrowed local invocation such as cargo test --lib will not run a guard placed beside it; state the invocation that does.

Functionality, containment and unreachability

• Functionality preserved. With the registrar credential, minting a service identity over the socket succeeds and creates the derived role and policy for bootroot-service-<registration_id> (exact spellings from service_role_name / service_policy_name, src/commands/service/approle.rs:103/:107); deregistering succeeds; both are idempotent on a second invocation. The caller passes only the identity's parts and never the internal prefix.
• Derived policy only. Every role the registrar mints reads back from OpenBao with token_policies == ["bootroot-service-<registration_id>"] and nothing else.
• Authority containment. The registrar's client certificate is accepted at the mint and deregister verbs and refused everywhere else it can be presented on this host, which today is exactly: auth/cert/login (covered above), and any operation identifier on the registrar socket other than the two verbs (an unknown or third operation is rejected, not dispatched). It is also not usable as a service identity: the recognition predicate from the client-identity issue rejects it, and service add refuses the reserved bootroot- prefix. If the implementer finds any other inbound daemon surface that accepts a client certificate, it must be covered too — the list above is complete as of this issue, not by construction.
• CA / responder-HMAC / EAB reads are not grantable, at named paths. The three reads are the ones POLICY_BOOTROOT_RUNTIME_SERVICE_ADD already grants today (src/commands/init/steps/openbao_setup.rs:458), at paths built from the constants at src/commands/init/constants.rs:178–:180: {kv_mount}/data/{PATH_CA_TRUST} and {kv_mount}/metadata/{PATH_CA_TRUST} (bootroot/ca), {kv_mount}/data/{PATH_RESPONDER_HMAC} (bootroot/responder/hmac), and {kv_mount}/data/{PATH_AGENT_EAB} (bootroot/agent/eab). Assert that a mint succeeds — so those reads demonstrably happened inside the verb under the internal credential — and that a direct read of each of the four paths with the registrar's own material fails, for the same reason every other red-team attempt fails: there is no token to carry. Build the paths from the constants and the scenario's kv_mount, not from retyped literals.
• The per-service KV the mint writes is likewise not directly reachable. A mint writes under {kv_mount}/data/bootroot/services/<registration_id>/ (SERVICE_KV_BASE, src/trust_bootstrap.rs:7, with the trust / http_responder_hmac / eab / secret_id suffixes at :27, :38, :45 and :32). Assert the registrar credential cannot read or write any of them directly, including the identity it just legitimately minted — receiving wrapped bootstrap material from a verb is not the same authority as holding the path.

Audit and capacity under a real stack

• The mandatory OpenBao file audit device still holds. Reuse assert_openbao_audit_log from scripts/impl/lib/audit-log.sh against the scenario's OpenBao container — do not write a second audit-device assertion.
• Refusal leaves a record and no OpenBao write. Drive a refused mint over the socket; assert a paired intent/outcome record exists in the verb-level audit store, and assert the OpenBao audit device shows no role, policy or KV write for that registration_id.
• Unwritable audit store refuses the mint with nothing created. Induce it by mounting the configured audit_store_dir on a small tmpfs sized in the scenario's compose overrides and filling it with dd before the call. Assert the mint is refused and that OpenBao holds no role, policy or KV path for that registration_id.
• The capacity alarm reaches the health response. With the same tmpfs-mounted store filled to within the configured threshold, the endpoint's health response carries registrar_health.audit_capacity with state = low_water before it ever reads exhausted. The scenario sets audit_store_reserve_bytes and audit_store_low_water_bytes to small scenario-local values; the thresholds themselves are the capacity issue's config keys and its unit tests' business — do not restate or re-assert the shipped defaults here.

Socket, ownership and pinning — the cases that need root

• Socket and directory ownership and mode — asserted differently on the two paths, deliberately. Read the numeric owner and octal mode of both (stat -c '%u %g %a' inside the container), never a "looks restrictive" heuristic, then assert:
  • the socket — owner uid 0, mode exactly 0700. An equality assertion, because the endpoint issue specifies that one mode and any other value is a deviation worth failing on.
  • the parent directory — owner uid 0, and mode with both the group-write and other-write bits clear (mode & 0o022 == 0). A bitmask assertion, not equality: the endpoint issue constrains the directory by what must not be writable rather than to one mode, so 0700 and 0755 are both conformant and pinning either would fail a compliant deployment. What is being proved is that nobody but root can replace the socket path — that is the occupation attack the directory exists to stop, and a group-writable directory defeats it just as a world-writable one does.
• The daemon inherits the listener and never binds or unlinks the path. Drive the activation mechanism the endpoint issue ships. Where the scenario's image cannot run the production unit manager, drive the same inherited-descriptor contract with systemd-socket-activate or an equivalent small parent that creates the listener and execs the daemon with LISTEN_FDS=1 and LISTEN_PID — but do not substitute a daemon that binds the path itself, which asserts the opposite of the property. The observable: the socket path's inode is byte-identical across a daemon restart, and the path still exists and is still root-owned while the daemon is down.
• The path cannot be occupied. With the daemon stopped, and again after a deliberately failed start (a daemon started against a broken configuration), an unprivileged process in the container can neither connect to the socket nor create or replace the path. This is the full assertion; the weaker "the parent directory is not writable" does not satisfy it.
• The callee is authenticated from peer credentials, not from a pre-connect stat. Construct the discriminating case deterministically rather than racing it: as root, create the socket directory 0700; have an unprivileged process bind a listener inside it; as root, chown 0:0 and chmod 0700 the resulting socket file. The path metadata now satisfies any stat-based check while SO_PEERCRED on the connected socket reports a non-zero uid. The caller must refuse.
• Endpoint identity pinning. The caller refuses a server identity it cannot match against the written fingerprint: point it at an endpoint presenting a leaf issued for a different name, and at a fingerprint file that does not match the presented leaf, and assert a pinning refusal in both directions rather than a successful connection.

Documentation

Document the scenario's threat-model boundary in the script header and the wrapper's module documentation — an exploited request handler, a compromised control plane and root on the bootroot host are deliberately not modelled — and tell a reader who opens only this file what the other two arms own and where each one lives:

• The sibling cargo arm owns every case that needs no Docker: exactly the categories the Out of scope list below assigns to it, the live-listener cases among them. It has no single file to point at, and that is deliberate rather than unsettled — its cases live as #[cfg(test)] modules inside the crate that owns the surface under test, beside the verb layer in the binary crate under src/commands/ and beside the endpoint in the library's private daemon module (src/lib.rs:24, src/daemon.rs), because an integration test under tests/ reaches neither. It uses tests/*.rs only for what the library's public surface exposes or what can be driven by spawning env!("CARGO_BIN_EXE_bootroot"), as tests/bootroot_service.rs:40 already does. Name that placement rule and those two module locations; do not invent a tests/ filename for this arm.
• The sibling endurance scenario owns the two cases whose cost is wall-clock — outliving a certificate to prove renewal, and asserting no AppRole is read across that renewal — and ships as scripts/impl/run-registrar-endurance.sh with the wrapper tests/docker_e2e_registrar_endurance.rs, at the extended tier, which does not gate pull requests. Name both paths and say that it sources this scenario's helper.

Those are the sibling issues' own deliverables, so they may not be in the tree yet when this work starts; that is expected and is not a reason to block or to leave the cross-reference out. Write the documentation against the names above, and if a sibling has already landed somewhere else, use the path that is actually in the tree and report the discrepancy on this issue. Do not coin a path that is neither in the tree nor named here.

Acceptance criteria

• Every bullet in the case lists above has at least one assertion in this scenario, and each test or assertion's name identifies the property it defends.
• The escalation-denied test asserts all four of its sub-points — the refused auth/cert/login, the absent credential material, and both attack shapes — against a live OpenBao, driving attack shape (a) from tests/e2e/registrar/privileged-policies.txt rather than from names retyped in the scenario script.
• The completeness guard passes and is load-bearing: every POLICY_BOOTROOT_* string literal declared in src/commands/init/constants.rs appears in tests/e2e/registrar/privileged-policies.txt and nothing else does. Adding a new POLICY_BOOTROOT_* constant without updating the file fails the guard — demonstrate that by adding one temporarily and observing the failure.
• The guard runs without Docker: it is not #[ignore]d, plain cargo test executes it, and a test run with Docker unavailable still fails when the file and the constants disagree. A guard reachable only through the Docker matrix does not satisfy this.
• The registrar's own material cannot read {kv_mount}/data/bootroot/ca, {kv_mount}/metadata/bootroot/ca, {kv_mount}/data/bootroot/responder/hmac, {kv_mount}/data/bootroot/agent/eab, or any {kv_mount}/data/bootroot/services/<registration_id>/* path, including one it has just legitimately caused to be written; the paths are built from the constants and the scenario's kv_mount.
• Minting and deregistering over the socket succeed, are idempotent, and every minted role reads back with token_policies == ["bootroot-service-<registration_id>"] and nothing else.
• The registrar's certificate is refused at every inbound surface other than the two verbs, and is not usable as a service identity.
• The scenario exists as scripts/impl/run-registrar-redteam.sh with the #[ignore]d wrapper tests/docker_e2e_registrar_redteam.rs, is invoked by scripts/preflight/ci/e2e-matrix.sh, is a test-docker-e2e-matrix matrix entry in .github/workflows/ci.yml, and runs green there on the pull request that adds it — within the job's existing timeout-minutes: 20, with its wall-clock reported.
• The shared bring-up helper lives under scripts/impl/lib/ and is written so the sibling endurance scenario can source it without copying.
• The socket is asserted by equality (uid 0, mode exactly 0700) and its parent directory by bitmask (uid 0, mode & 0o022 == 0), never by a "looks restrictive" heuristic.
• The socket path's inode is byte-identical across a daemon restart and the path survives while the daemon is down, proving the daemon neither binds nor unlinks it.
• An unprivileged process can neither connect to nor occupy the socket path with the daemon stopped and after a deliberately failed start.
• The peer-credential case is constructed deterministically as described and the caller refuses, proving authentication comes from the connected socket rather than a pre-connect stat.
• The scenario leaves no residue in a shared OpenBao instance and cannot collide with a developer's install: run-scoped instance names and free-port selection, as run-two-instance-isolation.sh does.
• The script header and the wrapper's module documentation state what the suite deliberately does not model — an exploited request handler, a compromised control plane, and root on the bootroot host — and point a reader at both sibling arms: the endurance scenario by path (scripts/impl/run-registrar-endurance.sh, tests/docker_e2e_registrar_endurance.rs) and the cargo arm by the case categories Out of scope assigns it — the live-listener cases included — plus its in-crate placement rule (#[cfg(test)] modules beside the verb layer under src/commands/ and beside the endpoint in src/daemon.rs), which by design has no single file path.
• cargo clippy is warning-free and cargo fmt --check passes.

Constraints

• Assert the guarantee, not the implementation: prefer observable behaviour — a refused login, a rejected connection, a record on disk — over inspecting internals.
• Do not weaken any assertion to make it pass. If a property cannot be asserted as written, that is a finding about the implementation, and the issue should report it rather than quietly narrowing the test.
• Do not add cases outside the stated threat model. The scenario is credential-level by design.
• Do not move a case out of this scenario into the extended tier to fit the matrix budget. If it does not fit timeout-minutes: 20, that is a finding to report on this issue: .github/workflows/e2e-extended.yml has no pull-request trigger, so relocating a security assertion there silently removes the gate while leaving the test file in the tree.
• Do not move a case here that needs no Docker. Cases that run in-process belong in the sibling cargo arm, where they gate every pull request cheaply.
• Assert socket-path occupation against the decided mechanism — a listener created by socket activation ordered ahead of the registrar and inherited by the daemon, which binds and unlinks nothing. The assertion is that the path cannot be taken across a restart or a failed start, not the weaker "the parent directory is not writable". Do not soften it.
• Treat the wire-contract issue's checked-in protocol reference under docs/ as the sole source of truth for any wire error identifier this scenario asserts; do not retype a spelling.
• Do not restate or re-assert the capacity thresholds' shipped defaults; the scenario sets small local values and asserts the alarm ordering.
• Keep shared setup in a sourced helper under scripts/impl/lib/ rather than copied between scripts.
• No new fault-injection mechanism, test hook or production introspection point.

Out of scope

• Everything that runs in-process without Docker: derivation from parts, the identity-shape and safe-set refusals, the synthetic collision and serialization races, the namespace-key-versus-SAN-label cases, the wire-visible audit and limiter error shapes, lapse-by-variant, and the three pre-derivation refusal requests. All belong to the sibling cargo arm.
• Everything a live listener can prove without root, which this arm deliberately does not absorb: the accept path itself, the mTLS handshake, request/response round-trips, and the wire and codec cases. An enabled endpoint under a non-zero effective uid warns at startup and serves rather than refusing, so that path stays reachable from an unprivileged cargo test against a listener bound inside a tempfile::tempdir(), and these cases need neither root nor a container. They belong to the sibling cargo arm too, and this is settled rather than pending — do not pull them here on the theory that a socket needs Docker. What genuinely needs root and a container, and therefore stays here, is the socket and parent-directory ownership and mode assertions, the socket-activation inode across a restart, path occupation, and the peer-credential discrimination constructed as root.
• Everything whose cost is wall-clock: outliving a certificate to prove renewal, and asserting no AppRole is read across that renewal. Both belong to the sibling extended-tier scenario, which sources this scenario's helper.
• Changing any production code to make a test pass — except where a test reveals a genuine defect, which should be fixed with the smallest correct change and called out explicitly.
• Adding test hooks, injection points or introspection to production code.
• Re-asserting sibling issues' unit-level behaviour: the audit store's startup ownership checks, the limiter's state and hysteresis tables, and the derivation library's own tests.
• Performance or load testing.
• Anything in another repository.

Test plan

The scenario is the deliverable; the criteria above enumerate it. In addition:

• Demonstrated green locally through scripts/preflight/ci/e2e-matrix.sh and in CI on the pull request, with its wall-clock reported against the job's timeout-minutes: 20.
• scripts/preflight/run-all.sh passes end to end, since it invokes the preflight script this scenario is added to.
• The completeness guard is demonstrated load-bearing by temporarily adding a POLICY_BOOTROOT_* constant and observing the failure — observed under plain cargo test with Docker unavailable, which is where it is meant to fire.
• Each assertion is independently runnable within the scenario and leaves no residue in a shared OpenBao instance — identities it mints are torn down, including on failure paths.
• A second run on the same machine, concurrent with a developer's real install, neither collides nor leaves residue.

Dependencies

Depends on the endpoint mTLS termination and on the endpoint's wire protocol — and, through them, the listener and the wire contract; on the bootroot-internal privileged credential, whose auth/cert trusted-certificate entry the core assertion probes and whose policy constant the completeness guard covers without naming; on the registrar client identity's issuance, since the scenario presents that leaf; on the reserved audit store's enforced reserve and the OpenBao audit device's bound, which the tmpfs and audit-device cases stand on; on the audit capacity probe for the health member the alarm case reads; on the audit record's verb integration for the refusal-leaves-a-record case; and on the verb rate limiter, whose presence must not perturb these assertions. Part of the registrar acceptance suite under the registrar-surface umbrella.

The sibling endurance scenario depends on this issue for the shared bring-up helper. Confirm each seam this scenario consumes is reachable before starting; where one is not, report it on this issue rather than working around it.

Pointers

• scripts/impl/run-two-instance-isolation.sh and tests/docker_e2e_two_instance_isolation.rs — the closest existing scenario, and the shape to copy: run-scoped naming, free-port selection, and the developer-install safety contract in its header comment
• scripts/impl/lib/audit-log.sh (assert_openbao_audit_log) — the existing OpenBao file-audit-device assertion to reuse, and the precedent for the shared helper's placement
• scripts/preflight/ci/e2e-matrix.sh and .github/workflows/ci.yml:382 (test-docker-e2e-matrix, timeout-minutes: 20, and the matrix.scenario list at :391 showing the label/script/artifact entry shape)
• scripts/preflight/run-all.sh:23–:27 — invokes the preflight scripts
• src/commands/init/constants.rs:133–:144 (the POLICY_BOOTROOT_* names the escalation test enumerates) and :178–:180 (PATH_RESPONDER_HMAC, PATH_AGENT_EAB, PATH_CA_TRUST)
• src/commands/init/steps/openbao_setup.rs:458 (POLICY_BOOTROOT_RUNTIME_SERVICE_ADD's body — the existing authority envelope)
• src/trust_bootstrap.rs:7 (SERVICE_KV_BASE) and :27/:32/:38/:45 — the per-service paths a mint writes and the registrar must not reach directly
• src/commands/service/approle.rs:15 (ensure_service_approle), :103 (service_role_name), :107 (service_policy_name), src/commands/service.rs:26 (SERVICE_ROLE_PREFIX)
• src/openbao.rs:537 (ensure_approle_auth), :765 (login_approle), :608 (write_policy), :642 (create_approle) — the OpenBao surfaces the red-team cases attack
• tests/e2e/docker_harness/scenarios/ — existing checked-in E2E data under tests/e2e/, the precedent for the attack list's placement
• src/lib.rs:24 (mod daemon, private) and tests/bootroot_service.rs:40 (env!("CARGO_BIN_EXE_bootroot")) — where the sibling cargo arm's cases live, for the documentation cross-reference
• docs/rfcs/0001-registrar-role-and-non-self-propagation.md §6 and §3 (the threat-model boundary to document)