Skip to content

Registrar acceptance suite: the three test arms #784

Description

@sehkone

Registrar acceptance suite: the three test arms

Summary

The registrar surface is assembled from many parts — the namespace key, the derivation library, the two verbs, the socket endpoint, the wire codec, mTLS, two certificates, the audit record and its limiter. Each ships its own unit tests. What no part can assert is the assembled behaviour: that they compose into the identity rules, the refusal taxonomy and the wire-visible outcomes the design promises, and above all that the guarantee holds as a system property. This is a grouping issue and carries no work of its own.

The three children split by what each case needs in order to be true, not by subject matter. That is the whole design of this group, and mixing the axes is how a suite rots:

  • The cargo arm needs nothing — no Docker, no network, no root. Ordinary cargo test files running the verb layer in-process with an injected OpenBao client and, where a case needs the wire, the endpoint bound to a unix socket inside a temporary directory. Because it is cheap it gates every pull request, which is exactly why every case that can live here must.
  • The per-PR Docker red-team scenario carries the cases where a real OpenBao enforcing real ACLs, a real process boundary, or root-owned files is what makes the assertion true — plus everything else that costs one stack bring-up and then runs in seconds. It also gates every pull request, because a security guarantee that only re-proves itself on a manual dispatch is not a regression gate.
  • The extended-tier endurance scenario carries only the cases whose cost is wall-clock, because the only way to prove a certificate was renewed is to outlive the one it replaced.

The rule that keeps the split honest, and that every child restates: a case belongs in a Docker arm only when a live OpenBao, a process boundary or root-owned files is what makes it true. Moving a case into a heavier arm by weakening it is the failure this split exists to prevent; moving one out of the per-PR arms is worse, and is governed below.

That rule is about cases, and a scenario's own supporting machinery is not a case. A case asserts something about the registrar; supporting machinery asserts something about the suite — that a scenario's data file is complete, that a helper is sourced rather than forked. Machinery ships with the scenario that owns it, even when it happens to run in a cheaper tier, because splitting a file from the guard that keeps it honest and from the script that consumes it puts one mechanism in two issues and leaves each reviewer looking at a third of it. The concrete instance is the red-team scenario's privileged-policy attack list: the file, the scenario's use of it, and the guard asserting it still matches the constants module are one unit owned by that scenario — and the guard runs as an ordinary non-#[ignore]d test needing no Docker, precisely so it fails fast. That placement is not an exception to the tiering rule; it is outside its subject. The cargo arm neither owns nor duplicates it.

Children

  • Add the cargo acceptance arm for the registrar surface — Verification
  • Add the registrar-redteam Docker scenario and gate every pull request on it — Verification
  • Add the registrar-endurance scenario for the renewal properties — Verification

Shared background

The tiering has a sharp consequence that governs placement. .github/workflows/e2e-extended.yml has no pull-request trigger — only workflow_dispatch and a nightly schedule. A green gh pr checks therefore proves nothing about the endurance scenario, and a linked manual dispatch is the only evidence that counts. Two rules follow and are binding on all three children: nothing load-bearing for the escalation guarantee may live at the extended tier, and no case may be relocated there to fit the per-PR job's time budget. If the per-PR scenario does not fit timeout-minutes: 20, that is a finding to report — relocating a security assertion silently removes the gate while leaving the test file in the tree, which reads like coverage and is not.

The threat model is credential-level by design, and the suite must not be written as if it were broader. The core case models a credential leaked or copied off the host — a disk image, a filesystem backup restored elsewhere. It deliberately does not model an exploited request handler (that handler runs as root on this host, so exploiting it is root here), a compromised control plane (which never holds this credential and is bounded by the verb's input contract instead), or root on the bootroot host (which can read the daemon's internal credential directly, and is out of scope). No child may add cases outside that model, and none may describe its cases as covering more than it does.

The single fact that decides how the red-team cases are written: the registrar holds no OpenBao credential of any kind. Its only credential is the mTLS client key and certificate for the daemon's unix socket. So "escalation denied" is not "the registrar's token was rejected by an ACL" — it is "there is no token, and the registrar's certificate cannot obtain one." A suite that asserts a refused token is asserting something the design does not claim.

Where this repository's half of a property ends, the suite stops — and says so rather than leaving a criterion looking unasserted. Two of the design's acceptance criteria describe paths that leave this repository: the health signals an operator ultimately reads travel from the daemon's endpoint through the co-located registrar's AgentInfo tail to the control plane's rendering, and a package whose registration template deviates from the safe-set is rejected at another repository's store receiver. bootroot's half of the first ends at the endpoint's own health container, which the arms do assert; the relay and the rendering are not this repository's code and no arm may claim to cover them. The second is not bootroot's at all. A reader mapping criteria to cases should find those boundaries stated, not find a case missing.

Externally owned spellings are asserted against the checked-in reference, never retyped. The wire-contract issue's protocol reference under docs/, and the golden serialization fixtures pinned to it, are the sole source of truth for every wire error identifier any arm asserts. No child may coin, rename or extend an identifier or add a reason to unblock a test — a guessed spelling is a cross-repo break that surfaces only at deploy time. If the reference disagrees with a spelling an issue expected, the reference wins and the disagreement is a finding to report.

Assert the guarantee, not the implementation, and never weaken an assertion to make it pass. Prefer observable behaviour — a record on disk, a refused login, a rejected connection, a state store's contents — over inspecting internals. If a property cannot be asserted as written, that is a finding about the implementation, and the child should report it rather than quietly narrowing the test. No child may add test hooks, injection points or production introspection to make its own job easier, and none may change production code to make a case pass except where a case reveals a genuine defect, which is fixed with the smallest correct change and called out explicitly.

That rule bans instrumentation compiled into bootroot; it does not ban a harness watching the process from outside. The distinction matters because one case genuinely needs observation: the endurance scenario's no-AppRole assertion has to establish that a renewal read no role_id and no secret_id anywhere, and behaviour alone cannot show a negative. Observing which files the process opens — from the harness, around a process it already runs in a container it already controls — is exactly the "prefer observable behaviour" this rule asks for, and it adds nothing to production code. The single named exception in this suite is therefore harness-level, and it is not a licence to add a production seam: if the negative cannot be established from outside the process, that is a finding to report on that issue, not a hook to compile in. No other child needs observation of this kind, and none may add one.

Where a case can live is constrained by the crate layout, not only by its tier. The surfaces this suite exercises are pub(crate) by design: src/lib.rs does not export commands at all — it lives only in the bootroot binary — and it declares daemon as a private module, so an integration test under tests/ reaches neither the verb layer nor the endpoint. The cargo arm therefore places cases inside the crate that owns them and uses tests/*.rs only for what the library exposes or what can be driven by spawning a binary, as tests/bootroot_service.rs already does through CARGO_BIN_EXE_bootroot. No arm may widen a production item's visibility to relocate a test — every issue in this effort specifies pub(crate) unless the compiler forces wider, and relaxing it for placement is a production change made for a test's convenience. One consequence is worth knowing before a narrowed invocation hides a green run: CI's unit job runs bare cargo test, which covers the library, the binaries and every tests/ target, while cargo test --lib runs none of the binary crate's tests.

Shared machinery rather than copied machinery — and the two Docker arms share different machinery from the cargo arm. They are not one convention with three instances, because the cargo arm has no scenario, no script and no container to conventionalise.

  • The two Docker scenarios follow the repository's existing E2E harness conventions rather than inventing a parallel one: scripts/impl/run-<scenario>.sh plus a tests/docker_e2e_<scenario>.rs wrapper plus a preflight step, in the shape of the two-instance-isolation scenario. Running at a different tier is not a reason to introduce a new shape. The red-team child places the registrar-surface bring-up both need in a sourced helper under scripts/impl/lib/; the endurance child sources it and extends it where it falls short rather than forking it. Both derive run-scoped instance names and pick free host ports so they cannot collide with a developer's real install.
  • The cargo arm shares none of that and must not acquire it. It is plain cargo test — no scenario script, no docker_e2e_* wrapper, no preflight step, no container to name or port to reserve. What it shares instead is the fixture builder the config-and-derivation issue exports, which it uses to build the rendered registrar config rather than hand-writing another approximation of a file this repository never renders in production, and the placement rules above that decide which crate each case lives in. Adding a script or a wrapper to it to look like its siblings would give it a Docker-shaped harness for a tier that has no Docker in it.
  • One artifact does cross the tiers, and it is machinery rather than a case: the red-team scenario's privileged-policy attack list and the guard that keeps it complete, which runs as an ordinary non-#[ignore]d test in the same tier the cargo arm runs in while belonging to the scenario that consumes it, exactly as the case/machinery distinction above sets out.

Ordering. The cargo and red-team arms are independent of each other and each depends on the parts it exercises. The endurance scenario depends on the red-team scenario for the shared helper, and nothing depends on the endurance scenario — it is the tier of last resort for two properties, and its value is entirely in not letting them go unasserted because they are slow.

Execution order

Issues in the same wave have no unmet dependencies among these children and can run in parallel.

External dependencies:

graph TD
  issue781["#781 Add the cargo acceptance arm for the registrar surface [phase: Verification]"]
  issue782["#782 Add the registrar-redteam Docker scenario and gate every pull request on it [phase: Verification]"]
  issue783["#783 Add the registrar-endurance scenario for the renewal properties [phase: Verification]"]
  issue758["#758 Implement the restricted registrar mint and deregister verbs [phase: Verb building blocks]"]
  issue758 -.-> issue781
  issue757["#757 Read the bootler-rendered registrar config and derive registration identities [phase: Verb building blocks]"]
  issue757 -.-> issue781
  issue762["#762 Implement the registrar endpoint's versioned wire protocol and its codec [phase: Surface]"]
  issue762 -.-> issue781
  issue765["#765 Add the in-repo client for the registrar endpoint [phase: Surface]"]
  issue765 -.-> issue781
  issue759["#759 Settle and transcribe the registrar endpoint's caller-facing wire contract [phase: Verb building blocks]"]
  issue759 -.-> issue781
  issue769["#769 Report the registrar certificates' lifetime and turn a lapse into a typed non-retryable error [phase: Surface]"]
  issue769 -.-> issue781
  issue777["#777 Write intent and outcome records around both registrar verb arms [phase: Detection]"]
  issue777 -.-> issue781
  issue779["#779 Scan the audit store for anomalies and report them on `bootroot status` [phase: Detection]"]
  issue779 -.-> issue781
  issue787["#787 Record limited registrar invocations and report the throttle [phase: Detection]"]
  issue787 -.-> issue781
  issue774["#774 Probe the reserved audit store's capacity and report it on the endpoint [phase: Detection]"]
  issue774 -.-> issue781
  issue764["#764 Terminate mTLS on the registrar endpoint and scope it to the registrar identity [phase: Surface]"]
  issue764 -.-> issue782
  issue762 -.-> issue782
  issue765 -.-> issue782
  issue766["#766 Provision the bootroot-internal privileged credential and run the verbs under it [phase: Surface]"]
  issue766 -.-> issue782
  issue767["#767 Self-issue the registrar surface's two certificates from the daemon [phase: Surface]"]
  issue767 -.-> issue782
  issue773["#773 Make the audit store's reserve a kernel-enforced ceiling [phase: Detection]"]
  issue773 -.-> issue782
  issue789["#789 Bound the OpenBao audit device's growth [phase: Detection]"]
  issue789 -.-> issue782
  issue774 -.-> issue782
  issue787 -.-> issue782
  issue777 -.-> issue782
  issue768["#768 Renew both registrar leaves on the daemon's existing loop and honour the reload contract [phase: Surface]"]
  issue768 -.-> issue783
  issue782 --> issue783
Loading

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions