Skip to content

Bound the OpenBao audit device's growth #789

Description

@sehkone

Bound the OpenBao audit device's growth

Context

bootroot requires a file-based OpenBao audit device: openbao/openbao.hcl declares audit { type = "file" ... file_path = "/openbao/audit/audit.log" }, verify_audit_file queries sys/audit at init and bootroot init aborts when no file-type device is present (src/openbao.rs:560, propagated at src/commands/init/steps/openbao_setup.rs:162). That device is the substrate the registrar's whole detection argument rests on — every OpenBao write a mint performs is recorded there, under the bootroot-internal credential, where the registrar cannot forge or erase it.

It is also the one audit artifact on the bootroot host with no bound on its size. OpenBao's file audit device does not rotate itself, and bootroot has never rotated it. The registrar's own verb records are self-bounding — their store caps them at audit_max_file_bytes × (audit_max_retained_files + 1), about 136 MiB on the shipped defaults — so on a store with a 2 GiB reserve the records cannot exhaust it. The device can.

That is not merely a lost log. OpenBao fails requests it cannot audit, so a full audit filesystem stops OpenBao serving — and OpenBao is the renewal path for every certificate in the deployment, on the one host running a shamir-sealed instance that must not be restarted. A reserved store bounds the blast radius to the audit artifacts; it does not stop the mandatory device from filling its own reserve. This issue closes that: it is what makes "exhausting the audit artifacts cannot stop OpenBao serving" true rather than aspirational, and without it the reserved store is a smaller container for the same unbounded writer.

The device's backing directory is a host bind mount under the reserved store, established by the store-layout issue, so rotation can be driven from the host side without reaching into the container's filesystem semantics.

Scope

The bound

  • Bound the OpenBao audit device to a configured total footprint through size- and age-bounded rotation driven from the bootroot host, with a bounded retained set, so the device's contribution to the reserved store has a ceiling rather than a trend.
  • The size ceiling is hard; the retention age is soft, exactly as it is for the verb-record store: when trimming to the retained-generation bound would drop a generation younger than the minimum retention age, drop it anyway. An unbounded store on the bootroot host is the outage the ceiling exists to prevent, and a scheme whose bound yields to its own retention target has no bound. On a busy deployment the ceiling will routinely win and the retention target will go unmet; that is the intended behaviour, not a fault, and it raises no alarm here.
  • The check is periodic, so the bound is enforced at passes rather than at writes, and the issue states that rather than overclaiming. bootroot is not the process appending to this file — OpenBao is — so a pass runs on a timer and the active log may overshoot openbao_audit_max_file_bytes by up to one interval's worth of OpenBao writes before anything rotates it. Run the check on a dedicated task spawned from run_daemon (src/daemon.rs::run_daemon:68) at a fixed interval declared as a file-top constant — 60 seconds — with the interval injectable for tests. Do not hang it off the per-profile check_interval loop: that interval is per-profile, jittered and operator-configurable up to hours, while the device is a single global object, so a deployment with no profiles would never rotate and a deployment with several would rotate several times over.
  • A pass is: rotate if the active log is at or over openbao_audit_max_file_bytes, then — only if the rotation is confirmed — trim. Trimming never runs on the back of a rotation that failed or was recovered, because the retained set is unchanged in that case and trimming it would drop history for nothing.
  • The invariant this issue guarantees, stated precisely. Let S = openbao_audit_max_file_bytes and N = openbao_audit_max_retained_files. The retained set has a hard byte budget of S × N (448 MiB on the shipped defaults), and trimming enforces it — see "The retained set" below. At the end of any pass the active log has just been emptied, so the device's footprint is at or under the nominal ceiling S × (N + 1) = 512 MiB shipped. Between passes the active log grows back to at most S plus one interval's writes, so the instantaneous footprint is the nominal ceiling plus that overshoot. Both numbers are stated in the documentation; do not claim the nominal ceiling as an instantaneous bound.
  • The overshoot is sized against the reserve rather than waved at. On shipped defaults the two bounded writers claim 648 MiB inside a 2 GiB store whose low-water alarm fires at 1536 MiB of usage, leaving about 888 MiB of margin — so a 60-second overshoot would have to reach 888 MiB, roughly 15 MiB/s of sustained audit output, before it could even reach the alarm. If an implementer changes the interval or the bounds, that margin is the number to re-check.

Configuration

Add three keys to the [registrar] table in the daemon's own configuration (src/config.rs, i.e. agent.toml) — the same table the record store's audit_record_dir and rotation bounds land in, and the same table the reserved-store keys land in. Sibling issues add their own keys to it and none of them depends on this one, so whichever lands second must be purely additive: if [registrar] already exists, add these keys to it; if it does not, define the table here. Do not reshape, rename or re-validate what is already there.

  • openbao_audit_max_file_bytesu64, the size bound on the active log in bytes. Default 67108864 (64 MiB). Validation rejects any value below the 1 MiB floor (1048576).
  • openbao_audit_max_retained_filesusize, the maximum number of retained rotated generations, not counting the active log. Default 7. Validation rejects 0.
  • openbao_audit_min_retain_daysu32, the soft minimum retention age in days. Default 90. Validation rejects 0.

Each rejection is a configuration error whose message names the offending key, in the same shape as the existing [registrar] validation.

The names are deliberately prefixed and the defaults are deliberately not the record store's. audit_max_file_bytes and audit_max_retained_files are already taken by the verb-record store in this same table and must not be reused, reinterpreted or shared: the two writers have unrelated volumes — the record store writes one bounded line per registrar invocation, the device writes an entry per OpenBao request across the whole deployment — so one pair of bounds cannot serve both.

The defaults are derived from the reserve, and the derivation is what an implementer should reproduce rather than the numbers. The shipped ceiling here is 64 MiB × (7 + 1) = 512 MiB. Against the reserved store's shipped audit_store_reserve_bytes of 2 GiB, the two bounded writers together claim at most 512 MiB + 136 MiB = 648 MiB, leaving about 1.4 GiB of headroom — comfortably clear of the shipped audit_store_low_water_bytes of 512 MiB. That margin is the point: with both writers bounded and the arithmetic done, the sibling capacity issue's low-water alarm can only fire from something other than these two, which is what makes the alarm worth reading. Record this arithmetic in the documentation so a later change to any of the five numbers is visibly a change to the same budget.

The floor is 1 MiB rather than the record store's 64 KiB, and for a different reason. The record store's floor is derived from its own maximum record size, which it controls. bootroot does not control the size of an OpenBao audit entry — request and response bodies are HMAC'd, but paths, metadata and error strings are copied through — so there is no maximum-entry constant to multiply. The floor instead exists so that the bound is large enough that a rotation pass is a rare event rather than something the 60-second check does on every tick, and so that the overshoot between passes is small relative to the bound. Declare the floor as a file-top constant and document it as such.

The retained set

  • The active log stays exactly where it is: audit.log in the device's directory, /openbao/audit/audit.log as the container sees it. openbao/openbao.hcl's file_path is not edited, so this name is fixed.
  • Rotated generations live in that same directory, beside the active log<audit_store_dir>/openbao/ on the host, /openbao/audit/ as the container sees it. That is settled here, not left open. The alternative considered and rejected was a root-owned sibling directory the container cannot reach, which would shield the retained set from anything running as the container's uid; it is rejected for two reasons. First, it buys nothing the RFC's argument needs: the actor that must not be able to forge or erase these entries is the registrar, which holds no OpenBao credential, has no access to this directory and does not run in that container at all — while an attacker who is inside the OpenBao container already holds the sealed instance and every key the deployment renews, next to which the retained audit set is not the asset in question. Second, a second directory would have to be created and permissioned by the reserved store's install path, which is a sibling issue's; requiring that here would make this leaf's on-disk layout depend on a change it does not own. Same-directory also keeps a rotation to one rename within one directory — same filesystem, no cross-directory failure mode — and keeps every generation reachable both on the host and through docker exec on /openbao/audit, where the existing documentation already points operators. This issue therefore needs no directory the store-layout issue does not already create.
  • A rotated generation is named audit-<YYYYMMDDTHHMMSSZ>.log — UTC, second precision, with a -1/-2 numeric suffix on collision — stamped at the moment of rotation. This mirrors the verb-record store's registrar-audit-<YYYYMMDDTHHMMSSZ>.jsonl naming deliberately: it sorts lexicographically in time order, so "newest retained generation" and "oldest retained generation" are a sort of the device directory's listing and not a stat of every file, and an operator reading the two artifacts side by side reads one convention. Ordering is defined by the name, not by mtime. The glob that distinguishes a generation from the active log is audit-*.log; audit.log never matches it, so listing, sorting and trimming can never touch the active file.
  • Rotated generations are not compressed, renamed on subsequent rotations, or shifted .1.2 logrotate-style. Timestamped names are written once and never rewritten, so a rotation is O(1) renames rather than O(N), and a crash mid-rotation cannot leave two generations claiming one name.
  • Trimming enforces two bounds, and the byte bound is the one that actually holds the ceiling. Drop generations oldest-first by name order until both hold: at most openbao_audit_max_retained_files generations remain, and their total size is at or under openbao_audit_max_file_bytes × openbao_audit_max_retained_files.
  • Count-based trimming alone would not hold the byte ceiling, which is why the byte bound is stated separately. A generation is exactly as large as the active log was at the moment it was rotated, and because the check is periodic that can be openbao_audit_max_file_bytes plus one interval's writes. Seven such oversized generations exceed S × N while the count bound is satisfied, so a count-only trim would let the device drift past the ceiling the whole issue exists to establish. The byte trim closes exactly that gap.
  • The pathological case is specified rather than left to the implementer. If, after dropping every older generation, the single newest generation by itself still exceeds openbao_audit_max_file_bytes × openbao_audit_max_retained_files, drop it too and log at warn with its size, the budget and the interval. One interval produced more audit output than the entire retained budget, which is a misconfigured bound or a runaway writer, and in that state the ceiling is the property worth keeping — that is the same "the size ceiling is hard" rule that already resolves the retention-age conflict, applied to its extreme. Never truncate, split or rewrite a rotated generation to make it fit: a partially rewritten generation is a corrupted JSON Lines file, and "written once, never rewritten" is what makes concatenation safe.
  • Reading a rotated generation is reading a file: the format is unchanged JSON Lines exactly as OpenBao wrote it, and concatenating the retained set in name order followed by the active log reconstructs the retained history in write order. Document that, including the one-line cat <audit_store_dir>/openbao/audit-*.log <audit_store_dir>/openbao/audit.log an operator would use.

The rotation mechanism, and what "verified" means

The device must be reopened or truncated without restarting, resealing or otherwise disturbing OpenBao. Not restarting the shamir-sealed OpenBao is a standing operational rule for this deployment; a rotation scheme that requires a restart-and-unseal cycle trades a contained problem for an unsealing risk and is not acceptable here. Two mechanisms satisfy the rule, and the implementation picks whichever the pinned OpenBao image actually supports, verifying it rather than assuming:

  • Reopen on signal. Rename the active log aside and signal the container's main process to reopen its audit file. This is the conventional form for a Vault-lineage file audit device and preserves every byte.
  • Truncate in place. Copy the active log aside and truncate the original to zero length. The device is opened for appending, so writes continue to land correctly at the new end of file and no descriptor is invalidated; the cost is a small window in which records written between the copy and the truncate are lost, which the implementation minimises and the documentation states.

Establish which mechanism the pinned image supports as the first step of the work. This issue therefore contains discovery work, and the bar for "verified" is spelled out here so two implementers cannot satisfy it differently:

  • The signal is SIGHUP, delivered to the container's main process with docker kill --signal=HUP <instance>-openbao, where the container is named ${BOOTROOT_INSTANCE:-bootroot}-openbao (docker-compose.yml:4). The compose entrypoint is sh -c "chown … && exec docker-entrypoint.sh server -config=…", so PID 1 should end up being the bao process — but a wrapper anywhere in that chain that does not exec swallows the signal, and confirming it does not is part of the probe rather than something to reason about from the compose file.
  • The evidence that a reopen worked is positive and file-level, not a zero exit status: after renaming audit.log aside and signalling, an OpenBao request driven immediately afterwards produces a new audit.log at the same path carrying that request's entry, while the renamed file receives nothing further.
  • "No container restart" is measured, not observed. Capture docker inspect -f '{{.State.StartedAt}} {{.RestartCount}} {{.State.Pid}}' for the OpenBao container before and after, and assert all three are identical; and assert sys/seal-status reports sealed=false and initialized=true both before and after.
  • The probe lives in scripts/impl/lib/audit-log.sh, beside the existing assert_openbao_audit_log, and is invoked from the Docker E2E lifecycle so it re-runs on every CI pass against whatever OPENBAO_IMAGE is in effect. A one-off finding pasted into a document goes stale at the next image bump and unbounds the device silently; a check wired into the lifecycle turns that bump into a CI failure. This is the difference between "verified once" and "verified", and the wiring is the deliverable.
  • The finding is written down with the tag it was established againstopenbao/openbao:2.5.5 today (docker-compose.yml:3, overridable through OPENBAO_IMAGE at docker-compose.deploy.yml:20) — on the en/ko operations pages, naming the mechanism chosen, the evidence above, and the fallback if the signal form did not hold.
  • If neither mechanism works against the pinned image, that is a finding to report on this issue with that same evidence — do not ship a rotation that restarts OpenBao, and do not silently leave the device unbounded while claiming a bound.

Conditionality, safety and the checks that must keep passing

  • Rotation is conditional in the same way the reserved store is, and the predicate is exactly [registrar_endpoint] enabled = true in the daemon's agent.toml — the Settings::registrar_endpoint.enabled boolean the endpoint-listener issue adds in src/config.rs, default false, with an absent [registrar_endpoint] table parsing as disabled rather than as a configuration error. Read that one field and nothing else: not the presence of the socket, not whether the unit pair is installed, not whether the store directory exists on disk. The device's backing directory when that predicate holds is <audit_store_dir>/openbao, the host bind mount the store-layout issue mounts at /openbao/audit through a rendered Compose override; audit_store_dir defaults to /var/lib/bootroot/audit-store. Where the predicate does not hold, /openbao/audit is still backed by the openbao-audit named volume, there is nothing on the host to rotate, and this issue changes nothing: no rotation task is spawned, the three new keys have no effect, and the deployment keeps today's behaviour. State that residual in the documentation rather than hiding it.
  • Rotation must never make the device unwritable, even transiently, in a way that fails an OpenBao request. Order the steps so the device always has a writable target: rename-or-copy first, then reopen or truncate, and never unlink the active path out from under a running OpenBao.
  • Ownership and mode are load-bearing and must survive rotation. The device's directory is chowned to the OpenBao container's uid:gid and chmod 700 (docker-compose.yml:17), and the daemon doing the rotating runs as root on the host, so it can rename inside that directory freely — which is exactly why it can also break it. Under the signal form, let OpenBao create the new active log itself; do not pre-create audit.log from the daemon, because a root-owned 0600 file in a directory the container cannot chown is a device that cannot write, i.e. an OpenBao that cannot serve. If the implementation must create or replace the active path for any reason, it chowns it to the container's uid:gid first, following the pattern bootroot already uses for container-written paths (issue_openbao_tls_cert's output-directory chown). Under the truncate form the original file's owner and mode are preserved by construction, and a test pins that.
  • A failed rotation is logged and retried on the next tick. It does not fail an OpenBao request, does not stop the daemon, and does not take the registrar endpoint down. Rotation is an operational duty, not a fail-closed control: the fail-closed artifact is the verb-record store, which is a sibling's and is unaffected by a failure here.
  • Partial failure has a defined recovery, and restoring the active path is part of the failing pass rather than of the next one. Both mechanisms have an intermediate state in which half the work is done, and "log it and retry next tick" is not sufficient on its own — retrying from an unrecovered intermediate state either leaves the active path missing for a full interval or duplicates a generation. The recovery runs before the failure is reported, and it is what the fault-injection test pins:
    • Signal form — the dangerous state is a rename that succeeded and a reopen that did not. OpenBao still holds a descriptor on the renamed inode and keeps auditing into it perfectly well, but nothing exists at the expected active path, so an operator — and assert_openbao_audit_log's docker exec … test -s /openbao/audit/audit.log — sees an absent audit log on a healthy deployment. Recovery is: look at the active path, then rename the generation back. If audit.log exists, OpenBao did reopen and the pass in fact succeeded whatever the evidence check reported — leave it, and treat the pass as successful. If it does not exist, rename the generation back to audit.log, which restores the exact pre-rotation state: the descriptor OpenBao holds does not care about the name, so no write is lost and there is no instant in which the device has no writable target. The rename-back must not overwrite an existing audit.log — on Linux renameat2 with RENAME_NOREPLACE gives that atomically, and a plain check-then-rename would race OpenBao creating the new file and clobber it.
    • Truncate form — order is copy, fsync the copy, then truncate, and never truncate before the copy is complete and durable. The dangerous state is a copy that succeeded and a truncate that did not: the active path is intact and OpenBao never stopped writing to it, but a generation now duplicates bytes that are still in the active log, so the next successful pass would emit two generations overlapping on the same records. Recovery is: unlink the copy and retry at the next tick.
    • A recovery that itself fails is logged at error naming the file it left behind, so an operator gets one message identifying exactly which path to move back by hand. Trimming does not run after a failed or recovered pass.
  • verify_audit_file's contract must keep holding across a rotation. That check queries sys/audit for a file-type device rather than inspecting the file, so rotation must not disable, re-register or re-path the device — the device's configuration is untouched and only its backing file is rotated.
  • assert_openbao_audit_log reads the active log only (docker exec … test -s /openbao/audit/audit.log, then greps that one file for an AppRole login response and a secret/data/ read — scripts/impl/lib/audit-log.sh:21). Rotation moves prior entries into a retained generation, so the helper passes on a rotated deployment exactly when the entries it looks for are in the current active log. The E2E arm must therefore drive a fresh AppRole login and KV read after the rotation and before asserting, rather than assuming the pre-rotation entries are still where the helper looks. Do not relax the helper to search rotated generations: it is an existing assertion with its own contract, and widening it here would weaken an unrelated test.

Documentation

Document, in both docs/en/ and docs/ko/ on the existing pages with no mkdocs.yml nav change — the bound and the three configuration keys with their defaults, units and floor on docs/en/configuration.md / docs/ko/configuration.md; the budget arithmetic against the reserve, both the end-of-pass ceiling and the instantaneous ceiling-plus-overshoot figure, the mechanism chosen and its evidence, the retained-set naming, where the generations live and how to read one, the residual for deployments without the registrar endpoint, and — if the truncate form is used — the small loss window it implies on docs/en/operations.md / docs/ko/operations.md. The shipped documentation is a mirrored en/ko pair, so a change to one without the other is incomplete.

Acceptance criteria

  • The OpenBao audit device has a bounded total footprint: a test drives the active log past openbao_audit_max_file_bytes, asserts a rotation occurred, and asserts that at the end of the pass the total of the active log plus its retained generations is at or under openbao_audit_max_file_bytes × (openbao_audit_max_retained_files + 1).
  • Trimming is by bytes as well as by count: a test builds a retained set of oversized generations — each larger than openbao_audit_max_file_bytes, as a periodic check legitimately produces — whose count is already within openbao_audit_max_retained_files, and asserts the pass still drops oldest-first until the retained total is at or under openbao_audit_max_file_bytes × openbao_audit_max_retained_files.
  • The pathological case holds the ceiling: a test rotates a single generation larger than the whole retained byte budget and asserts it is dropped, the drop is logged at warn with its size and the budget, and no rotated generation is ever truncated or rewritten in place.
  • Rotation happens without restarting, resealing or unsealing OpenBao: an E2E test rotates the device on a running deployment and asserts StartedAt, RestartCount and Pid are unchanged for the OpenBao container and that sys/seal-status reports sealed=false and initialized=true throughout.
  • The device stays writable across a rotation: an OpenBao request driven immediately after a rotation succeeds and its entry appears in the new active log, while the rotated generation receives nothing further.
  • The new active log is writable by the container: after a rotation it is owned by the OpenBao container's uid:gid, and no root-owned audit.log is left in the device directory.
  • verify_audit_file still succeeds after a rotation, and the device's registration in sys/audit is unchanged — a test asserts the device was not disabled, re-registered or re-pathed.
  • The existing assert-openbao-audit-log helper still passes on a deployment where rotation has occurred, with the E2E arm driving a fresh AppRole login and KV read after the rotation; the helper itself is unmodified.
  • The size ceiling wins over the retention age: a test drives the conflict and asserts a generation younger than openbao_audit_min_retain_days is dropped and the total stays inside the bound.
  • openbao_audit_max_file_bytes, openbao_audit_max_retained_files and openbao_audit_min_retain_days load with the documented defaults 67108864, 7 and 90; validation rejects an openbao_audit_max_file_bytes below the 1 MiB floor, a zero openbao_audit_max_retained_files and a zero openbao_audit_min_retain_days, each with an error naming the key; and the [registrar] additions leave any pre-existing key in that table untouched.
  • Rotated generations are named audit-<YYYYMMDDTHHMMSSZ>.log with a numeric suffix on collision, sort lexicographically in time order, are never renamed after they are written, and trimming drops the oldest by name order; the active log keeps the name audit.log and is never matched by the audit-*.log glob.
  • Rotated generations sit in the device's own directory beside the active log, and the implementation creates no directory the store-layout issue does not already create — a test asserts a rotation against a freshly provisioned <audit_store_dir>/openbao needs nothing else on disk.
  • Rotation is conditional on [registrar_endpoint] enabled = true: a test asserts that with it false or its table absent, no rotation task runs, the three new keys have no effect, and the deployment's audit log is unchanged and still backed by the openbao-audit named volume.
  • A failed rotation is logged and retried on the next tick rather than failing an OpenBao request or stopping the daemon; a test injects a rotation failure and asserts OpenBao keeps serving and the daemon's other duties keep running.
  • Partial failure recovers to the pre-rotation state before the failure is reported: with the reopen forced to fail after a successful rename, a test asserts audit.log is back at its expected path with its original contents, the generation is gone, the retained set was not trimmed, and the failure is logged and retried on the next tick; with the reopen forced to "fail" while a new audit.log nonetheless exists, the pass is treated as successful and the rename-back does not clobber it.
  • Under the truncate form the ordering and its recovery hold: a test asserts the copy is complete and fsynced before any truncate, and that a truncate forced to fail leaves the active log intact and unlinks the copy so no generation duplicates records still in the active log.
  • The chosen reopen mechanism is verified against the pinned OpenBao image by a probe wired into the Docker E2E lifecycle — so an OPENBAO_IMAGE bump that breaks it fails CI — and the finding, naming the image tag and the evidence, is recorded in the documentation.
  • The bound, the three keys with their defaults and the budget arithmetic against the reserve, the mechanism, the retained-set layout and how to read a generation, the residual for endpoint-less deployments and any loss window are documented in both docs/en/ and docs/ko/, with no mkdocs.yml nav change.
  • cargo clippy is warning-free and cargo fmt --check passes.

Constraints

  • Do not restart, reseal or unseal OpenBao as part of rotation, and do not require an operator to. Not restarting this instance is a standing rule for the deployment.
  • Do not edit openbao/openbao.hcl's audit stanza, do not change the device's file_path, and do not disable and re-enable the device to rotate it. Only the backing file is rotated; the device's registration is untouched.
  • Do not weaken or bypass the mandatory-audit-device check at init, and do not modify assert_openbao_audit_log to make it pass on a rotated deployment.
  • Do not unlink the active log out from under a running OpenBao, do not pre-create the active path as a root-owned file, and do not order the steps so that the device has no writable target at any instant.
  • Do not leave a failing pass in its intermediate state for the next tick to find: restore the active path (signal form) or unlink the copy (truncate form) before reporting the failure, and never let a rename-back overwrite an audit.log that OpenBao has already recreated.
  • Do not truncate, split, compress or otherwise rewrite a rotated generation, including to make it fit the byte budget — drop it whole instead.
  • Do not enforce the retained set by count alone; the byte budget is what holds the ceiling when a pass rotates an oversized active log.
  • Do not create or require any directory beyond the <audit_store_dir>/openbao the store-layout issue already provisions, and do not place rotated generations outside it.
  • Do not change behaviour for a deployment where [registrar_endpoint] enabled is not true, and do not add a second enablement gate, infer enablement from the socket or the unit pair, or gate on the store directory already existing.
  • Do not hang the rotation check off the per-profile check_interval loop, and do not add a configuration key for the rotation interval — it is a file-top constant, injectable for tests.
  • If neither reopen mechanism works against the pinned image, report it as a finding with the evidence rather than shipping a restart-based rotation or claiming a bound that does not hold.
  • Rotation failure is an operational event, not a fail-closed one: it must never fail an OpenBao request, never stop the daemon and never take the registrar endpoint down.
  • Do not define, rename, reuse or re-validate audit_record_dir, audit_max_file_bytes, audit_max_retained_files, audit_min_retain_days or any audit_store_* key; those belong to sibling issues and are read here only where a bound is sized against them. The three keys this issue adds carry the openbao_audit_ prefix precisely so no name collides.
  • No unwrap() in production code; no [] indexing.
  • Tests use tempfile::tempdir() and never a fixed path.

Out of scope

  • The reserved store's directory layout, its configuration keys, the Compose override that binds the device's directory out of its named volume, and the kernel-enforced reserve — all owned by sibling issues that land first and that this issue builds on.
  • Measuring the store's capacity, the low-water alarm and the registrar_health.audit_capacity member. This issue only sizes its defaults so that the alarm has margin; it neither reads nor reports capacity.
  • The registrar's own verb-record store, its format, its rotation bounds or its reader — that store is already self-bounding and is not what this issue rotates.
  • Compressing, indexing, shipping or externally forwarding the OpenBao audit log.
  • Changing OpenBao's audit device type, adding a second device, or altering what OpenBao writes into it.
  • Migrating audit records already written to the openbao-audit named volume.
  • Any retention shortfall signal for the device. The retention target here is soft and expected to go unmet on a busy deployment; deriving and reporting a shortfall is the verb-record reader's, over its own store.

Test plan

  • Rotation test driving the active log past openbao_audit_max_file_bytes and asserting rotation, the retained-generation count, and the end-of-pass total footprint at or under openbao_audit_max_file_bytes × (openbao_audit_max_retained_files + 1).
  • Byte-trim test with a within-count retained set of oversized generations, asserting oldest-first dropping until the retained total is inside openbao_audit_max_file_bytes × openbao_audit_max_retained_files.
  • Pathological-generation test: one generation larger than the whole retained budget is dropped whole and the drop is logged at warn; no generation is rewritten in place.
  • Naming and ordering test: generations are named audit-<YYYYMMDDTHHMMSSZ>.log, collide into a numeric suffix, sort lexicographically in time order, and trimming drops the oldest by name order while leaving audit.log in place.
  • E2E test on a running deployment asserting StartedAt, RestartCount and Pid are unchanged and sys/seal-status stays sealed=false / initialized=true across a rotation.
  • Post-rotation write test: an OpenBao request immediately after a rotation succeeds, its entry lands in the new active log, and the rotated generation gains nothing further.
  • Ownership test asserting the post-rotation active log is owned by the container's uid:gid, driven through the real rotation path so a root-owned re-creation fails the test.
  • Device-registration test asserting sys/audit still reports the file device with the same path and that verify_audit_file passes after a rotation.
  • E2E test asserting the unmodified assert_openbao_audit_log helper passes on a rotated deployment, with a fresh AppRole login and KV read driven after the rotation.
  • Retention-conflict test asserting the size ceiling wins over openbao_audit_min_retain_days.
  • Config tests: the three keys load with defaults 67108864, 7 and 90; an openbao_audit_max_file_bytes below 1048576, a zero openbao_audit_max_retained_files and a zero openbao_audit_min_retain_days each fail validation with an error naming the key; a [registrar] table already carrying the sibling keys keeps them.
  • Conditionality test asserting a deployment with [registrar_endpoint] enabled false or absent spawns no rotation task and is otherwise unchanged.
  • Fault-injection test making a rotation step fail and asserting OpenBao keeps serving, the daemon's other duties keep running, and the failure is logged and retried on the next tick — driven through an injected interval so the test does not wait 60 seconds.
  • Signal-form recovery tests: reopen forced to fail after a successful rename restores audit.log with its original contents, removes the generation and skips the trim; reopen "failing" while a new audit.log exists is treated as a successful pass and the rename-back does not clobber that file.
  • Truncate-form recovery test: the copy is fsynced before any truncate, and a failed truncate leaves the active log intact and unlinks the copy.
  • Mechanism-verification probe in scripts/impl/lib/audit-log.sh, invoked from the Docker E2E lifecycle, establishing which reopen form the pinned image supports with the file-level evidence above.
  • Tests use tempfile::tempdir() and never a fixed path.

Dependencies

Depends on the reserved-store layout issue, which creates <audit_store_dir>/openbao and renders the Compose override that makes it back /openbao/audit, and whose [registrar_endpoint] enabled predicate this issue's rotation is gated on. The predicate and the directory are restated above so this issue is implementable from its own text; where the two disagree, the layout issue owns them. This issue asks nothing further of that one: rotated generations live inside the directory it already creates, so no additional path, owner or mode is required from the install path. Part of the registrar-surface umbrella.

Nothing depends on this issue's mechanism, but the reserved store's guarantee is incomplete without it: the store bounds where the artifacts can spill, and this issue is what bounds the one writer that would otherwise fill the store itself.

Pointers

  • src/openbao.rs:560 (verify_audit_file) — the mandatory-device check that must keep passing, and the sys/audit query it makes
  • src/commands/init/steps/openbao_setup.rs:162 — the caller that aborts init when the device is absent
  • src/commands/init/steps/openbao_tls.rs — the rewrite that preserves the audit stanza, and the tests asserting it is not dropped
  • openbao/openbao.hcl — the audit stanza whose file_path must stay /openbao/audit/audit.log
  • docker-compose.yml:3 (the pinned openbao/openbao:2.5.5 tag), docker-compose.yml:12 and :17 (the audit mount and the entrypoint that chowns it), docker-compose.deploy.yml:20 (the OPENBAO_IMAGE override)
  • src/commands/clean.rs:34 (OPENBAO_NAMED_VOLUMES) — the openbao-audit volume that stays declared
  • src/daemon.rs::run_daemon:68 — where the dedicated rotation task is spawned, and src/daemon.rs:183 (check_interval) — the per-profile loop it must not use
  • src/config.rs — the [registrar] table the three new bounds join, and Settings::registrar_endpoint.enabled, the gating boolean added by the endpoint-listener issue
  • src/commands/infra.rs:800 (sweep_secrets_ownership) and issue_openbao_tls_cert's output-directory chown — the existing patterns for giving a container-written path the right owner
  • scripts/impl/lib/audit-log.sh:21 (assert_openbao_audit_log, active-log only) and scripts/impl/run-local-lifecycle.sh:1087 — the E2E assertion that must still pass, and where the mechanism probe is wired in
  • docs/en/operations.md:95 and docs/ko/operations.md:96 — the audit-logging sections; docs/en/configuration.md and docs/ko/configuration.md — the key reference
  • docs/rfcs/0001-registrar-role-and-non-self-propagation.md §5.6 and §6

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions