You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Record limited registrar invocations and report the throttle
Context
The sibling limiter issue bounds how fast one client identity may drive the two registrar verbs. It does so by suppressing work: on the pre-derivation arm a limited invocation writes neither of its two audit records, and on the post-derivation arm it is not attempted at all. That suppression is what stops a flood from exhausting the record store — and it is also what would make the flood invisible, because the mechanism that would have recorded it is exactly the mechanism being skipped.
So the limited invocations have to leave a trace of their own, in the same store, without reintroducing the per-invocation cost the limiter exists to avoid. A single counted record per window per key does that: a sustained flood costs one record per window rather than one per request, while the fact of the flood, who drove it and which bucket ran dry all survive.
Two more consequences of suppression land here. A caller throttled at the admission check point must be told something structurally distinct from a permanent failure, or the control plane classifies it as generic, retries, and feeds the flood. And a flood in progress must be visible on the endpoint's own responses rather than only in the record store after the fact.
Two words, used precisely throughout this issue, because they are not synonyms here. An invocation is limited when the limiter suppressed its audit records — true at both check points, and the condition the counted record and the health counters describe. An invocation is throttled when it is limited at the admission check point and therefore returns the retryable throttle outcome; a limited pre-derivation invocation is not throttled — it still receives its real, permanent refusal, and the sibling limiter issue owns that rule. So every throttled invocation is limited and the reverse does not hold. Read every criterion below with that distinction: the record and the health counters cover limited, and only the wire-mapping work covers throttled. (That split is ratified: the RFC's unqualified "a throttled invocation returns RegistrarBusy" language is being amended to name the admission check point, and the bucket-key question that also touched this body is settled too — see below. Neither re-specifies anything here.)
This issue owns all three: the counted record, the caller-facing throttle identifier for the admission case, and the health member. The limiter mechanism itself — the buckets, the check points, the sizing and the four bucket configuration keys — is the sibling issue's and is consumed here through the sink and the typed outcome it exposes.
Both rulings settled, and what they fix in this body
Two rulings reached this body. The bucket key is settled: the RFC's owner ratified per-(client identity, verb, bucket) buckets over the two-value enum, which is the shape this issue already reuses as limited_bucket. So the coalescing key (client identity, verb, limited_bucket), the counted record's limited_bucket field and its golden fixture, and the two per-bucket health counters are fixed — build them as written and do not treat the unamended RFC sentence as reopening the question, since amending that sentence is the decision issue's job rather than this one's.
The ruling's reasoning is worth carrying where this issue documents the two buckets, because it is narrower than "the RFC's shape is not implementable": the two-value enum is the maximal refinement knowable at the point the limiter is charged — a pre-derivation refusal is knowable there, an admitted invocation's outcome is not, since spec-conflict, name-collision and host-mismatch are settled by OpenBao work that follows the intent write. What that buys, positively stated, is that the cheapest flood path cannot starve accepted mints; what it leaves is expensive refusals consuming admission budget, bounded by attacker cost rather than by the limiter. The sibling limiter issue owns both statements; this issue neither restates them as its own guarantee nor contradicts them.
The caller answer is settled too, and it is settled the way this body already reads: the throttle is mapped from the admission check point only. A limited pre-derivation invocation keeps its real, permanent refusal. So The caller-facing throttle below stands as written, and the acceptance criterion and test asserting that a limited pre-derivation invocation reaches the caller as its real permanent refusal — written specifically to fail if the mapping is widened — are now guarding a ruling rather than a proposal. Do not widen the mapping to both check points, and do not read the unamended RFC sentence as licence to: amending it is the decision issue's job.
The ground for that ruling belongs where the mapping is defined, so carry it into the wire-mapping section rather than leaving it here as background: RegistrarBusy is the family's only retryable identifier — the enroll wire issue exposes retryability as a classification method on the error type, true for it and false for the other five — while this arm's refusals are classified deterministic and permanent by RFC-D2 §4b, which terminates the attempt and spends no retry budget, and by RFC-E §9, which states ServiceInstanceMismatch is not retryable because clicking again sends the same request. Mapping this arm onto RegistrarBusy would be a type-level contradiction, and RFC-E §9's rendering rule makes it operator-visible: a RegistrarBusy response is drawn as an in-progress wait honouring retry_after, never as a failure and with no retry button, so a malformed service_name would sit on screen as "waiting" indefinitely with its real cause unreachable.
One thing the suppression must keep, and this issue is where it is kept. A limited pre-derivation invocation writes no audit records, but it still increments the coalescing counter this issue drains into a counted record, and the reader's amended anomaly definition still counts it — RFC-F §5.6 puts it as "one coalesced counter increment rather than a durable record". Dropping the increment along with the records would make a flood least visible exactly while it is worst, and these records are the only detection the RFC's "detected, not prevented" argument rests on.
Both rulings are now bookkeeping rather than open questions: what remains is the decision issue's RFC amendment carrying them, which this issue's dependency list already gates on. This subsection is deleted when that lands.
Scope
The coalescing window and the counted record
Coalescing covers exactly the invocations the limiter stopped from writing records — the suppressed pre-derivation refusals and the throttled admissions. Every invocation the limiter admits keeps the audit-record work's ordinary two-phase behaviour, unchanged.
Implement the sibling issue's limited-invocation sink: within one coalescing window, limited invocations sharing the same (client identity, verb, limited_bucket) key increment one counter and produce exactly one counted record, written when the window closes. Flush any open window on daemon shutdown so a counted record is never lost to a restart.
Wrap the limiter's default counting sink; do not replace it. The limiter holds one sink, and the health member below publishes its two per-bucket counters — so construct this sink around the default counting one and delegate every event to it before coalescing. That keeps one counting implementation in the tree rather than two that can disagree, and it is why the member can be described as reading the limiter's counters rather than counters this issue keeps. Do not add a second per-bucket counter here, and do not install a sink that drops the counting one.
Close windows on the daemon's existing tick; add no second scheduler. The window is a duration, so something has to notice it elapsed — reuse the loop the daemon already runs rather than spawning a timer task per key, which would put one task per (identity, verb, bucket) behind a caller's ability to create keys. A window whose deadline has passed is written on the next tick; the resulting record's window_end is the deadline, not the tick that noticed it, so a slow tick does not stretch the window a record claims to cover.
Bound the open-window map explicitly, for the same reason the limiter bounds its bucket map. Windows are keyed on the client identity, and the sibling issue documents the concrete hazard that an identity may not be stable across connections — a peer-credential value rendering the connecting pid varies by construction. An unbounded map of open windows is then exactly the memory exhaustion the limiter's own cap exists to prevent, arriving through the mechanism built to record it. Cap the map with a file-top constant, and on overflow flush the oldest open window early rather than dropping it or refusing to open a new one: an early flush costs one extra record and loses nothing, while dropping loses the fact of a flood and refusing makes the newest traffic — the traffic most likely to be the attack — the traffic that goes unrecorded. Note in the documentation that an early flush is why a record's window may be shorter than the configured one.
A window that closes before its deadline carries window_end at the instant it closed, not at the deadline it never reached. The tick bullet above fixes window_end at the deadline for every window closed at or after it, including one a slow tick noticed late. The two closes that can precede the deadline — an overflow flush when the open-window map hits its cap, and the shutdown flush — take the flush instant instead, because the window stopped accepting invocations there and a record must never claim a span over which it counted nothing. So window_end - window_start is at most the configured window in every case, and is always exactly the span the record's count covers.
Add rate_limit_coalesce_window_seconds to the [registrar] configuration table — u32, default 60, validated against zero, because a zero window makes every suppressed invocation its own record, which defeats the coalescing. A negative or non-integer value produces a named configuration error identifying the key, in the same shape as the table's other validation failures; unrelated keys in the table survive the load untouched.
limited_bucket is the bucket that had no token — the sibling issue's two-value enum, predecision_refusal or admission, reused rather than redeclared. It is deliberately not a refusal class: at neither check point has a refusal been classified, and at the admission check point the invocation might have been a perfectly good mint. Naming the field for the bucket says exactly what is known and nothing more, and it makes the two cases legible to an operator: predecision_refusal means those callers received their real permanent refusal and only the records were suppressed, admission means those callers were throttled and never attempted.
The counted record's serialized field set is exactly this, and every spelling is fixed here rather than by whoever implements it first. This is a persistent on-disk audit contract, so the golden fixture must pin a spelling this issue chose, not one an implementation invented and the fixture then blessed:
record_version — integer, 1, unchanged.
phase — "limited".
ts — the instant the record was written, RFC 3339 UTC at the store's existing millisecond precision, like every other record's. It is deliberately not the same value as window_end: a window closed on a later tick is written after its deadline, and carrying both is what lets a reader see that lag instead of inferring it.
verb and caller_identity — the two identity fields of the coalescing key, spelled and bounded exactly as the store already spells and bounds them. The client identity is caller_identity; do not introduce client_identity as a second name for it.
limited_bucket — "predecision_refusal" or "admission", the coalescing key's third field, a Rust enum like every other enum-valued field in these types.
count — integer, at least 1: how many limited invocations this record coalesces.
window_start and window_end — RFC 3339 UTC at the same precision as ts.
truncated — exactly as on any other record, present only when caller_identity was capped.
request_id, requested and registration_id — omitted entirely. A counted record covers many invocations, so no per-invocation id identifies it, and at the pre-derivation check point nothing was parsed that could fill requested or registration_id. Omit them rather than writing null, an empty string or a synthesized id — the same omission discipline the store already applies to registration_id, outcome and truncated.
outcome — absent. That field is present only on the outcome phase, and a counted record has no outcome class.
Express those presence rules inside the store's existing types — Option fields skipped when absent, the way outcome, registration_id and truncated already are — rather than declaring a second record struct that happens to serialize compatibly.
The counted record does not carry a per-invocation refusal reason — unknown at one check point and nonexistent at the other.
It lands as the third phase variant — Rust Limited, serialized "limited" by the store's existing #[serde(rename_all = "snake_case")] — inside the audit-record store's own serde types, at the same record_version.limited is the word Context defines precisely (the limiter suppressed this invocation's records, at either check point) and the word limited_bucket is already named for, so the on-disk value says what the line is without a second vocabulary. That store ships phase as an enum with exactly two values precisely so this record can be added as a third, and it forbids the catch-all arm that would let a new variant pass unnoticed — so extend those types rather than declaring a parallel shape, and do not introduce a second file, a second directory, a second encoding or a new record_version. Bumping the version would be the worst option available: the store's reader rejects an unrecognised version as a malformed line, so every counted record would land in the count that means "someone is forging records", which is the opposite of what this record says.
The client identity in this record is subject to the store's field bound like any other: the 512-byte cap, UTF-8-boundary truncation, and the per-field truncated entry carrying the full value's SHA-256 and byte length. A counted record must fit the store's declared maximum record size, whose arithmetic assumes every attacker-influenced field is capped. It has exactly one such field where an intent record has five, and everything else it carries is a bounded integer, timestamp or enum, so it fits with room to spare — the criterion below still asserts that rather than leaving it assumed.
Extend the store's golden fixtures additively with one counted record, so this variant's serialized bytes are pinned the same way every other record's are and a later rename surfaces as a fixture diff.
A counted record that cannot be written is logged and does not block anything, and that is not a hole in the fail-closed rule. That rule refuses an invocation whose intent is unrecordable; a counted record is written when a window closes or at shutdown, with no invocation left to refuse — the callers were answered long ago. So report the failure through tracing and continue. Do not reach for a best-effort or skip mode on the record store to express this: that store has none by design, and adding one would let suppression leak into the verb path the rule protects. Write the counted record through the same fallible API every other record uses and handle its error here.
A sustained flood therefore costs one record per window per key, and the key includes the client identity: at the default 60-second window that is at most 60 records per hour per key, so at most 240 per hour for one client identity across its four keys. The deployment-wide figure is that number times the number of distinct client identities, which in this design is effectively one. Either way it is negligible against the record store's ceiling.
The reader's two classification rules
Adding the variant breaks the record reader's exhaustive matches, and fixing them is this issue's work, not a follow-up. The compile error is the intended mechanism — it is why phase is an enum — and it must be resolved by stating this record's classification in both of the reader's counters, not just one:
Intent-without-outcome. A counted record is self-contained: it is its own intent and its own outcome. It is never an unpaired intent and must not be reported as an anomaly, while a lone intent left by a crash mid-mint still must be. Limited invocations write no intent at all, so the pairing stays 1:1 over the intent/outcome records that exist and the reader's settle grace is untouched.
Malformed. A counted record is a well-formed record of a known variant at a known version and must not be counted as malformed either — it is not a line the reader "could not parse". Two of the reader's malformed triggers need their per-variant reading stated, because a counted record trips both if they are read as flat rules over one field set. Its missing-required-field rule is evaluated per phase: a "limited" line that omits request_id, requested and registration_id is complete for its variant, not short. And its duplicate-request_id rule cannot reach a counted record at all, because a counted record carries no request_id — so it can neither be counted as a duplicate itself nor make a genuine intent/outcome pair look like one.
Leaving either arm to a default would report a working limiter as an attack in progress, on the one artifact the whole detection argument rests on. This is the only amendment to the record store's behaviour: its rule that the pre-derivation arm writes both phases under its global lock continues to hold for every invocation the limiter admits.
The caller-facing throttle — the admission check point only
This section covers exactly one of the two check points. The sibling limiter issue produces its typed retryable throttle outcome only at admission; a limited pre-derivation invocation returns its real permanent refusal, which is already mapped by the endpoint's existing error mapping and is not touched here. Nothing in this section applies to it, and no criterion below may be written so that it appears to.
Map the sibling issue's typed retryable throttle outcome onto its own wire identifier, structurally distinct from the permanent unavailability variants whose reasons all mean until an operator acts. Without a distinct identifier the throttle arrives as a generic error, a caller classifies generic as transient and retries, and the retry storm feeds the limiter. The transient/permanent class must be preserved through the mapping.
What is already done, and what this issue actually adds — because the wire form is not new here. The endpoint protocol issue settled this identifier's assignment in advance, precisely because the verb-layer variant did not exist yet: it defines the throttle's wire form and its retryable class and covers them with a golden fixture, and records the assignment in the reference's local half beside the mapping. So do not introduce a second definition, a second class or a competing fixture. What lands here is the match arm for the newly added verb-layer variant — the arm that issue's exhaustive, catch-all-free mapping forces the moment the variant appears — together with any fixture extension the arm's real payload now makes possible. If the pre-recorded assignment turns out to disagree with what the reference spells, the reference wins and the disagreement is a finding against the wire-contract issue, not something to resolve by spelling it differently here.
The identifier and its payload field spelling come from the wire-contract issue's checked-in docs/ reference; the semantics come from the sibling limiter issue. That reference transcribes an externally owned error set and is authoritative over every identifier and payload name in it — the design prose renders this error as RegistrarBusy { retry_after } in one place and retry_after_seconds in another, and the reference settles which. Ship what the reference spells; retry_after_seconds is the fallback only where it spells nothing. Pin whichever spelling lands with a golden fixture.
The payload crosses the wire as the unsigned whole-second duration the limiter produced — never an absolute timestamp, and never a value below 1.
The health member
A flood in progress must be visible on the endpoint's own responses rather than only in the record store after the fact, so this issue publishes the limiter's counter there.
The field is registrar_health, a container of named members on the endpoint's responses, reserved by the endpoint protocol issue, which also owns the rule about which responses carry it. This issue adds exactly one member, limiter. Always present; its values are counters, not scan results, so serving it costs no store read.
The member reports the two buckets separately, because one collapsed number erases the distinction this issue spends a paragraph establishing.limited_bucket exists on the counted record precisely because the two cases mean different things to an operator — predecision_refusal means callers received their real permanent refusal and only the records were suppressed, admission means callers were throttled and never attempted — and those warrant opposite responses: the first says someone is flooding malformed input, the second says the control plane itself is being held back and a bring-up may be stalling. Publishing their sum would put the less actionable of the two on the wire while leaving the actionable one in a store on the bootroot host that the console cannot read. So limiter carries limited_predecision_refusal and limited_admission, both u64, both the count since daemon start, read from the sibling issue's in-process counters. Two named fields keep the member bounded and fixed-shape as the container requires — the bucket enum has exactly two values and gaining a third is a compile error there, not a silent widening here.
The member is purely additive. Do not reshape the container, do not decide which responses carry it, and do not read, write, reorder or otherwise touch any sibling member — the certificates member the certificate work populates, or the audit_capacity member the reserved-store work adds. None of those issues depends on this one, so whichever lands last must be additive; they also pin the encoded messages with golden fixtures, so a reshape here surfaces as a fixture diff and a caller-visible wire change.
The snapshot the response serializes is shared, and this issue may be the one that creates it. The protocol issue requires the container to be serialized from a snapshot the daemon refreshes on its own tick rather than computed per request, but it populates no member and therefore ships no holder; the three member-owning issues do not depend on each other, so whichever lands first meets the need. If no holder exists when this work starts, add one — a single daemon-held snapshot of the whole container, refreshed on the existing tick — rather than a holder private to limiter, and shape it so a sibling adds its member without touching this one. If a holder already exists, use it and add nothing. Three independently written holders refreshed on three cadences is the outcome to avoid: the container is one wire object and would then be assembled from values of three different ages with nothing on the wire to say so.
Populating the member changes the encoded messages, so extend the golden serialization fixtures additively across every shape that carries the container, and extend the docs/ protocol reference's local half with the limiter schema.
Document the coalescing window and its key, the counted record's shape and what limited_bucket means, and what the throttle identifier means for a caller, in bothdocs/en/ and docs/ko/, on the existing pages beside the audit-record documentation, with no mkdocs.yml nav change.
Acceptance criteria
A flood of limited invocations produces one counted record per window per (client identity, verb, limited_bucket) key: a test asserts the record store does not grow proportionally to the flood, and that records for the two buckets do not merge.
A counted record serializes exactly the field set above — record_version1, phase"limited", ts, verb, caller_identity, limited_bucket, count (≥ 1), window_start and window_end, plus truncated only when the identity was capped — carries no per-invocation refusal reason, and omitsrequest_id, requested, registration_id and outcome entirely rather than emitting them as null, "" or a synthesized value.
The counted record is the third phase variant in the audit-record store's own serde types, serialized "limited", at the unchangedrecord_version, written to its existing store: no second file, directory, encoding or version appears, and a golden fixture pins the variant's serialized bytes including that phase value.
An open coalescing window is flushed to a counted record on daemon shutdown rather than lost, and that record's window_end is the flush instant rather than the deadline it never reached.
rate_limit_coalesce_window_seconds loads with its default, is rejected at zero with an error naming the key, and produces the same named error for a negative or non-integer value; unrelated [registrar] keys survive the load.
The counted record's caller_identity obeys the store's field bound: an over-bound identity is carried as a leading prefix of at most 512 bytes, cut on a UTF-8 character boundary — so a multibyte character straddling the cap is dropped rather than split, and the prefix may be shorter than 512 bytes — plus a truncated entry whose SHA-256 and byte length are those of the complete original value, and the record fits the store's declared maximum record size.
A counted record that cannot be written is logged and blocks nothing: a failed write at window close and at shutdown refuses no invocation, panics not at all, and no best-effort or skip mode was added to the record store.
A counted record is counted by neither of the reader's signals: a flood producing counted records leaves both the intent-without-outcome count and the malformed-line count at zero for them — including that its omitted request_id, requested and registration_id raise no missing-required-field verdict, and that many counted records with no request_id raise no duplicate-request_id verdict — while a genuine crash-mid-mint and a genuinely unparseable line are each still counted by the right one.
An admitted pre-derivation invocation still writes the paired intent and outcome records the audit-record work specifies; the amendment above changes nothing for admitted invocations.
An invocation throttled at the admission check point reaches the caller as its own wire identifier, structurally distinct from every permanent unavailability variant and classified transient, with the payload spelled as the wire-contract reference spells it and pinned by a golden fixture.
A limited pre-derivation invocation does not reach the caller as that identifier: a test drains the predecision_refusal bucket and asserts the caller still receives its real permanent refusal on the wire, with its records suppressed. This is the criterion that fails if the wire mapping is widened to cover both check points.
The wire payload is an unsigned whole-second duration of at least 1 and never an absolute timestamp.
The coalescing sink wraps the limiter's default counting sink rather than replacing it: a test drives a flood through the installed production sink and asserts both that counted records are produced and that the limiter's two per-bucket counters still advance, and a reviewer can confirm no second per-bucket counter was introduced here.
registrar_health.limiter carries limited_predecision_refusal and limited_admission, both u64, wherever the health container is carried; a test floods each bucket separately and asserts only that bucket's counter moves, so a collapsed sum would fail. The member round-trips alongside any sibling member and leaves every sibling member and entry unchanged.
The health snapshot the response serializes is a shared holder, not one private to this member: a test asserts the container serialized on a response is read from that holder rather than computed per request, and a reviewer can confirm the holder is shaped so a sibling member can be added without touching limiter.
Window closing runs on the daemon's existing tick with no second scheduler and no per-key timer task; a record's window_end is the window's deadline rather than the tick that noticed it.
The open-window map is capped by a file-top constant: driving more distinct keys than the cap leaves it bounded and flushes the oldest window early rather than dropping it or refusing the newest traffic; a test asserts the flood's records all exist and that an early-flushed window carries window_end at the flush instant, so its bounds are shorter than the configured window.
The coalescing window, the counted record shape and the throttle's meaning for a caller are documented in bothdocs/en/ and docs/ko/, on the existing pages, with no mkdocs.yml nav change.
cargo clippy is warning-free and cargo fmt --check passes.
Constraints
Coalescing must not silently discard the fact of a limited invocation — the counted record must carry the count, the window bounds, the identity, the verb and the limited bucket.
Do not key a coalescing window or a counted record on the verb layer's fine-grained refusal reason. It is unknown at one check point and nonexistent at the other, and a key that varied with it would let a caller multiply its records by varying its refusal.
Do not introduce a second record file, directory, encoding or record_version, and do not add a best-effort or skip mode to the record store.
The counted record carries the field set fixed in Scope and nothing else: do not add a field to it, do not re-spell or rename a key the store already defines (caller_identity, verb, ts, truncated), and do not fill an omitted key with null, an empty string or a synthesized value. The phase value is "limited"; a golden fixture blessing some other spelling an implementation happened to emit is a defect, not a decision.
Do not weaken the "unrecordable intent implies refused" rule. A counted record's write failure is not an invocation's write failure, and the two must not be collapsed into one code path.
The throttle identifier must be structurally distinct on the wire from the permanent unavailability variants, and must not be expressed by overloading one error type with a boolean.
Use limited and throttled as defined in Context and do not treat them as interchangeable: the counted record and the health counters cover limited invocations at both check points, while the throttle identifier covers the admission one only. Do not widen the wire mapping to the pre-derivation check point — that would report a determined refusal as retryable, and the split has already been ruled: the group's decision issue transcribes it into the RFC, and neither that issue nor this one may reopen it.
Do not re-spell, rename or reclassify anything the docs/ protocol reference transcribes from the externally owned contract; mirror it.
The health member must be additive. Do not reshape the registrar_health container, do not decide its placement, and do not touch a sibling member's shape, contents or ordering.
Do not add a health snapshot holder private to this member. One shared holder for the whole container, or the one a sibling already added.
Do not implement, re-tune or duplicate the limiter itself — the buckets, the check points, their ordering, the four bucket configuration keys and the sizing formula are the sibling issue's, consumed here through its sink and its typed outcome.
Do not define a second wire form, class or fixture for the throttle identifier. The endpoint protocol issue settled and fixture-covered it ahead of the variant existing; this issue supplies the match arm that produces it.
Do not add a second scheduler or a timer task per key for window closing, and do not leave the open-window map unbounded. Both would put growth on the daemon behind a caller's ability to create keys, which is what the limiter exists to stop.
Do not collapse the two buckets into one published number. The record carries limited_bucket for a reason and the health member must preserve it.
No unwrap() in production code; no [] indexing.
Out of scope
The limiter mechanism: the two token buckets, their key, the bounded bucket map, the monotonic-clock refill, both check points and their ordering, the four rate_limit_* bucket keys, the sizing formula and the retry-after arithmetic. All the sibling limiter issue's.
Defining the record encoding, the rotation bounds, the anomaly scan's own machinery, its lookback window constant or audit_record_dir itself — all owned by the audit-record issues; this issue adds one record variant in that existing encoding and amends the two classification rules for it, and changes nothing else there.
The reserved audit store in every part: the audit_store_* configuration keys, the store directory layout and its ownership, relocating audit_record_dir onto it, the Compose override that moves the OpenBao audit device, the capacity probe, the headroom arithmetic, the low-water alarm and the registrar_health.audit_capacity member.
Relaying the audit-record anomaly, malformed-line or retention signals to the endpoint — the reserved-store work owns that relay.
A per-(component, host) ceiling on how many instances a deployment may legitimately run.
The control-plane side of rendering the relayed health values.
Test plan
Coalescing test: a flood on each bucket produces one counted record per window per key, the two do not merge, and the counted record carries the count, window bounds, identity, verb and limited_bucket.
Shutdown-flush test asserting an open window is written rather than lost.
Counted-record encoding test: the variant round-trips through the record store's own serde types at the unchanged record_version, its phase serializes as "limited", the omitted keys (request_id, requested, registration_id, outcome) are absent from the emitted line rather than present-and-empty, a golden fixture pins its bytes, and no second file, directory or encoding appears.
Reader-classification test: counted records raise neither the intent-without-outcome signal nor the malformed-line signal, while a crash-mid-mint intent and an unparseable line are each still counted by the right one.
Field-bound test: an over-bound caller_identity whose byte 512 falls inside a multibyte character is cut on the character boundary below the cap, carries a truncated entry whose digest and length are the complete original value's, and leaves the record inside the declared maximum record size.
Write-failure test: a failed counted-record write at window close and at shutdown is logged, refuses no invocation, and adds no skip mode to the store.
Config test: rate_limit_coalesce_window_seconds loads with its default; zero, negative and non-integer values each fail validation with an error naming the key; unrelated [registrar] keys survive.
Wire tests, one per check point: an admission-throttled invocation arrives as the distinct transient identifier with an unsigned whole-second payload of at least 1, pinned by a golden fixture and distinguishable from every permanent unavailability variant; a limited pre-derivation invocation arrives as its real permanent refusal and never as that identifier.
Health-member test asserting registrar_health.limiter carries both per-bucket counters wherever the container is carried, that flooding one bucket moves only its own counter, that the member round-trips alongside a sibling member, and that every sibling member and entry is unchanged.
Shared-snapshot test: the registrar_health container serialized on a response is read from the shared daemon-refreshed holder rather than computed per request — moving the limiter's counters without letting the daemon tick leaves the served container at the previously refreshed values, and it changes on the next tick — and no holder private to limiter exists.
Window-lifecycle tests, one per kind of close: a window whose deadline passed is written on the next daemon tick with window_end at the deadline; the open-window map driven past its cap stays bounded and flushes the oldest window early with window_end at the flush instant rather than dropping or refusing a key; and the shutdown flush likewise stamps window_end at the flush instant.
Tests use tempfile::tempdir() and never a fixed path.
Dependencies
Depends on the group's limiter bucket-shape decision issue, which carries two rulings, both now settled in favour of what this body already specifies: the bucket key as per-(client identity, verb, bucket) buckets over the two-value enum reused below as limited_bucket, and the caller answer as the throttle mapped from the admission check point only, with a limited pre-derivation invocation keeping its real permanent refusal. Neither re-specifies anything here. What this dependency gates on is the RFC amendment that issue must land carrying both, since until it does the accepted document still states two things this group's code will not deliver. Depends on the sibling limiter issue for the two-value bucket enum, the limited-invocation sink this implements, the in-process limited-invocation counter and the typed retryable throttle outcome this maps onto the wire. Depends on the audit-record store for its serde types, its JSON Lines encoding, its phase enum and record_version, its attacker-influenced-field bound and its golden fixtures; and on the store's reader, because the two classification rules amended here are defined there and adding a phase variant breaks that reader's matches at compile time. Depends on the registrar endpoint's wire protocol issue for the registrar_health container, the fixtures and the error identifier set the throttle joins. Part of the registrar verb rate-limiting umbrella.
Pointers
src/daemon.rs::run_daemon:68 — the daemon loop the coalescing window's close and shutdown flush hang off
src/config.rs — the configuration load and validation shape rate_limit_coalesce_window_seconds follows
docs/ — the checked-in protocol reference whose transcribed half spells the throttle identifier and whose local half gains the limiter schema
docs/en/operations.md:95 and docs/ko/operations.md:96 — the audit-logging sections this documentation sits beside
docs/rfcs/0001-registrar-role-and-non-self-propagation.md §5.6 and §6
Record limited registrar invocations and report the throttle
Context
The sibling limiter issue bounds how fast one client identity may drive the two registrar verbs. It does so by suppressing work: on the pre-derivation arm a limited invocation writes neither of its two audit records, and on the post-derivation arm it is not attempted at all. That suppression is what stops a flood from exhausting the record store — and it is also what would make the flood invisible, because the mechanism that would have recorded it is exactly the mechanism being skipped.
So the limited invocations have to leave a trace of their own, in the same store, without reintroducing the per-invocation cost the limiter exists to avoid. A single counted record per window per key does that: a sustained flood costs one record per window rather than one per request, while the fact of the flood, who drove it and which bucket ran dry all survive.
Two more consequences of suppression land here. A caller throttled at the admission check point must be told something structurally distinct from a permanent failure, or the control plane classifies it as generic, retries, and feeds the flood. And a flood in progress must be visible on the endpoint's own responses rather than only in the record store after the fact.
Two words, used precisely throughout this issue, because they are not synonyms here. An invocation is limited when the limiter suppressed its audit records — true at both check points, and the condition the counted record and the health counters describe. An invocation is throttled when it is limited at the admission check point and therefore returns the retryable throttle outcome; a limited pre-derivation invocation is not throttled — it still receives its real, permanent refusal, and the sibling limiter issue owns that rule. So every throttled invocation is limited and the reverse does not hold. Read every criterion below with that distinction: the record and the health counters cover limited, and only the wire-mapping work covers throttled. (That split is ratified: the RFC's unqualified "a throttled invocation returns
RegistrarBusy" language is being amended to name the admission check point, and the bucket-key question that also touched this body is settled too — see below. Neither re-specifies anything here.)This issue owns all three: the counted record, the caller-facing throttle identifier for the admission case, and the health member. The limiter mechanism itself — the buckets, the check points, the sizing and the four bucket configuration keys — is the sibling issue's and is consumed here through the sink and the typed outcome it exposes.
Both rulings settled, and what they fix in this body
Two rulings reached this body. The bucket key is settled: the RFC's owner ratified per-
(client identity, verb, bucket)buckets over the two-value enum, which is the shape this issue already reuses aslimited_bucket. So the coalescing key(client identity, verb, limited_bucket), the counted record'slimited_bucketfield and its golden fixture, and the two per-bucket health counters are fixed — build them as written and do not treat the unamended RFC sentence as reopening the question, since amending that sentence is the decision issue's job rather than this one's.The ruling's reasoning is worth carrying where this issue documents the two buckets, because it is narrower than "the RFC's shape is not implementable": the two-value enum is the maximal refinement knowable at the point the limiter is charged — a pre-derivation refusal is knowable there, an admitted invocation's outcome is not, since spec-conflict, name-collision and host-mismatch are settled by OpenBao work that follows the intent write. What that buys, positively stated, is that the cheapest flood path cannot starve accepted mints; what it leaves is expensive refusals consuming admission budget, bounded by attacker cost rather than by the limiter. The sibling limiter issue owns both statements; this issue neither restates them as its own guarantee nor contradicts them.
The caller answer is settled too, and it is settled the way this body already reads: the throttle is mapped from the admission check point only. A limited pre-derivation invocation keeps its real, permanent refusal. So The caller-facing throttle below stands as written, and the acceptance criterion and test asserting that a limited pre-derivation invocation reaches the caller as its real permanent refusal — written specifically to fail if the mapping is widened — are now guarding a ruling rather than a proposal. Do not widen the mapping to both check points, and do not read the unamended RFC sentence as licence to: amending it is the decision issue's job.
The ground for that ruling belongs where the mapping is defined, so carry it into the wire-mapping section rather than leaving it here as background:
RegistrarBusyis the family's only retryable identifier — the enroll wire issue exposes retryability as a classification method on the error type, true for it and false for the other five — while this arm's refusals are classified deterministic and permanent by RFC-D2 §4b, which terminates the attempt and spends no retry budget, and by RFC-E §9, which statesServiceInstanceMismatchis not retryable because clicking again sends the same request. Mapping this arm ontoRegistrarBusywould be a type-level contradiction, and RFC-E §9's rendering rule makes it operator-visible: aRegistrarBusyresponse is drawn as an in-progress wait honouringretry_after, never as a failure and with no retry button, so a malformedservice_namewould sit on screen as "waiting" indefinitely with its real cause unreachable.One thing the suppression must keep, and this issue is where it is kept. A limited pre-derivation invocation writes no audit records, but it still increments the coalescing counter this issue drains into a counted record, and the reader's amended anomaly definition still counts it — RFC-F §5.6 puts it as "one coalesced counter increment rather than a durable record". Dropping the increment along with the records would make a flood least visible exactly while it is worst, and these records are the only detection the RFC's "detected, not prevented" argument rests on.
Both rulings are now bookkeeping rather than open questions: what remains is the decision issue's RFC amendment carrying them, which this issue's dependency list already gates on. This subsection is deleted when that lands.
Scope
The coalescing window and the counted record
(client identity, verb, limited_bucket)key increment one counter and produce exactly one counted record, written when the window closes. Flush any open window on daemon shutdown so a counted record is never lost to a restart.(identity, verb, bucket)behind a caller's ability to create keys. A window whose deadline has passed is written on the next tick; the resulting record'swindow_endis the deadline, not the tick that noticed it, so a slow tick does not stretch the window a record claims to cover.window_endat the instant it closed, not at the deadline it never reached. The tick bullet above fixeswindow_endat the deadline for every window closed at or after it, including one a slow tick noticed late. The two closes that can precede the deadline — an overflow flush when the open-window map hits its cap, and the shutdown flush — take the flush instant instead, because the window stopped accepting invocations there and a record must never claim a span over which it counted nothing. Sowindow_end - window_startis at most the configured window in every case, and is always exactly the span the record'scountcovers.rate_limit_coalesce_window_secondsto the[registrar]configuration table —u32, default60, validated against zero, because a zero window makes every suppressed invocation its own record, which defeats the coalescing. A negative or non-integer value produces a named configuration error identifying the key, in the same shape as the table's other validation failures; unrelated keys in the table survive the load untouched.limited_bucketis the bucket that had no token — the sibling issue's two-value enum,predecision_refusaloradmission, reused rather than redeclared. It is deliberately not a refusal class: at neither check point has a refusal been classified, and at the admission check point the invocation might have been a perfectly good mint. Naming the field for the bucket says exactly what is known and nothing more, and it makes the two cases legible to an operator:predecision_refusalmeans those callers received their real permanent refusal and only the records were suppressed,admissionmeans those callers were throttled and never attempted.record_version— integer,1, unchanged.phase—"limited".ts— the instant the record was written, RFC 3339 UTC at the store's existing millisecond precision, like every other record's. It is deliberately not the same value aswindow_end: a window closed on a later tick is written after its deadline, and carrying both is what lets a reader see that lag instead of inferring it.verbandcaller_identity— the two identity fields of the coalescing key, spelled and bounded exactly as the store already spells and bounds them. The client identity iscaller_identity; do not introduceclient_identityas a second name for it.limited_bucket—"predecision_refusal"or"admission", the coalescing key's third field, a Rustenumlike every other enum-valued field in these types.count— integer, at least1: how many limited invocations this record coalesces.window_startandwindow_end— RFC 3339 UTC at the same precision asts.truncated— exactly as on any other record, present only whencaller_identitywas capped.request_id,requestedandregistration_id— omitted entirely. A counted record covers many invocations, so no per-invocation id identifies it, and at the pre-derivation check point nothing was parsed that could fillrequestedorregistration_id. Omit them rather than writingnull, an empty string or a synthesized id — the same omission discipline the store already applies toregistration_id,outcomeandtruncated.outcome— absent. That field is present only on the outcome phase, and a counted record has no outcome class.Optionfields skipped when absent, the wayoutcome,registration_idandtruncatedalready are — rather than declaring a second record struct that happens to serialize compatibly.phasevariant — RustLimited, serialized"limited"by the store's existing#[serde(rename_all = "snake_case")]— inside the audit-record store's ownserdetypes, at the samerecord_version.limitedis the word Context defines precisely (the limiter suppressed this invocation's records, at either check point) and the wordlimited_bucketis already named for, so the on-disk value says what the line is without a second vocabulary. That store shipsphaseas an enum with exactly two values precisely so this record can be added as a third, and it forbids the catch-all arm that would let a new variant pass unnoticed — so extend those types rather than declaring a parallel shape, and do not introduce a second file, a second directory, a second encoding or a newrecord_version. Bumping the version would be the worst option available: the store's reader rejects an unrecognised version as a malformed line, so every counted record would land in the count that means "someone is forging records", which is the opposite of what this record says.truncatedentry carrying the full value's SHA-256 and byte length. A counted record must fit the store's declared maximum record size, whose arithmetic assumes every attacker-influenced field is capped. It has exactly one such field where an intent record has five, and everything else it carries is a bounded integer, timestamp or enum, so it fits with room to spare — the criterion below still asserts that rather than leaving it assumed.tracingand continue. Do not reach for a best-effort or skip mode on the record store to express this: that store has none by design, and adding one would let suppression leak into the verb path the rule protects. Write the counted record through the same fallible API every other record uses and handle its error here.The reader's two classification rules
Adding the variant breaks the record reader's exhaustive matches, and fixing them is this issue's work, not a follow-up. The compile error is the intended mechanism — it is why
phaseis an enum — and it must be resolved by stating this record's classification in both of the reader's counters, not just one:phase: a"limited"line that omitsrequest_id,requestedandregistration_idis complete for its variant, not short. And its duplicate-request_idrule cannot reach a counted record at all, because a counted record carries norequest_id— so it can neither be counted as a duplicate itself nor make a genuine intent/outcome pair look like one.Leaving either arm to a default would report a working limiter as an attack in progress, on the one artifact the whole detection argument rests on. This is the only amendment to the record store's behaviour: its rule that the pre-derivation arm writes both phases under its global lock continues to hold for every invocation the limiter admits.
The caller-facing throttle — the admission check point only
matcharm for the newly added verb-layer variant — the arm that issue's exhaustive, catch-all-free mapping forces the moment the variant appears — together with any fixture extension the arm's real payload now makes possible. If the pre-recorded assignment turns out to disagree with what the reference spells, the reference wins and the disagreement is a finding against the wire-contract issue, not something to resolve by spelling it differently here.docs/reference; the semantics come from the sibling limiter issue. That reference transcribes an externally owned error set and is authoritative over every identifier and payload name in it — the design prose renders this error asRegistrarBusy { retry_after }in one place andretry_after_secondsin another, and the reference settles which. Ship what the reference spells;retry_after_secondsis the fallback only where it spells nothing. Pin whichever spelling lands with a golden fixture.The health member
registrar_health, a container of named members on the endpoint's responses, reserved by the endpoint protocol issue, which also owns the rule about which responses carry it. This issue adds exactly one member,limiter. Always present; its values are counters, not scan results, so serving it costs no store read.limited_bucketexists on the counted record precisely because the two cases mean different things to an operator —predecision_refusalmeans callers received their real permanent refusal and only the records were suppressed,admissionmeans callers were throttled and never attempted — and those warrant opposite responses: the first says someone is flooding malformed input, the second says the control plane itself is being held back and a bring-up may be stalling. Publishing their sum would put the less actionable of the two on the wire while leaving the actionable one in a store on the bootroot host that the console cannot read. Solimitercarrieslimited_predecision_refusalandlimited_admission, bothu64, both the count since daemon start, read from the sibling issue's in-process counters. Two named fields keep the member bounded and fixed-shape as the container requires — the bucket enum has exactly two values and gaining a third is a compile error there, not a silent widening here.certificatesmember the certificate work populates, or theaudit_capacitymember the reserved-store work adds. None of those issues depends on this one, so whichever lands last must be additive; they also pin the encoded messages with golden fixtures, so a reshape here surfaces as a fixture diff and a caller-visible wire change.limiter, and shape it so a sibling adds its member without touching this one. If a holder already exists, use it and add nothing. Three independently written holders refreshed on three cadences is the outcome to avoid: the container is one wire object and would then be assembled from values of three different ages with nothing on the wire to say so.docs/protocol reference's local half with thelimiterschema.limited_bucketmeans, and what the throttle identifier means for a caller, in bothdocs/en/anddocs/ko/, on the existing pages beside the audit-record documentation, with nomkdocs.ymlnav change.Acceptance criteria
(client identity, verb, limited_bucket)key: a test asserts the record store does not grow proportionally to the flood, and that records for the two buckets do not merge.record_version1,phase"limited",ts,verb,caller_identity,limited_bucket,count(≥ 1),window_startandwindow_end, plustruncatedonly when the identity was capped — carries no per-invocation refusal reason, and omitsrequest_id,requested,registration_idandoutcomeentirely rather than emitting them asnull,""or a synthesized value.phasevariant in the audit-record store's ownserdetypes, serialized"limited", at the unchangedrecord_version, written to its existing store: no second file, directory, encoding or version appears, and a golden fixture pins the variant's serialized bytes including thatphasevalue.window_endis the flush instant rather than the deadline it never reached.rate_limit_coalesce_window_secondsloads with its default, is rejected at zero with an error naming the key, and produces the same named error for a negative or non-integer value; unrelated[registrar]keys survive the load.caller_identityobeys the store's field bound: an over-bound identity is carried as a leading prefix of at most 512 bytes, cut on a UTF-8 character boundary — so a multibyte character straddling the cap is dropped rather than split, and the prefix may be shorter than 512 bytes — plus atruncatedentry whose SHA-256 and byte length are those of the complete original value, and the record fits the store's declared maximum record size.request_id,requestedandregistration_idraise no missing-required-field verdict, and that many counted records with norequest_idraise no duplicate-request_idverdict — while a genuine crash-mid-mint and a genuinely unparseable line are each still counted by the right one.predecision_refusalbucket and asserts the caller still receives its real permanent refusal on the wire, with its records suppressed. This is the criterion that fails if the wire mapping is widened to cover both check points.registrar_health.limitercarrieslimited_predecision_refusalandlimited_admission, bothu64, wherever the health container is carried; a test floods each bucket separately and asserts only that bucket's counter moves, so a collapsed sum would fail. The member round-trips alongside any sibling member and leaves every sibling member and entry unchanged.limiter.window_endis the window's deadline rather than the tick that noticed it.window_endat the flush instant, so its bounds are shorter than the configured window.docs/en/anddocs/ko/, on the existing pages, with nomkdocs.ymlnav change.cargo clippyis warning-free andcargo fmt --checkpasses.Constraints
record_version, and do not add a best-effort or skip mode to the record store.caller_identity,verb,ts,truncated), and do not fill an omitted key withnull, an empty string or a synthesized value. Thephasevalue is"limited"; a golden fixture blessing some other spelling an implementation happened to emit is a defect, not a decision.docs/protocol reference transcribes from the externally owned contract; mirror it.registrar_healthcontainer, do not decide its placement, and do not touch a sibling member's shape, contents or ordering.matcharm that produces it.limited_bucketfor a reason and the health member must preserve it.unwrap()in production code; no[]indexing.Out of scope
rate_limit_*bucket keys, the sizing formula and the retry-after arithmetic. All the sibling limiter issue's.audit_record_diritself — all owned by the audit-record issues; this issue adds one record variant in that existing encoding and amends the two classification rules for it, and changes nothing else there.audit_store_*configuration keys, the store directory layout and its ownership, relocatingaudit_record_dironto it, the Compose override that moves the OpenBao audit device, the capacity probe, the headroom arithmetic, the low-water alarm and theregistrar_health.audit_capacitymember.(component, host)ceiling on how many instances a deployment may legitimately run.Test plan
limited_bucket.serdetypes at the unchangedrecord_version, itsphaseserializes as"limited", the omitted keys (request_id,requested,registration_id,outcome) are absent from the emitted line rather than present-and-empty, a golden fixture pins its bytes, and no second file, directory or encoding appears.caller_identitywhose byte 512 falls inside a multibyte character is cut on the character boundary below the cap, carries atruncatedentry whose digest and length are the complete original value's, and leaves the record inside the declared maximum record size.rate_limit_coalesce_window_secondsloads with its default; zero, negative and non-integer values each fail validation with an error naming the key; unrelated[registrar]keys survive.registrar_health.limitercarries both per-bucket counters wherever the container is carried, that flooding one bucket moves only its own counter, that the member round-trips alongside a sibling member, and that every sibling member and entry is unchanged.registrar_healthcontainer serialized on a response is read from the shared daemon-refreshed holder rather than computed per request — moving the limiter's counters without letting the daemon tick leaves the served container at the previously refreshed values, and it changes on the next tick — and no holder private tolimiterexists.window_endat the deadline; the open-window map driven past its cap stays bounded and flushes the oldest window early withwindow_endat the flush instant rather than dropping or refusing a key; and the shutdown flush likewise stampswindow_endat the flush instant.tempfile::tempdir()and never a fixed path.Dependencies
Depends on the group's limiter bucket-shape decision issue, which carries two rulings, both now settled in favour of what this body already specifies: the bucket key as per-
(client identity, verb, bucket)buckets over the two-value enum reused below aslimited_bucket, and the caller answer as the throttle mapped from the admission check point only, with a limited pre-derivation invocation keeping its real permanent refusal. Neither re-specifies anything here. What this dependency gates on is the RFC amendment that issue must land carrying both, since until it does the accepted document still states two things this group's code will not deliver. Depends on the sibling limiter issue for the two-value bucket enum, the limited-invocation sink this implements, the in-process limited-invocation counter and the typed retryable throttle outcome this maps onto the wire. Depends on the audit-record store for itsserdetypes, its JSON Lines encoding, itsphaseenum andrecord_version, its attacker-influenced-field bound and its golden fixtures; and on the store's reader, because the two classification rules amended here are defined there and adding aphasevariant breaks that reader's matches at compile time. Depends on the registrar endpoint's wire protocol issue for theregistrar_healthcontainer, the fixtures and the error identifier set the throttle joins. Part of the registrar verb rate-limiting umbrella.Pointers
src/daemon.rs::run_daemon:68— the daemon loop the coalescing window's close and shutdown flush hang offsrc/config.rs— the configuration load and validation shaperate_limit_coalesce_window_secondsfollowsdocs/— the checked-in protocol reference whose transcribed half spells the throttle identifier and whose local half gains thelimiterschemadocs/en/operations.md:95anddocs/ko/operations.md:96— the audit-logging sections this documentation sits besidedocs/rfcs/0001-registrar-role-and-non-self-propagation.md§5.6 and §6