Follow-up from #4373 (node-auth bearer JWTs, #355).
Problem
NodeJwtValidator::validate runs the full verification path on every request carrying a bearer token: x5c chain path-building against the root CA, the chain signature check, then a second P-256 verification of the JWT signature itself, plus DER parsing.
Measured on the merged implementation — release build, single-threaded, a valid token against a one-deep test PKI:
228 µs per token, ~4,400 validations/sec per core.
The comparison that matters is against what it replaces. mTLS pays a similar cost once per connection, and gRPC channels are long-lived, so the cost amortizes to near zero per call. Bearer auth pays it per call.
Rough scale for a site:
| Fleet |
Requests/DPU/s |
Auth overhead |
| 1,000 DPUs |
1 |
~0.23 cores |
| 1,000 DPUs |
10 |
~2.3 cores |
Not breaking anything today — node-auth is off by default and no site has enabled it at scale — but it grows linearly with fleet size and request rate, and it lands entirely on the API.
Why this is worth caching
The workload is unusually favourable: tokens have a 5-minute TTL and clients re-mint only when under 60 s remain, so the same token string arrives hundreds of times. Every one of those repeats redoes both signature verifications from scratch.
A cache keyed by a hash of the token, storing (spiffe_uri, exp), reduces a repeat hit to a hash plus a map lookup — on the order of a 200x reduction on the hot path.
Constraints
Two things the implementation has to get right, both because this sits on the authentication path:
- Only cache successful validations. Otherwise a flood of garbage tokens grows the map unboundedly. Successful entries are naturally bounded by fleet size (one live token per machine).
- Clear the cache in
refresh_roots. The validator reloads its trust anchors when the client CA rotates; without invalidation, tokens chaining to a rotated-out CA would keep passing until their own exp.
Never extend an entry beyond its own exp and the cache cannot make a token outlive its natural lifetime — the security bound is unchanged, only the recomputation is skipped.
Scope
Contained to crates/api-core/src/node_auth.rs plus tests. Worth a test that a rotated CA invalidates cached entries, since that is the failure mode with real consequences.
Follow-up from #4373 (node-auth bearer JWTs, #355).
Problem
NodeJwtValidator::validateruns the full verification path on every request carrying a bearer token: x5c chain path-building against the root CA, the chain signature check, then a second P-256 verification of the JWT signature itself, plus DER parsing.Measured on the merged implementation — release build, single-threaded, a valid token against a one-deep test PKI:
228 µs per token, ~4,400 validations/sec per core.
The comparison that matters is against what it replaces. mTLS pays a similar cost once per connection, and gRPC channels are long-lived, so the cost amortizes to near zero per call. Bearer auth pays it per call.
Rough scale for a site:
Not breaking anything today — node-auth is off by default and no site has enabled it at scale — but it grows linearly with fleet size and request rate, and it lands entirely on the API.
Why this is worth caching
The workload is unusually favourable: tokens have a 5-minute TTL and clients re-mint only when under 60 s remain, so the same token string arrives hundreds of times. Every one of those repeats redoes both signature verifications from scratch.
A cache keyed by a hash of the token, storing
(spiffe_uri, exp), reduces a repeat hit to a hash plus a map lookup — on the order of a 200x reduction on the hot path.Constraints
Two things the implementation has to get right, both because this sits on the authentication path:
refresh_roots. The validator reloads its trust anchors when the client CA rotates; without invalidation, tokens chaining to a rotated-out CA would keep passing until their ownexp.Never extend an entry beyond its own
expand the cache cannot make a token outlive its natural lifetime — the security bound is unchanged, only the recomputation is skipped.Scope
Contained to
crates/api-core/src/node_auth.rsplus tests. Worth a test that a rotated CA invalidates cached entries, since that is the failure mode with real consequences.