Skip to content

Cache validated node-auth JWTs to cut per-request verification cost #4388

Description

@wminckler

Follow-up from #4373 (node-auth bearer JWTs, #355).

Problem

NodeJwtValidator::validate runs the full verification path on every request carrying a bearer token: x5c chain path-building against the root CA, the chain signature check, then a second P-256 verification of the JWT signature itself, plus DER parsing.

Measured on the merged implementation — release build, single-threaded, a valid token against a one-deep test PKI:

228 µs per token, ~4,400 validations/sec per core.

The comparison that matters is against what it replaces. mTLS pays a similar cost once per connection, and gRPC channels are long-lived, so the cost amortizes to near zero per call. Bearer auth pays it per call.

Rough scale for a site:

Fleet Requests/DPU/s Auth overhead
1,000 DPUs 1 ~0.23 cores
1,000 DPUs 10 ~2.3 cores

Not breaking anything today — node-auth is off by default and no site has enabled it at scale — but it grows linearly with fleet size and request rate, and it lands entirely on the API.

Why this is worth caching

The workload is unusually favourable: tokens have a 5-minute TTL and clients re-mint only when under 60 s remain, so the same token string arrives hundreds of times. Every one of those repeats redoes both signature verifications from scratch.

A cache keyed by a hash of the token, storing (spiffe_uri, exp), reduces a repeat hit to a hash plus a map lookup — on the order of a 200x reduction on the hot path.

Constraints

Two things the implementation has to get right, both because this sits on the authentication path:

  • Only cache successful validations. Otherwise a flood of garbage tokens grows the map unboundedly. Successful entries are naturally bounded by fleet size (one live token per machine).
  • Clear the cache in refresh_roots. The validator reloads its trust anchors when the client CA rotates; without invalidation, tokens chaining to a rotated-out CA would keep passing until their own exp.

Never extend an entry beyond its own exp and the cache cannot make a token outlive its natural lifetime — the security bound is unchanged, only the recomputation is skipped.

Scope

Contained to crates/api-core/src/node_auth.rs plus tests. Worth a test that a rotated CA invalidates cached entries, since that is the failure mode with real consequences.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions