Prevent concurrent redemption of rotating OAuth refresh tokens

[zackbart]

Motivation

Downstream OAuth connections created under different request scopes can independently observe and redeem the same expired refresh token. An authorization server that rotates refresh tokens may accept the first redemption and reject the second with invalid_grant; some providers also revoke the newly issued credential after reuse is detected.

Connecta's credential-generation fence prevents an older authorization flow from publishing over a newer generation. It does not establish ownership of an ordinary token refresh before the authorization server sees it.

Executor encountered this failure across concurrent MCP sessions and moved refresh ownership above the individual execution stacks: UsefulSoftwareCo/executor#1537. Connecta should establish its own guarantee without adopting Executor's session model.

Behavioral requirements

• For one connector credential generation within one registry runtime, concurrent callers must share one refresh attempt rather than redeeming the same refresh token independently.
• Every joined caller must observe the resulting credential or the same bounded failure.
• A failed refresh must release ownership so a later call can retry.
• A force reauthorization or issuer change must continue to invalidate work captured under an older generation.
• One caller's cancellation or deadline must not corrupt the credential or strand another caller that joined the refresh.
• The guarantee must remain honest about its coordination boundary. KVStorage has no atomic compare-and-set operation, so this issue does not claim exactly-once refresh across processes or Worker isolates.

Acceptance criteria

• A regression test starts two OAuth-backed downstream connection attempts under distinct request scopes with the same expired credential.
• The fake authorization server rotates refresh tokens and rejects a second redemption of the retired token.
• The test observes exactly one refresh grant within a registry runtime and both callers can continue with the resulting credential.
• Tests cover refresh failure followed by retry, cancellation of one joined caller, and a concurrent generation change.
• Existing generation-fence and force-reauthorization tests continue to pass.
• Downstream OAuth documentation states the runtime-local guarantee and the unresolved cross-runtime boundary.
• npm run check passes.

[zackbart]

Discovery: confirmed correctness bug

The current ownership boundaries permit the race:

• remoteMcp() keeps connecting and KvOAuthProvider inside the ConnectionState keyed by request scope.
• Two request scopes therefore have separate connection promises and provider instances over the same connector storage.
• MCP SDK v2 reads provider.tokens(), calls refreshAuthorization(), then calls provider.saveTokens() without coordination with another provider instance.
• On invalid_grant, the SDK calls invalidateCredentials("tokens") and retries the authorization flow.

I ran a deterministic reproduction using the installed MCP SDK's real auth() flow and two current KvOAuthProvider instances over one memoryStorage() instance. The fake authorization server held the first response until both callers had presented refresh-old, returned rotated credentials to the first caller, waited for that save, then returned invalid_grant to the second caller.

Observed result:

{
  "refreshRequests": 2,
  "results": [
    { "status": "fulfilled", "value": "AUTHORIZED" },
    { "status": "fulfilled", "value": "REDIRECT" }
  ],
  "storedTokens": null
}

The losing flow deleted the winning flow's newly rotated token before falling back to interactive authorization. Generation fencing does not prevent this because both flows belong to the same valid generation.

Narrowed boundary

The first guarantee should be one refresh owner per connector generation within one remoteMcp()/registry runtime. Provider-instance or request-scope ownership is too narrow. Cross-process and cross-isolate exactly-once refresh remains outside this issue because the public storage seam has no atomic coordination operation.

The regression should preserve three distinct properties:

1. concurrent ordinary refreshes join;
2. a generation change still fences the result; and
3. a failed shared refresh releases ownership for a later attempt.

No repository files were changed during this reproduction.

[zackbart]

Implementation verification

Updated after rebasing the implementation onto current origin/main at ce7058c (0.21.1). The earlier stale-base verification numbers are superseded by these results.

Implemented runtime-local refresh ownership per connector OAuth generation, including rotating, non-rotating, and byte-identical tokens; owner/follower cancellation; storage failure; stale-write fencing; SDK retry classification; and the identified TOCTOU/ABA windows. The port also preserves the current request-scope provider cache and awaits non-refresh passthrough failures so workerd keeps them attached to the SDK request.

Current-tip verification:

• focused OAuth suite: 77/77 in Node and 77/77 in workerd;
• full npm run release:check: 116 test files, 2,785 passed, 40 skipped;
• docs, operator UI, TypeScript, Oxlint, Knip, build, examples, production audit, and package smoke passed;
• production audit: 0 vulnerabilities;
• Docker unavailable, so generated-container smoke was skipped.

The documented boundary remains runtime-local. Cross-process and cross-isolate exactly-once refresh still requires a storage CAS/lease primitive that KVStorage does not provide.

[zackbart]

Shipped in v0.21.2. The trusted publish workflow passed and npm latest now resolves to 0.21.2.