fix(platform): revoke the gateway key on every session teardown - #3213
Merged
Conversation
A sandbox session's LLM-gateway virtual key outlived the session. The gateway has no native TTL on a virtual key, and `mintVirtualKey` gives every key a `reset_duration: '1M'` budget window on the assumption that teardown deletes it long before any reset matters. A key that survives teardown is therefore a permanent bearer credential with a self-refilling monthly allowance against the organization's own provider keys. ## One seam, every teardown edge `revokeSessionGatewayKeys` is the 0.5 twin of 0.4's `node_only/sandbox/session_teardown.ts` plus the revoke half of `session_admin_actions.destroySandbox`. All four edges route through it — TTL expiry, admin destroy, phantom heal, and a deadline-failed agent run — so they cannot drift apart. The election is the token flip. `UPDATE … WHERE revoked_at_ms IS NULL RETURNING llm_gateway_key_id` hands each live key to exactly one caller, so a watchdog sweeping the same session twice, or a destroy racing the expiry sweep, revokes once. `revokeVirtualKey` also treats a 404 as success, so even a genuine double DELETE cannot fail a teardown. Without `execId` the whole session is reclaimed. With one — a deadline-failed task-agent run — only that exec's minted key is taken, so a sibling turn still running on the same `pa-<agentId>` session keeps its credential. ## Failure posture Best-effort per key, as in 0.4. An unreachable gateway must never wedge a teardown, so the HTTP failure is caught — at `console.error` with the key id, because that key stays spendable and only an operator can delete it. ## Tests `gateway-keys.test.ts` covers the glue in CI; the SQL election stays the harness's job, since a mocked `sql` cannot prove a race. | Mutation | Went red | | --- | --- | | let a gateway failure throw | `never throws when the gateway fails`, `keeps revoking the remaining keys after one fails` | | clear the session row even when exec-scoped | `leaves the session row alone for an exec-scoped teardown` | `checkSandboxGatewayKeyReclaim` drives every edge against a fake gateway that records the DELETEs. It registers as a lane in the harness runner, so a throw is reported as a failed check naming the lane rather than truncating the run silently.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A sandbox session's LLM-gateway virtual key outlived the session. The gateway
has no native TTL on a virtual key, and
mintVirtualKeygives every key areset_duration: '1M'budget window on the assumption that teardown deletesit long before any reset matters. A key that survives teardown is therefore a
permanent bearer credential with a self-refilling monthly allowance against
the organization's own provider keys.
Replaces #3194, rebuilt on current main.
One seam, every teardown edge
revokeSessionGatewayKeysis the 0.5 twin of 0.4'snode_only/sandbox/session_teardown.tsplus the revoke half ofsession_admin_actions.destroySandbox. All four edges route through it — TTLexpiry, admin destroy, phantom heal, and a deadline-failed agent run — so
they cannot drift apart.
The election is the token flip.
UPDATE … WHERE revoked_at_ms IS NULL RETURNING llm_gateway_key_idhands each live key to exactly one caller, so awatchdog sweeping the same session twice, or a destroy racing the expiry
sweep, revokes once.
revokeVirtualKeyalso treats a 404 as success, so evena genuine double DELETE cannot fail a teardown.
Exec scope matters. Without
execIdthe whole session is reclaimed. Withone — a deadline-failed task-agent run — only that exec's minted key is
taken, so a sibling turn still running on the same
pa-<agentId>sessionkeeps its credential.
The unit coverage is new
The original PR proved this only through
integration-check.ts. That checkneeds
SANDBOX_LLM_GATEWAY_ADMIN_PASSWORD, and without it every revoke failsauthentication — so the check exercised its own fail-open lane and proved
nothing until that was fixed. Meanwhile a 487-line new module had no test in
CI at all.
gateway-keys.test.tscovers the glue, where the consequences are as sharpas in the SQL:
never throws when the gateway fails,keeps revoking the remaining keys after one failsleaves the session row alone for an exec-scoped teardownThe first is the one that matters most: an unreachable gateway wedging a
teardown is worse than a leaked key, so the failure is counted and logged —
at
console.error, notwarn, because the key stays spendable and only anoperator can delete it by hand.
The SQL election itself stays the harness's job; a mocked
sqlcannot provea race.
Gate
typecheck0 errors,oxlint --type-awareclean,oxfmt --checkclean,sandbox + tasks suites 24 passed (5 files).
Rebuilt rather than rebased
Main restructured the harness runner while this sat open: the flat sequence of
await checkX(...)calls is now aLanelist with truncation reporting and ashared-session liveness assertion after each lane. Rebasing produced a
500-line conflict across that restructure.
So this is the same change re-applied on top: the five source files as they
were, and
checkSandboxGatewayKeyReclaimregistered as a lane in the newstructure rather than appended to a list that no longer exists. A throw in it
now reports as a failed check naming the lane instead of truncating the run.
The
watchdogs.tsdocblock conflict was resolved by taking main's text — itgained the FAIR-walk detail and the whole RECLAIM bullet — and re-applying
only my clause about the expiry pass revoking gateway keys.
Both mutations re-verified after the rebuild.