Repository navigation
feat(control-api): push authorization and session changes over the GFS watch and retire the per-user entity-change stream - #1502
Open
zach88 wants to merge 8 commits into
Conversation
…FS watch Add migrations 0140-0145, one per table so each transaction holds a single table lock and cannot deadlock against an application writer: - gfs_authz_changed: one pg_notify per key on gfs_grants, gfs_shares, gfs_desktop_operator_links, control_admin_users and team_members. Subject keys carry the id only for user/team subjects of uuid length. - gfs_watch_session: users UPDATE (WHEN lifecycle changed) and DELETE. The migration runner retries a lost lock wait (55P03) for descriptors that declare lockRetries (validated to 0..3); only team_members and users do. Add total parsers parseAuthzKey and parseSessionHint. Listening on the new channels comes with their handlers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nges Listen on gfs_authz_changed and close affected streams with reauthorize: - a 250 ms authorization window collects direct user keys and expands team and admin keys with one query per kind on the watch pool, filtered against live connections at flush time; - a failed expansion is retried once; a second failure, an unreadable or throwing revocation hint, or more than 10,000 ids in a window fails closed: every connected user is held at once and reauthorized, at most once every 5 s per replica, spread over the closes; - a move or soft delete of an ancestor reauthorizes user connections whose accepted keys sit on that chain; linked operators keep their access; - listener loss and shutdown drop pending windows and release every hold. Tests run against real Postgres with the real triggers and writers, plus a model-based property test. The shared harness now finds the listener backend by its ping as well as its LISTEN, so the registry property test actually induces listener loss instead of skipping it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… watch
Roots: a change visible to a root listing (h=1) sends roots_changed {} to
the connected direct grantees with read (one query per 250 ms window, with
subject_id compared as text) and to every linked operator, at most once
every 2 s per connection with a trailing notice. Inserts skip the grantee
query. A held connection never receives it.
Session: listen on gfs_watch_session and treat each NOTIFY as a hint. An
affected user is held, the window confirms against users in one query and
closes with session_expired only when the row is gone, retired or carries
another lifecycle_version; a failed or unreadable confirmation closes with
reauthorize instead. A forged hint over an active row closes nothing.
Listener loss and shutdown cancel both windows and release every hold.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Anchor the 6 s suspicion window at the op's real start instead of at the instant the test sent the command. A delta queued behind an op in flight, or an open still authenticating, starts later; once more than 6 s have passed since the move its snapshot of recent ancestry changes is rightly empty, and the model demanded a suspicion the registry has no reason to raise. The start is the admission instant for an open (FakeSocket.admittedVt) and srvDeadline minus the delta deadline for a delta, read while the op is still the connection's. Check S4 on every decision, closed streams included. It was only checked on connections still ready at rest, so a suspect acceptance on a stream that later closed was never evaluated: a registry ignoring recentDirty, or with suspicion disabled in finishOp, passed T-M2. Whether a key's chain holds P comes from the fixture, since the indexed chain is gone once a stream closes. Class-check: windows anchored at a command's instant instead of its op's effect, in test/*property*.ts (grep "clock.now()" and "vt -") -> 2 sites, both in this file (openVt, puts[].vt) -> both fixed; the other oracles anchor at effect instants. Decision checks gated on "conn.phase === 'ready'" at rest in checkModel -> 2 (S4 and the I6 read check) -> S4 fixed; the I6 read check stays ready-only, left as a follow-up. Repro-test: control-api/test/gfsWatch.registry.property.realPostgres.integration.test.ts:178 (fails at ff18a96 with seed -1676849103 path "11", 3/3, and seed 1022089507 path "22", 2/3; passes 3/3 each with the fix). Negative control at :477: a registry ignoring recentDirty or with suspicion disabled in finishOp passes at ff18a96 and fails on I6/S4 with this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The GFS watch (`/external/gfs/watch`) now carries live entity changes, authorization and session changes for users, so the per-user 4-second entity-change stream is removed end to end: - control-api: drop `/external/entity-changes/stream`, its per-user open limiter and `entityChangeUserVisibilityRefreshMs`; `streamEntityChanges` becomes operator-only with the signature `(req, res, initialCursor, isAuthorized, principalId)`. The operator stream (`/gfs/entity-changes/stream`) keeps its limiters, admin auth and periodic recheck; `principal_kind` stays as a constant label. - external-rest-api: drop `/entity-changes/stream` and its tests. - Tests: session-currentness coverage rewritten against the observer and the watch; the stream-open fail-closed case now targets the watch and asserts the 503 on response headers (shared `openWatch` helper); control-ui GfsBrowser tests follow the new signature. New retirement tests pin 404 on both retired routes and a live operator stream. The desktop client still calls the retired route and moves to the watch in the follow-up desktop change; both must land together. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…treams included The I6 read check (a key may be accepted only after its op read the resource as a 2xx under `allow`) and the I6/I11 kind check were evaluated only on connections still ready at rest. An improper acceptance on a stream that later closed (a listener loss, a reauthorize close, the client leaving) was never evaluated: a registry that accepted a key after a deny or an unavailable read and then closed the stream passed T-M2. Both checks now run, next to S4, over every decision frame of every connection, with the attribution S4 already used: a key a frame accepts that the previous frame did not was decided by that frame's op (seq 0 = the open, otherwise the put's requestId). That holds in this model because a key is only accepted by the op that added it, and the one op that writes no frame (a retained user's delta) belongs to a stream that only leaves retention by closing. The ready-only block keeps I19/I21 and a single rest check: every key accepted at rest is in the last decision frame. The read rule changes from "the op's last read of the resource was allow" to "one of the op's reads of it was allow". gfsc sees the resource, not the key, so an op holding dir:P and file:P reads P twice; if P's decision turns to deny between the reads, dir:P is rightly accepted with a deny as the op's last read of P. The old rule flagged that legitimate case (the head oracle fails on it too, 2/2, while the connection is still ready). It is pinned as a third fixed example; with a single read per resource the two rules are the same. Class-check: decision checks gated on "conn.phase === 'ready'" at rest in test/*property*.ts (grep "phase !== 'ready'" / "phase === 'ready'") -> 2 sites in this file: checkModel (I6 read, I6/I11 kind, I19, I21) and connAt (command targeting, not an oracle) -> I6 read and kind moved to every decision; I19/I21 and the accepted-in-last-frame check stay ready-only (they are about rest state); connAt is not a check. "last read decides" (grep "reads.at(-1)") -> 1 site -> replaced as above. The other property suites have no I6 read check. Repro-test: control-api/test/gfsWatch.registry.property.realPostgres.integration.test.ts:491 (negative control: a registry that accepts on a non-principal 403 and closes the stream with 'reauthorize' after the frame passes 3/3 x 202 runs at 6cea012 and fails with this change, seed -1989312804 path "5"; the same with an unavailable (503) read: passes 3/3 at 6cea012, fails 2/2, seeds -1292543513 path "16" and -713675540 path "6". Dual-read example at :620 fails 3/3 with the last-read rule and passes 3/3 with this one.) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ubscriptions-authz Brings origin/dev (22dd421) through the synced parent branch. Conflicts: control-api package.json/package-lock.json version only, kept the higher version; external-rest-api releaseManifest.ts, took the v0.12.1 release coordinates from dev and kept this branch's externalRestApiVersion, which tracks external-rest-api/package.json. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ubscriptions-authz
This was referenced Nov 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Completes the server side of the GFS watch: authorization and session changes are pushed to open streams, and the per-user 4-second entity-change stream is retired. Control-api and ERA, plus test updates in control-ui.
0140to0145, one per table):gfs_authz_changed: onepg_notifyper key ongfs_grants,gfs_shares,gfs_desktop_operator_links,control_admin_usersandteam_members. Subject keys carry the id only for user/team subjects of uuid length.gfs_watch_session:usersUPDATE (when the lifecycle changed) and DELETE.55P03) for migrations that declarelockRetries(validated to 0..3); onlyteam_membersandusersdo.parseAuthzKeyandparseSessionHint.roots_changed {}to connected users with read through a direct or team grant (one query per 250 ms window) and to every linked operator, at most once every 2 s per connection with a trailing notice. Inserts skip the grantee query. A held connection never receives it.gfs_watch_sessionNOTIFY is a hint: the affected user is held, the window confirms againstusersin one query, and closes withsession_expiredonly when the row is gone, retired or carries anotherlifecycle_version. A failed or unreadable confirmation closes withreauthorize. A forged hint over an active row closes nothing./external/entity-changes/stream, its per-user open limiter andentityChangeUserVisibilityRefreshMsare removed.streamEntityChangesbecomes operator-only. The operator stream (/gfs/entity-changes/stream) keeps its limiters, admin auth and periodic recheck./entity-changes/streamis removed.Why
The per-user stream is a security fix that became the refresh mechanism: it sends a generic invalidation every 4 seconds by default with no ids, plus a session check per tick, so users are isolated from the timing of the global feed. With the watch carrying per-resource changes, revocations and session changes, that stream has no remaining job, and keeping it would leave two paths with different guarantees.
How
reauthorize, which makes the client reopen and re-authorize from scratch. Holds are released only by iterating their own structure.usersbefore anysession_expired.Testing
All server suites run against real Postgres with the real triggers and writers.
gfsWatch.authzSessionTriggers.realPostgres: one NOTIFY per key for every writer of every table, payload parsing, triggerWHENconditions.gfsWatch.authzWindow.realPostgres: expansion per kind, retry, fail-closed (second failure, unreadable hint, id ceiling) with its rate limit, ancestor chain revocation, linked operators.gfsWatch.rootsSession.realPostgres: root notices, their rate limit and trailing notice, session confirmation outcomes, forged hints.gfsWatch.authz.property.realPostgresandgfsWatch.rootsSession.property.realPostgres(fast-check commands and scheduler against a reference model). The registry property test now checks the read and kind of every decision on every connection, closed streams included, and anchors the suspicion window at the op's real start.migrationExecutionPolicy:lockRetriesvalidation and the retry on55P03.entityChangeStreamRetirement.realPostgresandroutes.entityChangesRetired(ERA): both retired routes answer 404 and the operator stream stays live.externalSessionCurrentness.realPostgresrewritten against the session observer and the watch.tsc --noEmitclean;npm test635 files / 8961 tests passed (real-Postgres suites skip in this lane); real-Postgres run of the watch suites (gfsWatch*,gfs.externalGfsRead*,entityChangeStreamRetirement,externalSessionCurrentness,routes.externalGfsRateLimitFailClosed): 27 files / 333 tests passed.db.realPostgresMigrationfails 41 of 75 cases on the test host because its production access verifier shells out topsql, which is not installed there; the same 41 fail ondev. external-rest-api at this head:tsc --noEmitclean,npm test56 files / 576 tests passed.Risks / follow-ups
/entity-changes/stream, which now answers 404.0140to0145add triggers on hot tables (team_members,users). Each one runs with a 10 slock_timeout; the two hot ones retry a lost lock up to their declared count.Stack
Live GFS subscriptions ship as a stack. The server half and the desktop half must reach
devtogether: the server PRs retire the route the current desktop uses, and the desktop PR needs the new watch route.devprincipal_deniedreasondevdevGET/PUT /external/gfs/watchGET/PUT /api/v1/me/gfs/watch1 and 2 are independent of the chain and safe to merge on their own (1 before 2, since the controller check in 2 mirrors the one in 1).
3 to 8 are one unit and should not be deployed partially:
gfs_resource_changedNOTIFY trigger, which nothing listens to yet.changedevents of a stream that is idle (no op in flight). Do not ship 5/6 without 7./api/v1/entity-changes/streamand control-api/external/entity-changes/stream), which the current desktop still uses. Do not ship 7 without 8.