Repository navigation
Conversation
Process-level building blocks for the live GFS watch, not wired yet: - runtime: timers whose handle is cleared on fire and cancelled through a single cancel form, guarded/guardedTick/bestEffort covering sync and async failures, total(), raceAbort and an injectable monotonic clock. - readSlots: process-wide FIFO semaphore of 64 gfsc reads with demand buckets and backlogMs for retry pacing. - recentDirty ring buffer and a dedicated watch pg pool with client-side query timeouts, keepalive and a permanent error listener. - listener: dedicated LISTEN client for the resource, ping and command channels; liveness by echo of its own ping, own-state-first teardown on loss, capped reconnect backoff, total per-channel dispatch, and fail-closed handling of unreadable revocation payloads through onExpansionFailed. - gfs_watch_* metrics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds the per-process watch registry that the HTTP routes will mount on:
- Connections with a single socket writer gated by phase, backpressure
(64 KiB / 5 s -> slow_consumer), heartbeat, max lifetime and token exp.
- closeConn that never throws: registry memory is released first, side
effects run best-effort, and the client retry is spread.
- The authorization op (ancestry CTE, per-op read token, bounded gfsc
reads) with a gate after every await, auth and server deadlines,
suspicion on concurrent ancestry changes, and deferred keys on
transient failures.
- Mutable subscription set via gfs_watch_cmd: binding check
(userId, authGeneration), seq window, and a single queued command.
- gfs_resource_changed delivery with a 250 ms fixed window that only
emits accepted keys outside the op in flight.
- open() as the registry API for the GET handler, with key and drive
ceilings enforced before admission.
readGfsc now caps the gfsc body at 64 KiB and replaces a non-uuid
request id, which gfsc rejects. Stream headers are exported as a
constant. New metric gfs_watch_cmd_dropped_total{reason}.
Revocation and session push hooks are left as no-ops for the follow-up
authz/session change; both ship together.
Tests: model-based run (fast-check commands + scheduler) against real
Postgres, the real trigger and real session/mint, with a fake gfsc built
from its own envelope helpers; 45 scenarios asserting observable frames.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…is set
The import sorter run by the pre-commit hook placed the clock helper,
which reaches src/db.ts through the watch runtime, ahead of the module
that points the core pool at the suite's database. The model-based
registry suite then bound the core pool to the default connection string
and failed in beforeAll on every run. The scenario suite only passed
because its env import happened to sort first.
Both suites now import only the env helper and types statically, and
load the harness and clock with a dynamic import from beforeAll, which
is the pattern the other real-Postgres suites already use.
Class-check: `process.env.CONTROL_API_PG_CONNECTION_STRING =` and
`gfsWatchRegistryEnv` under control-api/test → 3 sites (the two registry
suites via the env helper, gfsWatch.rateLimitCheck) → fix the two
registry suites; rateLimitCheck already sets the env and then uses
`await import('../src/db.js')`.
Repro-test: control-api/test/gfsWatch.registry.property.realPostgres.integration.test.ts
beforeAll (fails in f445e5a: "Connection terminated due to connection
timeout" from createUser, 5/5 runs on an isolated Postgres 16)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ent-subscriptions-api-registry Brings origin/dev (22dd421) through the synced parent branch. Conflicts: control-api package.json/package-lock.json version only; kept the higher version. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ent-subscriptions-api-registry
This was referenced Nov 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The per-process GFS watch registry in control-api, plus the process-level pieces it runs on. Not mounted on any route yet (#1500 does that).
runtime.ts: a singlecancel()form for timers (callers clear the handle when a timer fires);guarded/guardedTick/bestEffortcovering both sync throws and async rejections;total(),raceAbort, and an injectable monotonic clock.readSlots.ts: process-wide FIFO semaphore of 64 concurrent gfs-controller reads, with a demand counter and response-rate buckets feeding abacklogMs()estimate used to pace retries.watchPool.ts: a dedicated pg pool for the watch, with client-side query timeouts, keepalive and a permanent error listener.listener.ts: a dedicated LISTEN client for the resource, ping and command channels. Liveness is checked by the echo of its own ping; on loss it tears down its own state first, then reconnects with capped backoff. Dispatch is total per channel, and an unreadable payload on a revocation channel fails closed throughonExpansionFailed.recentDirty.ts: ring buffer of recent ancestry changes, used to flag an authorization decision that may have raced a move.registry.ts:slow_consumer), heartbeat, max lifetime and token expiry.closeConnnever throws: registry memory is released first, side effects run best-effort, and the client's retry delay is spread.deferredkeys on transient failures.gfs_watch_cmdchannel: binding check(userId, authGeneration), a seq window and a single queued command.gfs_resource_changeddelivery with a 250 ms fixed window that only emits accepted keys outside the op in flight.open()as the API for the GET handler, with key and drive ceilings enforced before admission.readGfsccaps the gfs-controller response body at 64 KiB and replaces a non-uuid request id, which gfs-controller rejects.gfs_watch_*, includinggfs_watch_cmd_dropped_total{reason}.The authorization and session push hooks (
resetWindows,onExpansionFailed) are required dependencies of the listener and are no-ops here. #1502 fills them in.Why
A watch stream has to keep three things true while requests, NOTIFYs, timers and socket events interleave across awaits: a client only ever receives events for keys it was authorized for, a change is not lost while a subscription is being re-authorized, and a stuck client or a burst of changes cannot exhaust the process. The registry is written as an explicit state machine so each of those can be checked at rest.
How
gfs_watch_cmd, so a PUT can land on a replica that does not hold the stream.Testing
gfsWatch.registry.property.realPostgresdrives the real registry with fast-check commands andfc.scheduler()for promise ordering, against real Postgres, the real trigger and real session/mint, with a fake gfs-controller built from its own envelope helpers. Invariants are checked at rest against a reference model.gfsWatch.registry.realPostgres(45 cases) asserts the frames the client observes.runtime,readSlots,recentDirty,watchPoolandlistener(including listener loss through a controllable TCP proxy).tsc --noEmitclean;npm test634 files / 8951 tests passed (real-Postgres suites skip in this lane); real-Postgres run of the watch suites (gfsWatch*,gfs.externalGfsRead*): 15 files / 206 tests passed.db.realPostgresMigrationfails 41 of 75 cases on the test host because its production access verifier shells out topsql, which is not installed there; the same 41 fail ondev.Risks / follow-ups
changedevents on a stream that has no op in flight. This PR and the HTTP route are not meant to be deployed without feat(control-api): push authorization and session changes over the GFS watch and retire the per-user entity-change stream #1502.Stack
Live GFS subscriptions ship as a stack. The server half and the desktop half must reach
devtogether: the server PRs retire the route the current desktop uses, and the desktop PR needs the new watch route.devprincipal_deniedreasondevdevGET/PUT /external/gfs/watchGET/PUT /api/v1/me/gfs/watch1 and 2 are independent of the chain and safe to merge on their own (1 before 2, since the controller check in 2 mirrors the one in 1).
3 to 8 are one unit and should not be deployed partially:
gfs_resource_changedNOTIFY trigger, which nothing listens to yet.changedevents of a stream that is idle (no op in flight). Do not ship 5/6 without 7./api/v1/entity-changes/streamand control-api/external/entity-changes/stream), which the current desktop still uses. Do not ship 7 without 8.