Add KV Connect protocol v4: multiplexed watch channel over WebSocket - #166
Draft
piscisaureus wants to merge 4 commits into
Draft
piscisaureus wants to merge 4 commits into
piscisaureus wants to merge 4 commits into
Conversation
piscisaureus
force-pushed
the
watch-channel-v4
branch
from
June 10, 2026 22:38
db8dcba to
784f85c
Compare
Member
Author
|
Pushed a revision addressing findings from a deeper review pass:
One known follow-up not addressed here: the npm client throws |
piscisaureus
force-pushed
the
watch-channel-v4
branch
2 times, most recently
from
June 10, 2026 22:55
3bb0b70 to
7b708ab
Compare
Watching n keys currently costs one long-lived streaming request per watch() call. Protocol version 4 adds a watch channel: a single WebSocket connection per database that key watches can be added to and removed from at any time, with support for resuming after a reconnect without duplicate deliveries. * proto: add WatchChannelClientMessage (add/remove a watched key; add carries an optional resume baseline: the last seen versionstamp or "absent") and WatchChannelServerMessage (key-tagged outputs carrying only state that differs from what the client has) to datapath.proto. Spec the endpoint and message flow in kv-connect.md and list version 4 in the protocol versions section. The limits module of denokv_proto is now public. * remote: RemoteTransport gains optional WebSocket support through defaulted methods, so existing transport implementations keep compiling and keep negotiating version 3 with the per-watch code path. Transports that implement websocket() advertise version 4 during the metadata exchange. A per-database channel manager refcounts watched keys across watch() calls, demultiplexes updates to the per-call streams (entries shared via Arc, copied only at the caller-facing conversion), and reconnects with jittered backoff, re-adding every key with its last seen state as the baseline. * remote: when the channel cannot be made to work, watch() falls back to per-watch streaming instead of retrying forever: after five consecutive failed or short-lived connections (e.g. an intermediary strips the Upgrade header, or the server closes the channel with a policy violation), subscriptions fail with a marker error that watch() converts into the version 3 code path. The same fallback covers endpoints introduced by metadata refreshes that have not passed the caller's permission check: the driver only connects to endpoints approved in watch(), and the per-watch path re-checks permissions on every request. Watches with more than MAX_WATCHED_KEYS keys or an empty key list also use version 3 semantics. * server: add the /watch_channel WebSocket endpoint, gated on x-denokv-version: 4. The server diffs watcher output against what each channel client is known to have, validates key sizes, enforces a per-channel key limit (close code 1008), batches bursts of add/remove messages into a single watcher rebuild (removals need no rebuild at all), and pings every 5s. The metadata endpoint now negotiates up to version 4. * The watch() stream contract is unchanged regardless of negotiated version: the first item is a full snapshot and later items mark untouched keys as Unchanged, in request key order. Both protocol paths share one entry conversion.
piscisaureus
force-pushed
the
watch-channel-v4
branch
from
June 10, 2026 23:02
7b708ab to
38328ea
Compare
Member
Author
|
Review findings addressed in the latest commits:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds KV Connect protocol version 4: a multiplexed watch channel.
Today every
watch()call opens its own long-lived streaming request,so a client watching many key sets holds many concurrent streams, and
every reconnect re-delivers state the client already has. Version 4
multiplexes all watches for a database over one WebSocket connection:
WatchChannelClientMessage), so thewatched set changes without new connections. The
addmessagecarries an optional baseline (last seen versionstamp, or "absent");
the server only sends state that differs from it. Re-adding all keys
with their baselines after a reconnect makes resumption lossless and
duplicate-free.
unlike the positional v3
WatchOutput.RemoteTransportgains optional WebSocket support via defaultedmethods, so this is semver-compatible: existing transports keep
compiling, keep advertising
[1, 2, 3], and keep the per-watch v3path. Only transports that implement
websocket()negotiate v4.Nothing changes for any current consumer until its transport opts
in.
denokvserver gains the/watch_channelendpoint (axum ws)behind the same auth, diffs updates against per-client known state,
enforces a per-channel key cap (1024, close code 1008), and pings
every 5s. Metadata negotiation now prefers v4.
watch()stream contract is identical under both protocols:first item is a full snapshot, later items mark untouched keys
Unchanged, in request key order — covered by integration teststhat exercise the channel path (shared channel across watches,
initial snapshots, per-key updates, re-subscription) next to the
existing v3 test.
Spec for the new endpoint and messages is in
proto/kv-connect.md("Watch Channel (version 4)").