Skip to content

M31 · Speak gRPC, without charging anyone who does not - #63

Merged
tonytonycoder11 merged 6 commits into
mainfrom
feat/m31-grpc-transport
Jul 31, 2026
Merged

M31 · Speak gRPC, without charging anyone who does not#63
tonytonycoder11 merged 6 commits into
mainfrom
feat/m31-grpc-transport

Conversation

@tonytonycoder11

Copy link
Copy Markdown
Contributor

Closes #52.

The seam has been in place since the first release and has had one implementation. This is the second,
and what it proves is that the public API really did carry no wire-protocol knowledge: not one line of
kdrant-core changed to add a protocol.

KdrantGrpc(host) returns the same QdrantClient the REST factory does, over Qdrant's Collections,
Points, Snapshots and Health services on 6334. The shared client contract from #62 runs against
both engines, against the same server.

Against the exit criterion"a kdrant-transport-grpc module passes the same client test suite as
the REST engine, and a REST-only build pulls in none of its dependencies"
:

  • GrpcClientContractIntegrationTest is four lines: it extends QdrantClientContract and picks port
    6334. Same file the REST engine is held to.
  • verifyRestEngineFootprint runs on every check and fails if the REST engine's runtime classpath
    resolves anything from io.grpc, com.google.protobuf, io.netty or com.google.guava. It was
    confirmed to fail by adding grpc-stub to the REST module and watching it report the five artifacts
    that arrived, then reverting. The README's footprint table rests on this, so it is checked rather
    than argued.

Where the seam is wider than the protocol. QdrantTransport was shaped by Qdrant's REST API, and
eleven of its operations exist over HTTP only: telemetry, metrics, issues, recoverSnapshot, snapshot
download and upload, and the five shard-scope snapshot operations. Each throws an
UnsupportedOperationException naming itself and naming REST. A snapshot download that returned an
empty flow would be a backup that quietly does not exist, and a KdrantException would have been the
wrong type — nothing failed on the wire, and no retry makes the call work. The three streaming
downloads throw at the call rather than on first collect, so the failure lands where the mistake is.

Health is the one place the protocols disagree and both are right. There is no gRPC /healthz,
/readyz or /livez; there is the standard gRPC health service, which answers one status for the
node. The three probes answer from it and keep the REST engine's false-on-failure contract.

Behavioural parity. Retries mirror the REST engine's, because maxRetries is a client setting and
an engine that ignored it would be a difference nobody asked for. Deadlines are per call rather than
per channel, because requestTimeout is documented as per attempt. gRPC statuses map onto the same
KdrantException hierarchy, including the INVALID_ARGUMENT Qdrant returns on some write paths for a
collection that does not exist.

Tests. Three layers, and only the first needs Docker: the shared contract against a real Qdrant;
GrpcQdrantTransportTest, which runs the transport against fake services over gRPC's in-process
transport and asserts the messages it built and the models it read back; and focused tests for the
error mapping and for the eleven operations that cannot work. Aggregate line coverage is 80.1%.

Also updates the README's module table and footprint paragraph, and the migration guide — "Kdrant does
not speak gRPC" stopped being true.

Merge after #61 and #62.

The seam has been in place since the first release and has had one
implementation. This is the second, and what it proves is that the public
API really did carry no wire-protocol knowledge: not one line of
kdrant-core changed to add a protocol.

KdrantGrpc(host) returns the same QdrantClient the REST factory does, over
Qdrant's Collections, Points, Snapshots and Health services on 6334. The
same shared contract runs against both engines, against the same server,
so the choice between them is a footprint and throughput decision rather
than a feature one.

The seam is wider than the protocol, and that is the interesting part.
QdrantTransport was shaped by Qdrant's REST API, and eleven of its
operations exist over HTTP only: telemetry, metrics, issues, snapshot
recovery, snapshot transfer, and the shard-scope snapshots. Each of them
throws here, naming itself and naming REST. A snapshot download that
returned an empty flow would be a backup that quietly does not exist, and
a KdrantException would have been the wrong type: nothing failed on the
wire, and no retry will make the call work.

Health is the one place the two protocols disagree and both are right.
Qdrant has no gRPC /healthz, /readyz or /livez; it has the standard gRPC
health service, which answers one status for the node. The three probes
answer from it, and keep the REST engine's false-on-failure contract
rather than throwing.

Retries mirror the REST engine's, because maxRetries is a client setting
and an engine that ignored it would be a behaviour difference nobody
asked for. Deadlines are per call rather than per channel, because
requestTimeout is documented as per attempt.

A REST-only build still resolves no gRPC, no protobuf and no Netty. That
was the whole argument for making this opt-in, and it is asserted rather
than assumed.
@tonytonycoder11
tonytonycoder11 requested a review from a team as a code owner July 31, 2026 13:17
The BOM exists so the modules cannot drift apart, and a module missing from
it drifts by default. Importing the platform and then naming
kdrant-transport-grpc without a version would resolve nothing.
The shared contract caught this on its first run against a real server, in
the one test that reads a payload back from a scroll nobody configured.

A scroll and a retrieve default with_payload to true on the server. The
engine was translating "the caller said nothing" into an explicit
enable = false, which is a different request: every scroll and every
retrieve that did not ask for a payload silently came back without one. The
REST engine omits the field and gets the default, which is why the two
disagreed and why only a test that ran against both could see it.

Absent now stays absent, on the payload selector and the vector selector
alike, and a unit test asserts the fields are unset rather than false.
@tonytonycoder11
tonytonycoder11 merged commit d4f8c26 into main Jul 31, 2026
6 checks passed
@tonytonycoder11
tonytonycoder11 deleted the feat/m31-grpc-transport branch July 31, 2026 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

M31 · Opt-in gRPC engine

1 participant