From e9cb01a32ade6fedd3537a0358b820c74c8a61be Mon Sep 17 00:00:00 2001 From: TonyTonyCoder11 Date: Fri, 31 Jul 2026 16:27:31 +0200 Subject: [PATCH] Say what the client is now, not what it was against The README's opening argument was that gRPC is what the official client costs you. Kdrant has a gRPC engine now, so that argument is half wrong and reads as an omission. The pitch is the wire being yours to pick: REST by default with no gRPC, protobuf or Netty on the classpath, and the gRPC engine one dependency away when throughput is the bottleneck. The bigger gap was discoverability. Six published modules were reachable only from a paragraph of roadmap prose, so someone arriving from a search for a Spring AI or LangChain4j vector store found no coordinate to copy. There is a module table with one line each now, a BOM snippet, and a short section on choosing an engine that names the two things that actually differ: the port, and the eleven operations Qdrant serves over HTTP only. The Architecture section had become a second copy of that table. It says the thing the table cannot instead: adding a second engine changed no line of kdrant-core, which is what the seam was for, and is the same reason the core compiles for eight native targets. STABILITY.md was written to argue for cutting 1.0 and then to justify shipping 1.x changes as minors. Both readings are history. The gates for a release three majors ago are gone, the 0.x section with them, and in their place is what a reader upgrading actually needs: the JVM API is unchanged byte for byte, a Gradle build changes only the version number, and a Maven build naming kdrant-core has to move to kdrant-core-jvm. The eleven HTTP-only operations are named there too, because "both engines behave the same" is only a contract if the exception is written down. --- README.md | 158 +++++++++++++++++---------- STABILITY.md | 84 ++++++-------- docs/migrating-from-qdrant-client.md | 2 +- 3 files changed, 140 insertions(+), 104 deletions(-) diff --git a/README.md b/README.md index 0bac4f9..f24e24e 100644 --- a/README.md +++ b/README.md @@ -13,10 +13,13 @@ [![API docs](https://img.shields.io/badge/API%20docs-Dokka-232B45?labelColor=0B0E17)](https://nacode-studios.github.io/Kdrant/) [![Website](https://img.shields.io/badge/website-nacodestudios.it-232B45?labelColor=0B0E17)](https://nacodestudios.it/en/project/kdrant) -Qdrant's official JVM client is built for Java: every call returns a `ListenableFuture`, requests -are assembled with protobuf builders, and it pulls a large gRPC/Netty stack onto your classpath. -Kdrant is the client you'd actually want to write Kotlin against: `suspend` functions, a type-safe -DSL, `kotlinx-serialization` models, and a small, coroutine-native footprint. +Qdrant's official JVM client is built for Java: every call returns a `ListenableFuture`, requests are +assembled with protobuf builders, and the wire is decided for you — gRPC, with a shaded Netty on your +classpath whether you needed the throughput or not. Kdrant is the client you'd actually want to write +Kotlin against: `suspend` functions, a type-safe DSL, `kotlinx-serialization` models, and a wire you +pick. The default engine is pure-Kotlin REST and pulls in no gRPC, no protobuf and no Netty; the gRPC +engine is one dependency away when throughput is the bottleneck. Both are the same `QdrantClient` and +are held to the same behavioural test suite. ```kotlin val qdrant = Kdrant(host = "localhost", port = 6333) { @@ -44,11 +47,12 @@ your own embedding model; Kdrant does not generate embeddings. > **See it end to end.** [`example-rag`](example-rag/) is a small runnable Retrieval-Augmented-Generation > service (ingest, embed, store, retrieve) built on Kdrant, with a `docker-compose` for Qdrant. -> **Status — `1.2`, stable.** The REST client is feature-complete: collections, `upsert`, the modern +> **Status — `2.0`, stable.** Both engines are feature-complete: collections, `upsert`, the modern > `/points/query` search (nearest, hybrid fusion, recommend/discover/context, batch, groups), sparse -> and multi-vectors, `scroll`, payload and vector management, aliases, snapshots, service/analytics -> endpoints, resilient retries, and the full filter DSL, plus Spring Boot, Spring AI and LangChain4j -> integrations. The public API is stable under SemVer; see [STABILITY.md](STABILITY.md). +> and multi-vectors, `scroll`, payload and vector management, aliases, snapshots, cluster and sharding, +> service/analytics endpoints, resilient retries, and the full filter DSL, plus Spring Boot, Spring AI, +> LangChain4j, Koog and Micrometer modules. `kdrant-core` builds for the JVM and eight Kotlin/Native +> targets. The public API is stable under SemVer; see [STABILITY.md](STABILITY.md). ## Why Kdrant @@ -56,10 +60,13 @@ your own embedding model; Kdrant does not generate embeddings. `CancellationException` is always propagated. - Collections, points, payloads and filters are built declaratively through scope-isolated builders (`@DslMarker`) rather than verbose request objects. -- The engine is pure Kotlin REST on Ktor and kotlinx-serialization, which keeps gRPC, Netty and - protobuf off your classpath. -- Failures surface as a sealed `KdrantException` you can handle exhaustively. -- The wire protocol sits behind a `QdrantTransport` seam, so the public API stays independent of it. +- The default engine is pure Kotlin REST on Ktor and kotlinx-serialization, so gRPC, Netty and + protobuf reach your classpath only if you ask for the gRPC engine by name. +- Failures surface as a sealed `KdrantException` you can handle exhaustively, whichever engine raised + them. +- The wire protocol sits behind a `QdrantTransport` seam. That is not a claim about layering: it is why + a second engine could be added without changing one line of `kdrant-core`, and why the models and the + query DSL compile for iOS, macOS, Linux and Windows as well as the JVM. ### Footprint vs the official client @@ -85,13 +92,52 @@ Requires JDK 17+. Artifacts are published to Maven Central under `io.github.naco ```kotlin dependencies { - implementation("io.github.nacode-studios:kdrant-transport-rest:1.2.0") + implementation("io.github.nacode-studios:kdrant-transport-rest:2.0.0") } ``` -`kdrant-transport-rest` brings in `kdrant-core` transitively; it is the only dependency you add. To use -the gRPC engine instead, depend on `kdrant-transport-grpc` and build the client with `KdrantGrpc(host)`; -nothing else in this README changes, because the API above the wire is the same API. +`kdrant-transport-rest` brings in `kdrant-core` transitively; it is the only dependency you add. + +### The modules + +Everything below is optional and additive. Take the engine you want and nothing else. + +| Artifact | What you get | +| --- | --- | +| `kdrant-transport-rest` | **The one to start with.** The REST engine on Ktor CIO and the `Kdrant(...)` factory. Brings `kdrant-core` with it. | +| `kdrant-transport-grpc` | The opt-in gRPC engine and the `KdrantGrpc(...)` factory. Reach for it when throughput or long-lived streaming is the bottleneck. | +| `kdrant-core` | The public API, models, DSLs and the `QdrantTransport` seam, with no wire-protocol knowledge. Multiplatform. You rarely depend on it directly. | +| `kdrant-spring-boot-starter` | Spring Boot auto-configuration: `kdrant.*` properties and a ready `QdrantClient` bean. | +| `kdrant-spring-ai` | A Spring AI `VectorStore` backed by Kdrant, metadata filters included. | +| `kdrant-langchain4j` | A LangChain4j `EmbeddingStore` backed by Kdrant, metadata filters included. | +| `kdrant-koog` | A [Koog](https://github.com/JetBrains/koog) document storage where Qdrant runs the search instead of the agent scoring in memory. | +| `kdrant-micrometer` | Request timings and outcomes per Qdrant operation, tagged by route template rather than by URL. | +| `kdrant-bom` | A platform that keeps the versions above aligned. Import it and drop the versions. | + +```kotlin +dependencies { + implementation(platform("io.github.nacode-studios:kdrant-bom:2.0.0")) + implementation("io.github.nacode-studios:kdrant-transport-rest") + implementation("io.github.nacode-studios:kdrant-spring-ai") +} +``` + +### Choosing an engine + +REST is the default and the right answer for most applications: it is a smaller dependency set, it +needs no native configuration under GraalVM, and it is the engine every operation is available on. + +```kotlin +val qdrant: QdrantClient = + if (useGrpc) KdrantGrpc(host = "localhost") // gRPC, port 6334 + else Kdrant(host = "localhost") // REST, port 6333 +``` + +Every example below reads the same either way, because the API above the wire is the same API. Two +differences are worth knowing before you switch. The **port** is 6334, not 6333, and nothing rewrites +it for you. And Qdrant serves eleven operations over HTTP only — telemetry, Prometheus metrics, the +issues endpoint, snapshot recovery, snapshot download and upload, and the shard-scope snapshots — which +the gRPC engine refuses by name rather than degrading quietly. `kdrant-core` is a Kotlin Multiplatform library and publishes one artifact per target. A Gradle build resolves the right one from the `kdrant-core` coordinate and needs no change. A Maven build names the @@ -100,9 +146,11 @@ artifact directly and wants `kdrant-core-jvm`. The engines and adapters are JVM- You also need a running Qdrant. For local development: ```bash -docker run -p 6333:6333 qdrant/qdrant +docker run -p 6333:6333 -p 6334:6334 qdrant/qdrant ``` +6333 is the REST port and 6334 the gRPC one; map both and either engine connects. + ## Usage ### Connecting @@ -280,48 +328,48 @@ val hits = catching { qdrant.search("articles") { query(queryVector) } }.getOrEl ## Architecture -Three modules keep protocol concerns out of the public API: +The wire lives behind one interface, `QdrantTransport`, and everything above it is protocol-neutral: -| Module | Contents | -| --- | --- | -| `kdrant-core` | Public API (`QdrantClient`), models, DSLs, error hierarchy, and the `QdrantTransport` seam, with no wire-protocol knowledge. | -| `kdrant-transport-rest` | The default REST engine (Ktor CIO) implementing `QdrantTransport`, plus the `Kdrant(...)` factory. | -| `kdrant-transport-grpc` | The opt-in gRPC engine over Qdrant's own protobuf services, plus the `KdrantGrpc(...)` factory. Depend on it only if you want it. | - -`kdrant-core` builds for the JVM and for eight Kotlin/Native targets: iOS, macOS, Linux and Windows. -That is what the transport seam bought — the models, DSLs and client logic never knew about a wire, so -they compile anywhere Kotlin does. The engines stay JVM-only, because Ktor CIO and grpc-java are, so a -multiplatform consumer today shares its models and its query building and supplies its own transport. -Kotlin/JS is left out for that reason rather than an accident of effort: with no JS engine the target -would ship a DSL with nothing to send. - -The DSLs and client logic live in `kdrant-core` and are independent of the protocol; only an engine -module knows about a wire. Both engines are held to the same behavioural test suite, so the choice is -a footprint and throughput decision rather than a feature one — with the exception of the operations -Qdrant serves over HTTP only, which the gRPC engine names rather than degrading. +``` +kdrant-core QdrantClient, models, DSLs, KdrantException, QdrantTransport + | no wire-protocol knowledge · JVM + 8 Kotlin/Native targets + +-- kdrant-transport-rest Ktor CIO Kdrant(host) JVM + +-- kdrant-transport-grpc grpc-kotlin KdrantGrpc(host) JVM +``` + +That is the arrangement the second engine tested. Adding gRPC changed **no line of `kdrant-core`**, and +the same behavioural suite runs against both engines against the same Qdrant, so the choice between them +is a footprint and throughput decision rather than a feature one — except for the operations Qdrant +serves over HTTP only, which the gRPC engine names. + +It is also why the core compiles for iOS, macOS, Linux and Windows: code that never knew there was a +wire has nothing platform-specific to port. The engines stay JVM-only, because Ktor CIO and grpc-java +are, so a multiplatform consumer today shares its models and its query building and supplies its own +transport. Kotlin/JS is left out for that reason rather than an accident of effort: with no JS engine +the target would ship a DSL with nothing to send. ## Roadmap -**Shipped (`1.2.0`).** Tier 6 closes the gaps that made adoption harder than it needed to be: -metadata-filter translation in the Spring AI and LangChain4j adapters, so a filtered RAG application is -a genuine drop-in swap; contract tests validating every request body against Qdrant's OpenAPI document, -so a wire change is a failing build rather than a silent difference; `ensureCollection`, an ordered -`scroll` that resumes, and `batchUpdate` for rerunnable bootstrap scripts and resumable ETL; a -`kdrant-micrometer` module, `X-Request-Id` correlation and connection-pool settings; and a -[migration guide from `io.qdrant:client`](docs/migrating-from-qdrant-client.md) with -[measured latency](benchmarks/README.md#measured-latency) behind it. On top of the earlier -`1.x` line: collection aliases, snapshots with streaming backup and restore, the service, health and -analytics endpoints, a granular transport seam with a `FloatArray` no-boxing hot path, the modern -`/points/query` engine (hybrid RRF/DBSF fusion, sparse and multi-vectors, recommend / discover / -context, batch and grouped search), payload and vector management, and typed-payload DX. Upgrading -from `1.1.0` is a recompile rather than a jar swap — see -[STABILITY.md](STABILITY.md#what-may-still-change-in-a-minor); the [CHANGELOG](CHANGELOG.md) has the -version-by-version detail. - -**Merged, unreleased.** The opt-in gRPC engine, `kdrant-transport-grpc`, and the move of `kdrant-core` -to Kotlin Multiplatform. Both are in `main` and neither has shipped; see the -[CHANGELOG](CHANGELOG.md#unreleased) for what they change and what the multiplatform move does to -`kdrant-core`'s artifact coordinates. +**Shipped (`2.0.0`).** Tier 5 closes the two things the transport seam existed to make possible. +`kdrant-transport-grpc` is an opt-in gRPC engine behind the same `QdrantClient`, generated from Qdrant's +own `.proto` files rather than wrapping the official client, so a REST build still resolves no gRPC, no +protobuf and no Netty — checked on every build, not asserted. And `kdrant-core` moved to Kotlin +Multiplatform: the JVM plus eight Kotlin/Native targets, with the JVM public API unchanged byte for +byte. Both engines are now held to one shared behavioural suite against a real Qdrant. The major bump is +for the artifact layout, not the API: `kdrant-core`'s JVM classes moved to `kdrant-core-jvm`, which a +Gradle build does not notice and a Maven build does. See +[STABILITY.md](STABILITY.md#upgrading-from-1-x). + +On top of the `1.x` line: metadata-filter translation in the Spring AI and LangChain4j adapters, +contract tests validating every request body against Qdrant's OpenAPI document, `ensureCollection`, an +ordered `scroll` that resumes, `batchUpdate`, the `kdrant-micrometer` module, `X-Request-Id` +correlation, cluster and sharding, collection aliases, snapshots with streaming backup and restore, the +service, health and analytics endpoints, a `FloatArray` no-boxing hot path, the modern `/points/query` +engine, and typed-payload DX. The [CHANGELOG](CHANGELOG.md) has the version-by-version detail, and the +[migration guide from `io.qdrant:client`](docs/migrating-from-qdrant-client.md) has +[measured latency](benchmarks/README.md#measured-latency) behind it. + +**Next.** Nothing is claimed yet; the board is where it gets decided. The plan lives on the [Kdrant board](https://github.com/orgs/NaCode-Studios/projects/4) — one item per milestone, each with its exit criterion — and every tier is a [milestone](https://github.com/NaCode-Studios/Kdrant/milestones) in this repository. See diff --git a/STABILITY.md b/STABILITY.md index 3a8b4fa..43c9cab 100644 --- a/STABILITY.md +++ b/STABILITY.md @@ -1,7 +1,7 @@ # Stability & versioning -This document is the written stability contract for Kdrant — what "stable" means, what changes are allowed -in which releases, and the plan for cutting `1.0`. It complements the [board](https://github.com/orgs/NaCode-Studios/projects/4) (where the +This document is the written stability contract for Kdrant — what "stable" means, what changes are +allowed in which releases, and what a major bump costs you. It complements the [board](https://github.com/orgs/NaCode-Studios/projects/4) (where the project is going) and the [CHANGELOG](CHANGELOG.md) (what has already shipped). ## Semantic Versioning @@ -15,16 +15,25 @@ Kdrant follows [Semantic Versioning 2.0.0](https://semver.org). Given `MAJOR.MIN ### What counts as "public API" The public API is exactly what the [binary-compatibility-validator](https://github.com/Kotlin/binary-compatibility-validator) -tracks in the committed `*.api` files (`kdrant-core/api`, `kdrant-transport-rest/api`, and the ecosystem -modules). Every public/protected symbol is in those dumps; `./gradlew apiCheck` fails the build on any +tracks in the committed `*.api` files (`kdrant-core/api`, both transport engines, and the ecosystem +modules; the gRPC module's generated protobuf and stub classes are excluded, because their surface is +Qdrant's to change and not ours to promise). Every public/protected symbol is in those dumps; `./gradlew apiCheck` fails the build on any untracked change, so **API breakage is never silent**. Anything not in the `*.api` files — `internal` declarations, or symbols annotated `@InternalKdrantApi` — is not public API and may change at any time. -The **wire behaviour** of the REST engine (the requests it sends and the responses it parses) is also part -of the contract: a change that alters the bytes on the wire for an existing operation is treated as a +The **wire behaviour** of an engine (the requests it sends and the responses it parses) is also part of +the contract: a change that alters what goes on the wire for an existing operation is treated as a breaking change unless it is a bug fix bringing Kdrant in line with Qdrant's documented API. Contract -tests validate every request body the engine builds against Qdrant's own OpenAPI document, pinned to the -version the CI matrix runs against, so a wire change is a failing build rather than a silent difference. +tests validate every request body the REST engine builds against Qdrant's own OpenAPI document, pinned +to the version the CI matrix runs against, and a shared behavioural suite runs both engines against a +real Qdrant, so a wire change is a failing build rather than a silent difference. + +**The two engines are held to the same behaviour, with one stated exception.** Qdrant serves eleven +operations over HTTP only — telemetry, Prometheus metrics, the issues endpoint, `recoverSnapshot`, +snapshot download and upload, and the five shard-scope snapshot operations. On the gRPC engine each +throws an `UnsupportedOperationException` naming itself and naming REST. That list is part of this +contract: an operation leaving it is an additive change, and an operation joining it would be a +breaking one. ### What may still change in a minor @@ -38,31 +47,15 @@ so the compiler tells you what a new release added; if you need a stub in a test third-party wire engine is a supported use of `QdrantTransport`, but it is a use that recompiles against each minor. -**A multiplatform artifact is named per target.** `kdrant-core` publishes Gradle module metadata under -its own coordinate and the JVM classes under `kdrant-core-jvm`. Gradle reads the metadata and resolves -the variant; Maven does not, and a Maven build naming `kdrant-core` gets no classes. That is a -coordinate change rather than an API change — the JVM public API is unchanged — but it is the kind that -belongs in a major, which is where it landed. - **Adding a field to a public `data class` changes its generated `copy` and `componentN`.** New response fields arrive as Qdrant returns more, and while the constructor keeps its defaults and source keeps compiling, code that called `copy()` against an older jar needs recompiling. Kdrant does not add fields -gratuitously and each one is listed in the [CHANGELOG](CHANGELOG.md), but a `1.x` upgrade is a +gratuitously and each one is listed in the [CHANGELOG](CHANGELOG.md), but a minor upgrade is a recompile, not a jar swap. -## Pre-`1.0` (the `0.x` line) - -While Kdrant is `0.x`, the public API may change between **minor** versions. Breaking changes are: - -- called out in the [CHANGELOG](CHANGELOG.md) under **Changed** / **Removed**, and -- always visible as a diff in the `*.api` files. - -We still avoid gratuitous breakage and prefer additive evolution, but `0.x` minors are the window for -getting the surface right before it is frozen. - -## Post-`1.0` +## The guarantee -From `1.0.0` onward: +Within a major version: - **No breaking public-API change without a major bump.** Source and binary compatibility are maintained across a major version. @@ -70,41 +63,36 @@ From `1.0.0` onward: at least one minor release before removal, and removal happens only in a major release. - **Coroutine contract.** Every operation stays a `suspend` function or a `Flow`; cancellation is cooperative and `CancellationException` is always propagated. -- **Wire compatibility.** Kdrant tracks Qdrant's stable REST API; new Qdrant features arrive as additive +- **Wire compatibility.** Kdrant tracks Qdrant's stable API; new Qdrant features arrive as additive minor releases. -## The `1.0` release +## Upgrading from `1.x` -`1.0.0` ships once the **REST core is feature-complete and stable** — it does **not** wait for the optional -gRPC engine (post-`1.0`, see M25). The gates, all met: +`2.0.0` breaks one thing, and it is not the API. **The JVM public API is unchanged, byte for byte** — +`apiCheck` reports no diff against `1.2.0`. What changed is where `kdrant-core`'s JVM classes are +published. -1. **Feature completeness (met).** Collections, the modern `/points/query` engine (nearest, hybrid fusion, - recommend/discover/context, batch, groups, sparse & multi-vectors), payload & vector management, payload - indexes, collection config, **aliases**, and **snapshots** are all shipped — the operational surface an - application needs is complete. -2. **Quality gates (met).** ktlint + detekt, a JDK and Qdrant-version CI matrix, dependency review, - Dependabot, and property-based serialization tests are in place; the public API is tracked. -3. **This stability policy (this document).** -4. **Benchmarks.** A reproducible JMH harness for upsert/search latency ships in [`benchmarks/`](benchmarks/); - the [published numbers](benchmarks/README.md#measured-latency) come from running it in CI against a pinned - Qdrant, alongside the footprint table in the [README](README.md#footprint-vs-the-official-client), honest - about where gRPC/HTTP2 wins. +`kdrant-core` is now a Kotlin Multiplatform library, so its own coordinate carries Gradle module +metadata and the JVM classes live in `kdrant-core-jvm`. Gradle reads that metadata and resolves the +variant, so **a Gradle build changes nothing but the version number**. Maven does not read it, so a +Maven build naming `kdrant-core` gets no classes and must move to `kdrant-core-jvm`. Depending on +`kdrant-transport-rest` or `kdrant-transport-grpc`, which is what the README recommends, is unaffected +either way. -With those gates met, **`1.0.0` is this release** — built on `0.2.0`, adding M19–M24. From here the public -API is stable under SemVer; new capabilities arrive as additive `1.x` minors, and breaking changes wait for -a `2.0`. +The rest of `2.0.0` is additive: the gRPC engine is a module you do not have, and every `1.x` call site +compiles unchanged. ## Java interoperability Kdrant is deliberately **Kotlin-coroutine-first** — that is the wedge (see the [README](README.md)). The public API is `suspend` functions and `Flow`s, which are callable from Java but not idiomatic there. -**Decision for `1.0`: no bundled `CompletableFuture` facade.** Mirroring ~40 suspend operations into a +**No bundled `CompletableFuture` facade.** Mirroring ~40 suspend operations into a blocking or future-returning Java API is a large, duplicated surface to maintain, and it is not the audience Kdrant optimises for. Java callers who need it should bridge with the standard tools: - `kotlinx-coroutines-jdk8`'s `future { }` to turn a `suspend` call into a `CompletableFuture`, or - `runBlocking { }` for a simple synchronous call. -A dedicated `kdrant-jdk` facade remains a **post-`1.0`, on-demand** option if there is real Java demand; it -would be additive and would not change the Kotlin API. +A dedicated `kdrant-jdk` facade remains an **on-demand** option if there is real Java demand; it would be +additive and would not change the Kotlin API. diff --git a/docs/migrating-from-qdrant-client.md b/docs/migrating-from-qdrant-client.md index 6d1db00..41cc187 100644 --- a/docs/migrating-from-qdrant-client.md +++ b/docs/migrating-from-qdrant-client.md @@ -34,7 +34,7 @@ Replace the dependency. Kdrant needs JDK 17, where the official client needs 8. ```kotlin dependencies { // implementation("io.qdrant:client:1.18.3") - implementation("io.github.nacode-studios:kdrant-transport-rest:1.2.0") + implementation("io.github.nacode-studios:kdrant-transport-rest:2.0.0") } ```