Add Alternator vector search support

[dkropachev]

Replaces #87.

This carries the vector search support from #87, scoped to the vector API/client integration work:

• Add com.scylladb.alternator.vectorsearch request/response support for VectorIndexes, VectorIndexUpdates, VectorSearch, FLOAT32VECTOR, and query scores.
• Auto-register VectorSearchInterceptor in the sync and async Alternator client builders.
• Add unit coverage for vector JSON conversion, sync/async response transformation, async query result extraction, checksum/header handling for modified vector responses, and vector integration helpers.
• Add integration tests that skip cleanly when the target Scylla/Alternator instance does not support vector indexes or has vector store disabled.

Split out of this PR:

• Demo2 shutdown cleanup: separate PR.
• Live-nodes polling client shutdown cleanup: separate PR.
• Compression/vector interceptor chaining: separate stacked PR based on vector-1.

Tests:

• JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:$PATH mvn -q fmt:check
• JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:$PATH mvn -q -Dtest=VectorSearchInterceptorTest,VectorSearchHttpInterceptorTest test
• JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:$PATH mvn -q test

[m-szymon]

Regarding conversions of FLOAT32VECTOR consider also some kind of compatibility switch in client.

Default: Keep the current behavior.
Server FLOAT32VECTOR -> interceptor -> DynamoDB L of N values
Applications can use: List<AttributeValue> values = item.get("embedding").l();

Opt-in: preserve mode
Expose server vectors as the existing magic-prefixed B attribute:
Server FLOAT32VECTOR -> interceptor -> magic-prefixed DynamoDB B
This is the same local representation created by: Float32Vector.toAttributeValue(0.25f, -1.5f, 3.0f)
It permits lossless read-modify-write:

Map<String, AttributeValue> item = client.getItem(request).item();
client.putItem(PutItemRequest.builder().tableName("copy").item(item).build());

In preserve mode, embedding.l() will not not work for optimized vectors.
Instead, Float32Vector should offer representation-independent accessors:

float[] values = Float32Vector.toFloats(item.get("embedding")); // decodes B-magic or converts List of Numbers
List<AttributeValue> numbers = Float32Vector.toNumberList(item.get("embedding")); // directly returns List of Numbers or converts B-magic

Both forms can be distinguished explicitly: boolean optimized = Float32Vector.isFloat32Vector(embedding);

[dkropachev]

Thanks. Since FLOAT32VECTOR is new and there is no released behavior to preserve, I propose avoiding a compatibility switch: always preserve server FLOAT32VECTOR as the existing magic-prefixed B, while real L remains L. Float32Vector.toFloats() and toNumberList() can accept both forms, and isFloat32Vector() can identify marker B. Callers that specifically need l() can explicitly convert to an L AttributeValue. We should not populate B and L together because AttributeValue is a union (type() becomes null and the server rejects multi-member values).

[swasik]

@m-szymon could you take a look at fixes by @dkropachev?

@dkropachev is something wrong with CI that it timed-out after 6 hours?

[dkropachev]

@m-szymon could you take a look at fixes by @dkropachev?

@dkropachev is something wrong with CI that it timed-out after 6 hours?

There was bug in the demo app, now it is gone

[Copilot]

Pull request overview

Adds Alternator vector-index and similarity-search support through AWS SDK interceptors and helper APIs.

Changes:

• Adds vector models, request helpers, response extraction, and FLOAT32VECTOR conversion.
• Auto-registers ordered vector interceptors in sync and async clients.
• Adds documentation and extensive unit/integration coverage.

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
README.md	Documents vector-search usage.
pom.xml	Updates SDK and adds Jackson.
.github/workflows/continuous-integration.yml	Adds integration-test timeout.
src/main/java/com/scylladb/alternator/AlternatorDynamoDbClient.java	Registers sync vector phases.
src/main/java/com/scylladb/alternator/AlternatorDynamoDbAsyncClient.java	Registers async vector phases.
src/main/java/com/scylladb/alternator/GzipRequestInterceptor.java	Caches compressed request bodies.
src/main/java/com/scylladb/alternator/ResponseCompressionInterceptor.java	Preserves unchanged response content.
src/main/java/com/scylladb/alternator/VectorSearchInterceptorPhases.java	Splits request/response interception.
src/main/java/com/scylladb/alternator/vectorsearch/CreateVectorIndexAction.java	Models index creation.
src/main/java/com/scylladb/alternator/vectorsearch/DeleteVectorIndexAction.java	Models index deletion.
src/main/java/com/scylladb/alternator/vectorsearch/Float32Vector.java	Encodes vector markers.
src/main/java/com/scylladb/alternator/vectorsearch/VectorAttribute.java	Models vector attributes.
src/main/java/com/scylladb/alternator/vectorsearch/VectorIndex.java	Models vector indexes.
src/main/java/com/scylladb/alternator/vectorsearch/VectorIndexUpdate.java	Models index updates.
src/main/java/com/scylladb/alternator/vectorsearch/VectorQueryResult.java	Wraps query results and scores.
src/main/java/com/scylladb/alternator/vectorsearch/VectorSearch.java	Models search parameters.
src/main/java/com/scylladb/alternator/vectorsearch/VectorSearchInterceptor.java	Rewrites vector HTTP payloads.
src/main/java/com/scylladb/alternator/vectorsearch/VectorSearchResultHolder.java	Carries extracted response metadata.
src/main/java/com/scylladb/alternator/vectorsearch/VectorSearchSupport.java	Provides sync/async helper APIs.
src/test/java/com/scylladb/alternator/AlternatorInterceptorOrderTest.java	Tests interceptor ordering.
src/test/java/com/scylladb/alternator/GzipRequestInterceptorTest.java	Tests compressed requests.
src/test/java/com/scylladb/alternator/ResponseCompressionInterceptorTest.java	Tests response chaining.
src/test/java/com/scylladb/alternator/VectorSearchInterceptorTest.java	Tests vector serialization.
src/test/java/com/scylladb/alternator/vectorsearch/Float32VectorTest.java	Tests marker validation.
src/test/java/com/scylladb/alternator/vectorsearch/VectorSearchHttpInterceptorTest.java	Tests HTTP transformations.
src/test/java/com/scylladb/alternator/vectorsearch/VectorSearchSupportTest.java	Tests facade behavior.
src/integration-test/java/com/scylladb/alternator/AlternatorDynamoDbAsyncClientIT.java	Adjusts async compression expectations.
src/integration-test/java/com/scylladb/alternator/AlternatorDynamoDbClientIT.java	Adjusts sync compression expectations.
src/integration-test/java/com/scylladb/alternator/HttpClientImplementationAsyncIT.java	Updates async transport handling.
src/integration-test/java/com/scylladb/alternator/HttpClientImplementationSyncIT.java	Updates sync transport handling.
src/integration-test/java/com/scylladb/alternator/VectorSearchIT.java	Adds live vector-search coverage.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[m-szymon]

As AWS announces Vector Search for DynamoDB, we should reconsider this PR.
It seems many things are implemented in SDK now and we shouldn't conflict with that, e.g. https://docs.aws.amazon.com/java/api/latest/software/amazon/awssdk/enhanced/dynamodb/internal/SearchVectorUtils.html#toSearchVector(float[])

[swasik]

As AWS announces Vector Search for DynamoDB, we should reconsider this PR. It seems many things are implemented in SDK now and we shouldn't conflict with that, e.g. https://docs.aws.amazon.com/java/api/latest/software/amazon/awssdk/enhanced/dynamodb/internal/SearchVectorUtils.html#toSearchVector(float[])

I would rather leave it for the next iteration. We need a support for our API anyway and it is already long till we launched. And we can do refactoring in the follow-up (we will need the follow-up for API anyway).

[m-szymon]

I would rather leave it for the next iteration. We need a support for our API anyway and it is already long till we launched. And we can do refactoring in the follow-up (we will need the follow-up for API anyway).

@swasik I don't mean re-implement it but review if we can adjust something now, to avoid some breaking changes in next iteration.

I am not Java expert, I am not sure if this is a real issue in practice - AI suggests:

Namespace the proprietary Alternator models before vector search becomes stable.
Recommended package:
com.scylladb.alternator.vectorsearch.model
Recommended names:
AlternatorVectorIndex
AlternatorVectorAttribute
AlternatorVectorIndexUpdate
AlternatorCreateVectorIndexAction
AlternatorDeleteVectorIndexAction

[m-szymon]

It accepts NaN/infinieties.

[swasik]

@swasik I don't mean re-implement it but review if we can adjust something now, to avoid some breaking changes in next iteration.

Yes, in this context I agree - we should not introduce anything we will need to break later because of new SDK.

[dkropachev]

@swasik I don't mean re-implement it but review if we can adjust something now, to avoid some breaking changes in next iteration.

Yes, in this context I agree - we should not introduce anything we will need to break later because of new SDK.

created a core issue - https://scylladb.atlassian.net/browse/SCYLLADB-3633 to do that in core, will check how if I can find middle ground between two implementations