Skip to content

docs: added opensearch-jvector plugin docs - #12866

Merged
kolchfa-aws merged 9 commits into
opensearch-project:mainfrom
kush992:opensearch-jvector-plugin-docs
Aug 26, 2026
Merged

docs: added opensearch-jvector plugin docs#12866
kolchfa-aws merged 9 commits into
opensearch-project:mainfrom
kush992:opensearch-jvector-plugin-docs

Conversation

@kush992

@kush992 kush992 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Description

Added opensearch-jvector plugin documentation.

Issues Resolved

Closes #[Replace this text, including the brackets, with the issue number. Leave "Closes #" so the issue is closed properly.]

Version

List the OpenSearch version to which this PR applies, e.g. 2.14, 2.12--2.14, or all.

3.5 and above

Frontend features

opensearch-jvector_plugin_doc.mp4

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions github-actions Bot added the In progress Issue/PR: The issue or PR is in progress. label Jul 28, 2026
@github-actions

Copy link
Copy Markdown

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference).

Comment thread _install-and-configure/additional-plugins/opensearch-jvector.md Outdated
@kush992
kush992 marked this pull request as ready for review July 28, 2026 15:53
@github-actions github-actions Bot added Tech review PR: Tech review in progress and removed In progress Issue/PR: The issue or PR is in progress. labels Jul 28, 2026
@kolchfa-aws kolchfa-aws added release-notes PR: Include this PR in the automated release notes v3.8.0 labels Jul 30, 2026
Comment thread _install-and-configure/additional-plugins/index.md Outdated
Comment thread _mappings/supported-field-types/knn-methods-engines.md Outdated
kush992 and others added 4 commits August 13, 2026 16:00
---------

Signed-off-by: Kush Bhalodi <kush.bhalodi@ibm.com>
Co-authored-by: Kush Bhalodi <kush.bhalodi@ibm.com>
Signed-off-by: Kush Bhalodi <kush.bhalodi@ibm.com>
Signed-off-by: Kush Bhalodi <kush.bhalodi@ibm.com>
Signed-off-by: Kush Bhalodi <kush.bhalodi@ibm.com>
@kush992
kush992 force-pushed the opensearch-jvector-plugin-docs branch from 30b2a74 to dd76baf Compare August 13, 2026 14:01
@kush992
kush992 requested a review from kolchfa-aws August 13, 2026 14:06
Comment thread _install-and-configure/additional-plugins/opensearch-jvector.md Outdated

@kolchfa-aws kolchfa-aws left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @kush992! A couple of clarifying questions for you before I start rewriting.

| **Vector engines** | `Nmslib`, `Faiss`, `lucene` | `jvector` (primary), `lucene` |
| **Concurrent ingestion** | Supported by some vector engines | `jvector` supports concurrent inserts, enabling high-throughput ingestion |
| **Index update cost** | Usually a full rebuild required on merge | Incremental merges — no full rebuilds for updates |
| **Memory efficiency** | In-memory indexing | DiskANN-style quantization |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this line accurate? k-NN plugin supports disk-based vector search with binary quantization (on_disk mode) https://docs.opensearch.org/latest/vector-search/optimizing-storage/disk-based-vector-search/

@akash-shankaran akash-shankaran Aug 20, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be re-phrased. On opensearch-jvector, it is: diskAnn-style vector search with PQ (product quantization)

- **Incremental Index Updates** - `jvector` allows incremental insertion of vectors into existing indexes. This providing efficiency gains over full index rebuilds for workloads involving frequent updates, particularly for large graph-based indexes.
- **Quantized DiskANN with Reranking** - `jvector` supports DiskANN-style quantization combined with reranking, delivering significant performance improvements for datasets larger than available memory. This approach is particularly effective for large-scale deployments where traditional in-memory indexing is not feasible.
- **Product Quantization (PQ)** - PQ is implemented with high-performance SIMD optimizations and separate codebooks to enable fast vector searches with low memory usage.
- **Advanced Quantization Techniques** - `jvector` includes advanced capabilities such as Non-Vector Quantization (NVQ), and Anisotropic PQ, enabling more efficient and accurate similarity computations beyond standard quantization approaches.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does NVQ stand for non-uniform vector quantization and not non-vector quantization?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kolchfa-aws . Good catch. I missed this. It should be non-uniform vector quantization.
Here is the source: https://arxiv.org/pdf/2509.18471

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it should be Non-Uniform Vector Quantization (as per the feature proposal). Change is coming right away

bin/opensearch-plugin install opensearch-jvector
```

## OpenSearch compatible features

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean k-NN features that jVector supports?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

@kolchfa-aws kolchfa-aws added Doc review PR: Doc review in progress Awaiting response and removed Tech review PR: Tech review in progress labels Aug 20, 2026
kush992 and others added 2 commits August 21, 2026 10:41
Signed-off-by: Kush Bhalodi <kush.bhalodi@ibm.com>
Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

@kolchfa-aws kolchfa-aws left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @kush992! Doc review complete - I pushed my changes into your PR. Please review my changes and see my outstanding questions. Thank you!


Repeat the following steps on every node in the cluster:

1. Stop OpenSearch on the node.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kush992 Installing the plugin by name only didn't work for me. These are the steps that worked (installing from Maven Central, as specified in the plugin repo user guide). If you have any suggestions for these steps, please let me know.

@reta reta Aug 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, I was not able to have it done for 3.8.0 (see please opensearch-project/opensearch-build#6251), so we would need to follow this guide [1] (the plugin is published in Maven [2]):

bin/opensearch-plugin install org.opensearch.plugin:opensearch-jvector-plugin:3.8.0.0

@kush992 could you please update the instructions? thank you

[1] https://docs.opensearch.org/latest/install-and-configure/plugins/#core-plugins
[2] https://repo1.maven.org/maven2/org/opensearch/plugin/opensearch-jvector-plugin/

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @reta! Updated the steps in a new commit.


The `disk_ann` method supports the following parameters.

Parameter name | Default | Description

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This table doesn't have the "Required" and "Updatable" columns, as do the other parameter tables. Should we add these here?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these values are 'updateable'.
If they're not set, they get set to the 'default' values. I'm not sure if these columns would help here.

The only place where 'required' is helpful would be in the table above this for 'disk_ann' property.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a sentence about them being optional. This assumes they get default values. The Supported methods table, which has Requires training, not Required, doesn't map onto an obvious edit. And I believe "Updatable" in the other tables refers to "can be changed after creating an index", so maybe we should add an "Updatable" column?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kolchfa-aws - sorry, responded to this on slack with suggestions. Kush made changes to the PR before it got merged, so we should be good

kolchfa-aws and others added 3 commits August 25, 2026 09:48
Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
Signed-off-by: Kush Bhalodi <kush.bhalodi@ibm.com>
Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

@kolchfa-aws kolchfa-aws left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the updates, @kush992! LGTM

@kolchfa-aws
kolchfa-aws merged commit 0ca6d3b into opensearch-project:main Aug 26, 2026
4 checks passed
kolchfa-aws added a commit that referenced this pull request Aug 26, 2026
* docs: added opensearch-jvector plugin docs

---------




* chore: corrected name in opensearch-jvector docs



* docs: rephrased plugin installation



* docs: updated dead link in performance-tuning



* fix: typos in opensearch-jvector document



* Doc review



* Address comments



* docs: added updatable and required columns for disk ann params



* Update column name and add defaults for num pq spaces



---------





(cherry picked from commit 0ca6d3b)

Signed-off-by: Kush Bhalodi <kush.bhalodi@ibm.com>
Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com>
Co-authored-by: Kush <85061303+kush992@users.noreply.github.com>
Co-authored-by: Kush Bhalodi <kush.bhalodi@ibm.com>
Co-authored-by: Fanit Kolchina <kolchfa@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport 3.8 Doc review PR: Doc review in progress

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants