Skip to content

Delegate CAGRA build heuristics to cuVS instead of hardcoding them - #177

Draft
imotov wants to merge 30 commits into
NVIDIA:release/26.08from
imotov:issue-149/use-cuvs-heuristics-in-cuvs-lucene
Draft

Delegate CAGRA build heuristics to cuVS instead of hardcoding them#177
imotov wants to merge 30 commits into
NVIDIA:release/26.08from
imotov:issue-149/use-cuvs-heuristics-in-cuvs-lucene

Conversation

@imotov

@imotov imotov commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Replace the hand-ported heuristic and hard-coded 5M-row algorithm threshold in CagraIndexParamsFactory with cuVS's own APIs, so cuVS stays the single source of truth.

Fixes #149

@copy-pr-bot

copy-pr-bot Bot commented Jul 20, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@imotov imotov added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Jul 20, 2026
@imotov

imotov commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test f320ada

@imotov

imotov commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 6d7fc62

@imotov

imotov commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test f9fd999

@imotov
imotov marked this pull request as ready for review July 21, 2026 01:06
@imotov
imotov requested a review from a team as a code owner July 21, 2026 01:06
@achirkin

achirkin commented Jul 22, 2026

Copy link
Copy Markdown

Let's get NVIDIA/cuvs#2345 merged and then use cagra::index_params::from_dataset for constructing CAGRA parameters independently from HNSW. The reason for this is that we will soon introduce a variable-degree graph building for HNSW by default (NVIDIA/cuvs#2031); the feature is useful for HNSW but detrimental for CAGRA GPU search.

paul-aiyedun
paul-aiyedun reviewed Jul 23, 2026
View reviewed changes

@paul-aiyedun paul-aiyedun left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes overall look good to me. I just had a few questions.

Comment thread src/main/java/com/nvidia/cuvs/lucene/GPUSearchParams.java Outdated
Comment thread src/main/java/com/nvidia/cuvs/lucene/CagraIndexParamsFactory.java Outdated
Comment thread src/main/java/com/nvidia/cuvs/lucene/CagraIndexParamsFactory.java
Comment thread src/test/java/com/nvidia/cuvs/lucene/TestCagraIndexParamsFactory.java
cjnolet
cjnolet approved these changes Jul 23, 2026
View reviewed changes

@cjnolet cjnolet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@imotov

imotov commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Let's get NVIDIA/cuvs#2345 merged and then use cagra::index_params::from_dataset for constructing CAGRA parameters independently from HNSW. The reason for this is that we will soon introduce a variable-degree graph building for HNSW by default (NVIDIA/cuvs#2031); the feature is useful for HNSW but detrimental for CAGRA GPU search.

@achirkin, it looks like your PR keeps the fromHnswParams signature intact and only refactors its internal implementation. Because of that, I'm not entirely sure why this PR needs to wait for yours. Wouldn't this PR automatically benefit from your changes once your PR is merged? As far as I can tell, there doesn't appear to be a direct dependency between the two. What am I missing?

imotov added 6 commits July 24, 2026 07:29
@imotov
Delegate CAGRA build heuristics to cuVS instead of hardcoding them
c803a00 
Replace the hand-ported heuristic and hard-coded 5M-row algorithm
threshold in CagraIndexParamsFactory with cuVS's own APIs, so cuVS stays
the single source of truth.

Fixes NVIDIA#149
@imotov
Fix checkstyle
98cea18
@imotov
Fix typo
cd9eee9
@imotov
Remove redundant NN_DESCENT parameter
9c57e39
@imotov
Add test for the writer threads with custom strategy
d6b66df
@imotov
Temporary switch to use NVIDIA/cuvs#2345
2b55668 
Switches to build against NVIDIA/cuvs#2345
@imotov
imotov force-pushed the issue-149/use-cuvs-heuristics-in-cuvs-lucene branch from a3df879 to 2b55668 Compare July 24, 2026 20:24
@imotov
imotov requested a review from a team as a code owner July 24, 2026 20:24
@imotov
imotov changed the base branch from main to release/26.08 July 24, 2026 20:31
@imotov

imotov commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 692229e

@imotov
imotov requested review from a team as code owners July 24, 2026 21:21
@imotov
imotov requested a review from jameslamb July 24, 2026 21:21
@imotov
imotov marked this pull request as draft July 24, 2026 21:23
@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test e8167e2

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 1b0fd04

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 3e7c48d

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 8e6ebe3

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test ca62402

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 6132d5f

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 1ce03b4

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 7b31187

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 1161c7d

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test df35c3f

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test a6d6b94

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test eb77f70

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 73c5828

@imotov

imotov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 1a19fdd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cjnolet cjnolet cjnolet approved these changes
@jameslamb jameslamb Awaiting requested review from jameslamb jameslamb is a code owner automatically assigned from NVIDIA/adi-packaging-codeowners
+1 more reviewer
@paul-aiyedun paul-aiyedun paul-aiyedun left review comments
Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

improvement Improves an existing functionality non-breaking Introduces a non-breaking change

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Heuristics in cuvs-lucene are being directly hardcoded based on cuVS heuristics instead of using cuVS heuristics.

4 participants

@imotov @achirkin @cjnolet @paul-aiyedun