Add support for RTX 4500 and RTX 6000 (Blackwell)

[jrbourbeau]

Description

This PR adds Blackwell arch support for running RTX 4500 and RTX 6000 GPUs

cc @navneet1v @rchitale7 @cjnolet

Issues Resolved

Closes #143

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[jrbourbeau]

Adding a few notes to help reviewers

[Rajrahane]

Suggestion to build confidence on this PR

We verify changes to faiss-base image by building the downstream images and running an index build

Please try building the core, e2e image and running https://github.com/opensearch-project/remote-vector-index-builder/blob/main/.github/workflows/run_e2e_tests.yml

Add logs to successful index build on the RTX Blackwell, and a g5,2xlarge instance for validating backward compat

[jrbourbeau]

Thanks @Rajrahane

We verify changes to faiss-base image by building the downstream images and running an index build

I've rebuild the api-latest image (and the other images it depends on) and ran an index build on a g7 and g7e instance on AWS. I can do the same on a g5 to confirm backwards compatibility.

Please try building the core, e2e image and running https://github.com/opensearch-project/remote-vector-index-builder/blob/main/.github/workflows/run_e2e_tests.yml

I may be missing something but it looks like that workflow won't run against this PR. Is that the case? I can try running the steps manually on a g6.xlarge to try to reproduce

[Rajrahane]

If required please ask @peterzhuamazon to add a gpu runner of type g7, g7e and trigger an index build
https://aws.amazon.com/ec2/instance-types/g5/
we use g6(corrected) self-hosted runners Github runners for e2e

[jrbourbeau]

we use g5 self-hosted runners Github runners for e2e

Ah, based on this

remote-vector-index-builder/.github/workflows/run_e2e_tests.yml

Line 32 in a10104e

labels: g6xlarge

I thought it was g6.xlarge. I'm fine running it on either instance type, just wanted to know what folks here prefer

[Rajrahane]

Thanks @Rajrahane

We verify changes to faiss-base image by building the downstream images and running an index build

I've rebuild the api-latest image (and the other images it depends on) and ran an index build on a g7 and g7e instance on AWS. I can do the same on a g5 to confirm backwards compatibility.

Please try building the core, e2e image and running https://github.com/opensearch-project/remote-vector-index-builder/blob/main/.github/workflows/run_e2e_tests.yml

I may be missing something but it looks like that workflow won't run against this PR. Is that the case? I can try running the steps manually on a g6.xlarge to try to reproduce

E2e looks good, I approved the workflow run for you

[rchitale7]

Thank you @jrbourbeau for making this change! Yes, verifying this on both g5 and g6 would be ideal. If you can do that and paste the logs, that would be great.