fix: add NVIDIA NIM provider profile for input_type embedding field#268
fix: add NVIDIA NIM provider profile for input_type embedding field#268vicjayjay wants to merge 4 commits intoCortexReach:masterfrom
Conversation
NVIDIA NIM rejects the `task` field and requires `input_type` instead. This adds a proper "nvidia" provider profile following the architecture introduced in CortexReach#216, rather than hardcoding URL checks in buildPayload. Detection: matches *.nvidia.com base URLs, nvidia/* model prefixes, and nv-embed* model names. Capabilities: sends input_type instead of task, maps retrieval.query → query and retrieval.passage → passage, supports encoding_format: float. Includes 5 automated tests covering: - NVIDIA sends input_type (not task) - retrieval.passage → passage value mapping - nvidia/ model prefix detection - Jina still sends task field - Generic providers send neither Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ad8c5ddd19
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
src/embedder.ts
Outdated
| "query": "query", | ||
| "passage": "passage", | ||
| }, | ||
| dimensionsField: null, |
There was a problem hiding this comment.
dimensions forwarding for dynamic NVIDIA models
For the new nvidia profile, setting dimensionsField to null means buildPayload() never forwards embedding.dimensions (src/embedder.ts:646-647), even though the constructor still treats that config as the expected vector size (src/embedder.ts:470). NVIDIA's OpenAI-compatible docs explicitly use a dimensions parameter for dynamic models such as nvidia/llama-3.2-nv-embedqa-1b-v2; with a config like dimensions: 384, this change now requests the model's default 2048-d output and immediately fails with Embedding dimension mismatch, or creates the wrong-sized table.
Useful? React with 👍 / 👎.
NVIDIA NIM's OpenAI-compatible API supports a `dimensions` parameter for dynamic models like nvidia/llama-3.2-nv-embedqa-1b-v2. Setting dimensionsField to null prevented buildPayload() from forwarding the configured dimensions, causing dimension mismatch errors. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
rwmjhb
left a comment
rwmjhb
left a comment
There was a problem hiding this comment.
Nice work — the provider profile approach following #216 is clean and extensible. Two small things:
Suggestion: update PR description
The summary still says "disables dimensionsField", but the second commit intentionally changed it to "dimensions" for dynamic models. Worth syncing the description so it doesn't confuse future readers.
Nits:
- Detection regex
/\.nvidia\.com|integrate\.api\.nvidia\.com/i— the second alternative is already matched by the first, could simplify to/\.nvidia\.com/i - Consider adding a test that detects NVIDIA via a
.nvidia.combaseURL (currently all tests use model name patterns)
Added a test case to detect NVIDIA from a .nvidia.com baseURL and modified import to include formatEmbeddingProviderError.
Simplifies the NVIDIA provider baseURL detection pattern now that .nvidia.com already covers integrate.api.nvidia.com.
|
@rwmjhb synced this up. I updated the PR description so it now reflects that NVIDIA dynamic models still forward The follow-up commits are:
I also re-checked the earlier |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 15079944d9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return "voyage-compatible"; | ||
| } | ||
|
|
||
| if (/\.nvidia\.com/i.test(base) || /^nvidia\//i.test(model) || /^nv-embed/i.test(model)) { |
There was a problem hiding this comment.
input_type
This detection is broader than the capability it enables. Any *.nvidia.com endpoint or nvidia/*/nv-embed* model now gets the nvidia profile, and buildPayload() will therefore add input_type whenever embedding.taskQuery or taskPassage is set. That also sweeps in NVIDIA-hosted embeddings such as BAAI/bge-m3, snowflake/arctic-embed-l, nvidia/nv-embed-v1, and nvidia/nv-embedcode-7b-v1, whose model docs describe plain text (or task-specific instructions) rather than the retriever-style query/passage contract. Those configs previously behaved as generic OpenAI-compatible embeddings, so this heuristic can turn valid requests into 400s or wrong embeddings for non-retriever models.
Useful? React with 👍 / 👎.
|
Follow-up commits are pushed and the PR is ready for re-check. There are currently 3 workflow runs awaiting maintainer approval on this PR before they can execute: |
2 participants
Summary
Supersedes #219. Implements the fix properly using the provider profile architecture from #216.
EmbeddingProviderProfiletype*.nvidia.combase URLs,nvidia/*model prefixes, andnv-embed*model namestaskField: "input_type"with value mapping (retrieval.query→query,retrieval.passage→passage)encoding_format: "float"and forwardsdimensionsfor NVIDIA dynamic modelsChanges
src/embedder.ts: Added the NVIDIA provider profile, keptdimensionsforwarding for dynamic models, and simplified the NVIDIA baseURL detection regex to/\.nvidia\.com/itest/nvidia-nim-provider-profile.test.mjs: 6 automated tests covering all maintainer requirements:input_type(nottask)retrieval.query/retrieval.passageare mapped correctly.nvidia.combaseURLtasktasknorinput_typeTest plan
node --test test/nvidia-nim-provider-profile.test.mjs)node test/embedder-error-hints.test.mjs)🤖 Generated with Claude Code