Skip to content

QVAC-22141 feat[api]: add fallbackSrc to loadModel - #3845

Merged
arun-mani-j merged 5 commits into
mainfrom
arun-mani-j/fallbackSrc
Aug 18, 2026
Merged

QVAC-22141 feat[api]: add fallbackSrc to loadModel#3845
arun-mani-j merged 5 commits into
mainfrom
arun-mani-j/fallbackSrc

Conversation

@arun-mani-j

Copy link
Copy Markdown
Contributor

🎯 What problem does this PR solve?

  • On some networks the model registry is unreachable, so loadModel() with a built-in catalog constant fails with no alternative and the app cannot load the model (#3224).
  • There is no supported way to point a catalog model at an alternate download source while still getting the catalog's checksum and metadata.

📝 How does it solve it?

  • Adds an optional fallbackSrc (an HTTP URL or a local file path) to loadModel(). When a built-in registry model's download fails, the SDK loads that model from fallbackSrc instead.
  • The bytes from fallbackSrc are validated against the catalog model's sha256 checksum before use, so an alternate source is trusted to the same degree as the registry copy.
  • fallbackSrc is honored only when modelSrc is a built-in catalog model — that constant supplies the checksum. It is rejected for a local-path/URL modelSrc, and a registry:///pear:// fallback is rejected (it would re-enter the transport that just failed).
  • Error surfaces are clear: a missing fallback file returns a not-found error; a mismatched file returns a checksum-validation error. The switch to the fallback is logged at info.
  • Scope: the fallback engages when the registry download fails (rejects). It does not cover a network that black-holes the P2P transport indefinitely without erroring.

🧪 How was it tested?

  • Unit tests: fallbackSrc is accepted on the wire and carried through the options→request transform.
  • Bare-runtime tests: the primary/fallback decision (primary success, fallback on failure, neutral error when no fallback, cancel is never overridden, unknown-model rethrow) and the fallback resolver (checksum match/mismatch, P2P-source rejection, no-checksum skip).
  • End-to-end on macOS: with a built-in Llama constant, simulated an unreachable registry and confirmed loadModel() falls back to a local copy of the model, validates it against the catalog checksum, and streams a completion. Also confirmed a nonexistent fallbackSrc surfaces a not-found error and a wrong-content file surfaces a checksum-validation error.

🔌 API Changes

import { loadModel, LLAMA_3_2_1B_INST_Q4_0 } from "@qvac/sdk";

// Load a catalog model, falling back to an alternate source when the registry is unreachable.
const modelId = await loadModel({
  modelSrc: LLAMA_3_2_1B_INST_Q4_0,
  fallbackSrc: "https://mirror.example.com/llama-3.2-1b-instruct-q4_0.gguf", // or a local file path
});

@arun-mani-j
arun-mani-j requested review from a team as code owners August 13, 2026 12:40
@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Review Status

Current Status: ✅ APPROVED
Approvals so far: Team Lead: 1, Member: 1

@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

License compliance — clean

No new dependency license findings in this PR.

Warn-only (shadow) mode — this check does not block merges yet.

Updated automatically by the canonical license compliance workflow.

NOTICE presence (advisory)

Missing NOTICE (advisory, does not block):

  • ./.github/actions/release-merge-guard
  • ./docs/website
  • ./packages/ggml-coload-smoke
  • ./packages/fabric/test/integration
  • ./packages/inference-addon-cpp/mobile
  • ./packages/sdk/e2e
  • ./packages/llm-llamacpp/benchmarks/performance
  • ./packages/llm-llamacpp/benchmarks/server
  • ./packages/vla-ggml/sim/server
  • ./packages/embed-llamacpp/benchmarks/performance
  • ./packages/embed-llamacpp/benchmarks/server
  • ./packages/asr-ggml/benchmarks/server

@opaninakuffo opaninakuffo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adds fallbackSrc for catalog loadModel() with checksum validation — right shape for #3224, and the bare tests cover the decision matrix well.

two blockers before merge: (1) Generate + test is red — schema.json gained fallbackSrc but packages/sdk-python/.../_generated wasn't regenerated (scripts/generate.py --check); (2) any primary failure for a known catalog model without fallbackSrc gets rewritten into a "registry on this network" ModelLoadFailedError, which masks typed errors like ChecksumValidationFailedError.

Comment thread packages/sdk/server/rpc/handlers/load-model/resolve.ts Outdated
Comment thread packages/sdk/contract/schema.json
Comment thread packages/sdk/server/rpc/handlers/load-model/resolve.ts
Comment thread packages/sdk/server/rpc/handlers/load-model/resolve.ts
opaninakuffo
opaninakuffo previously approved these changes Aug 17, 2026
lauripiisang
lauripiisang previously approved these changes Aug 17, 2026

@lauripiisang lauripiisang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, although the error message can be improved. Please resolve comment if you see it as not necessarily valuable.

Comment thread packages/inference/src/handlers/load-model/resolve.ts Outdated
Comment thread packages/sdk/server/rpc/handlers/load-model/resolve.ts Outdated
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
@arun-mani-j
arun-mani-j merged commit be7f9dc into main Aug 18, 2026
55 checks passed
@arun-mani-j
arun-mani-j deleted the arun-mani-j/fallbackSrc branch August 18, 2026 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants