Skip to content

fix(install): pin embeddings snapshot prefetch to the resolved commit - #5754

Open
tang-vu wants to merge 1 commit into
Osmantic:public-betafrom
tang-vu:fix/hf-snapshot-revision
Open

tang-vu wants to merge 1 commit into
Osmantic:public-betafrom
tang-vu:fix/hf-snapshot-revision

Conversation

@tang-vu

@tang-vu tang-vu commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Why this matters

When EMBEDDING_MODEL_REVISION is unset � the default for fresh installs � the phase-11 embeddings prefetch calls download-hf-snapshot.py with no --revision, so snapshot_download() resolves the repository's mutable default branch. The installed embeddings artifact can silently change between two installs of the same ODS release, weakening reproducibility and supply-chain integrity (Bandit B615, high confidence). Fixes #5658.

What changed

ods/scripts/download-hf-snapshot.py � when no revision is given, the helper now resolves the default branch to its current commit SHA via HfApi().model_info(repo_id).sha and always passes that immutable revision to snapshot_download(). The resolved SHA is printed so the install log records exactly which artifact was fetched. An explicit --revision (e.g. EMBEDDING_MODEL_REVISION) still passes through unchanged � no caller changes needed.

Design choice per the issue's suggested fix: resolve-and-persist a commit SHA before download rather than reject omitted revisions � hard-failing would break every default install, while resolution keeps behavior identical and pins the artifact.

Regression test

test_hf_download_helper.py::test_download_snapshot_pins_resolved_commit_when_revision_omitted � fakes huggingface_hub with a HfApi returning a fixed SHA; asserts snapshot_download receives revision=<sha> (never absent, never a branch name) and that the SHA is reported on stdout. The existing explicit-revision test still passes unchanged.

Overlap check

Validation

  • Standalone harness exercising download_snapshot with faked huggingface_hub: omitted revision ? snapshot_download(revision=<40-hex sha>), resolution printed; explicit revision="v1" ? passes through. (Environment lacks pytest; the committed pytest test covers the same assertions.)
  • git diff --check: clean
  • Not verified: a real HF download (network/credentials not available in this environment).

When EMBEDDING_MODEL_REVISION is unset, the phase-11 prefetch called
download-hf-snapshot.py without --revision, so snapshot_download()
resolved the repository's mutable default branch. The installed
embeddings artifact could silently change between installs, weakening
reproducibility and supply-chain integrity (Bandit B615, issue Osmantic#5658).

The helper now resolves the default branch to its current commit SHA
via HfApi.model_info() and always passes that immutable revision to
snapshot_download(). The resolved SHA is printed so the install log
records exactly which artifact was prefetched. An explicit --revision
still passes through unchanged.

Fixes Osmantic#5658.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant