Upgrade hf-hub to 1.0.0-rc.1#865
Draft
assafvayner wants to merge 3 commits into
Draft
Conversation
The 1.0 release of hf-hub is a full API rewrite — the `Api`/`ApiBuilder`/
`ApiRepo` types are gone, replaced by `HFClient`/`HFRepository`. This change
migrates the workspace to the new surface and switches the TLS backend to
rustls.
- Workspace dep: `hf-hub = "0.4"` (features `tokio`) → `1.0.0-rc.1`
(features `rustls-tls`).
- `backends/candle` dev-dep: feature `ureq` → `blocking` (sync API).
- `core` gains a dev-dep on `hf-hub` with `blocking` for the in-tree
tokenization test that uses the sync surface directly.
- New `text_embeddings_backend::ModelRepo` bundles
`(HFRepository<RepoTypeModel>, Option<revision>)` to replace the old
`ApiRepo` (which carried the revision internally — the new API takes it
per-call).
- Calls migrated:
- `ApiBuilder::from_env().with_progress(false).with_*` →
`HFClient::builder().cache_dir(..).token(..).build()`. The
`HF_HUB_USER_AGENT_ORIGIN` env var is read inside `build()`, so the
explicit `with_user_agent("origin", ..)` call is no longer needed.
- `api.repo(Repo::with_revision(id, RepoType::Model, rev))` →
`client.model(owner, name)` with revision threaded per-call.
- `api_repo.get(file).await` →
`repo.download_file().filename(file).maybe_revision(..).send().await`.
- `api.info().await` → `repo.info().maybe_revision(..).send().await`;
`info.siblings` is now `Option<Vec<RepoSibling>>`, handled with
`unwrap_or_default()`.
- `JoinHandle` results: `HFError` does not impl `From<JoinError>`, so
the awaited handle is mapped to `HFError::Other` explicitly.
- Tests in `backends/candle/tests/common.rs` switched to the sync
`HFClientSync`/`HFRepositorySync` handles and a small `fetch` helper
to keep call sites readable.
Verified: `cargo check --tests --workspace` and
`cargo clippy --workspace --tests -- -D warnings` both pass.
Per review: instead of a struct bundling `HFRepository<RepoTypeModel>` and `Option<String>`, plumb both as separate parameters through the download helpers. `HFRepository` is already cheap to clone (`Arc<HFClientInner>` inside), so the explicit `Arc<ModelRepo>` wrapping in the spawn closures is also gone — closures clone the repo handle directly and the revision `Option<String>` per task.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Upgrade
hf-hubfrom0.4to1.0.0-rc.1. The 1.0 release is a full API rewrite —Api/ApiBuilder/ApiRepoare gone, replaced byHFClient/HFRepository. This PR migrates the workspace to the new surface and switches the TLS backend to rustls.Filed as a draft so it can sit behind the final 1.0 release on crates.io if desired; ready for review against the RC.
Notable changes
hf-hub = "0.4"(featuretokio) →1.0.0-rc.1(featurerustls-tls).backends/candledev-dep: featureureq→blocking(sync API in tests).coredev-dep: newhf-hub = { workspace = true, features = ["blocking"] }for the in-tree tokenization test that uses the sync surface.text_embeddings_backend::ModelRepo: bundles(HFRepository<RepoTypeModel>, Option<String> /* revision */). The oldApiRepocarried the revision internally; the new API takes it per-call, so we keep them together for ergonomic plumbing throughdownload_artifactsetc.ApiBuilder::from_env().with_progress(false).with_*→HFClient::builder().cache_dir(..).token(..).build(). TheHF_HUB_USER_AGENT_ORIGINenv var is read insidebuild(), so the explicitwith_user_agent("origin", ..)call is no longer needed.api.repo(Repo::with_revision(id, RepoType::Model, rev))→client.model(owner, name)with revision threaded per-call.model_idis split on/(anyhow context if malformed).api_repo.get(file).await→repo.download_file().filename(file).maybe_revision(..).send().await.info():api.info().await→repo.info().maybe_revision(..).send().await.info.siblingsis nowOption<Vec<RepoSibling>>, handled withunwrap_or_default().HFErrordoes not implFrom<JoinError>, so the awaited handle is mapped toHFError::Other(...)explicitly instead ofhandle.await??.Test plan
cargo check --tests --workspacecargo clippy --workspace --tests -- -D warningshf-hub@1.0.0(final) is published, bumping the dep