Fix backend CI: upgrade datasets, use canonical squad id (closes #18) - #20
Merged
Merged
Conversation
CI's Backend tests check failed because `datasets` was capped `<3`, whose
dataset-loading path is incompatible with `huggingface_hub` 1.x — which
`transformers` 5.x requires (>=1.5.0). Resolving the bare `squad_v2` id under
hub 1.x raised HfUriError ("Repository id must be 'namespace/name'"). Pinning
hub down is impossible without breaking transformers.
Fix both halves of the conflict:
- requirements.txt: `datasets>=2.16,<3` -> `datasets>=3` (5.x supports hub 1.x).
- squad_v2 loader: load the canonical `rajpurkar/squad_v2` id, which hub 1.x
accepts. Dataset content and row shape are unchanged, so the frozen artifact
and the sampling tests still hold.
Verified: full suite 304 passing; the squad dataset tests pass from an empty
HF cache — the cold-start condition that fails in CI.
Closes #18.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Backend tests CI check has been red since before #17 (merged via admin override) — the pre-existing #18 dependency incompatibility. This fixes it at the root.
Root cause
A three-way version conflict:
requirements.txtcappeddatasets>=2.16,<3. The datasets 2.x load path resolves a baresquad_v2dataset id viahuggingface_hub.huggingface_hubfloated to 1.x (transitively), which rejects bare ids — it requiresnamespace/name. Soload_dataset("squad_v2")raisesHfUriErroron any machine without a warm cache (i.e. CI).<1.0is not an option:transformers 5.xrequireshuggingface_hub>=1.5.0.Fix
requirements.txt:datasets>=2.16,<3→datasets>=3— datasets 5.x supports huggingface_hub 1.x, so the whole stack (datasets / hub / transformers / sentence-transformers) is coherent (pip checkclean).src/eval/datasets/squad_v2.py: load the canonicalrajpurkar/squad_v2id, which hub 1.x accepts. Same underlying data and row shape — the checked-in frozen artifact and the sampling tests are unaffected.Verification
datasets 5.0.0+huggingface_hub 1.23.0.test_eval_datasets_squad.py) pass from an empty HF cache — the exact cold-start condition that fails in CI.Closes #18. Unblocks the required Backend-tests check so future PRs merge without admin override.