Repository navigation
Use the shared fixed-path registry and preserve legacy cache behavior - #620
Merged
Merged
Conversation
iskandr
marked this pull request as ready for review
September 29, 2026 15:50
Coverage Report for CI Build 36612670698Coverage increased (+0.01%) to 87.367%Details
Uncovered ChangesNo uncovered changes found. Coverage Regressions43 previously-covered lines in 1 file lost coverage.
Coverage Stats
💛 - Coveralls |
iskandr
changed the base branch from
refactor/589-datacache-downloads
to
main
September 29, 2026 17:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace hitlist's version resolution, path lookup, receipt hashing and manifest publication with datacache 1.15.0's shared
VersionedFileRegistry. Hitlist retains a small adapter for its public exception default, progress/cache messages, 300-second socket timeout and literal-URL decompression policy.Existing
<root>/<name>/<version>/<filename>files and root manifests remain in place; Path results, status keys and silentensure()cache hits are preserved. New receipts use bounded-memory hashing. Per-root writer serialization fixes the reproduced race where two successful downloads left one manifest entry.The generation-bundle registry was not a compatible storage replacement, so the reusable fixed-path implementation was moved upstream in openvax/datacache#84 and published as 1.15.0. Its 764 tests and full Linux/Python 3.9–3.14 plus macOS matrix passed; the two-process regression fails when writer serialization is removed. Published wheel/sdist hashes match the clean-master build.
Hitlist validation: format/lint and focused adapter/download/inventory/public-API checks pass against the published datacache wheel. Direct comparison with released hitlist 1.63.14 matches legacy paths, return values, status dictionaries and offline reuse, without new metadata. The full Python 3.9–3.12 unit matrix, all 45 corpus integration tests, lint, documentation build and combined coverage pass on final head 52e20fa (CI run).
The release workflow budget increases from 45 to 90 minutes to address #612: a passing serial unit phase previously took 37 minutes before integration/setup. The one-worker limit, complete test phases, memory guards and artifact/source verification remain intact. Its PR release-build gate passed in 47m54s, beyond the old timeout: 2,413 unit passes, one optional-data skip, all 45 integration passes, successful distributions and provenance artifacts. Heavy local validation was stopped after the user reported memory pressure; the partial run (1,185 passes and an ENOSPC failure) is not a passing gate. Remaining release validation runs in CI. Merged as 96035cf. The final clean-main Release build 36618631704 passed format, lint, full test.sh (2,413 unit passes, one optional-data skip, all 45 integration passes), build and distribution checks. Source/run provenance and both artifact hashes were verified against clean main before upload. Wheel metadata requires datacache >=1.15.0. PyPI 1.63.16 is published, and both published SHA-256 hashes and sizes match the tested artifacts. This stage is merged and deployed. The normal workspace environment also passes check_dev_install.py at 1.63.16 with datacache 1.15.0.
Closes #589. Closes #618. Closes #612.
Builds on the preceding downloader/inventory stage in #617, now published as 1.63.15 with verified PyPI hashes. Version 1.63.16. The separately reported training-data issues #614–#616, #18, #593, #588 and #599 remain open for the requested discussion; no training labels or exports change here.