Repository navigation
Hash path-based metadata cache keys - #578
Merged
Merged
Conversation
Contributor
|
This conflicts with main in |
Conan, CRAN, Conda, RPM, the main Debian archive and the default Maven repository keyed cached metadata by the request path with '/' replaced by '_'. Names and versions may contain '_', so foo/bar_1.0 and foo_bar/1.0 shared an entry, and with cache_metadata one reference's recipe files were served for the other. Conan also appended the query with '_', so a path without a query could collide with a shorter path plus one. Hash the upstream URL, the path and, for Conan, the query instead, as the APK, Arch, generic and Helm handlers already do. Each part is length-prefixed so no two inputs share a key.
Debian, RPM, APK, PyPI, CRAN and Maven build upstream URLs from the decoded request path, where a '?' or '#' would end the path and cache one file under another's name. Only the server middleware refused them; Arch and FreeBSD also check in the handler. Check in these handlers too, so they stay safe when mounted without the middleware.
simonchrz
force-pushed
the
fix/unambiguous-metadata-keys
branch
from
October 24, 2026 11:03
80409a9 to
d64bfc6
Compare
Contributor
Author
|
Rebased onto main, CI is green. |
Contributor
Author
|
The conda shards change added a second |
simonchrz
added
the
breaking
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Conan, CRAN, Conda, RPM, the main Debian archive and the default Maven repository keyed cached metadata by the request path with
/replaced by_. Conan names and versions may contain_, sofoo/bar_1.0andfoo_bar/1.0shared a cache entry, and withcache_metadata: trueone reference'slatestor recipe files were served for the other. Conan also appended the query with_, so/v2/conans/search?q=xand/v2/conans/search_q=xshared one too. The same can happen in the other ecosystems wherever_appears in a path.These keys are now a hash of the upstream URL, the path and, for Conan, the query, as the APK, Arch, generic and Helm handlers already do. The parts are length-prefixed before hashing so no two inputs share a key. Including the upstream URL also means repointing an upstream no longer serves what the old one cached.
Metadata cached under the old keys is not read any more and is simply fetched again on the next request. The old rows and files stay in storage; with a cache size limit, eviction removes them as least recently used, otherwise they stay until the cache is cleared. Debian's artifact filenames are unchanged.
A second commit adds the
?/#check that Arch and FreeBSD already do to the Debian, RPM, APK, PyPI, CRAN and Maven handlers. They build upstream URLs from the decoded path and relied only on the server middleware to refuse these characters.Tests:
TestConanMetadataKeysKeepReferencesApartrequests colliding Conan paths through the handler and checks each gets its own response;TestMetadataCacheKeysUnambiguouscovers the other handlers' keys;TestHandlersRejectQueryCharsInPathsends encoded?and#to each handler directly. All three fail without the changes.