Repository navigation
Build in version-pinned paths for well-known download hosts - #459
Conversation
Generic upstreams pointed at archive.apache.org, nodejs.org or dl.google.com/go now cache their versioned downloads as artifacts without any generic_artifacts patterns. Configured patterns add to the built-in ones.
andrew
left a comment
There was a problem hiding this comment.
The built-in patterns are added before the configured generic_artifacts in NewGenericHandler, and parseArtifact returns the first match, so a built-in overrides an overlapping user pattern. The current README recipe (dist/(?P<name>maven)/maven-3/... on https://archive.apache.org) will now resolve to maven/maven-3 instead of maven. The artifact cache is keyed on the version PURL, so everything already cached as pkg:generic/maven@... misses and is downloaded again, the old package stays listed alongside the new one, and any version deny or cooldown rule written against pkg:generic/maven stops matching.
Please compile the configured patterns first and the built-ins after them, so explicit config wins, and add a test case where a configured pattern overlaps a built-in and its name is the one returned.
The first matching pattern names the package, so an upstream's own patterns now take precedence over overlapping built-ins and keep the package names, cache entries and rules already keyed on them.
|
Thanks, good catch. Configured |
Follow-up to #424.
upstream.generic_artifactsmade non-GitHub downloads cacheable as artifacts, but every user caching mise's Maven, Node or Go downloads has to write the same regexes. Without them, those tarballs stay on the metadata route: revalidated aftermetadata_ttl, buffered againstmetadata_max_size, and not listed as packages.This adds built-in patterns keyed by upstream base URL, so naming the host is enough:
https://archive.apache.orgdist/{project dirs}/{version}/[binaries/]{file},{version}starting with a digitmaven/maven-3,maven/mvndhttps://nodejs.orgdist/v{version}/{file}https://dl.google.com/gogo{version}.{os}-{arch}.tar.gz,.zipand their.sha256/.generic_artifactspatterns add to the built-ins, so existing configs keep working, and duplicates of the built-in patterns become redundant.dist/index.json,dist/maven/maven-3/) don't match and stay on the metadata route.generic_artifactsexample now uses a host that isn't built in, and the README mise section gets a Maven/Node/Go recipe with no patterns.Tested with
go test ./...andgo vet, and by running the proxy with only thegeneric:block above against the real hosts. The first fetch ofapache-maven-3.9.9-bin.tar.gztook 79s and the second 8ms from cache. mvnd, NodeSHASUMS256.txtand Go.sha256files were also served from the artifact cache on the second fetch./api/search?ecosystem=genericlistsmaven/maven-3,maven/mvnd,nodejsandgolang. golangci-lint was not run locally.