Repository navigation
Apply cooldown to Maven - #573
Merged
Merged
Conversation
Cooldown needs to read maven-metadata.xml before serving it, with the same Gradle Plugin Portal fallback and error handling.
A version's publish time is the Last-Modified of its .pom, read with a HEAD request once per version and stored in the versions table. Named repositories store theirs under a repository_url qualifier. Versions in cooldown are left out of maven-metadata.xml, <latest> and <release> are moved to the last versions left, and the metadata's checksum files are computed from what is served. Files of a version in cooldown get 403. Only versions that can still be in cooldown are checked: none when <lastUpdated> is older than the window, otherwise from the end of the list back to the first old enough version. Part of git-pkgs#132.
Includes the caveats from git-pkgs#132: Nexus and Artifactory dates, Google's Maven Central mirror, version-ordered lists and SNAPSHOTs.
A build fetching pinned versions never reads maven-metadata.xml, so each version's time was read with a HEAD before its download, roughly doubling the requests to Maven Central, which answered a fresh build with 429 and a 30-minute Retry-After. The .pom's own response now gives the time; Maven fetches the .pom before a version's other files, so their check reads the stored time. A HEAD is only sent for a .pom served from the cache or a file requested before its .pom. A 429 on that HEAD now answers 503 with the upstream's Retry-After, as other held upstream requests do, instead of 502.
- Name the proxy in the User-Agent of publish-time HEAD requests, as its artifact downloads do, rather than Go's default, which Maven Central asks tools not to send. - Take a version's time from its Gradle .module as well as its .pom, so either one fetched first carries it. - Serve filtered metadata through the shared rewrite path, with gzip and the stale warning like the other ecosystems. - Say in the docs that a refused .pom or .module is still cached.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Maven part of #132, along the lines agreed there.
With cooldown on:
Last-Modifiedof its.pom, stored in theversionstable once per version, as npm and Go do. On Maven Central it is the deploy time. When the.pomor Gradle.moduleitself is downloaded, which clients do before a version's other files, the time comes from that response, so a build of pinned versions costs no extra requests. Otherwise it is read with a HEAD request that names the proxy in its User-Agent. A.pomor.modulerefused this way is still cached, so it counts towardmax_sizebefore the version leaves cooldown. Only a 2xx response is used. A missing.pomlets the version through; any other failure gives 502 instead of letting it through, or 503 with the upstream'sRetry-Afterwhen Central rate-limits the proxy.maven-metadata.xmlonly checks versions that can still be in cooldown. If<lastUpdated>is older than the window, nothing is checked. Otherwise the proxy walks the list from the end and stops at the first version that is old enough, so usually one or two HEAD requests per artifact, once.maven-metadata.xml.<latest>and<release>move to the last versions left, and the metadata's.md5/.sha1/.sha256/.sha512are computed from what is served. The document is edited in place, so the rest stays as upstream sent it, and served like other ecosystems' rewritten metadata, with gzip and the stale warning.upstream.maven_repositoriesstore their times under arepository_urlqualifier, so they don't share times with Central.The caveats are in
docs/configuration.mdnext to the Go one: Nexus/Artifactory may report when they cached the file; on Google's mirrorLast-Modifiedis when the file reached the bucket (the proxy doesn't readx-goog-meta-last-modified); and since Central lists versions in version order, a backport to an older line can stay listed while in cooldown, though its files are refused.Checked against Maven Central with a 10-day cooldown:
<latest>and<release>became 2.22.3, the served.sha1matched, and the 2.22.4.pomgot 403. That took two HEAD requests; a second metadata request took none..pom; 2.22.3 resolves.A first version read each pinned version's time with a HEAD before its download. A fresh Maven build through it sent about 150 requests to Central in five seconds and got a 429 with a 30-minute
Retry-After; reading the time from the.pomdownload removes those HEADs. I couldn't repeat the full build after that without risking another block, so the version-range and plugin cases are covered by the unit tests only.