Skip to content

Maven resolution: cool down after HTTP 429, dedupe repositories by URI, and let a silent mirror keep the mirrored repository's policy - #8683

Merged
jkschneider merged 3 commits into
mainfrom
worktree-8682
Aug 27, 2026
Merged

Maven resolution: cool down after HTTP 429, dedupe repositories by URI, and let a silent mirror keep the mirrored repository's policy#8683
jkschneider merged 3 commits into
mainfrom
worktree-8682

Conversation

@jkschneider

@jkschneider jkschneider commented Aug 27, 2026

Copy link
Copy Markdown
Member

1. An endpoint that answers HTTP 429 is not asked again for 60s

A 429 is transient, so it is deliberately never negative-cached, but nothing remembered it either: every later metadata or POM lookup re-asked the throttled host, and a failed metadata request was followed by a second request for the directory listing.

  • MavenExecutionContextView.getThrottledEndpoints(): a host:port -> skip-until map, sibling to getUnreachableEndpoints().
  • MavenPomDownloader.sendRequest records a 429 with a 60s cooldown; HttpSenderResponseException.isThrottled() added.
  • distinctNormalizedRepositories skips a cooling-down repository and reports it through ResolutionEventListener.repositoryAccessFailedPreviously; deriveMetadata does not follow a 429 with the listing request.

Retry-After is not honored yet, as the issue anticipated.

2. Repositories are deduplicated by URI, not by id

distinctNormalizedRepositories deduped on the post-mirror id, so two entries with different ids and the same URL cost a request each per lookup. The seen set is now keyed on the URI that would be asked (trailing slash trimmed, scheme/host case-folded); first occurrence wins so order and credentials are preserved. The "central" id check is untouched, since that is Maven's override rule for the implicit Central.

3. A mirror without a policy of its own keeps the mirrored repository's

MavenRepositoryMirror.apply widened the mirrored repository to releases=true, snapshots=true unless the mirror said false. Maven's DefaultMirrorSelector.getMirror copies the mirrored repository's policies onto the mirror instead, so mirroring Central through a <mirror> no longer turns it into a snapshot-accepting repository. Explicit values on the mirror still override.

Sections 2 and 3 are not quite independent

When several repositories collapse onto one mirror, Maven unions their policies (DefaultRemoteRepositoryManager.mergeMirrors); the old widening was accidentally approximating that. With per-repository policy on the mirror, "first occurrence wins" would have kept releases-only mirrored-Central and dropped the snapshot-accepting mirrored repository. download already passes the version into the iterator, so the first accepting occurrence wins — the union for a single lookup — but downloadMetadata filtered by version only after deduplication and would have stopped resolving -SNAPSHOT metadata through a * mirror. It now passes the version through like download does, and its redundant in-loop check is gone. mirrorKeepsThePolicyOfEachRepositoryItMirrors covers exactly this case.

Verification

  • Full :rewrite-maven:test: 1515 tests, 0 failures.
  • rewrite-gradle tests that go through mirrors (EffectiveGradleRepositoriesTest, FindRepositoryOrderTest, ChangeDependencyConcurrencyTest, UseJavaExtensionBlockTest): green.

A 429 is transient, so it is deliberately never negative-cached; but
nothing remembered it either, so every subsequent metadata or POM lookup
re-asked the throttled host, and a failed metadata request was followed
by a second request for the directory listing.

MavenExecutionContextView now keeps a host:port -> skip-until map next to
the unreachable-endpoints set. sendRequest populates it on a 429 with a
60s cooldown; distinctNormalizedRepositories skips a cooling-down
repository (reported through repositoryAccessFailedPreviously) and
deriveMetadata does not follow a 429 with a listing request.

Part 1 of #8682.
distinctNormalizedRepositories deduped on the post-mirror id, so two
entries with different ids and the same URL (generated settings, a POM
re-declaring a configured repository) cost a request each per lookup.
Key on the URI that would be asked instead, with the trailing slash
trimmed and the host compared case-insensitively; the first occurrence
wins so order and credentials are preserved. The "central" id check
stays on id, since that is Maven's override rule for the implicit
Central.

Part 2 of #8682.
…ored repository's

MavenRepositoryMirror.apply set the mirrored repository's releases and
snapshots to "true" unless the mirror explicitly said "false". Maven's
DefaultMirrorSelector copies the mirrored repository's policies onto the
mirror instead, so mirroring Central (releases-only) through a settings
<mirror> should not make it a snapshot-accepting repository that joins
every -SNAPSHOT lookup.

When several repositories collapse onto one mirror, Maven unions their
policies (DefaultRemoteRepositoryManager.mergeMirrors). The lazy
repository iterator already yields the first occurrence that accepts
the version being looked up, which is the same thing for a single
lookup, but downloadMetadata was filtering by version only after
deduplication and so would have dropped the snapshot-accepting mirrored
repository in favour of the releases-only one. It now passes the
version through to distinctNormalizedRepositories like download does.

Part 3 of #8682.
@github-project-automation github-project-automation Bot moved this to In Progress in OpenRewrite Aug 27, 2026
@jkschneider
jkschneider merged commit d88d458 into main Aug 27, 2026
1 check passed
@jkschneider
jkschneider deleted the worktree-8682 branch August 27, 2026 18:44
@github-project-automation github-project-automation Bot moved this from In Progress to Done in OpenRewrite Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Maven resolution: cool down endpoints after HTTP 429, dedupe repositories by URI, and let a silent mirror inherit the mirrored repository's policy

1 participant