Repository navigation
fix(hermetic): rewrite only Fedora-mirror URLs across retries - #1193
kylerankin wants to merge 3 commits into
Conversation
|
hive-base-moved: based on projectbluefin/main at Review feedback, addressed:
🐝 Hive Agent: |
|
Needs a decision, not a merge: check Head: verdict pending @ae8cec5, 27/32 checks green, red: Canary, soak 0s of 30m0s. Clear the cause and the hive re-evaluates on its next pass; the stage moves to Updated 2026-11-17 21:23 UTC. |
There was a problem hiding this comment.
Summary: This change makes the package build retry a failed download from a different public Fedora server, and it is careful to leave packages from other sources (such as Red Hat's Hummingbird repository) pointing where they already were. That part works. But as written, the build never tries the server that was originally chosen, even on the first attempt, so a build that would have succeeded can now fail. Also, the description says this only narrows an existing behavior, when in fact it adds the whole server-switching feature.
What I need from you: Make the first download attempt use the original server, and update the PR description to say this PR adds the retry-on-another-server feature.
correctness
- See the inline note on
tools/hermetic_build.sh:191. Before the first attempt, the lock's URLs are already rewritten tomirror.fcix.net, so the mirror that mock resolved is never used.
intent-alignment
- The PR body says "The mirror-rotation step rewrote every http(s) URL in the remote lock" and calls this PR "Split out from #1191". At the merge base,
tools/hermetic_build.shhas no mirror rotation. In this PR the script changes are +42/−0, and commitd5c049e0("rotate the remote lock's mirror across retries") adds the rotation. #1191 is an unrelatedtools/validate.pyratchet change. After the squash,mainwill get a new behavior: every remote lock URL under/fedora/linux/is redirected to one of five hard-coded third-party mirrors (tools/hermetic_build.sh:195-201). The description never mentions it. Suggested body opening:Add mirror rotation to the hermetic lane's remote fetch: before each attempt, rewrite Fedora-mirror URLs in the remote lock to the next host in a fixed list, normalising
/fedora/fedora/linux/to/fedora/linux/. Non-Fedora URLs (Hummingbird on packages.redhat.com, file://) are left verbatim.
docs-currency (advisory)
- The script header says the fetch step pulls "from Hummingbird's repositories" (
tools/hermetic_build.sh:37). Its outputs list (tools/hermetic_build.sh:21-24) does not name the newfetch.mirror.indexfile. Suggested change: add "and, for Fedora packages, from a fixed list of public Fedora mirrors rotated per retry", and addfetch.mirror.indexto the outputs.
Limits of this review: Unit tests pass at this head. The Canary check fails only because this is a fork PR (it needs the maintainer-run "Canary for a fork pull request"). So nothing has run the real mock-hermetic-repo fetch against the five listed mirrors. I did not check that each mirror serves /fedora/linux/ or that it carries updates/ without sync lag.
No findings from: security.
Confidence: 3/5 (needs attention) — 2 medium findings
— hive: agent=reviewer backend=claude model=claude-opus-5-5 effort=medium claude=2.1.325
| # mirror we are on; clean_repo removes only the repo, so the index | ||
| # survives and each call to this function tries a different one. | ||
| index_file="$H/fetch.mirror.index" | ||
| index=$(cat "$index_file" 2>/dev/null || echo 0) |
There was a problem hiding this comment.
correctness (medium): the first attempt already leaves the resolved mirror. There is no index file on the first call, so index is 0 and the heredoc rewrites every Fedora URL to mirror.fcix.net before the first mock-hermetic-repo run. The mirror that the metalink handed mock is never tried. That is the mirror that had these exact NEVRAs when the lock was resolved. A hard-coded mirror that lags on updates/ can 404 on a fresh RPM. With the default HERMETIC_NET_ATTEMPTS=3, only 3 of the 5 listed hosts are tried, so a build that would have passed on its first attempt can now fail. Fix: keep the lock as resolved for attempt 1 and rotate only on retries. The tests call the heredoc directly, so they are unaffected.
index_file="$H/fetch.mirror.index"
- index=$(cat "$index_file" 2>/dev/null || echo 0)
- python3 - "$H/lock/remote.json" "$index" "$index_file" <<'PY'
+ if [ ! -f "$index_file" ]; then
+ # First attempt: use the mirror the lock was resolved against.
+ echo 0 > "$index_file"
+ else
+ index=$(cat "$index_file")
+ python3 - "$H/lock/remote.json" "$index" "$index_file" <<'PY'
...
PY
+ fi| # point at the next working one. It must rewrite only URLs that already | ||
| # target a Fedora mirror path; a Hummingbird (packages.redhat.com) or | ||
| # other project URL is left exactly as-is, so a non-Fedora package never | ||
| # has its source hijacked to a public mirror. (Review on this PR: the |
There was a problem hiding this comment.
style (low): "(Review on this PR: the original rewrite() replaced every http(s) URL.)" describes history inside this PR. After the squash, no version of rewrite() ever replaced every URL, so readers will be confused. The two new tests also extract the same heredoc with slightly different slicing (lines 210-213 vs 257-259), and from pathlib import Path at line 254 repeats the module import at line 10. Suggested: drop the parenthetical and the local import, and share one _mirror_rewrite_body() helper.
| # has its source hijacked to a public mirror. (Review on this PR: the | |
| # has its source hijacked to a public mirror. |
|
@kylerankin this PR is back in your court ( Review asks:
Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path. Queued in your relay fix queue (4 waiting): fixes are handed out before new work as soon as you have capacity. |
|
@kylerankin the hive has adopted this PR ( |
|
The hive adopted this PR ( Blocker: this PR comes from a fork ( Moving to |
hanthor
left a comment
There was a problem hiding this comment.
Requesting changes. The first-attempt problem is fixed: the lock's own mirror is now tried first. But the direction needs to change:
- Trust change. Each retry rewrites every Fedora URL to one of five hard-coded third-party hosts (
mirror.fcix.net,mirror.servaxnet.com, ...). That's a decision about which servers the build trusts, and it shouldn't arrive in a retry fix. I'm escalating it to the maintainers rather than deciding it here. - Suggested alternative. On a retry, rewrite to Fedora's own redirector (
https://download.fedoraproject.org/pub/fedora/linux/...), which picks a live mirror from Fedora's mirror list. Or re-run the lock resolve so mock asks the metalink again. Either way, no host list gets hard-coded. - If a hard-coded list is kept, the PR body should say how each fetched RPM is verified against the lock (checksum or signature), so a reviewer can see that mirror choice doesn't affect integrity.
This doesn't overlap #1162, which covers the Hummingbird server, not Fedora mirrors.
Reviewed on behalf of @hanthor with Claude Code
ae8cec5 to
eca3f14
Compare
|
🤖 Hive base repair. Rebased this branch onto the current 🐝 Hive Agent: |
eca3f14 to
9b480c9
Compare
|
Base moved — this branch has been rebased onto the current
The new main commits touch only 🐝 Hive Agent: |
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. Reviewed by Reviewer notesSummary: This change makes the package build retry a download from a different public Fedora mirror when the first one is down, instead of retrying the same dead server. The latest update keeps the original mirror for the first try and switches only on retries. I re-checked just that update and found no problems. Delta re-review
Two notes for the maintainer, not blockers:
No findings from: correctness, security, intent-alignment, style, docs-currency. |
|
Follow-up on my earlier review. The retry is a good idea, but the five hard-coded third-party mirrors can't land. They add hosts we'd trust that Fedora's mirror list doesn't vouch for. I opened #1195, which keeps your retry and credits you as co-author. On a retry it sends Fedora URLs to Fedora's redirector, Once #1195 lands, I'd recommend closing this PR as superseded. If you'd rather finish it here, make the same change and drop the mirror tuple, and I'll review it. Reviewed on behalf of @hanthor with Claude Code |
The hermetic lane resolves its base RPMs through a single mirror the metalink handed mock. The failing run resolved to mirror.acm.umn.edu, which is down, and every online retry then hit that same dead host, so a flaky mirror failed the build. Rewrite the lock to the next working mirror before each attempt in fetch_remote. The mirror bakes in the non-standard /fedora/fedora/linux/ path prefix; strip the redundant fedora/ segment to the /fedora/linux/ prefix every other mirror serves, so the rewrite holds for the main repo (releases/44/os/...) and updates (updates/44/...) alike, and for any Fedora version. A small index file tracks which mirror we are on; clean_repo removes only the repo, so the index survives and each retry tries a different one. The online wrapper is untouched -- only fetch_remote changed -- so the retry count and cleanup contract are unchanged. Partially addresses projectbluefin#1189 Signed-off-by: kylerankin <kylerankin@users.noreply.github.com>
Review: rewrite() replaced every http(s) URL in the lock, hijacking a Hummingbird (packages.redhat.com) or other project source to a public mirror. Match only URLs that already carry a /fedora/linux/ mirror path and strip the redundant /fedora/fedora/ segment; leave every other URL verbatim. A Hummingbird fixture in the test lock confirms it comes through unchanged.
Skip the rewrite on the first attempt and rotate only on retries; document the rotation and fetch.mirror.index; share one heredoc helper in tests. Hive-Fix-Attempt: 1/6 Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
9b480c9 to
3a32ae9
Compare
|
hive-base-moved: rebased onto 🐝 Hive Agent: |
There was a problem hiding this comment.
Decision needed: @hanthor should decide whether to close #1193 in favour of #1195, which carries the same retry idea through Fedora's redirector instead of five hard-coded mirrors.
Superseded by #1195; the two conflict and cannot both merge.
I read both diffs against main (7f6c5e4). Both change fetch_remote() in tools/hermetic_build.sh to leave the first lock fetch attempt as resolved and to rewrite Fedora URLs in $H/lock/remote.json on every retry. They use the same FEDORA_PATH regex, so packages.redhat.com and file:// URLs are left alone in both. A three-way merge of the two heads conflicts in fetch_remote().
The difference that matters is where the retries go:
- #1193: rotates through five hard-coded third-party hosts (
mirror.fcix.net,volico.mm.fcix.net,paducahix.mm.fcix.net,mirror.servaxnet.com,fedora.mirror.constant.com). It tracks progress infetch.mirror.index. That adds five fixed hosts to the build's trust set, and each one can go stale or disappear. - #1195: sends retries to
download.fedoraproject.org. That redirector draws from the same MirrorManager pool the metalink already uses, so no new hosts are trusted and no index file is needed. It also resets its attempt marker on everymaterialize, and it updates thebuild-failure-triageskill.
#1195's body says it is the follow-up to this PR and that it drops the mirror list on purpose. It is also already reviewed. If you agree, this PR could be closed in favour of #1195. I have not closed anything.
Confidence: 0/5 (do not merge) — 2 medium findings; reject verdict
— hive: agent=reviewer-2 backend=claude model=claude-opus-5-5 effort=medium claude=2.1.326
…1195) The lock names whichever Fedora mirror the metalink handed mock, so a dead mirror failed every retry. The first fetch still uses the lock as resolved. Each retry rewrites Fedora mirror URLs to download.fedoraproject.org, which sends every request to a live mirror from the same MirrorManager pool the metalink uses. There's no host list of our own, and nothing new to trust. Hummingbird and file:// URLs are never rewritten. Based on the retry idea in #1193 by kylerankin, without its hard-coded third-party mirror list. Refs #1193 Assisted-by: Claude Opus 5.5 via Claude Code Signed-off-by: James Reilly <jreilly1821@gmail.com> Co-authored-by: kylerankin <kylerankin@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
Fixed by #1195 (merged): retries now rewrite only Fedora-mirror URLs. |
fix(hermetic): rotate the remote lock's Fedora mirror across fetch retries
The lock bakes in whichever mirror the metalink handed mock, so a dead mirror
failed every retry. This adds mirror rotation to the
lock fetchstep intools/hermetic_build.sh:URLs to the next of five public mirrors, tracked in
fetch.mirror.index./fedora/linux/path are rewritten, and theredundant
/fedora/fedora/segment is stripped. Hummingbird(packages.redhat.com) and
file://URLs are left verbatim.Tests cover first-attempt passthrough, rotation across hosts, and the
non-Fedora fixture.
🐝 Hive Agent:
contributor