Repository navigation
fix(hermetic): retry a failed lock fetch through Fedora's redirector - #1195
Conversation
The lock names whichever Fedora mirror the metalink handed mock, so a dead mirror failed every retry. The first fetch still uses the lock as resolved. Each retry rewrites Fedora mirror URLs to download.fedoraproject.org, which sends every request to a live mirror from the same MirrorManager pool the metalink uses. There's no host list of our own, and nothing new to trust. Hummingbird and file:// URLs are never rewritten. Based on the retry idea in #1193 by kylerankin, without its hard-coded third-party mirror list. Refs #1193 Co-authored-by: kylerankin <kylerankin@users.noreply.github.com> Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: James Reilly <jreilly1821@gmail.com>
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. ℹ️ Sentinel: informational, author is trusted (owner), merge is not blocked. Head
No |
There was a problem hiding this comment.
Summary: This change makes the build system try a different download server when fetching Fedora packages fails, instead of retrying the same broken server. It uses Fedora's own server picker rather than a list we maintain, which is what the earlier attempt's review (#1193) asked for. It looks correct and safe to merge. One small suggestion: when a retry can't switch servers because the server's web address has an unusual layout, the log says nothing about it.
Reviewed head 6f998c3f. I read tools/hermetic_build.sh (the online retry loop at 48-68, materialize at 173-226), tools/mock_config.py and the new test.
How it works: the first fetch uses the lock as written. On each retry, redirect_fedora_urls (157-171) rewrites matching URLs in remote.json only. buildroot_lock.json, which the offline build reads, is never touched. The rewrite is idempotent: a redirector URL matches again and comes out unchanged, so attempt 3 behaves the same as attempt 2. The marker is cleared per materialize (line 201), so one run's retries don't affect the next.
One advisory, posted inline: if a mirror's URL doesn't match the pattern, the retry quietly does nothing for that URL.
No findings from: security, intent-alignment, style, docs-currency. Security note: every Fedora/Hummingbird repo in tools/mock_config.py:95-151 already sets gpgcheck=0. The redirector serves from the same MirrorManager pool the metalink at 113/120 uses, so this PR adds no new trusted party. I did not check whether mock-hermetic-repo verifies digests from the lock. The PR says integrity checking is unchanged, and the diff agrees.
Confidence: 5/5 (safe)
— hive: agent=reviewer backend=claude model=claude-opus-5-5 effort=medium claude=2.1.326
| python3 - "$1" <<'PY' | ||
| import json, re, sys | ||
| path = sys.argv[1] | ||
| FEDORA_PATH = re.compile(r"^https?://[^/]+/(?:pub/)?fedora/(?:fedora/)?linux/") |
There was a problem hiding this comment.
correctness (low, advisory): This pattern only matches mirrors that serve Fedora under /fedora/linux/, /pub/fedora/linux/ or /fedora/fedora/linux/. A mirror with a different layout is left unchanged: for example /mirrors/fedora/linux/ or /linux/fedora/linux/ (I tested both against this regex). If the metalink picked a dead mirror with one of those layouts, all three attempts still hit it, and the ::warning ... retry line doesn't say that nothing was rewritten. Printing a count makes that visible in the log:
| FEDORA_PATH = re.compile(r"^https?://[^/]+/(?:pub/)?fedora/(?:fedora/)?linux/") | |
| FEDORA_PATH = re.compile(r"^https?://[^/]+/(?:pub/)?fedora/(?:fedora/)?linux/") | |
| REDIRECTOR = "https://download.fedoraproject.org/pub/fedora/linux/" | |
| lock = json.load(open(path)) | |
| moved = 0 | |
| for r in lock["buildroot"]["rpms"]: | |
| new = FEDORA_PATH.sub(REDIRECTOR, r["url"], count=1) | |
| moved += new != r["url"] | |
| r["url"] = new | |
| json.dump(lock, open(path, "w")) | |
| print(f"retry: {moved} Fedora URLs sent through download.fedoraproject.org") |
(Replace lines 164-169 with this.) Not blocking.
|
Ready for a human merge: verdict approved @6f998c3, 173/173 checks green, soak 30m0s elapsed, labels: hold. Held: the Confirm the intent (resolve the sentinel alert or lift the hold), then approve and merge it on GitHub if you accept it. |
…edirector Signed-off-by: James Reilly <jreilly1821@gmail.com> # Conflicts: # tools/hermetic_build.sh
Follow-up to #1193 by @kylerankin. It keeps that PR's retry idea and drops the hard-coded third-party mirror list, as the review there asked.
Change
tools/hermetic_build.sh: the firstlock fetchattempt uses the lock as resolved. Each retry rewrites Fedora mirror URLs (…/fedora/linux/…,…/pub/fedora/linux/…,…/fedora/fedora/linux/…) tohttps://download.fedoraproject.org/pub/fedora/linux/…. The redirector sends each request to a live mirror from the same MirrorManager pool the metalink already uses, so the set of hosts we trust doesn't change. Hummingbird (packages.redhat.com) andfile://URLs are never rewritten. There's no mirror index file. The attempt marker is reset on everymaterialize.tests/test_hermetic_lane.py: the first attempt is unchanged, the rewrite targets only the redirector, non-Fedora URLs are untouched, and no mirror hostnames are in the script.build-failure-triageskill: notes the redirector retry and says not to add mirror lists.Integrity checking is unchanged by this PR.
Verification
just check: 1654 passed.download.fedoraproject.org/pub/fedora/linux/releases/44/…/zlib-ng-compat-2.3.3-3.fc44.x86_64.rpmanswered 302 to a live mirror and the RPM downloaded, withrpmkeys -Kreporting digests OK.Refs #1193
Reviewed on behalf of @hanthor with Claude Code
🤖 Generated with Claude Code