Skip to content

fix(hermetic): retry a failed lock fetch through Fedora's redirector - #1195

Merged
hanthor merged 2 commits into
mainfrom
fix/hermetic-fedora-redirector
Nov 18, 2026
Merged

hanthor merged 2 commits into
mainfrom
fix/hermetic-fedora-redirector

Conversation

@hanthor

@hanthor hanthor commented Nov 18, 2026

Copy link
Copy Markdown
Member

Follow-up to #1193 by @kylerankin. It keeps that PR's retry idea and drops the hard-coded third-party mirror list, as the review there asked.

Change

  • tools/hermetic_build.sh: the first lock fetch attempt uses the lock as resolved. Each retry rewrites Fedora mirror URLs (…/fedora/linux/…, …/pub/fedora/linux/…, …/fedora/fedora/linux/…) to https://download.fedoraproject.org/pub/fedora/linux/…. The redirector sends each request to a live mirror from the same MirrorManager pool the metalink already uses, so the set of hosts we trust doesn't change. Hummingbird (packages.redhat.com) and file:// URLs are never rewritten. There's no mirror index file. The attempt marker is reset on every materialize.
  • tests/test_hermetic_lane.py: the first attempt is unchanged, the rewrite targets only the redirector, non-Fedora URLs are untouched, and no mirror hostnames are in the script.
  • build-failure-triage skill: notes the redirector retry and says not to add mirror lists.

Integrity checking is unchanged by this PR.

Verification

  • just check: 1654 passed.
  • In the pinned buildroot image, download.fedoraproject.org/pub/fedora/linux/releases/44/…/zlib-ng-compat-2.3.3-3.fc44.x86_64.rpm answered 302 to a live mirror and the RPM downloaded, with rpmkeys -K reporting digests OK.
  • I haven't seen it run through a real failed fetch.

Refs #1193

Reviewed on behalf of @hanthor with Claude Code

🤖 Generated with Claude Code

The lock names whichever Fedora mirror the metalink handed mock, so a dead
mirror failed every retry. The first fetch still uses the lock as
resolved. Each retry rewrites Fedora mirror URLs to
download.fedoraproject.org, which sends every request to a live mirror
from the same MirrorManager pool the metalink uses. There's no host list
of our own, and nothing new to trust. Hummingbird and file:// URLs are
never rewritten.

Based on the retry idea in #1193 by kylerankin, without its hard-coded
third-party mirror list.

Refs #1193

Co-authored-by: kylerankin <kylerankin@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 via Claude Code
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
@hivecommons-hive

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

ℹ️ Sentinel: informational, author is trusted (owner), merge is not blocked. Head 6f998c3f46e1 matched:

  • remote_code_execution: adds a remote-fetch-and-execute, decoded payload, reverse shell or embedded credential (tests/test_hermetic_lane.py)

No sentinel-alert label was added. The finding is recorded in the dashboard Security tab and /api/sentinel; set sentinel.trusted_authors_block: true to block trusted authors too.

@hivecommons-hive hivecommons-hive Bot added author/clanker A contributor's own hive authored this PR under their login dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 18, 2026
@hanthor hanthor added the hold Work is intentionally paused. label Nov 18, 2026
@hanthor

hanthor commented Nov 18, 2026

Copy link
Copy Markdown
Member Author

Holding this until a related build-hardening change lands first. That change has to be in place before retries go through the redirector. I'll lift the hold then, and #1193 gets closed once this merges.

Decided by @hanthor (posted with Claude Code)

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary: This change makes the build system try a different download server when fetching Fedora packages fails, instead of retrying the same broken server. It uses Fedora's own server picker rather than a list we maintain, which is what the earlier attempt's review (#1193) asked for. It looks correct and safe to merge. One small suggestion: when a retry can't switch servers because the server's web address has an unusual layout, the log says nothing about it.


Reviewed head 6f998c3f. I read tools/hermetic_build.sh (the online retry loop at 48-68, materialize at 173-226), tools/mock_config.py and the new test.

How it works: the first fetch uses the lock as written. On each retry, redirect_fedora_urls (157-171) rewrites matching URLs in remote.json only. buildroot_lock.json, which the offline build reads, is never touched. The rewrite is idempotent: a redirector URL matches again and comes out unchanged, so attempt 3 behaves the same as attempt 2. The marker is cleared per materialize (line 201), so one run's retries don't affect the next.

One advisory, posted inline: if a mirror's URL doesn't match the pattern, the retry quietly does nothing for that URL.

No findings from: security, intent-alignment, style, docs-currency. Security note: every Fedora/Hummingbird repo in tools/mock_config.py:95-151 already sets gpgcheck=0. The redirector serves from the same MirrorManager pool the metalink at 113/120 uses, so this PR adds no new trusted party. I did not check whether mock-hermetic-repo verifies digests from the lock. The PR says integrity checking is unchanged, and the diff agrees.

Confidence: 5/5 (safe)

— hive: agent=reviewer backend=claude model=claude-opus-5-5 effort=medium claude=2.1.326

Comment thread tools/hermetic_build.sh
python3 - "$1" <<'PY'
import json, re, sys
path = sys.argv[1]
FEDORA_PATH = re.compile(r"^https?://[^/]+/(?:pub/)?fedora/(?:fedora/)?linux/")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correctness (low, advisory): This pattern only matches mirrors that serve Fedora under /fedora/linux/, /pub/fedora/linux/ or /fedora/fedora/linux/. A mirror with a different layout is left unchanged: for example /mirrors/fedora/linux/ or /linux/fedora/linux/ (I tested both against this regex). If the metalink picked a dead mirror with one of those layouts, all three attempts still hit it, and the ::warning ... retry line doesn't say that nothing was rewritten. Printing a count makes that visible in the log:

Suggested change
FEDORA_PATH = re.compile(r"^https?://[^/]+/(?:pub/)?fedora/(?:fedora/)?linux/")
FEDORA_PATH = re.compile(r"^https?://[^/]+/(?:pub/)?fedora/(?:fedora/)?linux/")
REDIRECTOR = "https://download.fedoraproject.org/pub/fedora/linux/"
lock = json.load(open(path))
moved = 0
for r in lock["buildroot"]["rpms"]:
new = FEDORA_PATH.sub(REDIRECTOR, r["url"], count=1)
moved += new != r["url"]
r["url"] = new
json.dump(lock, open(path, "w"))
print(f"retry: {moved} Fedora URLs sent through download.fedoraproject.org")

(Replace lines 164-169 with this.) Not blocking.

@hivecommons-hive hivecommons-hive Bot added stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) stage/stalled In its current stage longer than expected stage/090-ready-for-human-merge Hive would merge this but trust/role blocks it; a maintainer's /lgtm /approve is all that is missing and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) stage/stalled In its current stage longer than expected labels Nov 18, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor

Ready for a human merge: verdict approved @6f998c3, 173/173 checks green, soak 30m0s elapsed, labels: hold. Held: the hold label was set by @hanthor. The hive never merges this head while the gate stands.

Confirm the intent (resolve the sentinel alert or lift the hold), then approve and merge it on GitHub if you accept it.

@hivecommons-hive hivecommons-hive Bot added stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) stage/090-ready-for-human-merge Hive would merge this but trust/role blocks it; a maintainer's /lgtm /approve is all that is missing stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/090-ready-for-human-merge Hive would merge this but trust/role blocks it; a maintainer's /lgtm /approve is all that is missing stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) labels Nov 18, 2026
@hanthor hanthor removed the hold Work is intentionally paused. label Nov 18, 2026
@hanthor

hanthor commented Nov 18, 2026

Copy link
Copy Markdown
Member Author

#1196 has merged, so every fetched RPM is now checked against the lock. Lifting the hold. #1193 gets closed when this lands.

Decided on behalf of @hanthor with Claude Code

@hanthor
hanthor enabled auto-merge (squash) November 18, 2026 17:16
…edirector

Signed-off-by: James Reilly <jreilly1821@gmail.com>

# Conflicts:
#	tools/hermetic_build.sh
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 18, 2026
@hanthor
hanthor merged commit 2383040 into main Nov 18, 2026
176 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/clanker A contributor's own hive authored this PR under their login dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant