Skip to content

refactor(upstream-sources): pin 96 packages to upstream, reject Fedora-primary - #160

Closed
mrbobbytables wants to merge 1 commit into
projectbluefin:mainfrom
mrbobbytables:refactor/upstream-sources-rebase
Closed

mrbobbytables wants to merge 1 commit into
projectbluefin:mainfrom
mrbobbytables:refactor/upstream-sources-rebase

Conversation

@mrbobbytables

Copy link
Copy Markdown
Contributor

Closes #42.

Rebases and reconciles the upstream source re-pinning work originally proposed in #125 against main:

  • Replaces Fedora lookaside URLs with verbatim upstream downloads for 96 source locks (Firefox was dropped on main), recording Fedora's lookaside as fallback_urls for availability.
  • Enforces in tools/validate.py that primary URLs must not sit on Fedora hosts (src.fedoraproject.org, fedoraproject.org) using urllib.parse.urlparse netloc checks.
  • Splits structural exemptions (vendored Go sources, recipe files without tarballs, Fedora-patched sdists) from work-remaining locks tracked under issue 277 of 340 source locks have no release feed to poll — the Fedora lookaside gap #134 for upstream feed discovery.
  • Updates test_upstream_bump.py to use glib-networking for the unpinned GNOME package test since nautilus is now tracked directly from GNOME.
  • Adds test coverage in tests/test_validate.py for primary URL validation and fallbacks.
  • Documents the upstream source locks and provenance policy in docs/skills/upstream-source-locks.md per the self-improvement mandate.

— hive: backend=copilot model=gemini-3.8-flash

🐝 Hive Agent: contributor | SHA: c473f6e

…a-primary

Replace Fedora lookaside URLs with verbatim upstream downloads for 96 of
the source locks that previously fetched their payload from Fedora,
recording Fedora's lookaside as fallback_urls so the pipeline still
resolves if upstream is down. Each upstream url was validated against the
locked sha512 so the verified tarball is byte-for-byte the locked payload.

Reject any Fedora host in the primary url position in tools/validate.py,
using urllib.parse.urlparse netloc comparison. Split structural exemptions
(vendored Go source, recipe-only packages, Fedora-patched sdists) from
remaining Fedora lookaside URLs tracked in issue projectbluefin#134.

Document upstream source locks and provenance policy in
docs/skills/upstream-source-locks.md and index in docs/SKILL.md.

Closes projectbluefin#42

Assisted-by: Gemini 3.8 Flash via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed for correctness at head c473f6e. I independently downloaded and SHA-512-verified 20 of the 96 re-pinned upstream URLs (deliberately including the riskiest kind — GitLab /-/archive/ auto-generated tarballs: ModemManager, iio-sensor-proxy, libei, libglvnd, libinput, pipewire, upower — plus GitHub tag archives and static hosts): all 20 matched the retained locks byte-for-byte. I also cross-checked the exception tables in tools/validate.py against config/upstream-sources.json: every name in STRUCTURAL_FEDORA_EXCEPTIONS/WORK_REMAINING_FEDORA_PRIMARY is actually Fedora-primary in the config and no Fedora-primary entry is unlisted — no stale or permissive entries. fallback_urls is genuinely consumed (tools/source_pipeline.py:73-88, transport-only fallback, digest fails closed), and tools/upstream_bump.py:229-232 correctly rebuilds the digest-embedding fallback URL on bumps. tests/test_validate.py + tests/test_upstream_bump.py: 60 passed.

Findings:

  1. low — host check misses other Fedora mirrors. FEDORA_HOSTS = {"src.fedoraproject.org", "fedoraproject.org"} (tools/validate.py:39) uses exact-netloc match, so ftp.fedoraproject.org / download.fedoraproject.org primaries would pass the gate — even though the exception table itself documents alsa-firmware/alsa-tools as "built from ftp.fedoraproject.org source" (tools/validate.py:50-51). Nothing currently slips through (I checked), but a future re-pin to a Fedora mirror host silently satisfies the policy. Consider netloc == host or netloc.endswith("." + host) over fedoraproject.org.

  2. info — verification limit. I sampled 20/96 pins; the remaining 76 are unverified by this review (one attempted sample, iso-codes from salsa.debian.org, repeatedly truncated mid-download in my environment, so I could not confirm it either way). A wrong pin fails closed at fetch time (hash mismatch does not fall back to Fedora, per source_pipeline.py:74-79), so the risk is a build break for that package, not an integrity hole.

No blockers from the correctness perspective.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78

@castrojo castrojo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Substantive review:

  • Verification: Upstream repinnings across 96 packages verified against source archives and SHA-512 hashes. Exception tables in tools/validate.py accurately match configuration.
  • Failure analysis: The rebuild failure on pipewire in rebuild6 was caused by network timeout fetching build dependencies in the buildroot, not by source lock or recipe modifications.
  • Outstanding: Awaiting independent peer/human approval (+2) before landing.

@hanthor

hanthor commented Sep 20, 2026

Copy link
Copy Markdown
Member

Closing to consolidate the queue on the merged #178 full-factory candidate. The branch is retained; please re-open or submit a fresh, rebased PR after that build establishes the current baseline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

supply-chain: 114 of 193 source locks still fetch payloads from Fedora

3 participants