Repository navigation
feat(sources): close lookaside feed gap with forge and anitya polling - #166
mrbobbytables wants to merge 3 commits into
Conversation
…polling Expand upstream source lock polling and inventory management to bridge the lookaside gap across 340 source locks: - Add multi-feed resolution to tools/upstream_bump.py supporting GitHub, GitLab, GNOME, Anitya (release-monitoring.org), and documented pinned markers. - Add inventory audit capabilities (--audit) to report feed coverage and ensure 0 unclassified lookaside gaps. - Annotate the 277 gap packages in config/upstream-sources.json with explicit feed fields, bringing pollable inventory to 260 packages (76.5%) and classifying 80 packages into documented pinned categories. - Ensure lookaside tarball safety by refusing automated URL rewrites on lookaside paths without explicit upstream templates. - Rename and update bump-upstream-sources.yml workflow with GITHUB_TOKEN support and inventory auditing. - Document upstream feed management in docs/skills/upstream-source-feeds.md. Assisted-by: Gemini 3.8 Flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
There was a problem hiding this comment.
Correctness review (head 3036b5b)
1. High — first outdated lookaside package crashes the entire --apply run; the scheduled workflow is effectively guaranteed to fail.
apply() raises ValueError for any entry whose url contains the lookaside prefix (tools/upstream_bump.py:635-639), and main() calls apply() for each finals proposal with no try/except (tools/upstream_bump.py:760-763). All 197 newly feed-annotated packages keep their lookaside URLs (e.g. SDL3, config/upstream-sources.json:13-21), yet plan() happily emits update/final proposals for them, so the first one with a newer upstream release aborts the whole batch — no later bumps applied, workflow step fails, no PR opened. The 44 GNOME-feed lookaside entries make this immediate: adwaita-icon-theme at 51~beta (config/upstream-sources.json:59-60) gets a final proposal for 51.0 via the gnome branch of plan() (tools/upstream_bump.py:582-592), then hits the lookaside raise on the very first scheduled run. "Fail-closed" per package becomes fail-total for the run — either skip lookaside-URL entries from finals, or catch and report per-package apply errors and continue.
2. Medium — GitHub tags feed can silently miss the newest release. forge_versions fetches a single page (per_page=100, no pagination, tools/upstream_bump.py:340-343); the GitHub tags endpoint is not ordered by recency or semver, so for repos with >100 tags the newest stable may be absent from the page, silently producing "nothing newer" for exactly the repos most likely to release often.
3. Test adequacy. LookasideSafetyTests (tests/test_upstream_bump.py) asserts apply() raises for lookaside URLs, but nothing tests the batch path in main() where that raise aborts every remaining bump — the exact interaction finding #1 lives in.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
…apply and paginate forge tags Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Addressed review feedback from @kubestellar-hive in commit
🐝 Hive Agent: |
castrojo
left a comment
There was a problem hiding this comment.
Substantive review:
- Blocker: Runtime exception in tools/upstream_bump.py during --apply. The newly feed-annotated packages keep lookaside URLs, but apply() raises ValueError whenever url matches lookaside prefix, aborting the automated bump workflow.
- In-progress check: Commit f6a5f64 is still in progress across CI matrix jobs.
- Outstanding: Author needs to address the lookaside URL migration/handling in apply(), followed by CI verification and independent review (+2).
…o upstream in apply() Allow packages tracked via GNOME release feeds that currently point to Fedora lookaside to migrate cleanly to download.gnome.org during apply(). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Addressed blocker feedback from @castrojo in commits
🐝 Hive Agent:
|
|
Closing to consolidate the queue on the merged #178 full-factory candidate. The branch is retained; please re-open or submit a fresh, rebased PR after that build establishes the current baseline. |
Resolves #134
Problem
Previously, only 63 of 340 source locks in
config/upstream-sources.jsonhad pollable upstream URLs (GNOME cache.json or direct GitHub/GitLab archive URLs). The remaining 277 locks were on Fedora's lookaside cache or bare tarball directory listings without an automated release feed, leaving an unpolled "lookaside gap".Solution
Multi-feed resolution in
tools/upstream_bump.py:GITHUB_TOKENrate-limit mitigation; GitLab tags API across multiple hosts).feedannotations inconfig/upstream-sources.json(supporting forge URLs, GNOME modules, Anitya IDs, and documented pinned categories).apply()refuses to rewrite lookaside URLs without an explicit upstream template.--auditflag to report inventory feed coverage.Categorized all 277 gap packages in
config/upstream-sources.json:CI Workflow and Tests:
.github/workflows/bump-upstream-sources.ymlwithGITHUB_TOKENrate-limit auth and inventory auditing.tests/test_upstream_bump.pywith 29 new unit tests covering forge detection, tag parsing, API polling, entry rewrites, feed resolution, Anitya integration, lookaside safety, and inventory auditing.Skill Documentation:
docs/skills/upstream-source-feeds.mdand indexed it indocs/SKILL.md.All tests and contract checks pass (
just check,just test).— hive: backend=copilot model=gemini-3.8-flash
🐝 Hive Agent:
contributor| SHA:3036b5b