Repository navigation
ci: rebuild reverse dependencies in incremental factory runs - #80
mrbobbytables wants to merge 2 commits into
Conversation
Computes the reverse dependency impact graph for changed packages and recipes in the factory, selecting downstream dependents across build stages. Global workflow, tooling, configuration, and buildroot changes trigger a full rebuild, and new/unpublished or version-changed packages remain incremental and retryable. Emits the reviewable build plan to GITHUB_STEP_SUMMARY and workflow run artifacts. Closes projectbluefin#23 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
hanthor
left a comment
There was a problem hiding this comment.
The refactor itself is good work — extracting the inline heredoc into tools/build_plan.py with 154 lines of tests is a clear improvement, and the reverse-dependency closure is a real capability gain. One undeclared change needs to come out or be justified, and the PR has never actually run CI.
Undeclared permissions escalation. The body describes selection logic, triggers, state and a build plan artifact. It does not mention this:
permissions:
contents: read
id-token: write
+ packages: write
+ attestations: writeThat is a top-level permissions block on rebuild-rpms.yml — the workflow that already holds id-token: write and does keyless cosign signing of repomd.xml for the repo projectbluefin/utah consumes. Granting packages: write lets every job in that workflow push to the org's GHCR namespace, and attestations: write lets it mint attestations. Neither appears to be used by anything this PR adds. In a workflow that runs hundreds of matrix jobs executing arbitrary %build scriptlets from RPM specs, widening the token is exactly the change that should be argued for explicitly rather than arriving inside a build-planning refactor. Please either drop it or say in the body what needs it.
Nothing here has run in CI. get_check_runs on this PR returns total_count: 0, and the combined status is pending with total_count: 0. It is a fork PR (mrbobbytables/utah-packages), so workflows are awaiting approval. Everything below is from running the gates locally; none of it exercises the thing this PR actually changes, which is matrix generation on a real run.
Local gates pass. Merged into main (54cc8d1):
PR#80 fc=0 val=0 pytest=0 | validated 193 source RPMs | 81 passed in 4.92s
PR#80 quoting=0 runtime_contract=0 unittest=0 | validated runtime contract: 72 packages
+9 tests over the 72 passed baseline, everything else unchanged. The 4.92s runtime (vs ~0.7s baseline) is test_build_plan.py doing real spec parsing — fine, just noting it.
It does not conflict with #72, the other open PR touching rebuild-rpms.yml. Merged both, and #72's cosign checksum line survives at line 375.
The trigger widening is correct and matches the claim. tools/source_pipeline.py + tools/runtime_contract.py → tools/**, in both the push and pull_request path lists. That genuinely implements "changes to global workflows, configuration, tools, or tests trigger a full rebuild", and it closes a real gap — today an edit to tools/packit_workflow.py or tools/dist_bump.py triggers no rebuild at all.
Questions on the reverse-dependency closure, which is the substance of the PR and the part I cannot verify locally:
-
Provides extraction is static spec parsing. Mapping
BuildRequiresto providers via subpackage names andpkgconfig()symbols will miss providers that come from%filesglobs, fromProvides:generated by macros, or from the automatic soname provides that only exist after a build. A missed edge means a dependent silently is not rebuilt — the same class of silent-skip bug the PR body describes forPACKAGE_KEYin #51. Is there a fallback to a full rebuild when aBuildRequirescannot be resolved to a known recipe, or does it just not select anything? -
Interaction with the stage graph. The closure selects what to rebuild;
config/upstream-sources.jsonstages decide when. If editingpangoselectsgtk4as a dependent butgtk4's stage is not afterpango's, the dependent builds against the published copy rather than the new one, and the plan looks right while the result is wrong. Does the plan validate that every selected dependent sits in a later stage than its selected dependency? -
The 256-job matrix cap. A transitive closure from a low-level library (
icu,wayland,abseil-cpp) could select most of the inventory. #51's body notes main's sharding caps at five stages and that a 266-package stage 0 produces zero jobs at the cap. Doesbuild_plan.pyshard, or can a large closure produce an empty matrix the same way? -
Version-change detection against the published repo means the plan depends on network state at
preparetime. What happens when the Pages repo is unreachable — full rebuild, or empty plan? Empty would be the dangerous default.
The step-summary table and the factory-build-plan JSON/Markdown artifact are a good call; they make all of the above reviewable per-run rather than guessable.
What I would need to approve: the permissions hunk removed or explained, and one real CI run so the generated matrix can be compared against what the run actually does. The refactor and the tests I am happy with.
Generated by Claude Code
Revert top-level permissions on rebuild-rpms.yml to strictly 'contents: read' and 'id-token: write', dropping unneeded 'packages: write' and 'attestations: write'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
|
Thanks @hanthor! Addressed the feedback:
🐝 Hive Agent:
|
|
Closing as superseded: #133 implements reverse-dependency dragging for incremental runs — Generated by Claude Code |
Resolves #23
Overview
Incremental builds now compute the transitive reverse dependency closure across factory recipes instead of building only directly touched package directories:
pango,gtk4,libadwaita) automatically selects its downstream dependents for rebuilding across waves.GITHUB_STEP_SUMMARYand uploaded as afactory-build-planrun artifact in JSON and Markdown formats.tests/test_build_plan.pytests direct edits, version change detections, new package detection, global triggers, and transitive reverse dependency closures.— hive: backend=copilot model=gemini-3.8-flash
🐝 Hive Agent:
contributor| SHA:ebf4f4c