Repository navigation
feat(build): rebuild what changed and what BuildRequires it, on every merge - #259
Merged
Merged
Conversation
hanthor
force-pushed
the
ci/incremental-builds
branch
from
September 25, 2026 18:34
74e5559 to
221dc34
Compare
… merge A push to main that changes packages/** or config/** now runs the factory, and the run builds only what changed since the published image plus the direct BuildRequires dependents of what changed, in waves solved from real BuildRequires. Everything else stays published and untouched. - tools/extract_buildrequires.sh runs rpmspec over every recipe in the build root (prepare, ~2 min on 4 cores); tools/build_graph.py maps each BuildRequires to the factory recipes that provide it (subpackage names, explicit Provides, and the published generated provides), orders a build set by longest path, and reverses the edges for selection. A config stage only orders the members of a BuildRequires cycle (malcontent-bootstrap -> flatpak -> malcontent); elsewhere it is ignored. The solved graph is 12 waves deep where hand stages went to 10, so the workflow now has rebuild0..rebuild13. - tools/factory_state.py records on the published image, as the label org.projectbluefin.factory.state, the input digest of every build in it (recipe files, inventory entry, build-root pin, Hummingbird repo) and the packages that failed at their current digest. prepare selects against it, so a run is the union of every change since the last publication, and a push holds a package that already failed at exactly these inputs while the daily run retries it. An image without the label falls back to the diff and NEVR comparison. A label adds no layer. - Reverse dependencies are direct only: following BuildRequires transitively dragged 167 packages into a one-line libical fix, through tool BuildRequires (47 recipes BuildRequire git) and a 55-package cycle. A consumer two hops away that really breaks surfaces as a stale published build and is rebuilt then. - Triggers: push (packages/**, config/**), daily 03:17 (was 06:41, the same minute as bump-upstream-sources.yml; a merged bump now builds via push), weekly full rebuild Sunday 01:23. Runs on a ref are serialized with cancel-in-progress false; a replaced pending run loses nothing because the next run selects against the published state. - prepare writes the plan, with the wave and the reason for every package, to the job summary, and uploads the graph as the build-graph artifact. The canary gains pass5: an incremental run against pass1's image, pinned by digest, with vulkan-headers perturbed. It must select exactly vulkan-headers and vulkan-loader, in solved waves 0 and 1 (not config stage 4), and leave libical alone; verify-publish now also checks the state label. Supersedes #195, which computed a transitive closure from a regex parse of the specs and dropped the Hummingbird-owned prune. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QMTLsnqW2jtz5QsNC3W3qv
hanthor
force-pushed
the
ci/incremental-builds
branch
from
September 25, 2026 18:55
221dc34 to
701f1e7
Compare
Member
Author
Canary proof: run 36176501278, gate and workflow green
Merging this adds the push trigger, which covers |
hanthor
marked this pull request as ready for review
September 25, 2026 19:13
This was referenced Sep 25, 2026
castrojo
added a commit
that referenced
this pull request
Sep 26, 2026
…ut format Assisted-by: Gemini via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
castrojo
added a commit
that referenced
this pull request
Sep 27, 2026
…ild_matrix.py (#192) * test: executed coverage for tools/rebuild_matrix.py — tests/test_rebuild_matrix.py tools/rebuild_matrix.py is the driver .github/workflows/rebuild-rpms.yml runs to decide which packages a rebuild builds, and it reported 0% of its 104 statements. tests/test_factory_witness.py names the file but only reads it as text, so nothing ever executed it. Adds tests/test_rebuild_matrix.py: 31 cases over changed_recipes, changed_inventory, fetch_primary, hummingbird_baseurl, fetch_published and main(). subprocess, urllib.request.urlopen and the module's ROOT are redirected at the module object, so no case touches the network or a real git repository. Covered decisions that had nothing holding them in place: the base-sha guard that keeps a scheduled run from shelling out to an empty diff range; the inventory reader that proves nothing rather than everything when the old config is absent or unparseable; the mirror failure that rebuilds everything instead of skipping; the stale and dependent-drag rebuild reasons; the >250-package chunk split; and the SystemExit on a stage with no job. Coverage for the module goes 0% -> 99%; the remaining line is the __main__ guard. Suite goes 396 -> 427 passed. Assisted-by: Claude Opus 5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> * fix(tests): adapt test_rebuild_matrix.py to hummingbird-provided-sources and prune repo fetch * test: adapt test_rebuild_matrix.py to post-#259 wave and trusted output format Assisted-by: Gemini via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> Co-authored-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: castrojo <castrojo@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Item 1 of the incremental-factory work. Stacked on #258 (incremental publish); retargets to
mainwhen that merges. Supersedes #195.What a merge now does
A push to
mainthat touchespackages/**orconfig/**runs the factory. Prepare selects:It orders them into waves solved from real BuildRequires. Everything else stays published and is not touched. The plan, with each package's wave and the reason it was picked, goes to the job summary. The graph is uploaded as the
build-graphartifact.How
tools/extract_buildrequires.shrpmspec -q --buildrequires / --provides / --qf %{NAME}over every recipe, in the build root, 4-way parallel. All 397 specs parse; 4 min on 2 local cores, so about 2 min on the runner.tools/build_graph.pypkgconfig(), files) from the published primary.xml. Solves waves by longest path over the edges inside the build set.stageis consulted only between members of one strongly connected component (the malcontent-bootstrap → flatpak → malcontent cycle-breaker). A cycle that config does not order builds side by side and is named in the log.tools/factory_state.pyorg.projectbluefin.factory.statelabel on the published image:inputs= input digest of every published build (recipe files + inventory entry +config/buildroot-image+config/hummingbird.repo);failed= packages whose last attempt at their current digest failed. Publish merges it: replaced → new digest, failed → keep the old entry and record the failure, trusted (everything prepare did not select) → current digest, pruned → removed. A label adds no layer, and the canary asserts the image is still exactly two layers.With the state label, a run builds the union of every change since the last publication, however many runs the queue replaced. An image without the label (the first run after merge) falls back to the old diff plus NEVR comparison. The first labelled publish learns the state of untouched packages from prepare's
trustedlist.Why direct reverse dependencies, not transitive
Measured on the real graph: following BuildRequires transitively selects 167 packages for a libical fix and 178 for vulkan-headers. The edges run through tool BuildRequires (47 recipes BuildRequire
gitfor%autosetup -S git, 27desktop-file-utils) and a 55-member BuildRequires cycle (cairo, gtk3, gobject-introspection, …). Taking direct dependents only, the median change selects 1 other package, 116 recipes have none, and the largest (gobject-introspection) selects 82. That is how Fedora and Hummingbird rebuild for an ABI change. A consumer two hops away that really does break shows up as a stale published build (an unsatisfied Requires) and is rebuilt then. Without the graph, the old transitive runtime closure is still the fallback.Waves
The solved graph is 12 waves deep (hwdata → libdisplay-info → mesa → libglvnd → mesa-libGLU → gstreamer1-plugins-base → pulseaudio → pipewire → ffmpeg → localsearch → nautilus → nautilus-python). The hand stages went to 10, put 321 packages in stage 0 where the graph puts 194, and had missed real edges. The workflow now declares
rebuild0…rebuild13, withpublish_gate.STAGES,rebuild_plan.STAGESand the tests updated to match. One consequence: a package that used to build next to a same-stage provider now builds after it, which is repeated mistake 3's own rule. It may surface failures that were hidden before. Under incremental publish those stay per-package.Triggers
main,packages/**orconfig/**bump-upstream-sources.yml)Top-level
concurrency: rebuild-rpms-<ref>-<prefix|factory>,cancel-in-progress: false, so every merge's run finishes. GitHub keeps one run pending and replaces an older pending run with a newer one. Nothing is lost when that happens, because the newer run selects against the published state. A merged bump PR now reaches the factory through the push trigger.Canary
The canary gains
pass5, an incremental run against pass1's image pinned by digest (a newfactory_taginput that also takessha256:…), withvulkan-headersperturbed.verify-incrementalasserts it selected exactlyvulkan-headers(wave 0) andvulkan-loader(wave 1, not its config stage 4), and leftlibicalalone. Without the state label this pass would select nothing: there is no diff range and the NEVR did not move.verify-publishnow also checks that the label records the set.Also
fix:stage_outputs/overflowtake the solved waves;reverse_closuretakes a depth.Proof
Canary on this PR, then a real one-package change on
mainwith timings. Numbers to follow.🤖 Generated with Claude Code