Repository navigation
ci: rebuild reverse dependencies in incremental factory runs - #155
mrbobbytables wants to merge 4 commits into
Conversation
Computes reverse dependency closures in incremental factory rebuilds, selecting downstream dependants across build waves for changed packages and recipes. Supports both runtime Provides/Requires from repodata primary.xml and spec-level BuildRequires/pkgconfig symbols. Global workflow, tooling, configuration, or buildroot changes trigger a full rebuild, while new and unpublished packages remain incremental and retryable. Emits the computed build plan to GITHUB_STEP_SUMMARY and workflow run artifacts. Closes projectbluefin#23 Assisted-by: Gemini 3.8 Flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
There was a problem hiding this comment.
Correctness review (head 085c431)
1. High — stale detection ignores Fedora, half the buildroot; most published packages will be flagged stale every run.
stale_from_primary (tools/rebuild_plan.py:315-350) marks a source stale when a Requires of its published binaries is satisfied neither by the factory primary nor by external, and main() builds external from the Hummingbird repo only (tools/rebuild_matrix.py:171-178). But the actual transaction pairs Hummingbird with Fedora — tools/mock_config.py renders [fedora]/[updates] into every buildroot and its header says "Fedora paired with Hummingbird", and Hummingbird itself is documented as ~3,500 base packages with "no desktop stack at all" (config/hummingbird.repo:1-4). Any factory package linking against a Fedora-provided soname (libX11, cairo, gtk, etc. — i.e., most of a desktop stack) will have unsatisfied Requires under this check, be selected as "stale published build" (tools/rebuild_plan.py:500-503), and then drag its reverse closure — turning every incremental/scheduled run into a near-full rebuild and defeating the PR's purpose. Either include Fedora provides in external, or restrict staleness to capabilities the factory repo itself once provided. The tests (tests/test_rebuild_plan.py StaleTests) only model a repo-internal soname bump, so this path is untested.
2. Medium — is_global_change forces a full rebuild for any tests/** or tools/** edit.
GLOBAL_REBUILD_PATTERNS (tools/rebuild_plan.py:416-423) includes ^tests/.* and ^tools/.*, so editing a unit test or a planning script (including this PR itself) now rebuilds the entire inventory. Previously such pushes triggered the workflow but skipped everything already published. Tests cannot affect build outputs; consider limiting global triggers to files that shape the buildroot/build (e.g., mock_config, build-stage) rather than all tooling and tests.
3. Low — dead duplicated block after return 0 in main().
tools/rebuild_matrix.py:277-294 repeats the overflow check, stage output loop, and GITHUB_OUTPUT write unreachably after the return 0 at line 275. Harmless today but a trap for the next edit — delete it.
4. Low — dead/typo assignment in spec_dependents.
tools/rebuild_plan.py:249 assigns symbol_to_pkg[f"{subname}{sfx}"] = pkg_specs (a dict) and is immediately overwritten by the correct = p on the next line. Remove line 249.
Not a finding: the build-stage.yml prior step (lines 51-55) actually fixes a pre-existing dangling steps.prior.outputs.pattern reference on main — good catch, and the [0-N] glob class is safe since stages cap at 10.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
hanthor
left a comment
There was a problem hiding this comment.
This is aimed at a real problem — a stale package that nothing rebuilds is exactly the class of failure that has been breaking the publish gate — and the two-graph approach (repodata for runtime, specs for build-time) is the right shape. Three things before it lands, plus one coordination note that matters more than any of them.
1. A dead assignment in spec_dependents that says this block has not been read.
symbol_to_pkg[f"{subname}{sfx}"] = pkg_specs
symbol_to_pkg[f"{subname}{sfx}"] = pThe first line assigns pkg_specs — the dict of spec contents — into the symbol map, and the second immediately overwrites it. Harmless as written, which is the problem: it means nothing exercised that path closely enough to notice a dict being used as a package name. Given this function decides what gets rebuilt, the whole block wants a careful pass rather than just deleting that line.
2. The symbol map guesses subpackage names as well as reading them, and the guesses can overwrite the real ones.
for sfx in ("-devel", "-libs", "-common", "-static", "-tools", "-utils", "-doc", "-docs"):
symbol_to_pkg[f"{p}{sfx}"] = p
symbol_to_pkg[f"pkgconfig({p})"] = p
if p.startswith("lib"):
symbol_to_pkg[f"pkgconfig({p[3:]})"] = p
else:
symbol_to_pkg[f"pkgconfig(lib{p})"] = pEvery factory package gets eight synthetic -suffix symbols and one or two synthetic pkgconfig() symbols whether or not it ships them, and these are written into the same flat dict as the real %package declarations. Since it is last-write-wins over an unordered set iteration, a synthetic symbol from one package can overwrite a genuine one from another, and which one wins is not stable between runs.
The specs already state the answer — %package and %package -n are parsed a few lines later, and Provides: pkgconfig(...) is declarative. Build the map from those only. A guessed edge that is wrong is worse than a missing one here: a false edge rebuilds something unnecessarily, which is merely slow, but a displaced real edge means a genuinely stale consumer is not rebuilt, which is the bug this PR exists to fix, now arriving non-deterministically.
3. Two smaller parsing points. re.sub(r"#.*$", "", line) strips from the first # anywhere on the line, including inside URLs with fragments; and the macro expansion does sub.replace(f"%{k}", v) unanchored, so a macro named name also rewrites the prefix of any longer %name… token. Neither is likely to bite on today's recipes, but both are the kind of thing that fails silently and shows up as a missing rebuild.
4. The coordination note — this collides head-on with #133, and the collision is not textual.
Neither provides_from_primary, stale_from_primary nor dependents_from_primary exists on main:
$ git show origin/main:tools/rebuild_plan.py | grep -n 'def provides_from_primary\|def stale_from_primary\|def dependents_from_primary'
(no output)
Both this PR and #133 introduce them independently, from scratch. git merge-tree between the two branches conflicts in four files:
CONFLICT (content): Merge conflict in .github/workflows/build-stage.yml
CONFLICT (content): Merge conflict in tests/test_rebuild_plan.py
CONFLICT (content): Merge conflict in tools/rebuild_matrix.py
CONFLICT (content): Merge conflict in tools/rebuild_plan.py
Good news that git flags it rather than silently keeping the later definition of a duplicated function, but whoever merges second has a semantic merge, not a textual one. Specifically, #133's provides_from_primary carries an EXCLUDED_EXTERNAL parameter that stops the staleness model counting an ICU build the consumer transaction will refuse; that behaviour has to survive the merge, and it is not obvious from the diff that it would.
I would rather not race you for it. If you want to land first I will rebase #133 onto this and re-apply the ICU exclusion on top of your provides_from_primary; if the reverse is easier, say so. What should not happen is both landing without someone deliberately reconciling those three functions.
One related thing worth knowing while you are in this code: reverse-dependency rebuild selection does not fix reverse-dependency stage ordering, and the latter is what has been poisoning the publish gate. bluez sat in the same stage as libical, so it resolved libical from Fedora, whose build requires libicuuc.so.77, and linked ICU 77 — which the Hummingbird-only consumer transaction then refuses. Same story for gvfs/samba and localsearch/nautilus. Rebuilding a dependent does not help if it rebuilds in a wave where its factory-built dependency still does not exist. #133 now has a test that derives those ordering violations from the specs; it may be worth the two graphs eventually feeding it.
CI is unstable on this head, which should be green before merge regardless.
Generated by Claude Code
…taleness - In spec_dependents: drop synthetic suffix guesses (-devel, -libs, pkgconfig) and build symbol map strictly from declared package and Provides tokens; remove dead assignment; improve macro expansion and avoid cutting URLs with fragment comments. - In stale_from_primary: restrict staleness to capability stems provided by the factory repository itself so base buildroot capabilities (e.g. from Fedora paired with Hummingbird) do not falsely mark packages stale. - In GLOBAL_REBUILD_PATTERNS: limit triggers to buildroot/workflow files rather than all tests and tooling. - In rebuild_matrix.py: remove dead duplicated code block after return 0. - Update unit tests in tests/test_rebuild_plan.py. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Addressed review feedback from @hanthor and @kubestellar-hive in commit
🐝 Hive Agent: |
castrojo
left a comment
There was a problem hiding this comment.
Substantive review:
- Diagnosis & Fix identified: The CI failure in the prepare job (TypeError/PatternError in tools/rebuild_plan.py:expand_spec_macros) is triggered when spec macro definitions containing backslashes (such as in grub2, git, hwinfo, or webkitgtk) are substituted via re.sub, where raw string replacements treat trailing backslashes as invalid escape sequences. Using replacement functions
lambda _: vresolves this across all specs. - Outstanding: Author @mrbobbytables or maintainer push to fork branch
mrbobbytables:ci/rebuild-reverse-dependenciesis required to apply the regex replacement fix, unblocking prepare, followed by independent peer review (+2).
castrojo
left a comment
There was a problem hiding this comment.
Pushed fix in commit ff05977:
- Root cause: Spec macro definitions containing backslashes (e.g. in grub2, git, hwinfo, or webkitgtk) caused re.sub in expand_spec_macros to raise re.PatternError on invalid escape sequences in string replacement templates.
- Fix: Use lambda _: v replacement callback in expand_spec_macros to treat macro values as verbatim text without regex template escape interpretation.
- Verification: rebuild_matrix.py and all 337 unit tests now pass cleanly locally.
- Outstanding: Awaiting CI completion of commit ff05977 and independent peer review (+2).
|
Closing to consolidate the queue on the merged #178 full-factory candidate. The branch is retained; please re-open or submit a fresh, rebased PR after that build establishes the current baseline. |
1 similar comment
|
Closing to consolidate the queue on the merged #178 full-factory candidate. The branch is retained; please re-open or submit a fresh, rebased PR after that build establishes the current baseline. |
Summary
Closes #23
Implements reverse dependency closure computation and reviewable build plan reporting for incremental factory rebuilds in
rebuild-rpms.yml:Provides/Requiresand sonames from repodataprimary.xml(dependents_from_primary).BuildRequiresand subpackage/pkgconfig symbols from package specs inpackages/(spec_dependents).merge_dependents) and computes the transitive reverse dependency closure (reverse_closure) so editing an upstream library rebuilds all downstream dependants across waves in proper stage order.tools/, non-inventoryconfig/, or tests (is_global_change), selecting a full rebuild with explicit cause tracking.$GITHUB_STEP_SUMMARY.work/reports/build-plan.json) and Markdown (work/reports/build-plan.md) reports uploaded as thefactory-build-planworkflow artifact.tests/test_rebuild_plan.pycovering direct edits, version changes, global changes, reverse dependency selection (runtime and spec BuildRequires), and report formatting..agents/skills/build-failure-triage/SKILL.md.Verification
just check just test just factory-check pre-commit run --all-filesAll local verification gates and unit tests pass (335 passed).
— hive: backend=copilot model=gemini-3.8-flash
🐝 Hive Agent:
contributor| SHA:085c431