Repository navigation
Build the whole Utah contract: 343 recipes, Firefox as a Flatpak, four stage failures fixed - #70
Conversation
The factory builds 193 packages; a Utah image needs 345. The missing
152 are what a real dnf transaction against Hummingbird names, found by
composing the image and reading the resolver until it stopped
complaining: ffmpeg, bluez, SDL3, flatpak, appstream, gnome-keyring,
Xwayland and its five X11 helpers, hunspell and its dictionaries, gcr3,
openjph, setools, hidapi, desktop-file-utils, soxr, and the rest of the
runtime closure behind them.
Every recipe arrives in this repository's shape rather than dist-git's:
one spec, sources, changelog, patches, and the upstream provenance file.
The tmt plans, monitoring.toml, per-package .packit.yaml and .gitignore
that the scaffolding purge removed are stripped on the way in, so
nothing lands that the purge would only have to remove again.
Three recipes carried build-time defines in the old factory, which the
Packit inventory has no field for. They move into the spec, where they
work whichever builder runs:
mdadm ABRT off -- its only unresolved dependency was
libreport-filesystem, for a crash-reporter hook, in an
image that ships no crash reporter
v4l-utils Qt off -- qv4l2 and qvidcap need Qt6, and the build root
cannot resolve it while Hummingbird offers both libicu-77
and a qt6-qtbase that requires libicuuc.so.78
protobuf Java off -- the bindings need a JDK the build root does
not carry and nothing here consumes them
color-filesystem has no upstream archive at all: it creates four
directories and an rpm macro file, and dist-git records no sources for
it. Rather than invent a URL to satisfy the shape of a lock, an entry
may now declare `no_upstream_source`, which fetches and verifies
nothing. colord requires it, and colord is what gtk3 and the desktop
hang off.
webkit2gtk4.1 is deliberately absent. It is a second entry that builds
webkitgtk's recipe with a different define, and the inventory has no
alias field; webkitgtk itself is here.
Verified: `tools/validate.py` reports 345, the inventory reports every
recipe source-locked and Packit-configured, and all 42 tests pass. Not
verified: none of these has been built under Mock. The counts the
inventory tests assert are updated to match.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
PACKAGE_KEY did not accept "_", so a package whose name contains one parses as no package at all. package_names() drives both the rebuild matrix and the inventory's packit_configured check, which means such a package is silently never built and nothing reports it missing. No name in the current 193 has an underscore, so this has been latent. It surfaces the moment lm_sensors or volume_key is added. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9 Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
GitHub caps a matrix at 256 jobs, and it does not fail when one would exceed that -- it expands to nothing. A stage holding more than 256 packages produces zero build jobs, silently, and every later stage then builds against an empty buildroot. This branch's cold stage 0 is 315 packages, so run 34171159421 spawned no rebuild0 jobs at all: 14 jobs at stage 1, 3 at stage 2, 1 each at 3 and 4, and none at 0. The stage-1 failures all read as missing dependencies -- vulkan-loader: No match for argument: vulkan-headers = 1.4.350.0 pipewire-libs-extra: No match for argument: pkgconfig(libfreeaptx) gdk-pixbuf2: glycin-2 found 2.1.0 but need >= 2.2.alpha.7 -- and all three name packages this factory builds at stage 0. They were not missing; they were never built. Hand each stage to build-stage.yml in chunks of 250 rather than in one piece, and fan the caller out over them. Stage 0 becomes two chunks (250 + 65); every other stage is unchanged at one. The flat stageN outputs stay, so the already-published skip logic and the empty-stage guards are untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
samba BuildRequires libtevent-devel >= 0.17.2 and python3-tevent >= 0.17.2. Fedora 44 ships 0.17.1, so those have to come from this factory's own libtevent 0.17.2 -- but libtevent sat at the same stage 1 as samba, and a stage cannot see its own output. samba failed with: No match for argument: libtevent-devel >= 0.17.2 Move samba to stage 2. libtalloc 2.5.0, its other version-floored dependency, is already at stage 0, so nothing else needs to move. The rest of that job's "missing dependency" errors were not ordering at all -- vulkan-headers, glycin, libfreeaptx and liblc3plus are all stage 0 packages that never built, because stage 0 exceeded GitHub's matrix cap and expanded to zero jobs. That is fixed separately. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Thirteen stage-0 jobs failed to resolve their buildroot, and none of them were
missing a package. Each hit the same shape: this factory rebuilds a library
with a new soname, Fedora's dependent still wants the old one, and the rebuilt
dependent -- which we also carry -- sits in the same stage, where a sibling job
cannot see it.
libheif 1.23.1 (so.2) vs Fedora glycin-loaders wanting libheif.so.1
-> pulseaudio, rest, sdl2-compat, tecla, wayland, wireplumber,
xdg-user-dirs-gtk (reached via graphviz -> gdk-pixbuf2 -> glycin)
abseil-cpp 20260526 vs Fedora webrtc-audio-processing wanting
libabsl_strings.so.2601
-> libepoxy, libglvnd, libayatana-ido, xdg-desktop-portal-gtk
pipewire 1.6.8 vs Fedora pipewire-jack-audio-connection-kit-devel
-> gstreamer1-plugins-good
Our glycin 2.2, webrtc-audio-processing 2.1 and pipewire 1.6.8 are all stage 0
and all build fine; they were simply invisible to their siblings. Move the
dependents to stage 1 so they see them.
nautilus-python goes to stage 2 rather than 1: it needs localsearch, which is
itself stage 1 -- Fedora's localsearch 3.11 wants libavformat.so.62, which our
ffmpeg 9.0.1 displaces.
Moving wayland to stage 1 is safe: the only versioned consumer is
wayland-protocols (>= 1.25.0), and Fedora 44 ships wayland-devel 1.25.0.
Also fix a lowercase month in dracut's changelog, which rpmbuild rejects
outright: "bad date in %changelog: Mon aug 10 2026". 10 Aug 2026 is a Monday,
so only the capitalisation was wrong.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Six stage-0 builds -- gcc, git, iputils, libvdpau, appstream and libtheora --
died with nothing useful in a tail:
Running transaction
Transaction failed: Rpm transaction failed.
dnf resolved the buildroot happily; rpm then refused to install it, and the
reason prints after the failure line rather than before it:
file /usr/bin/erb conflicts between attempted installs of
ruby4.0-default-gems-4.0.6-37.3.hum1 and ruby3.4-default-gems-3.4.10-31.7.hum1
Both sides are .hum1. The existing exclusions are per-repo -- fedora.excludepkgs
and updates.excludepkgs -- so they can express "prefer Hummingbird's copy over
Fedora's" but not "prefer one Hummingbird package over another". Hummingbird now
carries ruby3.4-default-gems and ruby4.0-default-gems, whose file lists collide.
Exclude the superseded ruby globally with --exclude, leaving 4.0 as the only
candidate in any repo. The per-repo lists are left alone: they still do the job
they were written for.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Both failed with a file that is sitting in the recipe directory:
error: Bad file: /work/staged/mdadm/9CE05640D703A136.asc: No such file
.gitignore carries `packages/*/*.asc`, meant for downloaded signatures. A
keyring a spec names as a SourceN is not a downloaded artefact -- it is part
of the recipe, read straight out of _sourcedir:
%gpgverify --keyring='%{SOURCE11}' --signature='%{SOURCE10}' --data=-
So the file existed locally, was never committed, and CI checked out without
it. git, wayland and zsh already carry theirs because someone force-added them
one at a time; mdadm and setxkbmap were simply missed, and nothing reported it.
Commit the three missing keyrings and narrow the ignore so the next one is not
lost the same way: keep ignoring downloaded .asc signatures, but never a file
named as a key ID or carrying "key"/"keyring" in its name.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Xwayland's meson.build asks for wayland-client >= 1.26; Fedora 44 ships 1.25.0, so only this factory's wayland 1.26.0 satisfies it: meson.build:189:4: ERROR: Dependency lookup for wayland-client with method 'pkg-config' failed: Invalid version, need 'wayland-client' ['>= 1.26...'] Both sat at stage 0, where a job cannot see its sibling's output, so Xwayland could never have found it. wayland has since moved to stage 1, so Xwayland goes to stage 2. The floor is declared in upstream's meson.build rather than the spec, which is why scanning BuildRequires does not find it -- the spec's wayland-devel carries no version at all. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
The openssh regress suite failed every parallel test job: Privilege separation user sshd does not exist FATAL: sshd_proxy broken make: *** [parallel_test.Makefile:3: t-exec-03] Error 2 The recipe carries openssh-server-systemd-sysusers.conf and creates the user from %pre at install time. A buildroot never installs the package it is building, so the user is absent exactly where the test suite expects it. Apply any sysusers config the recipe stages, before rpmbuild runs. This is generic -- any package shipping one gets the users it declares -- and it supplies what a real build root has rather than disabling the test that noticed the gap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
%prep died with all three hunks rejected: 3 out of 3 hunks FAILED -- saving rejects to file src/nasm.inc.rej error: Bad exit status from /var/tmp/rpm-tmp.bC0pTE (%prep) The patch was not stale. xvidcore ships src/nasm.inc with CRLF line endings -- all 213 lines of it -- while the patch carried LF-only context, so no context line ever matched. Nothing about the change itself was wrong. Regenerated against the pristine 1.3.7 tarball, with CRLF preserved on both sides of the diff. Same three hunks, same 23 inserted lines, same GNUHEADER macro and the two call sites. Verified by applying it to an untouched src/nasm.inc and comparing the result against the intended file byte for byte. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Git normalised the CRLF context out of the regenerated xvidcore patch the moment it was committed -- the blob went in with zero carriage returns, so the fix would have failed in CI exactly as before, and the working tree would still have looked correct. Mark patches, diffs, keys and signatures as -text so git leaves them alone. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
flac failed %check with the suite naming the cause itself:
ERROR: iterator claims file is writable when tester thinks it should not
be; are you running as root?
80% tests passed, 2 tests failed out of 10
The test is right. It chmods a file read-only and asserts the write fails,
and root writes it regardless of the mode bits. tools/build-rpm.sh already
fixed this for a local build, in as many words:
# Match mock's build user. FLAC's read-only-file tests correctly fail when
# rpmbuild runs as root, which can write files regardless of their mode bits.
useradd --create-home --home-dir /builddir mockbuild
build_rpm() { runuser -u mockbuild -- rpmbuild "$@"; }
The CI lane never got the same treatment and kept building as root out of
/root/rpmbuild. So the two lanes disagreed about who runs the build, and a
package could pass locally and fail here for a reason belonging to neither
the recipe nor the code.
Create the same mockbuild user, move _topdir to /builddir/rpmbuild, and run
both rpmbuild invocations through runuser. Fedora builds this way; so should
both of ours.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
evolution-ews sat at stage 0 while evolution-data-server, which supplies every
pkgconfig module it asks for, is at stage 1:
No match for argument: pkgconfig(camel-1.2) >= 3.61.3
No match for argument: pkgconfig(evolution-data-server-1.2) >= 3.61.3
No match for argument: pkgconfig(evolution-mail-3.0) >= 3.61.3
Move it to stage 2. evolution-data-server itself builds fine.
tailscale vendors go-json-experiment/json, whose alias.go forwards to the
standard library encoding/json/v2 when the jsonv2 experiment is on. Hummingbird
ships Go 1.27, whose v2 API has moved past that vendored shim:
vendor/github.com/go-json-experiment/json/alias.go:618:21:
undefined: json.SkipFunc
vendor/github.com/go-json-experiment/json/alias.go:957:14:
undefined: json.DiscardUnknownMembers
GOEXPERIMENT=nojsonv2 takes the build tag out, so the module uses the
implementation tailscale ships and tests against.
libnice lost one test of forty-two, on the clock rather than on a failure:
42/42 libnice:test-new-trickle TIMEOUT 30.03s killed by signal 15 SIGTERM
Fail: 0 Timeout: 1
test-new-trickle negotiates ICE candidates against timers and wants more than
the 30s meson allows by default on a loaded builder. Raise the multiplier
rather than drop the test that ran out of time.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
The Packit SRPM pilot has failed on every run since the monorepo passed 256 packages, and it failed invisibly: discover goes green, reports "discovered 345 packages", and the srpm matrix beneath it produces no jobs at all. GitHub caps a matrix at 256 and expands a larger one to nothing rather than rejecting it. The discover step already guards against a matrix that runs nothing -- it asserts the list is non-empty -- but a non-empty list of 345 is exactly the case that guard cannot see. Split the per-package steps into a reusable packit-srpm-chunk.yml and fan the pilot out over chunks of 250, the same shape rebuild-rpms.yml uses. Per-package parallelism is unchanged; only the number of matrices grows. Chunking lives in packit_workflow.py rather than inline in YAML so it can be tested, and a test now asserts every chunk fits under the cap and that chunking loses no package. This is what the pilot was built to answer and has never reached: whether packit srpm works against these recipes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Packit stays, scoped to what it does well: turning a recipe into an SRPM and proving the spec is well formed, in three or four seconds a package. That is a cheap gate -- dracut sat behind a lowercase month in a %changelog date, which an SRPM step catches at once instead of after a build root has been installed and the package compiled. Packit is not replacing the binary lane, and the disttags say why. packit srpm yields flac-1.5.0-9.fc43; this factory ships gnome-shell-extension-gsconnect-72-3.hum1.bfin. Rebuilding against Hummingbird so sonames match the image is the entire reason the factory exists, and a Fedora-chroot SRPM does not answer it. Copr is the shape in which Packit could take the binary lane, and we are not pursuing it now: it would need a custom chroot carrying the Hummingbird repository plus the exclusion rules the lane already encodes. That is a migration of the build root, not a change of build driver. Written down so the option is not rediscovered from scratch. Also records two things that were true but undocumented -- Packit does no source acquisition here either, and .packit.yaml carries no jobs section, so the service does no work on a pull request -- notes the matrix cap that kept the pilot from ever running, and refreshes the counts from 193 to 345. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Both were solved on fix/repeatable-local-builds (#45) and lost in the port to the Packit shape. Bringing the recipes across rather than rediscovering them. libheif is the one blocking ten packages in the current run. Its ffmpegdec plugin links Fedora libav, and the moment the factory promises its own ffmpeg 9.x the Fedora copy is excluded by name -- so the accumulator libheif becomes uninstallable and takes every buildroot that reaches gdk-pixbuf2, glycin, graphviz or doxygen with it. That is exactly what opus, libthai, libvpx, mpg123, openjpeg, spandsp, libnice, libnma and libtalloc are failing on: package graphviz requires libgdk_pixbuf-2.0.so.0, but none of the providers can be installed ... glycin-loaders requires libheif.so.1 ... The draft answers it with a narrow %bcond ffmpeg 0: the aom and dav1d decoder plugins stay, which is what actually decodes AVIF and HEIC here, and only the ffmpeg HEVC path goes. Narrower than %bcond bootstrap, which would also drop rav1e, SvtEnc and sdl2. tailscale builds from the official static release payload rather than a vendored Go tree, so it no longer depends on the Hummingbird Go version merely to reproduce an upstream artifact. This supersedes the GOEXPERIMENT=nojsonv2 workaround added earlier today, which was treating a symptom: Go 1.27 has moved past the vendored go-json-experiment shim. The spec goes from 1192 lines to 71, the vendor tarball script and the 1.92.5 systray patch go with it, and the source lock points at tailscale_1.98.8_amd64.tgz. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
…ildroot git was the last failing package, and not for a reason belonging to git. Six tests of t9168-git-svn-partially-globbed-names failed, every one of them on svn: E155007: ... is not a working copy with the cause further up the log: svn: E200029: atomic initialization could not be performed svn: E200030: SQLite compiled for 3.53.4, but running with 3.51.2 Hummingbird builds subversion against sqlite 3.53.4. Fedora 44 ships 3.51.2 and wins the buildroot, so subversion cannot open a repository at all, every checkout and copy fails, and it surfaces as a git test failure. sqlite therefore joins libicu, gpgme and qt6-qtbase in the list of names Fedora must not answer for -- the same mechanism, for the same reason, written down where the others already are. The subversion versions are otherwise identical (1.14.5 in both), which is what rules out a version incompatibility in the test itself and makes skipping it the wrong answer. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
spandsp was the last failure after the draft port, and again not for a reason
belonging to spandsp. Its buildroot reaches doxygen, and:
package libheif-1.23.1-1.hum1.bfin from stages requires
libopenjph.so.0.25()(64bit), but none of the providers can be installed
Same shape as the ffmpeg plugin, one layer down. The factory promises openjph
0.31, which provides libopenjph.so.0.31. libheif builds in the same stage, so
it cannot see that and links Fedora openjph 0.25 instead -- and at install time
the Fedora copy is excluded by name, because the factory answers for openjph.
libheif is then uninstallable and takes gdk-pixbuf2, glycin, graphviz and
doxygen down with it.
Ordering cannot settle this without restratifying five deep -- openjph,
libheif, glycin, gdk-pixbuf2, then everything that reaches doxygen -- which is
the entire stage budget for one plugin.
Decline the feature, as the ffmpeg bcond already does. This drops the HTJ2K
encoder plugin alone: AVIF and HEIC still decode through aom and dav1d, and
JPEG 2000 still works through OpenJPEG, which is a different library and stays
enabled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
The prior-artifact download ran unconditionally, so a stage 0 job collected whatever rpm-* artifacts its sibling stage 0 jobs had already uploaded. The buildroot then depended on the order those siblings finished in, which is why the failing set moved between runs of the same commit. It also breaks the stage contract in the direction that matters. A stage cannot see its own output -- that is the assumption every stage assignment in config/upstream-sources.json is made against -- and here stage 0 saw part of it, non-deterministically. The failure names nothing near the cause. Collecting a sibling openjph 0.31 puts openjph into the by-name Fedora exclusion list, so: package libopenjph-0.25.3-3.fc44 from fedora is filtered out by exclude package libheif-1.21.2-1.fc44 requires libopenjph.so.0.25 ... glycin-loaders requires libheif.so.1 ... gdk-pixbuf2 requires libglycin-2 graphviz requires libgdk_pixbuf-2.0.so.0 ... doxygen requires graphviz and opus, mesa-demos, passim and msgraph fail on a package none of them names. Guard the step on the stage number. Stage 0 resolves against Fedora and Hummingbird alone, which is reproducible and is what its stage assignments already assume. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
…-processing Reading #45's config/build-lanes.toml rather than rediscovering it. It already records why these three cannot build in the fast lane, and the reasoning holds under the numeric stages this branch uses. libheif now builds with %bcond bootstrap on, permanently. It build-requires sdl2, and sdl2 -> SDL3 -> libdecor -> gtk3 -> gdk-pixbuf2 -> glycin-libs -> glycin-loaders -> libheif closes a cycle. Fedora never trips it because its repository is self-consistent; this accumulator is not. Bootstrap costs exactly one binary, the heif-view example -- the encoders and every library stay. That also retires the %bcond openjph I added earlier today, which dropped the HTJ2K encoder to dodge the same problem. Ordering is the better answer: libheif moves to the wave after openjph, so it links the openjph this factory builds rather than the Fedora 0.25 it then excludes by name. The plugin comes back. ffmpeg and webrtc-audio-processing move with it, for the reason the draft gives: they link libraries the fast lane rebuilds with moved sonames, so building beside them leaves the consumer linked to sonames that never reach the repository. Their own consumers follow one wave later -- pipewire links webrtc-audio-processing, and libepoxy, libglvnd, libayatana-ido, xdg-desktop-portal-gtk and pipewire-libs-extra reach it through pipewire-libs. Stages stay within the five the workflow allows: 296/24/16/6/3. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Adding sqlite to the Fedora exclusion list changed nothing, because excluding a repository does not touch a package that is already installed. The base image arrives carrying Fedora sqlite-libs 3.51.2, so the resolver preferred Hummingbird while the build root kept Fedora, and subversion -- which Hummingbird builds against 3.53.4 -- still could not open a repository: svn: E200029: atomic initialization could not be performed svn: E200030: SQLite compiled for 3.53.4, but running with 3.51.2 which surfaced, six tests later and one package away, as t9168-git-svn-partially-globbed-names failing in the git build. Upgrade each name in HB_EXCLUDE that the image already has, so the decision that Hummingbird answers for it reaches the build root and not merely the resolver. Scoped to those names rather than a general upgrade: they are exactly the ones already declared, and a wholesale upgrade would move a build root that 238 packages currently build against. Also prints the resulting sqlite alongside the openssl line, so the next mismatch of this kind is visible in the log rather than inferred from a test failure three layers up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Guarding stage 0 fixed the worst case; the same race remained above it. Artifacts are named rpm-s<stage>-<package> and the download pattern was rpm-*, so a stage 2 job collected rpm-s2-* as well -- whichever of its own siblings had finished uploading by then. That is the same defect as the stage 0 one, and it fails the same way: the buildroot depends on the order siblings complete in, so the same commit produces different failures on different runs, each naming a package that is not the cause. Ask for strictly earlier stages: stage 3 collects rpm-s[0-2]-*. Every stage now sees exactly what its stage assignment promises -- everything built before it, and nothing built beside it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Seven of the eight stage 0 failures were not package failures at all: Failed to FinalizeArtifact: Received non-retryable error: (403) Forbidden grub2, gtk2, iio-sensor-proxy, liblc3, xkbcomp, xmodmap and xrdb each built their RPMs and then lost them on upload. Every job uploads its debuginfo and debugsource alongside the RPMs, which no BuildRequires ever names and which are more than half the bytes -- #45 already skips them when seeding a local prior for exactly that reason. Exclude them from the artifact so the run stops spending its budget on payload nothing downstream reads. python-evdev is the one real failure. Its %generate_buildrequires runs %pyproject_buildrequires, and that macro has to execute before the dependencies it generates can be resolved -- so with nothing named statically the first pass has nothing to run it with: pyproject-rpm-macros is needed by python-evdev python3-devel is needed by python-evdev python3dist(packaging) is needed by python-evdev and the retry loop then reinstalls what is already installed, three times, and gives up. Name them, the same way python-typing-inspection asks for python3-packaging. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
Sixteen of twenty-three stage 1 packages failed on one knot, and openjph tied it. This PR added openjph; libheif is the only recipe in the factory that build-requires it, and Hummingbird ships no libopenjph at all -- only Fedora does, at 0.25. Carrying openjph 0.31 therefore excluded Fedora libopenjph by name, which made Fedora libheif uninstallable, which made this factory own glycin-loaders uninstallable, which took gdk-pixbuf2 and everything behind it down: glycin-libs-2.2~beta from stages requires glycin-loaders = 2.2~beta glycin-loaders-2.2~beta from stages requires libheif.so.1 libheif-1.21.2-1.fc44 from fedora requires libopenjph.so.0.25 libopenjph-0.25.3-3.fc44 from fedora is filtered out by exclude filtering -- evolution-data-server, ffmpeg, flatpak, gdk-pixbuf2, gstreamer1-plugins-bad-free, gstreamer1-plugins-good, libtevent, localsearch, pulseaudio, rest, sdl2-compat, tecla, wayland, webkitgtk, wireplumber and xdg-user-dirs-gtk. Ordering cannot untie it within five stages. openjph, libheif, glycin, gdk-pixbuf2, then everything that reaches gdk-pixbuf2, is five waves before webkitgtk, gjs and gnome-shell have anywhere left to go. Declining the encoder removes the dependency instead: libheif no longer needs openjph at build or at run time, Fedora libopenjph is never excluded, Fedora libheif stays installable for glycin to link, and nothing has to move. libheif returns to stage 0. The cost is HTJ2K encoding. AVIF and HEIC decode through aom and dav1d, which is what this image actually shows, and JPEG 2000 keeps working through OpenJPEG -- a different library, still enabled. Counts drop to 344. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
…ed out Sixteen stage 1 packages failed on the openjph knot, and the reason ordering could not untie it was that five waves is not enough to express this dependency graph. openjph, libheif, glycin, gdk-pixbuf2 and then everything reaching gdk-pixbuf2 is already five, before webkitgtk, gjs, evolution-data-server, flatpak or gnome-shell have anywhere to go. fix/repeatable-local-builds solved that with sub-lanes -- stage0_late, stage0_late_b, stage0_late_c, stage1_heavy, stage2_late, stage3_late -- each its own wave with its own boundary. Rather than invent a second answer, take that lane order and flatten it into consecutive numbers here: 1 libheif (stage0_late) 2 webrtc-audio-processing (stage0_late_b) 3 ffmpeg, gstreamer1-plugins-good, xorg-x11-server-Xwayland (stage0_late_c) 5 webkitgtk (stage1_heavy) 7 gjs (stage2_late) 9 evolution-data-server, flatpak, gnome-shell (stage3_late) with the old numeric waves shifted into 4, 6, 8 and 10 around them. The reasoning behind each boundary is recorded in that file and holds unchanged: WebRTC before any consumer whose buildroot resolves PipeWire; libheif before its late consumers; Flatpak after the malcontent-bootstrap provider it needs and before the full malcontent build that needs Flatpak. rebuild-rpms.yml grows rebuild5 through rebuild10 on the same chunked reusable-workflow pattern, and KNOWN_STAGES widens to match so the inventory still refuses a stage no job can resolve. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VF8ttMH6WGQHeQ9hdMSY9
The factory built Firefox only because Bluefin's copied [fedora] section
names it. Utah does not want the RPM: projectbluefin/utah already lists
firefox in its unavailable set and carries org.mozilla.firefox in
contracts/bluefin-desktop.toml.
Building it was not free. The recipe took a 2.5-hour PGO/LTO job in the
last full run -- the longest of the 285 -- carried its own 12G swap
workaround in the build workflow, and still failed in %install:
cp: cannot create regular file
'objdir/dist/bin/browser/chrome/browser/content/browser/default-bookmarks.html'
So the recipe, its source lock, its .packit.yaml entry and the swap step
that existed for it alone all come out. firefox joins [unavailable] in
the runtime contract, which is what keeps the Bluefin-requested name from
failing the transaction. mozjs140 is untouched -- it is the embeddable
JavaScript runtime and unpacks a firefox tarball of its own.
Inventory 193 -> 192; the four test counts follow.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
…ures Merges the 151-recipe contract from #51 (which carries #52's matrix-cap sharding and eleven stages) and fixes every non-Firefox failure from run 34415980185. **The build root could not see the published repository.** `prepare` skips any package already published at the same version, but the [factory] repo block is guarded by `vars.FACTORY_REPO`, which is unset on pull_request runs -- so it never ran, and a skipped package was simply absent. That is why pipewire-libs-extra died on No match for argument: pkgconfig(libfreeaptx) while libfreeaptx-devel sat published and skipped. FACTORY_REPO now defaults to the same GitHub Pages URL `prepare` reads to decide what to skip, so the two halves of that decision cannot disagree. **Two recipes were staged before what they need.** - xorg-x11-server-Xwayland 3 -> 5. It needs wayland-client >= 1.26.0 and the buildroot had Fedora's 1.25.0; this factory builds wayland at 4. Its three stage-0 build consumers resolve Xwayland from Fedora, so moving it later changes nothing for them. - evolution-ews 6 -> 10. It needs libebook/libecal/libedata-* >= 3.61.3, and evolution-data-server builds at 9. **gstreamer1-plugins-bad-free declines chromaprint**, on exactly the reasoning ffmpeg.spec already records: Fedora's libchromaprint links libavutil.so.60, this factory's ffmpeg 9.0.1 provides so.61, and the factory excludes the Fedora libav by name, so libchromaprint-devel is uninstallable here. The spec disabled chromaprint only under `%if %{without extras}`; it is now its own bcond, off, with the BuildRequires and the plugin in %files following it. gcc was cancelled with the run, not failed, and needs only a re-run. Inventory 344 -> 343 with Firefox dropped; validate, the runtime contract, the quoting check and all 73 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
spirv-tools is the one failure in run 35200615124, and it cannot be fixed here. 3d2da97 set SPIRV_WERROR=OFF, which worked -- the build got past -Werror=switch and died one layer down on a real API break: core_tables_body.inc:1713:10: error: SPV_OPERAND_TYPE_GATHER_MODES was not declared in this scope core_tables_body.inc is generated from the external spirv-headers grammar. The newer headers emit an operand type that spirv-tools 2026.2 does not define, so the generated table references a symbol that does not exist. That is version skew between two separately released halves of one project, not a warning to relax: the only fixes are a spirv-tools release that knows the symbol or a vendored older headers copy, and neither belongs in this factory. Rebuilding it here buys nothing. Nothing in the runtime contract asks for it -- absent from bluefin-packages.toml, runtime-contract.toml and factory-contract.json. Its only consumer in the tree is mesa, through pkgconfig(SPIRV-Tools), and mesa sits in the same stage 0: it resolves spirv-tools-devel out of the build root before our rebuild exists and can never consume it. So the recipe is dead weight that fails a stage. Dropping it follows 20eae7a (Firefox), 4d2af41 (openjph) and 4bb0711 (gcc): the factory scope is the desktop stack Hummingbird does not ship, and a recipe no consumer here can use is out of scope. Removing spirv-tools from the rebuild set: - config/upstream-sources.json: drop the entry - .packit.yaml: drop the block - packages/spirv-tools/: delete the recipe, patches and sources - tests/: track the 341 -> 340 package count, and 340 -> 339 for the selection test that already skips dracut 267 tests pass, validate reports 340 source RPMs, the factory contract is satisfied across 2463 tracked files, and the quoting check clears all 12 workflows. The plan drops stage 0 from 290 to 289 packages and leaves every other stage untouched -- stage 6 still holds pipewire and samba. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
The ONNX fix in fa7c3fc worked. pipewire got through builddep, through the build, and into %check for the first time in this factory -- and 51 of its 52 tests passed. The one failure: 39/52 pipewire:pw-test-endpoint FAIL 5.01s killed by signal 14 SIGALRM Ok: 51 Fail: 1 That is not a timeout meson imposed. src/tests/test-endpoint.c:441 arms its own watchdog: alarm(5); /* watchdog; terminate after 5 seconds */ and then drives five sequential pw_main_loop_run round trips through a real pw_context with the session-manager modules loaded. The budget lives inside the test binary, which is why --timeout-multiplier is the wrong lever here and is deliberately not what this commit reaches for: meson never gets to apply a timeout, because the process SIGALRMs itself first. Failing at 5.01s against a 5-second alarm is the signature of a process that ran out of wall clock, not one that deadlocked. What is movable is the contention. %meson_test defaults to one process per core, so 52 tests share a 4-vCPU runner while this one needs five daemon handshakes inside five seconds. --num-processes 1 gives it the box. Nothing is skipped and nothing is made non-fatal. The same 52 tests run and the same exit status still fails the build -- tests_nonfatal is untouched, and it would be the wrong answer even though the spec offers it. This is the approach e6cf24a already took for librsvg2's remaining suites, where a parallel run was likewise what produced the failures. The cost is wall clock in one job. pipewire's %check took 7m22s parallel; one process at a time will be slower, and that is an acceptable trade against a stage 6 failure that blocks the publish gate. Verified from the primary source rather than inferred: the failing test name was read off run 35200615124 job 105241322510, and alarm(5) was read out of the pipewire-1.6.8 tarball this factory locks, whose sha512 matches config/upstream-sources.json. 267 tests pass, validate reports 340 source RPMs, the factory contract is satisfied across 2463 tracked files, and the quoting check clears all 12 workflows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
b6b5b4a did not fix pipewire, and it disproved my diagnosisSerializing
That rules out starvation. The neighbouring tests are the control: Which also kills the fallback I'd proposed. Patching The test emits nothing at all — meson records What this leaves
Some facts that bear on the options, measured rather than assumed:
So the honest options are:
Everything else in the contract is proven. Run 35200615124 completed all eleven stages with this as the only failure in 338 jobs — samba, webkitgtk, nautilus, mesa, flatpak, evolution-data-server, librsvg2, gjs, gdm and gnome-shell all built, and Generated by Claude Code |
|
Correcting myself on this, before anyone acts on it. My comment above offered "drop the Measured since:
This factory is the only source of the audio stack for Utah. And What stands from the earlier comment: the diagnosis. Serialized on an idle runner it still dies at 5.00s while neighbouring tests take 12s and 18s and pass, so it is a hang and not starvation, and raising the watchdog is no more a fix than serializing was. Also worth recording: the reason this surfaced only now is that Full write-up, with the measurements and the three real options, in #132. Generated by Claude Code |
Every run was a full rebuild. prepare decides what is already built by
reading a repository listing, and the URL it read -- the same one every
build root enabled as [factory] -- was the GitHub Pages mirror that README
says was removed. The deployment was never taken down, so it still answers,
frozen at 67 packages from before the 343-recipe contract. Both runs 375
and 377 say it plainly:
published repo has 67 source packages
will build 333 of 340 packages
Nothing publish wrote could ever reach that listing, so no run could ever
skip more than seven recipes. Eight hours for a one-line spec fix, and
webkitgtk alone five of them on the serial path at stage 5, with no change
to its recipe.
The image publish writes is the only thing that moves, so it is now the
only thing either half reads:
- prepare resolves ghcr.io/<owner>/utah-packages to a digest once (the ref's
own tag first, then latest; main reads only latest), extracts /repository
out of it, and runs the plan against that on disk. No image at all is a
warning and an empty witness, which rebuild_plan.py already treats as
"skip nothing".
- every build job, and preflight, extract the same digest with the new
load-factory-repo action and enable it as file:///work/factory. Same
bytes as the plan read, and nothing fetched over HTTP mid-build. The
vars.FACTORY_REPO override is gone: a second source of truth is how the
two halves first came to disagree.
Skipping correctly makes a second gap load-bearing. A fix to mutter would
build mutter alone and leave the published gnome-shell linked against the
libmutter it just replaced. The plan now reads Provides and Requires out of
the same primary.xml and drags every published dependent of anything being
rebuilt, transitively, in inventory order so the waves still hold. It is
the runtime closure; the repository carries no SRPMs, so a devel-only
BuildRequires is the residual gap.
Also on the critical path: debuginfo. The upload step excludes every
debuginfo and debugsource RPM by pattern, so they were built, stripped and
compressed only to be discarded -- 25 minutes of find-debuginfo and 10 of
compression on webkitgtk alone. Both lanes now build with
debug_package %{nil}.
And two things that made the runs read worse than they were: precedence
and publish ran after a failed prepare and died on a missing artifact
(372, 379), so they now require prepare to have succeeded; and a push to a
pull request left the previous run going for hours (369, 370, 374 were all
cancelled by hand), so pull_request runs now cancel their predecessor.
main and the schedule are untouched.
Verified: 293 tests pass, including new ones for the dependency closure
and for the workflow wiring; both workflows and the action parse; the
quoting check passes. The first run after this lands is still a near-full
build, because latest holds the 60-package image from before #70 -- the
last one.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
…n to use it (#178) * feat(sources): poll git forges for releases, not just GNOME The factory's stated purpose is to track the latest upstream release rather than whatever Fedora tagged -- docs/targeting-hummingbird.md is explicit that "a spec is a build recipe, not a release feed". Automation covered 17 of 340 locks, all of them download.gnome.org. 5%. This adds the next tranche that has a real release feed, taking it to 63: download.gnome.org 17 cache.json per-module index (unchanged) github.com 36 tags, or releases where published gitlab (3 hosts) 10 repository tags API The remaining 277 are locked on Fedora's lookaside cache or on a bare directory listing. A lookaside URL carries its own SHA-512 in the path, so there is nothing to poll for "latest" -- reaching those needs each project's real upstream feed identified per package, which is a different job and is filed separately. Three shapes are recognised, measured against the real inventory rather than guessed at: github /archive/<tag>/ (25), github /releases/download/<tag>/ (11), and gitlab /-/archive/ or /-/releases/<tag>/downloads/ (10). Two details came from reading the locks instead of assuming: evtest is pinned over http rather than https, and two freedesktop projects use the GitLab release shape rather than the archive one. Both would have been silent misses -- the first draft of the pattern found 7 of the 10 GitLab locks and said nothing about the other three. Where this differs from the GNOME path, and why: Release selection. GNOME has release cycles, so that path only moves inside the cycle a lock already names. A forge project has no cycle, so the major stands in for it: a newer stable release sharing the current major is an "update" and may be applied, one that crosses a major is "review" only and is never written. A major can move a soname and break every consumer in the graph; no comparison of version strings can make that call. URL rewriting. GNOME entries are rebuilt from a known template. A forge URL has no template this tool can own, because projects choose their own tag and asset names, so forge_planned_entry substitutes the old version for the new and the old digest for the new across url, filename, sha256_url and fallback_urls. Substituting the bare version fixes a v-prefixed tag in the same pass, since "v2.2.1" contains "2.2.1". It refuses outright when the current version does not appear in the URL, rather than writing a new version beside an old tarball. Prereleases. A maintainer's draft or prerelease flag on a GitHub release is honoured before the version string is examined, because some prereleases carry no alpha or beta suffix at all. GITHUB_TOKEN is wired into the workflow and is not optional: anonymous GitHub allows 60 requests an hour against 36 locks, so an unauthenticated run rate-limits part way down the list and reports the rest as unreachable -- which reads as "nothing to bump". A non-list API response is raised as an error for the same reason; treating GitHub's rate-limit object as an empty feed would report every package as current. The workflow is renamed bump-upstream-sources.yml, since it is no longer only GNOME. Nothing referenced the old filename. Verified. 287 tests pass, 20 of them new, all offline through the existing fake opener. The GitLab path was also exercised against the live API: it returns 19, 50, 50, 13 and 75 versions for libcamera, wayland-protocols, shared-mime-info, evtest and accountsservice, and correctly reports each as already at its newest stable -- so "nothing to bump" there is a real answer and not an empty list. validate reports 340 source RPMs, the factory contract is satisfied across 2463 files, and the quoting check clears all 12 workflows. NOT verified here: the GitHub path against the live API. api.github.com answers 403 in this environment -- a plain curl does too, so it is the network and not this code. Its tag, release, draft/prerelease and rate-limit behaviours are covered by unit tests through the fake opener, and the first real exercise will be the workflow's own run with a token. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * fix(ci): skip against the published image, not a retired Pages mirror Every run was a full rebuild. prepare decides what is already built by reading a repository listing, and the URL it read -- the same one every build root enabled as [factory] -- was the GitHub Pages mirror that README says was removed. The deployment was never taken down, so it still answers, frozen at 67 packages from before the 343-recipe contract. Both runs 375 and 377 say it plainly: published repo has 67 source packages will build 333 of 340 packages Nothing publish wrote could ever reach that listing, so no run could ever skip more than seven recipes. Eight hours for a one-line spec fix, and webkitgtk alone five of them on the serial path at stage 5, with no change to its recipe. The image publish writes is the only thing that moves, so it is now the only thing either half reads: - prepare resolves ghcr.io/<owner>/utah-packages to a digest once (the ref's own tag first, then latest; main reads only latest), extracts /repository out of it, and runs the plan against that on disk. No image at all is a warning and an empty witness, which rebuild_plan.py already treats as "skip nothing". - every build job, and preflight, extract the same digest with the new load-factory-repo action and enable it as file:///work/factory. Same bytes as the plan read, and nothing fetched over HTTP mid-build. The vars.FACTORY_REPO override is gone: a second source of truth is how the two halves first came to disagree. Skipping correctly makes a second gap load-bearing. A fix to mutter would build mutter alone and leave the published gnome-shell linked against the libmutter it just replaced. The plan now reads Provides and Requires out of the same primary.xml and drags every published dependent of anything being rebuilt, transitively, in inventory order so the waves still hold. It is the runtime closure; the repository carries no SRPMs, so a devel-only BuildRequires is the residual gap. Also on the critical path: debuginfo. The upload step excludes every debuginfo and debugsource RPM by pattern, so they were built, stripped and compressed only to be discarded -- 25 minutes of find-debuginfo and 10 of compression on webkitgtk alone. Both lanes now build with debug_package %{nil}. And two things that made the runs read worse than they were: precedence and publish ran after a failed prepare and died on a missing artifact (372, 379), so they now require prepare to have succeeded; and a push to a pull request left the previous run going for hours (369, 370, 374 were all cancelled by hand), so pull_request runs now cancel their predecessor. main and the schedule are untouched. Verified: 293 tests pass, including new ones for the dependency closure and for the workflow wiring; both workflows and the action parse; the quoting check passes. The first run after this lands is still a near-full build, because latest holds the 60-package image from before #70 -- the last one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * fix(ci): stop the native find-debuginfo too, or the MinGW specs orphan .debug files Run 380 failed ten stage-0 packages -- enchant2, hidapi, libsoup3, libvorbis, libxmlb, openjpeg, opus, protobuf3, taglib, wavpack -- all with the same error at the same step: error: Installed (but unpackaged) file(s) found: /usr/lib/debug/usr/lib64/libhidapi-hidraw.so.0.15.0-...x86_64.debug /usr/src/debug/hidapi-0.15.0-4.hum1.bfin.x86_64/hidapi/hidapi.h Those ten are exactly the specs that build a MinGW half and say %{?mingw_debug_package}; the eleventh such spec, abseil-cpp, has its mingw bcond off and passed. Nothing else in the run failed. debug_package %{nil} only removes the native -debuginfo subpackage. It does not stop find-debuginfo, which __spec_install_post runs whenever __debug_package is set -- and Fedora's %mingw_debug_package sets it itself (mingw-filesystem, macros.mingw line 20: %global __debug_package 1). So on these ten the native extraction still ran, wrote .debug files into the buildroot, and the subpackage that used to claim them was gone. The fix empties __debug_install_post as well, in both lanes. That stops the native extraction at its source. The MinGW halves are unaffected: their .debug files come from the explicit %mingw_debug_install_post call in %install and go into the mingw32/mingw64 -debuginfo subpackages, which still exist and which the upload filter drops like every other debuginfo RPM. Specs that reference __debug_install_post themselves (libcamera, libxcrypt, grub2) all guard it with %{?__debug_package:...}, which stays unset for them. Reproduced and fixed locally with a five-line spec that sets __debug_package the way the mingw macro does: debug_package %{nil} alone dies on the same "Installed (but unpackaged)" .debug file; with __debug_install_post %{nil} added it writes the RPM. The workflow test now pins both defines in both lanes. 300 tests pass; the quoting check clears all 12 workflows. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Revert "fix(spirv-tools): drop the recipe, let the build root supply it" This reverts commit 56b2973. * Restore spirv-tools at 2026.3 and rebuild stale published packages Run 381 got past every build stage but failed the publish job's "Validate Hummingbird-only consumer transaction" for two reasons. First, mesa-dri-drivers links libSPIRV-Tools.so and Hummingbird does not ship spirv-tools, so dropping the recipe (56b2973) made the runtime contract unresolvable. The recipe is back, moved to Fedora Rawhide's 2026.3 (SDK 1.4.357.0) because Fedora 44's spirv-headers 1.5.5 carries GatherModes, which spirv-tools 2026.2 cannot compile against. The tarball comes from the Fedora lookaside with its sha512 verified; the two downstream patches no longer apply and are gone with the spec. Second, several packages in the 67-package witness were built against sonames the factory no longer provides (libheif -> libavcodec.so.62, gvfs -> libbluray.so.3, mozc -> libabsl 2601, gnome-shell -> libedataserver-1.2.so.27). Their current specs already disable those inputs, but nothing changed in the recipes, so the plan trusted the published build and skipped them. The dependents-dragging rule cannot see this because the missing provider is absent from the witness. rebuild_plan now reads the published repo's Requires and Provides and marks a source stale when one of its binaries requires something that neither the published repo nor Hummingbird provides. rebuild_matrix fetches Hummingbird's primary.xml (baseurl from config/hummingbird.repo) for the external provides, prints the reason for each stale rebuild, and skips the rule with a warning if the fetch fails. Stale packages drag their dependents like any other rebuild. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * docs: record the mistakes the history already made, so they are not made again git log on main is a record of fixes that undid earlier fixes: fifteen commits fighting one ICU split, three digest repins that each died within days, a recipe dropped and restored, the same libheif bcond added and retired twice, four soname breaks that were one staging mistake, and two empty commits pushed to re-run CI. docs/skills/repeated-mistakes.md names each recurring pattern, the commits that made and unmade it, and the rule that settled it, with a checklist to run before pushing a fix. It is indexed in docs/SKILL.md and AGENTS.md so the router loads it for stage, pin, bcond, workflow and recipe-removal work. The triage skill still told readers to repin the build-root digest on exit 125 "manifest unknown"; f1f1e4e showed that cannot work for an image whose digests live hours, and moved the build root to a once-per-run pull. Both rows now say so instead. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Build the Wi-Fi supplicant, libdaemon and the nautilus closure Eight recipes imported from Fedora Rawhide dist-git, each pinned by commit in .hummingbird-upstream.json and source-locked to its upstream release: wpa_supplicant 2.11, wireless-regdb 2026.09.03, iw 6.17 Hummingbird ships NetworkManager-wifi, whose metadata requires wireless-regdb and (wpa_supplicant or iwd), and ships none of the three, so Wi-Fi cannot be installed on Utah at all (#136, utah#97). wireless-regdb in turn requires iw. The wpa_gui bcond is forced off: nothing in the contract wants it and the build root has no qt-devel. libdaemon 0.14 Hummingbird's avahi daemon requires libdaemon.so.0 and ships no libdaemon, so geoclue retries forever against no daemon (utah#104). inih 62 (stage 0), exiv2 0.28.9 (1), libgexiv2 0.16.2 (2), libportal 0.10.0 (5) The factory built nautilus with no way to install it: it needs libgexiv2-0.16.so.4, libportal.so.1 and libportal-gtk4.so.1, none in any repository (utah#100). exiv2 links inih, gexiv2 links exiv2, and libportal links gtk3 and gtk4 (gtk4 is stage 4), hence the stages; nautilus itself is at 6. libportal's Qt5 backend is off, since Hummingbird has no Qt5 and only the GTK4 backend is consumed. inih and exiv2 are GitHub archives and gexiv2 is on download.gnome.org; the bootstrap tool could not fetch them through this environment's proxy, so their locks are written by hand with the sha512 Fedora's sources file records, verified against the byte-identical lookaside copy, which is also the fallback URL, as for the other GitHub-hosted recipes. The runtime contract now asks for NetworkManager-wifi, wpa_supplicant, wireless-regdb, avahi and nautilus, so publication proves the three closures resolve from factory plus Hummingbird before Utah can pin them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Patch pw-test-endpoint's watchdog, and drop the serialization that did not work pipewire was the one failure in 333 jobs of the last run, and it failed the same way it did before b6b5b4a: 39/52 pipewire:pw-test-endpoint FAIL 5.01s killed by signal 14 SIGALRM Ok: 51 Fail: 1 b6b5b4a serialized %check and claimed that fixed it. It did not. The log shows --num-processes 1 in the meson test command line and the identical failure at the identical 5.01s, so contention was never the cause and the serialization was only making the longest job in stage 6 slower. It is reverted here. Reporting that fix without a green job to point at is the mistake docs/skills/repeated-mistakes.md records as pattern 13, made again by me. The real cause is in the test. src/tests/test-endpoint.c:441 arms alarm(5); /* watchdog; terminate after 5 seconds */ around setup that costs more than five seconds on this build root, so the process kills itself; meson never gets to apply a timeout, which is why --timeout-multiplier is not the lever either. The budget is out of line with the test's own siblings rather than with anything the test does: pw-test-stream and pw-test-filter are declared in the same test_apps list in src/tests/meson.build, with the same environment, and took 12.02s and 18.03s in this same run on this same machine. So the watchdog is patched to 60 seconds. Nothing is skipped, disabled or made non-fatal: every assertion still runs and still has to pass. The run is still bounded, by meson rather than by the alarm -- these tests declare no timeout: kwarg, and meson's default is 30 seconds (KwargInfo('timeout', int, default=30) in mesonbuild/interpreter/type_checking.py), so a genuine hang is killed at 30s, before the new alarm can fire. The alarm goes from being the tighter guard to being the redundant one. The patch was applied to the locked 1.6.8 tarball and checked rather than assumed: patch -p1 applies clean and leaves alarm(60) at line 441. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Seed publish from the image prepare witnessed, not from :latest Review finding on dae6241, and it is this change's own bug one layer down. prepare resolves the skip witness as the branch tag first and latest second. publish seeded from a hardcoded ghcr.io/<owner>/utah-packages:latest. So the two disagreed about what is already built: on a second push to a pull request, anything skipped because the *branch* tag carried it was in neither the seed nor that run's artifacts, and the republished branch image simply lost it -- or regressed to the stale latest copy for packages the first push had rebuilt. That is exactly the failure this change removed vars.FACTORY_REPO to prevent, reintroduced at the far end of the same pipeline. publish already needs prepare, so its resolved digest was there to use all along. The empty case is handled rather than assumed: when no image is published at all, prepare emits an empty witness and warns, and the seed now says so and publishes a new repository instead of trying to pull "". The concurrency group is scoped to pull requests too. The group covered every ref while only cancel-in-progress was conditional, so main and schedule runs queued behind one another and GitHub could drop a superseded pending run -- a behaviour change the block's own comment says is not wanted, since concurrent waves are safe by design. tests/test_factory_witness.py asserts the seed reads prepare's output and that no workflow names the latest tag directly. Reverting the seed to :latest fails it, so it is a gate and not a restatement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Build ffmpeg without libplacebo, which no consumer can provide The publish gate on run 35413902261 refused to publish, correctly. All 331 build jobs passed; the Hummingbird-only consumer transaction then failed: Problem 2: package nautilus-gsconnect ... requires gnome-shell-extension-gsconnect = 72-3.hum1.bfin, but none of the providers can be installed - package gnome-shell-extension-gsconnect ... requires /usr/bin/ffmpeg, but none of the providers can be installed - nothing provides libplacebo.so.360()(64bit) needed by ffmpeg-free-9.0.1-1.hum1.bfin.x86_64 from factory libplacebo exists in neither runtime repository. Hummingbird does not package it -- I read its primary.xml and there is no libplacebo of any version among the 20050 packages -- and the factory has no recipe. So the BuildRequires was satisfied by Fedora 44 in the build root alone, and the built ffmpeg-free carried a runtime dependency nothing could ever answer. That is precisely the build-only-becomes-runtime failure this factory exists to prevent, and it took gsconnect down with it because that needs /usr/bin/ffmpeg. Two ways out: add libplacebo to the factory, or stop linking it. libplacebo would need glslang, shaderc and spirv-headers first, none of which the factory builds, so that is four new recipes for Vulkan-backed libavfilter paths (placebo scaling and tone mapping) that nothing in Utah's desktop asks for yet. Turning the bcond off is one line and makes ffmpeg-free installable now; the comment records what it costs and what re-enabling needs. Release goes to 2 so the rebuilt RPM is not the same NVR with different dependencies. This does not address the other half of the gate failure -- nautilus requiring libicuuc.so.77 against a runtime that excludes libicu 77. That one needs a decision rather than a patch, and I have written the evidence up on the PR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Make the build root pick the same ICU the consumer transaction demands The other half of the publish failure on run 35413902261. All 331 builds passed; the Hummingbird-only transaction then refused: package nautilus-51~beta-1.hum1.bfin from factory requires libicuuc.so.77, but none of the providers can be installed - package libicu-77.1-2.1.hum1 from public-hummingbird-x86_64-rpms is filtered out by exclude filtering I read Hummingbird's primary.xml rather than reasoning from the comment history, and it settles the question. Hummingbird has migrated to ICU 78: harfbuzz-icu 14.1.0-1.1 .. 14.2.0-1 libicuuc.so.77 harfbuzz-icu 14.2.1-3 .. 14.3.1-1 libicuuc.so.78 boost-locale 1.90.0-6.1, -7 libicuuc.so.77 boost-locale 1.90.0-9 .. -13 libicuuc.so.78 libtinysparql 3.11.1-2 libicuuc.so.78 Every current build links 78; only superseded ones link 77. So my earlier note on the PR -- that Hummingbird is itself split across both ICUs -- was wrong: I had counted superseded builds alongside current ones. The consumer exclusion of libicu 77 is therefore correct, and nothing current needs 77. What went wrong is that the build root never made the same choice. The two libicu builds share a package name, so dnf installs exactly one and nothing said which. A root that resolved a superseded copy pulled libicu 77; libicu-devel followed it to 77.1 to match; and the package built there linked ICU 77, which its consumers then refuse. Two halves of one decision, disagreeing -- pattern 2 in docs/skills/repeated-mistakes.md. So HB_REPO_EXCLUDE gains the same exclusion, in the same spelling, with release as its own field because dnf splits a spec on dashes before globbing (1f88491). This is not the blanket exclusion 012cb6a reverted. That one reached Fedora, whose build-only deps do legitimately link ICU 77 -- raptor2, localsearch and pipewire-libs-extra transitively -- and it is scoped to the Hummingbird repository here. HB_EXCLUDE already keeps Fedora ICU out of the build root entirely, and the rendered mock config shows both halves of that: excludepkgs=...,libicu,icu,... (fedora, fedora-updates) excludepkgs=ruby3.3-...,ruby3.4-...,libicu-77.*-*hum1 (hummingbird) tools/mock_config.py carries it too, because tests/test_mock_config.py already asserts the mock root and the container root encode one policy -- that test caught this change when only the workflow had it, which is the guard working. New IcuAgreementTests then asserts the build root and the consumer transaction name the same ICU, so they cannot drift apart again. 309 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Stop the staleness model counting an ICU the gate will refuse Third place the same decision lives, and the one that would have let this failure repeat even with the other two fixed. rebuild_matrix builds `external` from every capability Hummingbird provides, libicu 77 included, and stale_from_primary asks whether a published binary's Requires are satisfied by factory plus that set. So a published package linked against libicuuc.so.77 looks satisfiable, is skipped as fresh, and then fails the consumer transaction -- which excludes libicu 77 -- at publish. The model that exists to predict that transaction disagreed with it. Run 35413902261 is what that costs: 331 green builds, precedence green, then publish refused, and nothing published. provides_from_primary now takes the builds the transaction refuses and skips their capabilities, driven by one module constant: EXCLUDED_EXTERNAL = (("libicu", "77."),) A package linked against the refused build is then reported stale, which makes it rebuild against the current root -- the only cure -- rather than being carried forward to fail the gate. The tests use a primary.xml shaped like the real thing, with libicu 77.1 and 78.3 under one name, and assert all three directions: 77 provides nothing, a published package needing .so.77 is stale, and one needing .so.78 is not, so the rule does not condemn everything that touches ICU. Emptying EXCLUDED_EXTERNAL fails three of them. 313 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Revert the build-root ICU exclusion: it stranded the last provider of .so.77 I added libicu-77.*-*hum1 to HB_REPO_EXCLUDE two commits ago and it made the factory strictly worse. bluez failed at stage 0 of run 35443117478: package libical-3.0.20-7.fc44 from fedora requires libicuuc.so.77, but none of the providers can be installed - package libicu-77.1-2.fc44 from fedora is filtered out by exclude filtering - package libicu-77.1-2.1.hum1 from public-hummingbird-x86_64-rpms is filtered out by exclude filtering The reasoning that produced it was checked against the wrong population. I read Hummingbird primary.xml, found that every current Hummingbird build links libicuuc.so.78 and only superseded ones link .so.77, and concluded that excluding 77 could not strand anything. That conclusion is sound about Hummingbird and says nothing about the build root, which also carries Fedora binaries Hummingbird has never rebuilt. libical is one of them and requires .so.77 outright. My own commit message contained the refutation: HB_EXCLUDE already keeps Fedora libicu out of the root, so Hummingbird superseded libicu-77 was the only remaining provider of .so.77. Removing it left none. I quoted that line as evidence the exclusion was safe when it is the reason it was not. 012cb6a reverted a blanket libicu exclusion for this same underlying reason, and scoping it to the Hummingbird repository did not change the reason -- it only changed which repository the last provider happened to live in. Trading a publish-gate failure for a stage-0 failure is a bad trade: the gate failure came after 331 successful builds and named one package, this one loses every build behind it. So the build-root half comes out, in the workflow and in tools/mock_config.py, and both places now carry a note saying why it must not come back. The publish-side half stays. EXCLUDED_EXTERNAL in tools/rebuild_plan.py and the consumer exclusion are about what may be installed on a Hummingbird system, where nothing current links ICU 77; the build root is a different question with a different answer. IcuAgreementTests asserted the two halves were symmetric; it now asserts the asymmetry and says why, so the next person to notice that Hummingbird has migrated to 78 does not make this change again. The underlying problem is unsolved and stated as such in both comments: a factory package can still link ICU 77 because Fedora build deps pull it into the root, and the consumer transaction then refuses that package. It has to be fixed without removing the .so.77 provider the root legitimately needs. 315 tests pass. check_workflow_quoting.py clean; the new comment contains no apostrophe. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Build samba before the packages that would otherwise take Fedora ICU 77 This is the actual cause of the publish failure the ICU exclusions were flailing at, and it is an ordering bug, not an exclusion one. Hummingbird ships libicu 77.1 beside 78.3 under one name, so a build root installs exactly one. dnf takes the newest unless something in the root requires libicuuc.so.77, and then it downgrades -- libicu-devel follows the base to match, because icu.spec pins it exactly: Requires: lib%{name}%{?_isa} = %{version}-%{release} so there is no devel-only split to exploit, which was the other fix I considered. Whatever builds in that root then links ICU 77, and the Hummingbird-only consumer transaction refuses it. The chain in run 35413902261 runs entirely through our own staging: Fedora samba-core-libs-4.24.6-1.fc44 requires libicuuc.so.77 -> localsearch (stage 4) resolved samba from Fedora, because the factory samba was stage 6 and did not exist yet. Its root installed libicu-77.1-2.1.hum1 over five available 78.3 builds, including our own libicu-78.3-1.hum1.bfin, and localsearch-3.12~beta-1.hum1.bfin came out requiring .so.77. -> nautilus (stage 6) requires localsearch, so its root downgraded too, and nautilus-51~beta-1.hum1.bfin required .so.77. -> publish step 6 refused nautilus. All 331 builds had passed. The factory samba is not the problem: it built against libicu-78.3-1.hum1.bfin with zero .so.77 anywhere in its root. Its own entry says it was rebuilt precisely to stop requiring .so.77. It was simply built four stages too late to be used by the packages that needed it. samba moves to stage 2 and libtevent, its only factory build dependency above stage 0, to stage 1. Both are safe by their own dependencies: libtevent is built from libtalloc and make, both stage 0; samba from libtevent plus libtalloc, libtdb, icu and libxcrypt, all stage 0 or now 1. Nothing moves later, so no existing consumer ordering can break -- pulling a provider earlier only ever adds packages to the roots that follow it. tests/test_icu_staging.py makes this a gate. It asserts samba precedes both known consumers and that samba still follows everything it is built from, and it fails against the previous stage numbers. The docstring carries the whole chain, because the next person to meet an ICU 77 requirement will otherwise reach for an exclusion again -- which is what de4ac4d had to revert, since Fedora packages the root legitimately needs (libical) require .so.77 and excluding it strands them outright. 318 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Build gvfs after samba too: it takes ICU 77 by the same route Found by asking which factory packages consume samba, which I should have done in the same breath as moving it. gvfs BuildRequires libsmbclient-devel -- a samba subpackage -- and sat at stage 1, so it resolved libsmbclient from Fedora and its root installed libicu-77.1-2.1.hum1, exactly as localsearch did: [ 34/485] Installing libicu-0:77.1-2.1. 36.4 MiB [132/485] Installing libicu-devel-0:77. 5.1 MiB That was already true before the previous commit -- gvfs stage 1 against samba stage 6 -- so this is not fallout from moving samba, it is the same bug one package over that moving samba to stage 2 did not by itself reach. gvfs goes to stage 3. It is unconstrained in both directions: no factory package BuildRequires gvfs at all, and its only build dependency above stage 0 is samba, now at 2. Everything else it needs -- libnfs, libsoup3, libsecret, udisks2, gcr, ibus, libcdio, libplist, libimobiledevice, gnome-online-accounts, libtalloc -- is stage 0. The dependency scan behind both moves now resolves pkgconfig() BuildRequires as well as plain names, which the first pass skipped; re-running it over samba confirms its factory dependencies are still only libtevent (1) and stage-0 packages, so stage 2 remains correct. tests/test_icu_staging.py covers gvfs in both directions and fails against the previous stage number. 318 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Find the remaining ICU-77 consumers by rule instead of one at a time Three contaminated packages had been found three different slow ways -- localsearch and nautilus from the publish failure, gvfs by asking who consumes samba, and each of those took a round trip. So I asked the general question instead: which factory packages link ICU at all, and does anything consume one of them from its own stage or earlier? Nine factory packages BuildRequire ICU (enchant2, evolution-data-server, libical, libphonenumber, localsearch, mozjs140, re2, samba, webkitgtk). Exactly one unfixed ordering violation fell out: bluez (stage 0) BuildRequires libical-devel from libical (stage 0) Same stage, so bluez cannot see the factory libical and takes Fedora's, which requires libicuuc.so.77. Confirmed against run 35413902261, where bluez root installed libical-0:3.0.20-7.fc44 from fedora and then libicu-0:77.1-2.1.hum1. The factory own libical is clean -- it built against libicu-78.3-8.hum1 -- it was simply not available in time, exactly as samba was not available in time for gvfs and localsearch. bluez moves to stage 1. Nothing constrains it in either direction: no factory package BuildRequires bluez, and its only factory build dependencies are libical and make, both stage 0. The more useful half is the new IcuLinkerOrderingTests, which derives this rather than restating it. It scans every recipe for an ICU BuildRequires, then for every factory consumer of one of those packages asserts the consumer builds strictly later. Verified it fails against bluez previous stage with the offending pair named. Not every same-stage pair would be a live bug -- it only bites when Fedora build of the provider actually requires .so.77 -- but every such pair is a place where a Fedora ICU can enter a build root unnoticed, and the fix costs one stage number. Finding the fourth case by hand was not going to be cheaper than encoding the rule. 319 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Add the cache key for per-package build reuse, and the rule for when to use it The factory rebuilds about 282 of its 349 packages on every run, because its only skip mechanism is the atomically-published consumer repository and publication needs every stage plus precedence plus the Hummingbird-only transaction. A failing gate therefore freezes the witness, and the witness has been ghcr.io/projectbluefin/utah-packages:latest @sha256:9cb66729 since 2026-08-30 with 67 of 349 packages in it. webkitgtk, which is not among them, built 20 times on 19 September at 182 minutes a build, with a recipe unchanged since 2026-09-05 and an inventory entry byte-identical across every commit since. Issue #177 has the measurements. This is the first half of the fix: a key for a cache that answers "have we already built this exact thing", kept strictly apart from "is the repository coherent enough to install from". Nothing installs from the cache. A hit is materialised as the stage's ordinary artifact, so precedence, the transaction check and publish see an identical repository either way -- the cache changes how the RPMs were obtained, never what the run validates. That is the property that makes a hit safe to trust. Keying on the recipe alone would be wrong, and would reintroduce exactly the incoherence the publish gate exists to catch. A stage-5 package builds in a root holding earlier stages fresh output, so the resolved build root is part of the key, which means the key is computed after dnf has resolved builddep rather than before: the resolution costs about a minute and the compile costs 182. The chain only holds across stages if an unchanged recipe yields an unchanged NEVR, or every root would differ from last run and nothing would ever hit. It does. 167 of 349 specs use %autorelease, which rpmautospec derives from the committed packages/<name>/changelog rather than from a build counter, and libical-3.0.20-1.hum1.bfin came out identical in run 35413902261 and run 35445712318, twelve hours and many commits apart. The changelog is therefore part of the recipe digest, and there is a test saying so. Recipes are hashed whole, every file, paths alongside contents so a rename is a change, for the reason utah's kernel-cache-tag.sh gives: it can only ever rebuild something that did not need rebuilding, never reuse something stale. rebuild_plan.cacheable() is the second half: which selected packages may consult the cache. It narrows nothing -- plan() still decides what to build. It excludes `changed`, because the author of a recipe edit is owed a real build (the key would miss anyway, so this is belt and braces), and `stale`, which matters: a stale published package requires a soname nothing provides any more and its recipe may not have moved, so its key can hit and hitting would hand back the very build that is broken. That is the one case where a hit is worse than a miss, so it is named rather than left to chance. Verified the key changes on every input -- package, recipe, buildroot digest, factory digest, resolved root, disttag, schema -- and does not change on transaction order or duplicates in the root. An empty resolved root is refused rather than keyed, because a key over nothing collides across genuinely different roots. Not in this commit: the workflow wiring that publishes and consults the cache. That is a separate change and wants this key reviewed first. 19 new tests, 338 in the suite. check_workflow_quoting, validate.py and the factory contract all clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Tag cache entries <package>-<key>, under a cache image name of its own Review suggestion, and better than the bare hash I had. A tag of nautilus-e02585b1... can be listed, filtered and reasoned about in the registry without a side database, which is what you want when the question is "why did this miss". A bare hash says nothing. The prefix adds no uniqueness -- the package name is already inside the hash, and a test asserts that two packages with otherwise identical inputs get different keys, so the prefix can never be the only thing separating them. It is metadata, not identity. Checked before adopting it, rather than after: all 349 recipe names are valid OCI tag components, and the longest tag is 64 of the 128 characters (gnome-shell-extension-gsconnect). But RPM names are laxer than tags -- gtk+ and libstdc++ are legal packages and illegal tag components -- so cache_tag refuses a name a tag cannot carry instead of emitting a mangled one. A silently mangled tag would be a permanent cache miss that reads as a correctness bug. No recipe needs it today; an import could. The one thing this must not do is share an image name with utah-packages. That name is the consumer repository, and prepare resolves utah-packages:latest and utah-packages:<branch> and treats whatever it gets as a whole repository. A cache entry holds one package's RPMs. If they shared a tag namespace, a resolve landing on a cache entry would compute `published` from a single package -- and publish seeds from that same image, so it would drop the other 66. The docstring says so at the point where someone would be tempted. --key-only prints the bare key for callers that want it, and still builds the tag first so the grammar is validated whichever output was asked for. 26 tests, 345 in the suite. One pre-existing test asserted the CLI printed a bare key; it now asserts the tag, since that is the contract the workflow will use. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Drop the package-name prefix: the key is the tag Right, and this reverses the previous commit. The prefix bought nothing the key does not already carry -- the package name is inside the hash, and a test asserts two packages with otherwise identical inputs get different keys -- while introducing a failure mode the bare key cannot have. That failure mode was mine to begin with. RPM names are laxer than OCI tags: gtk+ and libstdc++ are legal packages and illegal tag components, so the prefix needed a grammar check, a refusal path and a subtest over all 349 names, all to guard a hazard that only existed because of the prefix. A 32-character hex digest always matches [a-zA-Z0-9_][a-zA-Z0-9._-]{0,127}, so with the key as the tag there is nothing to validate and nothing an import can break. Verified across 20000 generated keys, and gtk+ now hashes like any other name. The legibility argument was the only one for the prefix, and it was aimed at the wrong place. Browsing a tag list is not how you answer "why did this miss" -- the build log already prints the package and the key it computed, which is what you actually read. For the registry side, the push step sets labels, which survive without constraining the tag grammar: org.opencontainers.image.title = <package> org.opencontainers.image.version = <nevr> org.opencontainers.image.revision = <commit> org.projectbluefin.factory.disttag = <.humN.bfin[.N]> skopeo inspect answers the question, and a tag listing still answers "how many entries exist", which is the other thing you would want from it. The note about not sharing an image name with utah-packages stays, because that hazard is real and independent of the tag shape: prepare resolves utah-packages:latest and :<branch> and treats what it gets as a whole repository, and publish seeds from that same image. 22 tests, 341 in the suite. The removed tag tests are replaced by TagIsTheKeyTests, which pins the property that makes the prefix unnecessary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * fix(ci): address package cache review findings --------- Co-authored-by: Claude <noreply@anthropic.com>
tools/rebuild_plan.py (from projectbluefin#70) reads spec_release() too and treats None as "no comparable release"; "" made it concatenate a suffix onto nothing and fail test_an_autorelease_recipe_falls_back_to_the_version. The plan now also short-circuits on None explicitly instead of relying on BumpError. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
…ries ffmpeg's recipe was first imported in projectbluefin#70; projectbluefin#414 only added ffmpegthumbnailer, so keep pr = 414 just for the thumbnailer names. Hive-Fix-Attempt: 3/6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Takes the contract from #51 and fixes everything that failed in its last run
(34415980185), so a single run can build what a Utah image actually needs.
That run's numbers: 285 build jobs, 279 succeeded, 5 failed, gcc cancelled
with the run.
publishwas skipped because the run ended cancelled, sonothing from it reached the repository. Of the five failures, one was a real
recipe defect and four were the build root not containing what the recipe
needed.
Firefox comes out
Utah ships Firefox as the
org.mozilla.firefoxFlatpak. The factory built thebrowser only because Bluefin's copied
[fedora]section names it, andprojectbluefin/utah already lists
firefoxas unavailable while carrying theFlatpak in
contracts/bluefin-desktop.toml.It was not a cheap recipe to keep: a 2.5-hour PGO/LTO job, the longest of the
285, carrying its own 12G swap workaround in the build workflow — and it
failed anyway, in
%install:So the recipe, its source lock, its
.packit.yamlentry and the swap step thatexisted for it alone are gone, and
firefoxjoins[unavailable]in theruntime contract — which is what keeps the Bluefin-requested name from failing
the transaction.
mozjs140is untouched: it is the embeddable JavaScriptruntime and unpacks a firefox tarball of its own.
The build root could not see the published repository
This is the interesting one.
prepareskips any package the publishedrepository already carries at the same version — but the
[factory]repo blockin
build-stage.ymlis guarded byvars.FACTORY_REPO, which is unset onpull_requestruns. The block never ran, so a skipped package was not "alreadyavailable", it was absent. 59 of the 344 recipes were in that state.
That is why
pipewire-libs-extrafailed onwhile
libfreeaptx-develsat published and skipped.FACTORY_REPOnowdefaults to the same GitHub Pages URL
preparereads to decide what to skip,so the two halves of that decision cannot disagree. The repository variable
still wins when set.
Two recipes were staged before what they need
xorg-x11-server-Xwaylandwayland-client >= 1.26.0, build root had Fedora's 1.25.0; this factory buildswaylandat 4evolution-ewslibebook/libecal/libedata-*>= 3.61.3;evolution-data-serverbuilds at 9Xwayland's three stage-0 build consumers (
libayatana-appindicator,libayatana-indicator,zenity) resolve it from Fedora and never saw ours, somoving it later changes nothing for them.
gstreamer1-plugins-bad-free declines chromaprint
On exactly the reasoning
ffmpeg.specalready records: Fedora'slibchromaprintlinkslibavutil.so.60, this factory's ffmpeg 9.0.1 providesso.61, and the factory excludes the Fedora libav by name — solibchromaprint-develis uninstallable here, permanently, not as a bootstrapphase. The spec disabled chromaprint only under
%if %{without extras}; it isnow its own bcond, off, with the
BuildRequiresand the plugin in%filesfollowing it. Nothing in the runtime contract links AcoustID fingerprinting.
gccwas cancelled with the run rather than failing, and needs only a re-run.Relationship to the open PRs
matrix-cap sharding and eleven stages. fix: shard rebuild stages past GitHub's 256-job matrix cap #52 was closed unmerged despite an
lgtm; main still caps at five stages, and this contract needs eleven.Verification
tools/validate.pyreports 343 source RPMs, the runtime contract resolves to71 packages against the pinned base image, the workflow quoting check passes
across all 11 workflows, both changed workflows parse, and all 73 tests pass.
Not verified locally: the build itself. That is what this PR's run is for.
The chromaprint bcond and the FACTORY_REPO default can only be confirmed by
Actions building against a real root.
🤖 Generated with Claude Code
https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
Generated by Claude Code