Repository navigation
Conversation
A pipeline change used to be proven by dispatching the full factory and reading it five to ten hours later. canary.yml calls rebuild-rpms.yml itself (now also a workflow_call target) over libtalloc, libtdb (stage 0) and libtevent (stage 1, BuildRequires libtalloc-devel): - pass1 builds, checks precedence and a Hummingbird-only transaction over every binary built, and publishes, signs and attests utah-packages:canary-<pr>, never latest. verify-publish reads the digest with Utah's own check-repo-availability.py fetched from Utah main, asserts two layers with repodata first and exactly the canary set, and verifies the cosign signature and the provenance attestation. - pass2 repeats the set and must compile nothing (cache key stable). - pass3 adds python-typing-inspection with a perturbed recipe: it compiles, every other package must still hit (no wholesale invalidation). The "Canary" job always runs and is the required check; it passes without building when a PR touches no pipeline path. Every new rebuild-rpms/build-stage input defaults to production behaviour: an empty cache salt leaves production keys byte-for-byte unchanged (tested), and call-only booleans are negative so null on schedule/dispatch reads false. Also fixes build-stage.yml downloading every artifact of the run into work/prior: it read steps.prior.outputs.pattern with no step of that id, so same-stage siblings leaked into the stages repository (repeated mistake 3). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QMTLsnqW2jtz5QsNC3W3qv
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QMTLsnqW2jtz5QsNC3W3qv
…ulkan-loader samba.org answered the runner with bytes that did not match the talloc lock while serving the right tarball everywhere else; the source pipeline rightly refuses that. A canary has to fail on the pipeline, not on an upstream host. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QMTLsnqW2jtz5QsNC3W3qv
Member
Author
Canary proof: run 36170094324, green
Cache verdict (from the run's own jobs API):
Published About the 15-minute target: pass1's critical path is libical (5 m 11 s with its Two things the first attempts turned up:
|
hanthor
marked this pull request as ready for review
September 25, 2026 18:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Top-priority item: a canary so every pipeline change is proven in minutes on the real jobs, before any full run.
What
.github/workflows/canary.ymlcallsrebuild-rpms.ymlitself (newworkflow_calltrigger), not a copy, over a fixed set that spans two stages with a reverse dependency:libtalloc,libtdb(stage 0) andlibtevent(stage 1,BuildRequires: libtalloc-devel). All three run-time close over Hummingbird alone (checked against the published repodata and Hummingbird's primary.xml), and built in about 4.3 min each in recent runs.pass1consumer: built), publish + cosign sign + provenance attestation toutah-packages:canary-pr<N>. Neverlatest: both prepare and publish refuse it.verify-publishscripts/check-repo-availability.py(fetched from Utahmainat run time,repository_metadata(): layer 0 < 64 MiB, onlyrepository/repodata/*), asserts exactly two layers with the small one first and that the repodata carries exactly the canary set, thencosign verifyagainst the workflow identity andgh attestation verify.pass2skip_publish: every build job must be a cache hit, zero compiles.pass3python-typing-inspectionwith its spec perturbed in the checkout: it compiles, the canary set must still hit.verify-cacheBuild the verified source…skipped +Restore package RPM cachesuccess), writing a table to the job summary.Canary.github/workflows/,.github/actions/,tools/,config/,Containerfile*); fails for fork PRs that do (they cannot publish).pass2andpass3only needpass1's cache entries, so they run in parallel.The cache is salted with a digest of the build path (
tools/canary.py salt:build-stage.yml, the load actions,hummingbird.repo,source_pipeline.py,dist_bump.py,package_cache_key.py). A PR that changes how packages compile compiles for real inpass1; anything else reuses the previous canary's builds. Dispatch withcompile=trueto force it.Production behaviour is unchanged
Every new input defaults to what a production run does:
cache_saltis left out of the key payload entirely, so production keys are byte-for-byte what ci(buildroot): factory-owned build-root mirror, and a cache key that survives a publish #252 made them (new test recomputes the unsalted key).artifact_prefixempty → identical artifact names.no_factory,skip_publish) sonullon schedule/dispatch reads as false; GitHub'snull == falseis true, which would have silently disabled publish.needs.prepare.outputs.seed_image, which equalsfactory_imageunless apublish_tagis given (repeated mistake 14 kept).One real bug fixed on the way
build-stage.ymldownloaded earlier stages withpattern: ${{ steps.prior.outputs.pattern }}but no step hadid: prior, so the pattern was empty and download-artifact pulled every artifact of the run intowork/prior, including same-stage siblings that finished first (repeated mistake 3) and the build-root tarball. Apriorstep now namesrpm-s{0,…,N-1}-*. This can surface a mis-staged package that only built because a same-stage sibling leaked in; that is the intended rule, and item 1 (solved waves) removes the hand staging anyway.After merge
I'll make
Canarya required status check onmain.Proof
The canary run on this PR is the proof; numbers to follow in a comment.
🤖 Generated with Claude Code