Repository navigation
fix(kernel-cache): hash Containerfile.kernel whole so a recipe edit rotates the tag - #171
Conversation
…otates the tag scripts/kernel-cache-tag.sh hashed only the ARG BASE_IMAGE= line of Containerfile.kernel, but the body of that file is the recipe: the single RUN decides what lands in /cache-out and the final stage decides what is copied out of the builder. Editing either left the tag unchanged, and CI builds the cache image only when its tag is not already published, so the gaming, nvidia and nvidia-gaming flavors would take a cache image built from the previous recipe and unpack stale artifacts without any failure. Hash the whole file instead. This strictly widens the key -- the base image pin is still covered, so Renovate bumps still rotate the tag -- and restores the bias the script documents, which is to over-rebuild rather than ever reuse something stale. tests/test_kernel_cache_tag.py holds the contract: every hashed input moves the tag, and a recipe edit that leaves ARG BASE_IMAGE= alone moves it too. Both new assertions fail against the previous implementation. The key rotates once as a result, so the next build of each cached flavor pays one ~45-minute cache rebuild. Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
|
Important Held for human review by the hive's ACMM level gate. This PR was opened by the "scanner" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the Hive will automatically remove the |
hanthor
left a comment
There was a problem hiding this comment.
Approving and merging. The repository owner has cleared me to act on hold items, so I am clearing the hold on this one rather than leaving it parked.
I reproduced the defect against main instead of taking the description's word for it — same probe, both implementations:
main before=a5439a843919c4cc after=a5439a843919c4cc same=YES
fixed before=5e80352e212ee6c5 after=508588d79be0da52 same=no
where the probe rewrites /cache-out throughout Containerfile.kernel and touches no ARG BASE_IMAGE= line. So on main the recipe that decides what fills the cache could be rewritten wholesale without moving the tag, CI would find the tag published, skip the rebuild, and hand gaming, nvidia and nvidia-gaming an image built by the previous recipe. Nothing would go red — the install scripts would find /utah-cache and unpack it. That is the bad direction of the bias the script's own header claims: reuse something stale rather than rebuild something that did not need it.
The widening is strictly a superset — the ARG line is still inside cat Containerfile.kernel — and test_base_image_pin_still_moves_the_tag guards precisely against the widening having dropped what the narrow key covered, which is the mistake worth guarding here.
Two things I checked beyond the diff:
kernel_cacheon this run took 24 minutes rather than the usual ~2, because rotating the key actually forced the rebuild. The one-time cost the description predicts is visible in the run, not just asserted.- This and #166 both touch
docs/skills/kernel-cache.md, so I merged them together locally before merging either: they auto-merge, and all 11 tests from both files pass on the combined tree. They are complementary rather than overlapping — this one covers the recipe body, #166 covers the content of files a futureCOPYmight add.
One small correction to fold in whenever AGENTS.md is next touched: the kernel-cache bullet lists Containerfile.kernel, the two install scripts and packages/hummingbird.repo, but the script also hashes packages/fedora-44.repo. The prose in docs/skills/kernel-cache.md has it right; the bullet is one repository short.
Generated by Claude Code
…ctually build (#182) * Bump the NVIDIA driver to the release that compiles against kernel 7.2 The base image pin has been frozen since Hummingbird moved to kernel 7.2. Every attempt to advance it -- #122 is the current one -- fails kernel_cache 22 minutes in, building the NVIDIA open module: nvidia/os-interface.c: In function 'os_get_current_process_name': nvidia/os-interface.c:764:5: error: implicit declaration of function 'strncpy' [-Wimplicit-function-declaration] 764 | strncpy(buf, current->comm, len - 1); make[4]: *** [.../7.2.5-200.fc44.x86_64/...: nvidia/os-interface.o] Error 1 A frozen base pin means no kernel security updates reach the flavored images, so this is worth fixing properly rather than pinning around. The cause is the kernel finishing its strscpy migration and removing strncpy, and the fix is upstream's, not ours. Read from the vendor source at each tag: 595.84 strncpy(buf, current->comm, len - 1); 595.91.07 strscpy 595.99.02 if (strscpy(buf, current->comm, len) < 0) { So 595.99.02 carries the migration, and it is also what https://download.nvidia.com/XFree86/Linux-x86_64/latest.txt designates today -- which is the rule the pin comment already states. This is that policy working, not a hand-picked version: read latest.txt, take the SHA-256 NVIDIA publishes beside the installer. The base bump rides along deliberately. On the current base the old driver compiles fine, so a driver-only change proves nothing -- the two can only be verified together, and kernel_cache on this PR is that verification. Both ARG BASE_IMAGE= lines move together, as `just check` requires. The cache tag rotates 1ce05c4b16fafe9c -> f1e31a1dee27ffaa, so the three cached flavors compile the kernel once. That is the intended cost: the tag moving is what proves the new base and driver were actually built, rather than a stale cache being reused (#166, #171). Closes #173. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Make the shim mirroring say what it could not find, and pair the kernel-devel hash with its kernel The base bump to a kernel-7.2 Hummingbird regressed build_main. It failed deterministically, three attempts out of three, and printed nothing: the last line of the log was "Utah desktop contract passed", then exit 1. The step was four predicates chained with && at the tail of a long RUN -- shim_version="$(rpm -q --qf '%{VERSION}-%{RELEASE}' shim-x64)" && \ test -d /usr/lib/bootupd/updates/EFI/fedora && \ install -d ... && cp -a ... -- so nothing distinguished "shim-x64 is not in this base" from "shim-x64 is installed but staged nothing", and those have different answers. The chain now lives in scripts/mirror-shim.sh, which names the failing predicate, lists what the bootupd tree actually contains when the payload is absent, and says that a base which dropped shim-x64 needs it added to the package contract rather than the step skipped. This does not yet fix the build; it makes the next run say which of the two happened, which the previous three runs could not. The second change is the review finding on the same bump. KERNEL_DEVEL_SHA256 guards an unsigned koji download and is documented as a constant precisely because "the kernel cannot change without BASE_IMAGE changing" -- which is the reason it must move when BASE_IMAGE does, and it did not. It stayed pinned to 7.1.8-100.fc43 while the base moved to 7.2.5-200.fc44. Nothing failed, because the dnf path satisfied 7.2 and the fallback never ran; it would have failed the first time the enabled repositories dropped the kernel, which is the single situation the fallback exists for. The hash is now the real one for kernel-devel-7.2.5-200.fc44.x86_64, taken from koji, and it is recorded beside KERNEL_DEVEL_NEVR. A mismatch between the recorded kernel and the booted one now fails immediately with both versions named, instead of after fetching 60 MB to compare against a hash that cannot match. Tests drive the real code: mirror-shim.sh runs against a stub rpm and a fake bootupd tree for all three outcomes, asserting the two failures do not share a message; the NEVR guard is extracted and executed for both the matching and mismatched kernel. 204 tests, OK. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Handle the base image that already mirrors the shim payload itself I did not need to wait for CI for this. Listing both base images layer by layer from the registry answers it outright, and the answer is that the new base has already done the work: old base sha256:c5539f9e... usr/lib/bootupd/updates/EFI/fedora/{shimx64,shim,mmx64,grubx64}.efi no usr/lib/efi/shim at all new base sha256:db1007fd... usr/lib/efi/shim/16.1-5/EFI/fedora/{shimx64,shim,mmx64}.efi, BOOTX64.CSV usr/lib/bootupd/updates/ holds only BIOS.json and EFI.json So the destination Utah was copying into is exactly what the new base now ships, and the source it was copying from no longer exists. The step was not broken by a missing package or a broken transaction; it was asserting a layout the base has moved on from, and failing on a job that was already done. mirror-shim.sh now looks for the payload before doing anything else and exits 0 when it is already in place. It searches for the EFI/fedora/shimx64.efi file rather than assuming a directory name, because the base names that directory for the shim version with no dist tag and rpm need not report the same string. Checking the filesystem first also means the base that ships the payload is the base whose shim-x64 rpm query we no longer depend on. The old layout still works: the copy path is unchanged and tested. The two real failures -- no package and no payload anywhere -- still fail, still say which, and still do not share a message. Tests drive the real script against synthetic filesystems for both layouts, including the case where rpm reports 16.1-5.fc44 while the base directory is 16.1-5. 206 tests, OK. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Make the kernel NEVR overridable, so the hash override is not dead code Review finding, and correct. I added a guard that exits when the booted kernel differs from KERNEL_DEVEL_NEVR, to turn a 60 MB download ending in a checksum mismatch into an immediate failure naming both versions. But UTAH_KERNEL_DEVEL_SHA256 exists for exactly one situation -- validating a kernel this file has not been re-recorded for -- and the guard runs before the hash check, so it blocked precisely the case the override is for. Correct override or not, that path exited 1. I turned an escape hatch into dead code while adding a guard meant to make failures clearer. KERNEL_DEVEL_NEVR now takes UTAH_KERNEL_DEVEL_NEVR the same way the hash takes its override, and the guard message names both variables, so the way past it is to set the pair deliberately rather than to edit the script. Overriding one without the other still fails, which is the intended behaviour: the hash belongs to a specific NEVR and the two only mean anything together. Three tests: both constants accept an override, the guard message names them, and the real guard extracted from the script reaches the download when both are set to a kernel that is not the recorded one. The first fails against the previous constant line. 209 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Cluster:
scripts/kernel-cache-tag.sh,docs/skills/kernel-cache.md§ What the tag hashes, the kernel-cache bullet inAGENTS.md, and newtests/test_kernel_cache_tag.py. Touches noContainerfile.kernelcontent, no install script, and no workflow, so it is disjoint from #133, #154, #158 and #162.Fix
scripts/kernel-cache-tag.shdeclared its own contract — "It must change whenever anything the cache image contains would change" — but hashed only one line ofContainerfile.kernel:grep -m1 '^ARG BASE_IMAGE=' Containerfile.kernelThe body of that file is the cache recipe. The single
RUNdecides what fills/cache-out; the final stage decides what is copied out withCOPY --from=builder. Edit either and the tag did not move:CI builds and pushes the cache image only when its tag is not already published, so after such an edit the old tag is found, the rebuild is skipped, and
gaming,nvidiaandnvidia-gamingtake the previous recipe's image as theirBASE_IMAGE. The install scripts find/utah-cacheand unpack instead of compiling, so stale artifacts ship with nothing red to show for it.This is the exact opposite of the bias the script documents — it hashes whole scripts, comments included, so it "can only ever rebuild something that did not need rebuilding, never reuse something stale".
What changed
scripts/kernel-cache-tag.sh:grep -m1 '^ARG BASE_IMAGE='→cat Containerfile.kernel. This strictly widens the key; theARG BASE_IMAGE=line is still hashed, so Renovate base-image bumps still rotate the tag. Header comment updated to say why the whole file is hashed.tests/test_kernel_cache_tag.py(new): copies the hashed inputs into a scratch tree and asserts each one moves the tag; asserts a recipe edit that provably leavesARG BASE_IMAGE=untouched still moves it; asserts the base pin alone still moves it, so the widening did not drop what the key already covered. Two of these fail against the previous implementation.docs/skills/kernel-cache.mdandAGENTS.md: corrected — both described the narrow behaviour as if it were the contract.Verification
(121 before, +5 new.) Against the pre-fix script the new suite is
FAILED (failures=2), so it is a gate and not decoration.python3 scripts/generate_skill_index.py --checkis clean.Expected one-time cost
Widening the key rotates the tag once (
791f9b07ddd07e54→5e80352e212ee6c5), so the first build of each cached flavor after this merges pays one ~45-minute cache rebuild. That is the correct price for a key that was previously able to reuse a stale image.Closes #170
Filed by scanner agent (ACMM L5 — hold-gated mode). Hold-gated: human review required.
— hive: agent=scanner backend=copilot model=claude-opus-5 copilot=1.0.78