Repository navigation
test: gate the kernel cache key against Containerfile.kernel's build inputs - #166
Conversation
…inputs scripts/kernel-cache-tag.sh hashes a hand-written list of inputs. Containerfile.kernel decides, separately, what the cache image is built from. Nothing tied the two together, so a COPY added there without a matching hash line yields a key that does not move when that input does -- and CI builds the cache image only when its tag is not already published, so gaming, nvidia and nvidia-gaming would keep unpacking a kernel and an NVIDIA module built from the superseded input, silently. tests/test_kernel_cache_key.py derives the input set from Containerfile.kernel (the ARG BASE_IMAGE= line and every non---from COPY source) instead of restating it, then mutates a copy of the tree and re-runs the script: every derived input must change the key, an unrelated file must not, and the key must be deterministic. A sandbox-versus-repo equality check keeps the mutation tests from passing against an incomplete tree. Closes #165 Assisted-by: Claude Opus 5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
|
Important Held for human review by the hive's ACMM level gate. This PR was opened by the "architect" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the Hive will automatically remove the |
hanthor
left a comment
There was a problem hiding this comment.
Approving on the substance. Not merging — the hold label says human review is required, and clearing it is not mine to do.
I verified the claim rather than taking it from the description. Checked out the head and ran the suite, then made it fail on purpose:
$ python3 -m unittest tests.test_kernel_cache_key
Ran 6 tests in 0.139s
OK
$ printf 'COPY packages/utah.toml /tmp/probe.toml\n' >> Containerfile.kernel
$ python3 -m unittest tests.test_kernel_cache_key
AssertionError: '791f9b07ddd07e54' == '791f9b07ddd07e54' : Containerfile.kernel copies
packages/utah.toml into the cache image, but editing it does not change the cache tag,
so CI reuses a cache image built from the old packages/utah.toml. Add it to the hashed
set in scripts/kernel-cache-tag.sh.
So the gate is real and the failure message names the fix. Two things make it worth having rather than decorative:
- It is behavioural.
scripts/kernel-cache-tag.shdoescd "$(dirname "$0")/..", so running the copy in the sandbox hashes the sandbox — the mutation tests genuinely re-derive the key rather than asserting on the script's text.test_sandbox_reproduces_the_real_keycloses the hole where an incomplete sandbox would make the mutations pass vacuously. - It asserts the "and not otherwise" half too. A key that moved on every edit would pass a naive version of this test while rebuilding the 45-minute cache image for nothing, which is the cost the cache exists to avoid.
The failure mode it closes is the quiet one, which is why it is worth a test: an unhashed input means the tag does not move, build.yml skips the rebuild because the tag is already published, and gaming, nvidia and nvidia-gaming keep unpacking a kernel built from the superseded input. Nothing goes red. This is pattern 4 in docs/skills/repeated-mistakes.md — a config change that alters a build has to invalidate the published copy — turned into a gate instead of a paragraph.
One limit worth stating so nobody reads more into the green than is there: the derivation covers ARG BASE_IMAGE= and non---from COPY sources. An input that enters the cache image some other way — a RUN curl, or a second ARG pin — is still outside the gate. That is a strict improvement over nothing, not a complete fence.
All fourteen checks are green and the head is based on current main, so it is ready for whoever clears the hold.
Generated by Claude Code
…ctually build (#182) * Bump the NVIDIA driver to the release that compiles against kernel 7.2 The base image pin has been frozen since Hummingbird moved to kernel 7.2. Every attempt to advance it -- #122 is the current one -- fails kernel_cache 22 minutes in, building the NVIDIA open module: nvidia/os-interface.c: In function 'os_get_current_process_name': nvidia/os-interface.c:764:5: error: implicit declaration of function 'strncpy' [-Wimplicit-function-declaration] 764 | strncpy(buf, current->comm, len - 1); make[4]: *** [.../7.2.5-200.fc44.x86_64/...: nvidia/os-interface.o] Error 1 A frozen base pin means no kernel security updates reach the flavored images, so this is worth fixing properly rather than pinning around. The cause is the kernel finishing its strscpy migration and removing strncpy, and the fix is upstream's, not ours. Read from the vendor source at each tag: 595.84 strncpy(buf, current->comm, len - 1); 595.91.07 strscpy 595.99.02 if (strscpy(buf, current->comm, len) < 0) { So 595.99.02 carries the migration, and it is also what https://download.nvidia.com/XFree86/Linux-x86_64/latest.txt designates today -- which is the rule the pin comment already states. This is that policy working, not a hand-picked version: read latest.txt, take the SHA-256 NVIDIA publishes beside the installer. The base bump rides along deliberately. On the current base the old driver compiles fine, so a driver-only change proves nothing -- the two can only be verified together, and kernel_cache on this PR is that verification. Both ARG BASE_IMAGE= lines move together, as `just check` requires. The cache tag rotates 1ce05c4b16fafe9c -> f1e31a1dee27ffaa, so the three cached flavors compile the kernel once. That is the intended cost: the tag moving is what proves the new base and driver were actually built, rather than a stale cache being reused (#166, #171). Closes #173. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Make the shim mirroring say what it could not find, and pair the kernel-devel hash with its kernel The base bump to a kernel-7.2 Hummingbird regressed build_main. It failed deterministically, three attempts out of three, and printed nothing: the last line of the log was "Utah desktop contract passed", then exit 1. The step was four predicates chained with && at the tail of a long RUN -- shim_version="$(rpm -q --qf '%{VERSION}-%{RELEASE}' shim-x64)" && \ test -d /usr/lib/bootupd/updates/EFI/fedora && \ install -d ... && cp -a ... -- so nothing distinguished "shim-x64 is not in this base" from "shim-x64 is installed but staged nothing", and those have different answers. The chain now lives in scripts/mirror-shim.sh, which names the failing predicate, lists what the bootupd tree actually contains when the payload is absent, and says that a base which dropped shim-x64 needs it added to the package contract rather than the step skipped. This does not yet fix the build; it makes the next run say which of the two happened, which the previous three runs could not. The second change is the review finding on the same bump. KERNEL_DEVEL_SHA256 guards an unsigned koji download and is documented as a constant precisely because "the kernel cannot change without BASE_IMAGE changing" -- which is the reason it must move when BASE_IMAGE does, and it did not. It stayed pinned to 7.1.8-100.fc43 while the base moved to 7.2.5-200.fc44. Nothing failed, because the dnf path satisfied 7.2 and the fallback never ran; it would have failed the first time the enabled repositories dropped the kernel, which is the single situation the fallback exists for. The hash is now the real one for kernel-devel-7.2.5-200.fc44.x86_64, taken from koji, and it is recorded beside KERNEL_DEVEL_NEVR. A mismatch between the recorded kernel and the booted one now fails immediately with both versions named, instead of after fetching 60 MB to compare against a hash that cannot match. Tests drive the real code: mirror-shim.sh runs against a stub rpm and a fake bootupd tree for all three outcomes, asserting the two failures do not share a message; the NEVR guard is extracted and executed for both the matching and mismatched kernel. 204 tests, OK. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Handle the base image that already mirrors the shim payload itself I did not need to wait for CI for this. Listing both base images layer by layer from the registry answers it outright, and the answer is that the new base has already done the work: old base sha256:c5539f9e... usr/lib/bootupd/updates/EFI/fedora/{shimx64,shim,mmx64,grubx64}.efi no usr/lib/efi/shim at all new base sha256:db1007fd... usr/lib/efi/shim/16.1-5/EFI/fedora/{shimx64,shim,mmx64}.efi, BOOTX64.CSV usr/lib/bootupd/updates/ holds only BIOS.json and EFI.json So the destination Utah was copying into is exactly what the new base now ships, and the source it was copying from no longer exists. The step was not broken by a missing package or a broken transaction; it was asserting a layout the base has moved on from, and failing on a job that was already done. mirror-shim.sh now looks for the payload before doing anything else and exits 0 when it is already in place. It searches for the EFI/fedora/shimx64.efi file rather than assuming a directory name, because the base names that directory for the shim version with no dist tag and rpm need not report the same string. Checking the filesystem first also means the base that ships the payload is the base whose shim-x64 rpm query we no longer depend on. The old layout still works: the copy path is unchanged and tested. The two real failures -- no package and no payload anywhere -- still fail, still say which, and still do not share a message. Tests drive the real script against synthetic filesystems for both layouts, including the case where rpm reports 16.1-5.fc44 while the base directory is 16.1-5. 206 tests, OK. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M * Make the kernel NEVR overridable, so the hash override is not dead code Review finding, and correct. I added a guard that exits when the booted kernel differs from KERNEL_DEVEL_NEVR, to turn a 60 MB download ending in a checksum mismatch into an immediate failure naming both versions. But UTAH_KERNEL_DEVEL_SHA256 exists for exactly one situation -- validating a kernel this file has not been re-recorded for -- and the guard runs before the hash check, so it blocked precisely the case the override is for. Correct override or not, that path exited 1. I turned an escape hatch into dead code while adding a guard meant to make failures clearer. KERNEL_DEVEL_NEVR now takes UTAH_KERNEL_DEVEL_NEVR the same way the hash takes its override, and the guard message names both variables, so the way past it is to set the pair deliberately rather than to edit the script. Overriding one without the other still fails, which is the intended behaviour: the hash belongs to a specific NEVR and the two only mean anything together. Three tests: both constants accept an override, the guard message names them, and the real guard extracted from the script reaches the download when both are set to a kernel that is not the recorded one. The first fails against the previous constant line. 209 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Refactor
Makes the kernel cache key a derived consequence of what
Containerfile.kernelis built from, instead of a second hand-maintained list.Structural problem.
scripts/kernel-cache-tag.shhashes a list it namesitself: the
ARG BASE_IMAGE=line ofContainerfile.kernel,scripts/install-ogc-kernel.sh,scripts/install-nvidia.sh,packages/hummingbird.repoandpackages/fedora-44.repo.Containerfile.kerneldecides independently what the cache image actuallycontains — its
ARG BASE_IMAGE=pin plus every non---fromCOPYsource. Thetwo lists agree today; nothing made them agree tomorrow.
Add a
COPYtoContainerfile.kernelwithout a matching hash line and the keystops moving when that input moves.
build.ymlbuilds the cache image only whenits tag is not already published, so an unchanged key means no rebuild, and
gaming,nvidiaandnvidia-gamingkeep unpacking a kernel and an NVIDIAmodule built from the superseded input. Nothing fails; the wrong bits ship.
What this adds.
tests/test_kernel_cache_key.pyderives the input set fromContainerfile.kernelrather than restating it, then asserts the contractbehaviourally against a writable copy of the tree:
scripts/kernel-cache-tag.sh's output;ARG BASE_IMAGE=pin, when repinned, changes it;scripts/flavors.py) does not change it — the "and nototherwise" half of the invariant the script's header states, so the key can't be
"fixed" by hashing the tree and rebuilding the ~45-minute cache on every edit;
^[0-9a-f]{16}$;vacuously against an incomplete tree; and the
COPYderivation is assertednon-empty, with each derived path required to exist.
No production code, no workflow, no behavior change: one new test file plus the
skill-file update
AGENTS.mdrequires.Verification
The gate is non-vacuous — appending
COPY packages/utah.toml /tmp/probe.tomlto
Containerfile.kernel(not committed) fails it with the message it existsto produce:
Scope and overlap
Claims exactly
tests/test_kernel_cache_key.py(new) and the "What the taghashes" section of
docs/skills/kernel-cache.md. Checked against every openPR on this repo: #122 and #133 edit
Containerfile.kernel; #154, #158 and #160edit
scripts/install-ogc-kernel.sh/scripts/install-nvidia.shanddocs/skills/kernel-cache.md. Those change the content of hashed inputs —which is the case this gate protects — and none of them adds a gate over the
cache-key input set. No overlap with
AGENTS.md(#142).Closes #165
Filed by architect agent (ACMM L5 — hold-gated mode). Hold-gated: human review required.
— hive: agent=architect backend=copilot model=claude-opus-5 copilot=1.0.78