Repository navigation
fix(nvidia): bump the driver for kernel 7.2, and make the base bump actually build - #182
Conversation
The base image pin has been frozen since Hummingbird moved to kernel 7.2. Every attempt to advance it -- #122 is the current one -- fails kernel_cache 22 minutes in, building the NVIDIA open module: nvidia/os-interface.c: In function 'os_get_current_process_name': nvidia/os-interface.c:764:5: error: implicit declaration of function 'strncpy' [-Wimplicit-function-declaration] 764 | strncpy(buf, current->comm, len - 1); make[4]: *** [.../7.2.5-200.fc44.x86_64/...: nvidia/os-interface.o] Error 1 A frozen base pin means no kernel security updates reach the flavored images, so this is worth fixing properly rather than pinning around. The cause is the kernel finishing its strscpy migration and removing strncpy, and the fix is upstream's, not ours. Read from the vendor source at each tag: 595.84 strncpy(buf, current->comm, len - 1); 595.91.07 strscpy 595.99.02 if (strscpy(buf, current->comm, len) < 0) { So 595.99.02 carries the migration, and it is also what https://download.nvidia.com/XFree86/Linux-x86_64/latest.txt designates today -- which is the rule the pin comment already states. This is that policy working, not a hand-picked version: read latest.txt, take the SHA-256 NVIDIA publishes beside the installer. The base bump rides along deliberately. On the current base the old driver compiles fine, so a driver-only change proves nothing -- the two can only be verified together, and kernel_cache on this PR is that verification. Both ARG BASE_IMAGE= lines move together, as `just check` requires. The cache tag rotates 1ce05c4b16fafe9c -> f1e31a1dee27ffaa, so the three cached flavors compile the kernel once. That is the intended cost: the tag moving is what proves the new base and driver were actually built, rather than a stale cache being reused (#166, #171). Closes #173. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
|
Verified: Three things worth drawing out, because a green tick alone does not distinguish a fix from a cache hit:
So the driver-and-kernel combination is proven, and the base pin is no longer frozen: 595.99.02's Remaining on this PR: the four image builds. I will report when they finish. #122 can close as superseded, and #173 is fixed by this. Generated by Claude Code |
There was a problem hiding this comment.
Correctness review (head 3a2cf0a)
1. High — the base bump breaks build_main; the PR's own verification criterion doesn't cover this failure.
build_main / image (main, utah, testing, x86_64) fails deterministically (3/3 retry attempts) on this head at the shim-mirroring tail of Containerfile:182-185:
shim_version="$(rpm -q --qf '%{VERSION}-%{RELEASE}' shim-x64)" &&
test -d /usr/lib/bootupd/updates/EFI/fedora && ...
Everything up through utah-verify-desktop-contract succeeds ("Utah desktop contract passed: 38 Flatpaks…"), then the step exits 1 with no diagnostic — consistent with either shim-x64 no longer being present or /usr/lib/bootupd/updates/EFI/fedora no longer existing in the new base (sha256:db1007fd…). The PR body names kernel_cache going green as "the real proof" — and it did pass — but the base bump regresses the main image build, which the body doesn't anticipate. The silent rpm -q/test -d chain also makes it impossible to tell which predicate failed from the log; whoever fixes this will want the step to say which check tripped.
2. Medium — KERNEL_DEVEL_SHA256 violates its own stated invariant after this bump.
scripts/install-nvidia.sh:117-118 pins the koji-fallback kernel-devel hash for 7.1.8-100.fc43 with the justification "the kernel cannot change without BASE_IMAGE changing." This PR changes BASE_IMAGE to a kernel-7.2 base (Containerfile:1, Containerfile.kernel:19) without touching that hash. kernel_cache passed, so the dnf install kernel-devel-${kernel} path evidently satisfied 7.2.x and the fallback never ran — but the moment the enabled repos drop that kernel (the exact scenario the comment at install-nvidia.sh:105-115 documents for 7.1.8), the fallback hard-fails against a hash for the wrong kernel. The same discipline the PR applies to NVIDIA_RUN_SHA256 (install-nvidia.sh:72) should apply here: move KERNEL_DEVEL_SHA256 with the base, or update its comment to say the constant is now stale for this base.
Not verified from here: the NVIDIA-published SHA-256 for 595.99.02 (install-nvidia.sh:72) and the latest.txt designation — taken on the CI sha-check's word, which kernel_cache exercised and passed.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
…el-devel hash with its kernel
The base bump to a kernel-7.2 Hummingbird regressed build_main. It failed
deterministically, three attempts out of three, and printed nothing: the last
line of the log was "Utah desktop contract passed", then exit 1. The step was
four predicates chained with && at the tail of a long RUN --
shim_version="$(rpm -q --qf '%{VERSION}-%{RELEASE}' shim-x64)" && \
test -d /usr/lib/bootupd/updates/EFI/fedora && \
install -d ... && cp -a ...
-- so nothing distinguished "shim-x64 is not in this base" from "shim-x64 is
installed but staged nothing", and those have different answers. The chain now
lives in scripts/mirror-shim.sh, which names the failing predicate, lists what
the bootupd tree actually contains when the payload is absent, and says that a
base which dropped shim-x64 needs it added to the package contract rather than
the step skipped. This does not yet fix the build; it makes the next run say
which of the two happened, which the previous three runs could not.
The second change is the review finding on the same bump. KERNEL_DEVEL_SHA256
guards an unsigned koji download and is documented as a constant precisely
because "the kernel cannot change without BASE_IMAGE changing" -- which is the
reason it must move when BASE_IMAGE does, and it did not. It stayed pinned to
7.1.8-100.fc43 while the base moved to 7.2.5-200.fc44. Nothing failed, because
the dnf path satisfied 7.2 and the fallback never ran; it would have failed the
first time the enabled repositories dropped the kernel, which is the single
situation the fallback exists for.
The hash is now the real one for kernel-devel-7.2.5-200.fc44.x86_64, taken from
koji, and it is recorded beside KERNEL_DEVEL_NEVR. A mismatch between the
recorded kernel and the booted one now fails immediately with both versions
named, instead of after fetching 60 MB to compare against a hash that cannot
match.
Tests drive the real code: mirror-shim.sh runs against a stub rpm and a fake
bootupd tree for all three outcomes, asserting the two failures do not share a
message; the NEVR guard is extracted and executed for both the matching and
mismatched kernel. 204 tests, OK.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
|
Both findings are correct, and the first one corrects me. I reported I also confirmed the regression is mine and not pre-existing: Finding 1 — the silent shim chain. Exactly as you described: three attempts out of three, last log line
This does not fix the build. It makes the next run say which of the two happened, which the previous three could not. I will act on whichever it reports. Finding 2 — Rather than update the comment to admit staleness, the hash is now the real one for I checked whether 204 tests, OK. Generated by Claude Code |
I did not need to wait for CI for this. Listing both base images layer by layer
from the registry answers it outright, and the answer is that the new base has
already done the work:
old base sha256:c5539f9e...
usr/lib/bootupd/updates/EFI/fedora/{shimx64,shim,mmx64,grubx64}.efi
no usr/lib/efi/shim at all
new base sha256:db1007fd...
usr/lib/efi/shim/16.1-5/EFI/fedora/{shimx64,shim,mmx64}.efi, BOOTX64.CSV
usr/lib/bootupd/updates/ holds only BIOS.json and EFI.json
So the destination Utah was copying into is exactly what the new base now
ships, and the source it was copying from no longer exists. The step was not
broken by a missing package or a broken transaction; it was asserting a layout
the base has moved on from, and failing on a job that was already done.
mirror-shim.sh now looks for the payload before doing anything else and exits 0
when it is already in place. It searches for the EFI/fedora/shimx64.efi file
rather than assuming a directory name, because the base names that directory
for the shim version with no dist tag and rpm need not report the same string.
Checking the filesystem first also means the base that ships the payload is the
base whose shim-x64 rpm query we no longer depend on.
The old layout still works: the copy path is unchanged and tested. The two real
failures -- no package and no payload anywhere -- still fail, still say which,
and still do not share a message.
Tests drive the real script against synthetic filesystems for both layouts,
including the case where rpm reports 16.1-5.fc44 while the base directory is
16.1-5. 206 tests, OK.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
|
Root cause found, and it is not either of the two possibilities I set the diagnostic up to distinguish between. I did not need to wait for the run: listing both base images layer by layer straight from the registry answers it outright. The new base already does the mirroring itself. That is the answer to your "either Fixed in 7b5204d.
The old layout still works — the copy path is unchanged and tested — so this is not a swap from one hardcoded assumption to another. Both genuine failures still fail, still name which one, and still do not share a message. Tests cover both real layouts against synthetic filesystems, including the case where rpm reports Your Generated by Claude Code |
|
Run 35445275258 on That is the verification this PR needed, and it is worth being specific about which claim each job settles, because I reported this PR as verified once before on much weaker evidence and was wrong:
So all four images build on Not settled by this run, and I would rather say so than let a green wall imply otherwise: nothing here boots an image or checks that the NVIDIA module actually loads at runtime. This needs an approving review from someone who is not me; I will not self-approve. #122 should be closed in favour of this once it lands. Generated by Claude Code |
There was a problem hiding this comment.
Correctness review (head 7b5204d)
1. The NEVR guard makes the UTAH_KERNEL_DEVEL_SHA256 override dead code — scripts/install-nvidia.sh:157-163. The guard [ "${kernel}" != "${KERNEL_DEVEL_NEVR}" ] && exit 1 runs before the hash check, and KERNEL_DEVEL_NEVR (line 127) is a plain constant with no env override. But the only situation in which overriding UTAH_KERNEL_DEVEL_SHA256 (line 128) is useful is precisely when the kernel differs from the recorded NEVR — and now that path exits unconditionally, correct override or not. Either make the NEVR overridable alongside the hash, or skip the guard when UTAH_KERNEL_DEVEL_SHA256 is explicitly set. (severity: medium, agent-fixable)
2. Body/diff gap: the shim-mirroring refactor is undisclosed. The body says "the only lines that change are the version and its digest, plus a comment," but the diff also replaces the inline shim mirroring with a new 69-line scripts/mirror-shim.sh plus 138 lines of tests. To be clear, the change is necessary — on the new base, /usr/lib/bootupd/updates/EFI/fedora no longer exists (per scripts/mirror-shim.sh:11-15), so the old inline test -d chain (removed at Containerfile:50-53 old lines) would have failed the build. The script handles both layouts correctly (find depth 4 matches <ver>/EFI/fedora/shimx64.efi) and the tests drive the real script. But the PR description should say this rider exists, since it changes runtime image layout behavior. (severity: low)
Not verified (outside what I can check): the new base digest db1007fd…, the NVIDIA 595.99.02 SHA-256, and the koji kernel-devel hash — those are only provable by the kernel_cache job this PR is designed to exercise, which the author correctly identifies as the real verification.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
Review finding, and correct. I added a guard that exits when the booted kernel differs from KERNEL_DEVEL_NEVR, to turn a 60 MB download ending in a checksum mismatch into an immediate failure naming both versions. But UTAH_KERNEL_DEVEL_SHA256 exists for exactly one situation -- validating a kernel this file has not been re-recorded for -- and the guard runs before the hash check, so it blocked precisely the case the override is for. Correct override or not, that path exited 1. I turned an escape hatch into dead code while adding a guard meant to make failures clearer. KERNEL_DEVEL_NEVR now takes UTAH_KERNEL_DEVEL_NEVR the same way the hash takes its override, and the guard message names both variables, so the way past it is to set the pair deliberately rather than to edit the script. Overriding one without the other still fails, which is the intended behaviour: the hash belongs to a specific NEVR and the two only mean anything together. Three tests: both constants accept an override, the guard message names them, and the real guard extracted from the script reaches the download when both are set to a kernel that is not the recorded one. The first fails against the previous constant line. 209 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
|
Both findings correct, both fixed. 1. The NEVR guard made the hash override dead code. You're right, and the shape of the mistake is worth naming: I added a guard to make a failure clearer and in doing so removed the only way to proceed deliberately past it. Fixed in da7bf08 by making the NEVR overridable alongside the hash rather than by skipping the guard: KERNEL_DEVEL_NEVR="${UTAH_KERNEL_DEVEL_NEVR:-7.2.5-200.fc44.x86_64}"
KERNEL_DEVEL_SHA256="${UTAH_KERNEL_DEVEL_SHA256:-02467ce3…}"I preferred this to your alternative of bypassing the guard when the hash is set, because the hash belongs to a specific NEVR — the two only mean anything as a pair, and overriding one without the other should still fail. The guard message now names both variables, so the way past it is to set the pair deliberately rather than to edit the script. Three tests: both constants accept an override, the guard message names them, and the real guard extracted from the script reaches the download when both are set to a kernel that is not the recorded one. The first fails against the previous constant line — verified, not assumed. 2. The body understated the diff. Also right. "The only lines that change are the version and its digest, plus a comment" was written before the shim work existed and I never revised it. The description is rewritten: the shim refactor now has its own numbered section saying it is a rider, why it is necessary (both base layouts listed from the registry), and what it changes about image layout behaviour. The verification section also now separates what the green run settles from what it does not. Thank you for the second one in particular. A body that undersells a 200-line rider is how a reviewer ends up approving something they did not read, and I would rather the merge record be accurate than flattering. On your "not verified" list — the base digest, the NVIDIA SHA-256 and the koji hash — 209 tests pass on Generated by Claude Code |
Danathar
left a comment
There was a problem hiding this comment.
What I checked, because this PR is three pins and a base bump and each has to be right independently:
- Driver: downloaded
NVIDIA-Linux-x86_64-595.99.02.runfrom us.download.nvidia.com and hashed it:e8747795…f6e1ec, matching the newNVIDIA_RUN_SHA256. 595.99.02 is the release that follows the kernel'sstrncpyremoval, which is what killed 595.84 on 7.2 (#173). - kernel-devel: downloaded
kernel-devel-7.2.5-200.fc44.x86_64.rpmfrom koji and hashed it:02467ce3…6a0703, matchingKERNEL_DEVEL_SHA256. The newKERNEL_DEVEL_NEVRguard that fails before the 60 MB download when the booted kernel doesn't match the recorded one is a good addition — that's the failure #122 hit blind. - Base:
sha256:db1007fd…is the digest Renovate proposed in #122, so this supersedes that PR rather than diverging from it. - mirror-shim: moving the bare
&&chain intoscripts/mirror-shim.shwith its own test is right; a silent failure there is exactly the kind of thing the old form produced.
All four flavored images built green on the new base, including both NVIDIA ones through kernel_cache, which is the proof #173's acceptance criteria asked for. Closes #173 and supersedes #122.
There was a problem hiding this comment.
correctness
scripts/mirror-shim.sh:47—shim_version="$(rpm -q --qf '%{VERSION}-%{RELEASE}' shim-x64 2>&1)"merges stderr into the captured version. If rpm succeeds but emits a warning (rpmdb rebuild notices do exactly this), the warning text lands inshim_versionand the mirror target becomes/usr/lib/efi/shim/<garbage>/EFI/fedora. Capture stdout only and test the exit code, or redirect stderr separately. Low severity — the early payload check means the new base never reaches this line — but it's the exact silent-corruption class this script was written to eliminate.
style
scripts/install-nvidia.sh:180— the hash-mismatch remedy still says "update KERNEL_DEVEL_SHA256" alone, while the new guard 15 lines above (and its comment) insist the constants move as a pair ("Override both, or neither"). The one path that can now reach this message is a pair that was re-recorded inconsistently, so it should name both constants.
No findings from: security, intent-alignment, docs-currency. (Docs checked at head: no stale 595.84/7.1.8/shim-layout references remain; run_suite.py auto-discovers the two new test files. Noting the CI run cited as verification (35445275258) ran on 7b5204d, not head — the body already discloses this and the guard change since is unit-tested.)
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
castrojo
left a comment
There was a problem hiding this comment.
Reviewed diff and CI results. Bumps BASE_IMAGE pin across Containerfile and Containerfile.kernel consistently to db1007fdcda076f2d7fd0e2adfe998141dd1908b8f66c732654f762c6a8b2728, adapts NVIDIA driver to kernel 7.2 with clean shim mirroring in scripts/mirror-shim.sh, updates documentation. All required checks pass.
#122 bumps the Hummingbird base from
c5539f9todb1007f, which moves the kernel from 7.1.8-100.fc43 to 7.2.5-200.fc44. That bump cannot land on its own: three separate things in this repository assume the old kernel, and each was found by a different failure. This PR carries the same digest bump plus all three fixes.1. NVIDIA 595.84 does not compile against kernel 7.2
Kernel 7.2 completed the
strscpymigration and removedstrncpy, so the module build dies inos-interface.c:Upstream followed in 595.99.02.
scripts/install-nvidia.shmoves to that release with its published SHA-256, and the comment records why rather than leaving a bare version bump for the next reader to archaeologise.2.
KERNEL_DEVEL_SHA256was pinned to the old kernelThat constant guards an unsigned koji download and is documented as a constant because "the kernel cannot change without BASE_IMAGE changing" — which is precisely why it has to move when BASE_IMAGE does. Nothing would have failed immediately: the dnf path satisfies 7.2 and the koji fallback never runs. It would have failed the first time the enabled repositories dropped the kernel, which is the single situation that fallback exists for.
It is now the real hash for
kernel-devel-7.2.5-200.fc44.x86_64, taken from koji, and paired with a newKERNEL_DEVEL_NEVRso a mismatch fails immediately with both versions named instead of after fetching 60 MB. Both constants accept an environment override, together — review caught that a NEVR guard which refused to proceed on any mismatch would makeUTAH_KERNEL_DEVEL_SHA256unreachable in the only case it is for, turning an escape hatch into dead code. Fixed in da7bf08.3. The shim mirroring assumed a layout the base has moved on from
This is a real rider on the diff, not a one-line change, so it deserves its own section: a new
scripts/mirror-shim.shplus tests, replacing an inline&&chain in the Containerfile.I listed both base images' layers directly from the registry:
The new base already does the mirroring itself.
/usr/lib/efi/shim/…is exactly the destination Utah's step copies into, and the source it copies from no longer exists. Nothing is missing — notshim-x64, not bootupd. The step was asserting the old shape and failing on a job already done.Because it was four predicates chained with
&&at the tail of a longRUN, it failed silently: the last line of the log wasUtah desktop contract passed, then exit 1, three attempts out of three, with nothing to say which predicate tripped.mirror-shim.shnow looks for the payload before anything else and exits 0 when it is already in place, finding theEFI/fedora/shimx64.efifile rather than assuming a directory name (the base uses16.1-5; rpm may report16.1-5.fc44). The old layout's copy path is unchanged and still tested, and the two genuine failures — no package, and no payload anywhere — still fail, still say which, and still do not share a message.Verification
Run 35445275258 on
7b5204d: completed success, all jobs green.build_maingreen settles the shim regression; the two nvidia flavors settle the driver bump. All four images build on kernel 7.2.5-200.fc44, which is what #122 could not do alone.Not settled by this run, stated plainly because I reported this PR as verified once before on weaker evidence and was wrong: nothing here boots an image or checks the NVIDIA module loads at runtime —
dispatch-isois gated torefs/heads/testing, so that is only observable after merge. The koji fallback is also unexercised, so the kernel-devel pin rests on a unit test against the base's actual kernel rather than on a download.209 tests pass.
#122 should be closed in favour of this once it lands.
🤖 Generated with Claude Code
https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M