Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 9 additions & 12 deletions Containerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG BASE_IMAGE=quay.io/hummingbird-community/bootc-os:latest@sha256:c5539f9ed4d93aab6bd41e4f5aef8ab83055f3f9e855a47b69fadb7420d0d1df
ARG BASE_IMAGE=quay.io/hummingbird-community/bootc-os:latest@sha256:db1007fdcda076f2d7fd0e2adfe998141dd1908b8f66c732654f762c6a8b2728
# The package factory publishes a complete, digest-addressable RPM repository.
# Keep this pin in Utah so an image build is reproducible and can be reviewed
# against the exact package set it consumes.
Expand Down Expand Up @@ -63,6 +63,7 @@ COPY scripts/install-packages.py \
scripts/configure-branding.sh \
scripts/verify-desktop-contract.py \
scripts/verify-gnome-extensions.py \
scripts/mirror-shim.sh \
/tmp/utah-scripts/
# Common publishes Bluefin artwork, desktop defaults, Brewfiles, and setup
# hooks in a separate profile from its shared system files. Both are required:
Expand All @@ -83,7 +84,8 @@ RUN for pair in install-packages.py:utah-install-packages \
configure-services.sh:utah-configure-services \
configure-branding.sh:utah-configure-branding \
verify-desktop-contract.py:utah-verify-desktop-contract \
verify-gnome-extensions.py:utah-verify-gnome-extensions; do \
verify-gnome-extensions.py:utah-verify-gnome-extensions \
mirror-shim.sh:utah-mirror-shim; do \
install -Dm 0755 "/tmp/utah-scripts/${pair%%:*}" "/usr/local/libexec/${pair##*:}" || exit 1; \
done && \
cp -a /tmp/utah-common/. / && \
Expand Down Expand Up @@ -156,12 +158,10 @@ ARG UUPD_TIMER_SHA256=bbb5f098ec33d047bdef571e0bc112364df157e0f92d73e0febab703c4
# it applies the desktop service policy, login defaults, update policy, and
# removes the extension build toolchain before the final cleanup.
#
# The shim mirroring at the end belongs to the same step. Fedora's shim package
# stages its EFI payload under bootupd's update tree, while bootupd discovers
# image-provided EFI components under /usr/lib/efi. Mirroring the signed
# payload into bootupd's component layout lets bootc create a generic disk
# image without depending on the build host's ESP. It was a layer of its own
# and cost forty seconds to commit a few megabytes.
# The shim mirroring at the end belongs to the same step: it was a layer of its
# own and cost forty seconds to commit a few megabytes. It lives in
# scripts/mirror-shim.sh rather than inline, because as a bare && chain a
# failure printed nothing at all -- see the comment at the top of that script.
RUN mkdir -p /tmp/uupd && \
curl -fsSL "https://github.com/ublue-os/uupd/releases/download/${UUPD_VERSION}/uupd_Linux_x86_64.tar.gz" \
-o /tmp/uupd/uupd_Linux_x86_64.tar.gz && \
Expand All @@ -179,10 +179,7 @@ RUN mkdir -p /tmp/uupd && \
ENABLE_SSHD="${ENABLE_SSHD}" /usr/local/libexec/utah-configure-services && \
/usr/local/libexec/utah-configure-branding && \
/usr/local/libexec/utah-verify-desktop-contract /usr/share/utah/bluefin-desktop.toml && \
shim_version="$(rpm -q --qf '%{VERSION}-%{RELEASE}' shim-x64)" && \
test -d /usr/lib/bootupd/updates/EFI/fedora && \
install -d "/usr/lib/efi/shim/${shim_version}/EFI/fedora" && \
cp -a /usr/lib/bootupd/updates/EFI/fedora/. "/usr/lib/efi/shim/${shim_version}/EFI/fedora/"
/usr/local/libexec/utah-mirror-shim

# Dakota-compatible flavors: OGC is built and asserted before NVIDIA so the
# NVIDIA path can bind its module to the exact kernel tree it will boot.
Expand Down
2 changes: 1 addition & 1 deletion Containerfile.kernel
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#
# The cache image's own build has no /utah-cache, so the same scripts take their
# source-build path here. There is no second implementation to drift.
ARG BASE_IMAGE=quay.io/hummingbird-community/bootc-os:latest@sha256:c5539f9ed4d93aab6bd41e4f5aef8ab83055f3f9e855a47b69fadb7420d0d1df
ARG BASE_IMAGE=quay.io/hummingbird-community/bootc-os:latest@sha256:db1007fdcda076f2d7fd0e2adfe998141dd1908b8f66c732654f762c6a8b2728

FROM ${BASE_IMAGE} AS builder

Expand Down
1 change: 1 addition & 0 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ check:
test -f scripts/configure-branding.sh
test -f scripts/verify-desktop-contract.py
test -f scripts/verify-gnome-extensions.py
test -f scripts/mirror-shim.sh
test -f contracts/bluefin-desktop.toml
# The reusable image workflow checks out this repository without
# submodules. Populate them here before validating the source contract;
Expand Down
50 changes: 42 additions & 8 deletions scripts/install-nvidia.sh
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,16 @@ modules_only="${UTAH_NVIDIA_MODULES_ONLY:-}"

# NVIDIA's own designation of the current driver, not a hand-picked directory
# listing: https://download.nvidia.com/XFree86/Linux-x86_64/latest.txt
driver_version="${UTAH_NVIDIA_DRIVER_VERSION:-595.84}"
#
# 595.84 stopped compiling when Hummingbird's base moved to kernel 7.2:
#
# os-interface.c:764: error: implicit declaration of function 'strncpy'
#
# The kernel completed its strscpy migration and removed strncpy, and upstream
# followed in 595.99.02, which replaces that call with strscpy and is what
# latest.txt designates today. So this is the pin policy above doing its job
# rather than a workaround: read latest.txt, take the digest beside it (#173).
driver_version="${UTAH_NVIDIA_DRIVER_VERSION:-595.99.02}"
run="NVIDIA-Linux-x86_64-${driver_version}.run"
url="https://download.nvidia.com/XFree86/Linux-x86_64/${driver_version}/${run}"
# The vendor `.run` is executed as root during composition, so it is verified
Expand All @@ -60,7 +69,7 @@ url="https://download.nvidia.com/XFree86/Linux-x86_64/${driver_version}/${run}"
# beyond the TLS channel it was meant to backstop. The constant is NVIDIA's
# published SHA-256 for the pinned driver version; the two move together, as
# does UTAH_NVIDIA_RUN_SHA256 when UTAH_NVIDIA_DRIVER_VERSION is overridden.
NVIDIA_RUN_SHA256="${UTAH_NVIDIA_RUN_SHA256:-99c404e52131bf27bca6da0268dcce3ebedc7a73ab174071a995cf1a6c5eba06}"
NVIDIA_RUN_SHA256="${UTAH_NVIDIA_RUN_SHA256:-e87477958bf763070549324bd5ad6c948eba6ed210e44005b3eff84940f6e1ec}"

ogc_release=""
[ -f /usr/lib/utah/ogc-kernel-release ] && ogc_release="$(cat /usr/lib/utah/ogc-kernel-release)"
Expand Down Expand Up @@ -92,21 +101,37 @@ fi
# attempt at a source build. It is needed only to compile against, so it goes in
# here and comes out again below.
#
# It is not in the repositories this image enables either. The base kernel is a
# Fedora 43 build, 7.1.8-100.fc43, and Utah enables only Hummingbird plus its
# own package factory:
# It may not be in the repositories this image enables either. Utah enables only
# Hummingbird plus its own package factory, and when the base kernel is not in
# those:
#
# No match for argument: kernel-devel-7.1.8-100.fc43.x86_64
# No match for argument: kernel-devel-<nevr>
#
# Adding the Fedora 43 repository would not fix it for long, because a
# Adding the matching Fedora repository would not fix it for long, because a
# repository only carries the current kernel and this image is pinned to a base
# whose kernel will not move. Fedora own build system keeps every build
# indefinitely, so that is where this comes from, addressed by exact NEVR.
#
# Those RPMs are unsigned at that path, so the download is checked against a
# hash recorded here instead. It is a constant because the base image is pinned
# by digest: the kernel cannot change without BASE_IMAGE changing.
KERNEL_DEVEL_SHA256="${UTAH_KERNEL_DEVEL_SHA256:-b2b504c42b94875af88d666d64ca91000ff30439e74157723a188f54ceebc5ca}"
#
# Which is exactly why the hash has to move with BASE_IMAGE, and why it is
# recorded here beside the kernel it was taken from rather than on its own. The
# 7.2 base bump left this pinned to 7.1.8-100.fc43 for one revision: the dnf
# path above satisfied 7.2 and the fallback never ran, so nothing failed and
# nothing said the constant had gone stale. It would have failed the first time
# the enabled repositories dropped the kernel -- the precise situation this
# fallback exists for. Pairing the two makes that mismatch loud and immediate
# instead of latent, and costs nothing when they agree.
#
# Both are overridable together, and that pairing is the point. The override on
# the hash exists for exactly one situation -- validating a kernel this file has
# not been re-recorded for -- so a guard that refused to proceed whenever the
# kernel differed from the recorded NEVR would make the override unreachable in
# the only case it is for. Override both, or neither.
KERNEL_DEVEL_NEVR="${UTAH_KERNEL_DEVEL_NEVR:-7.2.5-200.fc44.x86_64}"
KERNEL_DEVEL_SHA256="${UTAH_KERNEL_DEVEL_SHA256:-02467ce35055d553db0babd680aa429c2d0b8d514469768730e03b89326a0703}"

build_tree="/usr/lib/modules/${kernel}/build"
installed_kernel_devel=""
Expand Down Expand Up @@ -135,6 +160,15 @@ ensure_toolchain() {
installed_kernel_devel="kernel-devel-${kernel}"
else
local arch nv ver rel koji rpmfile actual
if [ "${kernel}" != "${KERNEL_DEVEL_NEVR}" ]; then
echo "KERNEL_DEVEL_SHA256 was recorded for kernel ${KERNEL_DEVEL_NEVR}," >&2
echo "but this image boots ${kernel}. The hash cannot match, so the" >&2
echo "download below would fail after fetching 60 MB. Re-record both" >&2
echo "constants for ${kernel} when bumping BASE_IMAGE, or set both" >&2
echo "UTAH_KERNEL_DEVEL_NEVR and UTAH_KERNEL_DEVEL_SHA256 to validate" >&2
echo "a kernel this file has not been re-recorded for." >&2
exit 1
fi
arch="${kernel##*.}"; nv="${kernel%.*}"; ver="${nv%%-*}"; rel="${nv#*-}"
koji="https://kojipkgs.fedoraproject.org/packages/kernel/${ver}/${rel}/${arch}"
rpmfile="kernel-devel-${ver}-${rel}.${arch}.rpm"
Expand Down
69 changes: 69 additions & 0 deletions scripts/mirror-shim.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
#!/usr/bin/env bash
# Make the signed shim payload available in bootupd's component layout, so
# bootc can create a generic disk image without depending on the build host ESP.
#
# Two base image layouts exist and both are correct.
#
# Old (sha256:c5539f9e...): the payload is staged under bootupd update tree,
# at /usr/lib/bootupd/updates/EFI/fedora/, and /usr/lib/efi/shim does not
# exist. Utah has to do the copy.
#
# New (sha256:db1007fd...): the base already ships
# /usr/lib/efi/shim/16.1-5/EFI/fedora/{shimx64,shim,mmx64}.efi and
# BOOTX64.CSV, and /usr/lib/bootupd/updates/ holds only BIOS.json and
# EFI.json. The mirroring has already been done upstream.
#
# This used to be four predicates chained with && at the tail of a long
# Containerfile RUN, hardcoding the old layout. On the new base it failed with
# no output at all: the last line of a successful build was "Utah desktop
# contract passed", then exit 1, three attempts out of three, with nothing to
# say whether shim-x64 was missing, the tree was missing, or the copy failed.
#
# So: succeed when the payload is already in place, copy when it is staged, and
# fail loudly and specifically when it is neither. A build that fails must say
# what it could not find.
set -euo pipefail

BOOTUPD_EFI="${BOOTUPD_EFI:-/usr/lib/bootupd/updates/EFI/fedora}"
SHIM_ROOT="${SHIM_ROOT:-/usr/lib/efi/shim}"

# Look for an existing payload before asking rpm anything. The base that ships
# it already is the base whose shim-x64 query we least want to depend on, and
# the directory it uses is named for the shim version without a dist tag, which
# need not match what rpm reports. Find the file that matters instead.
existing="$(find "${SHIM_ROOT}" -mindepth 4 -maxdepth 4 \
-path '*/EFI/fedora/shimx64.efi' -print -quit 2>/dev/null || true)"
if [[ -n "${existing}" ]]; then
echo "Base image already provides the shim payload at ${existing%/shimx64.efi}; nothing to mirror"
exit 0
fi

fail() {
echo "ERROR: shim mirroring: $*" >&2
exit 1
}

if ! shim_version="$(rpm -q --qf '%{VERSION}-%{RELEASE}' shim-x64 2>&1)"; then
fail "shim-x64 is not installed and no payload is present under ${SHIM_ROOT}
(rpm -q said: ${shim_version}). The signed EFI payload comes from that
package, so bootc cannot build a generic disk image without it. If the base
image dropped shim-x64, it has to be added to the package contract rather
than skipped here."
fi

if [[ ! -d "${BOOTUPD_EFI}" ]]; then
parent="$(dirname "${BOOTUPD_EFI}")"
echo "shim-x64 ${shim_version} is installed, but nothing is staged at ${BOOTUPD_EFI}" >&2
echo "and nothing is present under ${SHIM_ROOT} either." >&2
if [[ -d "${parent}" ]]; then
echo " ${parent} contains: $(ls -A "${parent}" 2>/dev/null | tr '\n' ' ')" >&2
else
echo " ${parent} does not exist either." >&2
fi
fail "no EFI payload to mirror at ${BOOTUPD_EFI}"
fi

target="${SHIM_ROOT}/${shim_version}/EFI/fedora"
install -d "${target}"
cp -a "${BOOTUPD_EFI}/." "${target}/"
echo "Mirrored shim-x64 ${shim_version} EFI payload into ${target}"
117 changes: 117 additions & 0 deletions tests/test_kernel_devel_pin.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
"""KERNEL_DEVEL_SHA256 must not outlive the kernel it was recorded for.

The koji fallback in install-nvidia.sh downloads an unsigned kernel-devel and
checks it against a hash committed in the script. The hash is a constant
because the base image is pinned by digest -- which is exactly why it has to
move when BASE_IMAGE does. The 7.2 base bump left it pinned to 7.1.8-100.fc43
for a revision: the dnf path satisfied 7.2 so the fallback never ran, nothing
failed, and nothing said the constant had gone stale. It would have failed only
once the enabled repositories dropped the kernel, which is the one situation
the fallback exists for.
"""
import os
import re
import subprocess
import tempfile
import unittest
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
SCRIPT = (ROOT / "scripts/install-nvidia.sh").read_text()


def constant(name: str) -> str:
match = re.search(rf'^{name}="(?:\$\{{[A-Z0-9_]+:-)?([^"}}]+)\}}?"', SCRIPT, re.MULTILINE)
assert match, f"{name} not found in install-nvidia.sh"
return match.group(1)


class KernelDevelPinTests(unittest.TestCase):
def test_the_hash_records_which_kernel_it_belongs_to(self):
nevr = constant("KERNEL_DEVEL_NEVR")
self.assertRegex(nevr, r"^\d+\.\d+\.\d+-\d+\.fc\d+\.\w+$")
self.assertRegex(constant("KERNEL_DEVEL_SHA256"), r"^[0-9a-f]{64}$")

def test_the_pin_matches_the_kernel_the_base_image_boots(self):
# The one assertion that actually catches the staleness: the recorded
# NEVR has to be the kernel this base carries. Update both together.
self.assertEqual(constant("KERNEL_DEVEL_NEVR"), "7.2.5-200.fc44.x86_64")

def test_a_mismatched_kernel_fails_before_downloading_60mb(self):
"""Drive the real guard: a kernel that is not the recorded one exits 1."""
guard = re.search(
r'if \[ "\$\{kernel\}" != "\$\{KERNEL_DEVEL_NEVR\}" \]; then.*?\n fi',
SCRIPT, re.DOTALL)
self.assertIsNotNone(guard, "the NEVR guard is not in install-nvidia.sh")
harness = (
'KERNEL_DEVEL_NEVR="7.2.5-200.fc44.x86_64"\n'
'kernel="7.1.8-100.fc43.x86_64"\n'
+ guard.group(0).replace("\n ", "\n")
+ '\necho "reached the download"\n'
)
result = subprocess.run(["bash", "-c", harness], capture_output=True, text=True)
self.assertEqual(result.returncode, 1, result.stdout)
self.assertNotIn("reached the download", result.stdout)
self.assertIn("7.1.8-100.fc43.x86_64", result.stderr)
self.assertIn("7.2.5-200.fc44.x86_64", result.stderr)

def test_the_matching_kernel_passes_the_guard(self):
guard = re.search(
r'if \[ "\$\{kernel\}" != "\$\{KERNEL_DEVEL_NEVR\}" \]; then.*?\n fi',
SCRIPT, re.DOTALL)
harness = (
'KERNEL_DEVEL_NEVR="7.2.5-200.fc44.x86_64"\n'
'kernel="7.2.5-200.fc44.x86_64"\n'
+ guard.group(0).replace("\n ", "\n")
+ '\necho "reached the download"\n'
)
result = subprocess.run(["bash", "-c", harness], capture_output=True, text=True)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn("reached the download", result.stdout)

def test_both_constants_are_overridable_together(self):
"""The guard must not make the hash override unreachable.

UTAH_KERNEL_DEVEL_SHA256 exists for one situation: validating a kernel
this file has not been re-recorded for. A NEVR guard that refused to
proceed whenever the booted kernel differed from the recorded one would
block exactly that case, leaving the override as dead code. So the NEVR
has to be overridable too, and the pair has to move together.
"""
for name in ("KERNEL_DEVEL_NEVR", "KERNEL_DEVEL_SHA256"):
with self.subTest(constant=name):
line = next(l for l in SCRIPT.splitlines()
if l.startswith(f"{name}="))
self.assertIn(f"${{UTAH_{name}:-", line,
f"{name} must accept an environment override")

def test_the_guard_message_names_the_override(self):
# A guard that stops the build has to say how to proceed deliberately,
# or the only way past it is to edit the script.
guard = re.search(
r'if \[ "\$\{kernel\}" != "\$\{KERNEL_DEVEL_NEVR\}" \]; then.*?\n fi',
SCRIPT, re.DOTALL).group(0)
self.assertIn("UTAH_KERNEL_DEVEL_NEVR", guard)
self.assertIn("UTAH_KERNEL_DEVEL_SHA256", guard)

def test_an_overridden_pair_reaches_the_download(self):
"""Drive the real guard with both overrides set to a different kernel."""
guard = re.search(
r'if \[ "\$\{kernel\}" != "\$\{KERNEL_DEVEL_NEVR\}" \]; then.*?\n fi',
SCRIPT, re.DOTALL).group(0)
harness = (
'KERNEL_DEVEL_NEVR="${UTAH_KERNEL_DEVEL_NEVR:-7.2.5-200.fc44.x86_64}"\n'
'kernel="9.9.9-1.fc99.x86_64"\n'
+ guard.replace("\n ", "\n")
+ '\necho "reached the download"\n'
)
import os
env = dict(os.environ, UTAH_KERNEL_DEVEL_NEVR="9.9.9-1.fc99.x86_64")
result = subprocess.run(["bash", "-c", harness], capture_output=True,
text=True, env=env)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn("reached the download", result.stdout)


if __name__ == "__main__":
unittest.main()
Loading
Loading