Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ FROM ${BASE_IMAGE}
# turn below them.
COPY packages/bluefin.toml packages/utah.toml contracts/bluefin-desktop.toml /usr/share/utah/
COPY packages/hummingbird.repo packages/nvidia-container.repo packages/utah-packages.repo /etc/yum.repos.d/
# Hummingbird signs its RPMs with Red Hat's release key 2 (fd431d51); the key
# lets packages/hummingbird.repo run with gpgcheck=1 here and in the live ISO
# build on top of this image.
COPY packages/RPM-GPG-KEY-redhat-release-2 /etc/pki/rpm-gpg/
# The package image is an RPM repository, not a runtime dependency. It is
# bind mounted into the two RUN steps that install from it and never copied
# into a layer: a COPY used to put the whole ~4 GB repository at
Expand Down
2 changes: 2 additions & 0 deletions Containerfile.kernel
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ FROM ${BASE_IMAGE} AS builder
# itself installs from: Hummingbird's overlay plus the pinned Fedora 44
# buildroot. Fedora is builder-only; it is not copied into the runtime stage.
COPY packages/hummingbird.repo /etc/yum.repos.d/hummingbird.repo
# hummingbird.repo runs with gpgcheck=1; the key it names has to exist here too.
COPY packages/RPM-GPG-KEY-redhat-release-2 /etc/pki/rpm-gpg/
COPY packages/fedora-44.repo /etc/yum.repos.d/fedora-44.repo
RUN mkdir -p /usr/local/libexec
COPY scripts/install-ogc-kernel.sh /usr/local/libexec/utah-install-ogc-kernel
Expand Down
14 changes: 12 additions & 2 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,14 @@ check:
grep -q -- '--mount=type=bind,from=packages,source=/repository,target=/etc/utah-packages,ro' Containerfile
# The package repository is bind mounted, never committed; a COPY would ship
# the whole ~4 GB RPM repository in every image and of every ISO (#128).
! grep -q 'COPY --from=packages' Containerfile
# `! cmd` is exempt from errexit, so a bare `! grep` neither stops this
# script nor changes its exit status unless it happens to be the last line
# of the recipe: it reads as a gate and enforces nothing. Use an explicit if.
if grep -q 'COPY --from=packages' Containerfile; then
echo 'Containerfile must bind-mount the package repository, not COPY it' >&2
exit 1
fi
test -f packages/RPM-GPG-KEY-redhat-release-2
# Every executable release asset fetched during composition must be pinned
# and verified; no build may resolve a mutable latest release.
python3 scripts/check-download-integrity.py
Expand All @@ -99,7 +106,10 @@ check:
# No workflow may carry its own copy of the flavor list. That drift is what
# config/flavors.json exists to stop: narrowing the build matrix while
# promote and release still name images nothing produces fails late.
! grep -rn 'utah-nvidia\|utah-gaming' .github/workflows/
if grep -rn 'utah-nvidia\|utah-gaming' .github/workflows/; then
echo 'no workflow may name a flavored image; read it from config/flavors.json' >&2
exit 1
fi

# Verify branding, desktop defaults, first-boot Flatpak policy, and service
# enablement in an already-composed image. The same verifier runs in the
Expand Down
10 changes: 10 additions & 0 deletions docs/skills/containerfile.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,16 @@ In summary:
layer uses it or not: with `VERSION` at the top, the package transaction
missed the registry layer cache on every commit, since `VERSION` carries
the date and the commit. Nothing above the branding step ever sees them.
- The package repository is bind mounted, never copied:
`RUN --mount=type=bind,from=packages,source=/repository,target=/etc/utah-packages,ro`
on the two steps that install from it. A `COPY --from=packages` used to put
the whole 4 GB repository into the image, nothing removed it, and it was two
thirds of every published image and of every ISO (#130). `just check`
refuses a `COPY --from=packages`. `packages/utah-packages.repo` is committed
with `enabled=1`, because the two mounted steps install from it; the flavor
step, the last one that installs anything, flips it to `enabled=0` so a later
dnf call on the finished image -- the live ISO build among them -- does not
fail on a baseurl that is no longer mounted (comment, the repo file itself).
- `ARG PACKAGE_IMAGE_REF=${PACKAGE_IMAGE}@${PACKAGE_IMAGE_SHA}` defaults to
the digest-pinned OCI package repository for reproducible CI builds, while
allowing local composition to inject a local image from containers-storage
Expand Down
17 changes: 15 additions & 2 deletions iso/live/Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,23 @@ RUN chmod 0755 /tmp/utah-live-src/configure-live.sh && \
# to open; the installed system boots the payload image's own initramfs, which
# this build never touches. Revisit if Utah ever wants TPM-bound LUKS, which
# needs tpm2-tools in the installed image, not in the live overlay.
RUN set -eux; \
# /root is a symlink to /var/roothome in a bootc image and the target does not
# exist at build time, so dracut's install of /root failed on every build:
# dracut-install: ERROR: installing '/root' to '/root'
# dracut[E]: FAILED: /usr/lib/dracut/dracut-install ... -f /root /root
# and dracut still exited 0, so the failure never stopped the build (#132).
# Give the symlink a target for the duration of the build, and treat any
# dracut[E] FAILED line as the error it is.
RUN set -euxo pipefail; \
mkdir -p /var/roothome; \
kernel="$(ls /usr/lib/modules | sort -V | tail -1)"; \
dracut --force --reproducible --no-hostonly \
--add dmsquash-live \
--omit 'tpm2-tss pcsc' \
--add-drivers 'squashfs overlay loop iso9660 sr_mod cdrom' \
"/usr/lib/modules/${kernel}/initramfs.img" "${kernel}"
"/usr/lib/modules/${kernel}/initramfs.img" "${kernel}" 2>&1 | tee /tmp/dracut.log; \
if grep -q 'dracut\[E\]: FAILED' /tmp/dracut.log; then \
echo 'dracut reported a failed install; refusing to ship the initramfs' >&2; exit 1; \
fi; \
rm -f /tmp/dracut.log; \
rmdir /var/roothome || true
33 changes: 33 additions & 0 deletions packages/RPM-GPG-KEY-redhat-release-2
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
pub 4096R/FD431D51 2009-10-22
Key fingerprint = 567E 347A D004 4ADE 55BA 8A5F 199E 2F91 FD43 1D51
uid Red Hat, Inc. (release key 2) <security@redhat.com>

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v2.0.22 (GNU/Linux)

mQINBErgSTsBEACh2A4b0O9t+vzC9VrVtL1AKvUWi9OPCjkvR7Xd8DtJxeeMZ5eF
0HtzIG58qDRybwUe89FZprB1ffuUKzdE+HcL3FbNWSSOXVjZIersdXyH3NvnLLLF
0DNRB2ix3bXG9Rh/RXpFsNxDp2CEMdUvbYCzE79K1EnUTVh1L0Of023FtPSZXX0c
u7Pb5DI5lX5YeoXO6RoodrIGYJsVBQWnrWw4xNTconUfNPk0EGZtEnzvH2zyPoJh
XGF+Ncu9XwbalnYde10OCvSWAZ5zTCpoLMTvQjWpbCdWXJzCm6G+/hx9upke546H
5IjtYm4dTIVTnc3wvDiODgBKRzOl9rEOCIgOuGtDxRxcQkjrC+xvg5Vkqn7vBUyW
9pHedOU+PoF3DGOM+dqv+eNKBvh9YF9ugFAQBkcG7viZgvGEMGGUpzNgN7XnS1gj
/DPo9mZESOYnKceve2tIC87p2hqjrxOHuI7fkZYeNIcAoa83rBltFXaBDYhWAKS1
PcXS1/7JzP0ky7d0L6Xbu/If5kqWQpKwUInXtySRkuraVfuK3Bpa+X1XecWi24JY
HVtlNX025xx1ewVzGNCTlWn1skQN2OOoQTV4C8/qFpTW6DTWYurd4+fE0OJFJZQF
buhfXYwmRlVOgN5i77NTIJZJQfYFj38c/Iv5vZBPokO6mffrOTv3MHWVgQARAQAB
tDNSZWQgSGF0LCBJbmMuIChyZWxlYXNlIGtleSAyKSA8c2VjdXJpdHlAcmVkaGF0
LmNvbT6JAjYEEwEIACACGwMGCwkIBwMCBBUCCAMEFgIDAQIeAQIXgAUCSuBJPAAK
CRAZni+R/UMdUfIkD/9m3HWv07uJG26R3KBexTo2FFu3rmZs+m2nfW8R3dBX+k0o
AOFpgJCsNgKwU81LOPrkMN19G0+Yn/ZTCDD7cIQ7dhYuDyEX97xh4une/EhnnRuh
ASzR+1xYbj/HcYZIL9kbslgpebMn+AhxbUTQF/mziug3hLidR9Bzvygq0Q09E11c
OZL4BU6J2HqxL+9m2F+tnLdfhL7MsAq9nbmWAOpkbGefc5SXBSq0sWfwoes3X3yD
Q8B5Xqr9AxABU7oUB+wRqvY69ZCxi/BhuuJCUxY89ZmwXfkVxeHl1tYfROUwOnJO
GYSbI/o41KBK4DkIiDcT7QqvqvCyudnxZdBjL2QU6OrIJvWmKs319qSF9m3mXRSt
ZzWtB89Pj5LZ6cdtuHvW9GO4qSoBLmAfB313pGkbgi1DE6tqCLHlA0yQ8zv99OWV
cMDGmS7tVTZqfX1xQJ0N3bNORQNtikJC3G+zBCJzIeZleeDlMDQcww00yWU1oE7/
To2UmykMGc7o9iggFWR2g0PIcKsA/SXdRKWPqCHG2uKHBvdRTQGupdXQ1sbV+AHw
ycyA/9H/mp/NUSNM2cqnBDcZ6GhlHt59zWtEveiuU5fpTbp4GVcFXbW8jStj8j8z
1HI3cywZO8+YNPzqyx0JWsidXGkfzkPHyS4jTG84lfu2JG8m/nqLnRSeKpl20Q==
=79bX
-----END PGP PUBLIC KEY BLOCK-----
9 changes: 8 additions & 1 deletion packages/hummingbird.repo
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ name=public-hummingbird-x86_64-rpms
baseurl=https://packages.redhat.com/api/pulp-content/public-hummingbird/x86_64/
enabled=1
sslverify=1
gpgcheck=0
# Hummingbird's own repo file ships gpgcheck=0 so that their lockfile tooling
# fails loudly instead of silently on a bad signature. Their RPMs are signed
# (RSA/SHA256, key ID 199e2f91fd431d51, Red Hat release key 2), so an image
# build has no reason to skip the check. The key is committed as
# packages/RPM-GPG-KEY-redhat-release-2 (fingerprint 567E 347A D004 4ADE 55BA
# 8A5F 199E 2F91 FD43 1D51) and installed by the Containerfile.
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release-2
priority=10
zchunk=false
4 changes: 4 additions & 0 deletions packages/utah-packages.repo
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@
# utah-install: true
name=Utah package factory
baseurl=file:///etc/utah-packages
# The path exists only while the Containerfile bind mounts the pinned package
# image there. The flavor step, the last one that installs anything, flips this
# to enabled=0 so later dnf calls on the image (the live ISO build included)
# do not fail on a baseurl that is not there.
enabled=1
gpgcheck=0
repo_gpgcheck=0
Expand Down
5 changes: 3 additions & 2 deletions scripts/configure-services.sh
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,10 @@ disable_unit() {
# Hummingbird intentionally does not ship every Bluefin integration package.
# Not enabled: rechunker-group-fix.service. It exists to repair the
# /usr/lib/{group,gshadow} damage left by the legacy rechunker
# (hhd-dev/rechunk) when rebasing to an image built without it. Utah rechunks
# with chunkah, through projectbluefin/actions' reusable-build, and has never
# (hhd-dev/rechunk) when rebasing to an image built without it. Utah has never
# been through legacy-rechunk -- so there is nothing here for it to repair.
# (Utah is not rechunked at all today: reusable-build skips chunkah on the
# testing stream, which is the only stream Utah builds. Tracked by #131.)
#
# It is not merely useless, it breaks the boot. The unit orders itself both
# After=local-fs.target and Before=systemd-sysusers.service, and local-fs
Expand Down
3 changes: 3 additions & 0 deletions scripts/kernel-cache-tag.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,7 @@ cd "$(dirname "$0")/.."
cat Containerfile.kernel
cat scripts/install-ogc-kernel.sh scripts/install-nvidia.sh
cat packages/hummingbird.repo packages/fedora-44.repo
# The builder imports this key to verify Hummingbird's RPMs, so a rotated key
# is a different build root (tests/test_kernel_cache_key.py enforces it).
cat packages/RPM-GPG-KEY-redhat-release-2
} | sha256sum | cut -c1-16
1 change: 1 addition & 0 deletions tests/test_kernel_cache_tag.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
"scripts/install-nvidia.sh",
"packages/hummingbird.repo",
"packages/fedora-44.repo",
"packages/RPM-GPG-KEY-redhat-release-2",
)


Expand Down
39 changes: 39 additions & 0 deletions tests/test_package_resolution.py
Original file line number Diff line number Diff line change
Expand Up @@ -232,5 +232,44 @@ def test_verifier_asserts_the_parity_section(self):
self.assertIn(f" - {target}\n", stderr.getvalue())


class ImageSizeTests(unittest.TestCase):
MOUNT = "--mount=type=bind,from=packages,source=/repository,target=/etc/utah-packages,ro"

def test_package_repository_is_mounted_not_copied(self):
# #130: a COPY put the whole 4 GB repository into every image and ISO.
source = (ROOT / "Containerfile").read_text()
self.assertNotIn("COPY --from=packages", source)
steps = [step for step in source.split("\nRUN ") if step.startswith(self.MOUNT)]
self.assertEqual(len(steps), 2, "both install steps must mount the repository")
self.assertIn("utah-install-packages", steps[0])
self.assertIn("utah-install-ogc-kernel", steps[1])
self.assertIn("utah-install-nvidia", steps[1])
# Nothing installs after the flavor step, so it is the one that turns
# the repository file off for the image's lifetime.
self.assertIn("sed -i 's/^enabled=1$/enabled=0/' /etc/yum.repos.d/utah-packages.repo", steps[1])

def test_package_repository_file_is_enabled_only_during_the_build(self):
text = (ROOT / "packages/utah-packages.repo").read_text()
self.assertIn("enabled=1", text)
self.assertIn("baseurl=file:///etc/utah-packages", text)
self.assertIn("utah-packages", installer.REPOS)

def test_hummingbird_packages_are_signature_checked(self):
text = (ROOT / "packages/hummingbird.repo").read_text()
self.assertIn("gpgcheck=1", text)
self.assertIn("gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release-2", text)
key = (ROOT / "packages/RPM-GPG-KEY-redhat-release-2").read_text()
self.assertIn("BEGIN PGP PUBLIC KEY BLOCK", key)
for containerfile in ("Containerfile", "Containerfile.kernel"):
self.assertIn("COPY packages/RPM-GPG-KEY-redhat-release-2 /etc/pki/rpm-gpg/",
(ROOT / containerfile).read_text(), containerfile)

def test_live_initramfs_build_fails_on_a_dracut_error(self):
source = (ROOT / "iso/live/Containerfile").read_text()
self.assertIn("mkdir -p /var/roothome", source)
self.assertIn("set -euxo pipefail", source)
self.assertIn("dracut\\[E\\]: FAILED", source)


if __name__ == "__main__":
unittest.main()
Loading