Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ FROM ${BASE_IMAGE}
# transaction reads. These, the pinned package image and the install script
# are the whole input to the expensive layer, so everything else waits its
# turn below them.
COPY packages/bluefin.toml packages/utah.toml contracts/bluefin-desktop.toml /usr/share/utah/
COPY packages/bluefin.toml packages/utah.toml packages/parity-exceptions.toml packages/parity-baseline.txt contracts/bluefin-desktop.toml /usr/share/utah/
COPY packages/hummingbird.repo packages/nvidia-container.repo packages/utah-packages.repo /etc/yum.repos.d/
# Hummingbird signs its RPMs with Red Hat's release key 2 (fd431d51); the key
# lets packages/hummingbird.repo run with gpgcheck=1 here and in the live ISO
Expand Down Expand Up @@ -68,6 +68,7 @@ COPY scripts/install-packages.py \
scripts/verify-desktop-contract.py \
scripts/verify-gnome-extensions.py \
scripts/mirror-shim.sh \
scripts/check-image-parity.py \
scripts/verify-efi-chain.sh \
/tmp/utah-scripts/
# Common publishes Bluefin artwork, desktop defaults, Brewfiles, and setup
Expand All @@ -91,6 +92,7 @@ RUN for pair in install-packages.py:utah-install-packages \
verify-desktop-contract.py:utah-verify-desktop-contract \
verify-gnome-extensions.py:utah-verify-gnome-extensions \
mirror-shim.sh:utah-mirror-shim \
check-image-parity.py:utah-check-image-parity \
verify-efi-chain.sh:utah-verify-efi-chain; do \
install -Dm 0755 "/tmp/utah-scripts/${pair%%:*}" "/usr/local/libexec/${pair##*:}" || exit 1; \
done && \
Expand Down Expand Up @@ -207,6 +209,11 @@ RUN --mount=type=bind,from=packages,source=/repository,target=/etc/utah-packages
esac && \
IMAGE_FLAVOR="${IMAGE_FLAVOR}" /usr/local/libexec/utah-verify-rpm-contract \
/usr/share/utah/bluefin.toml /usr/share/utah/utah.toml && \
rpm -qa --qf '%{NAME}\t%{VERSION}-%{RELEASE}\n' | sort > /usr/share/utah/packages.txt && \
# Report-only: without --strict this step records the gaps against Bluefin's
# image and exits 0 even when the inventory fetch fails, so neither a new gap
# nor an unreachable registry can fail the build.
/usr/local/libexec/utah-check-image-parity --report /usr/share/utah/parity-report.txt && \
# The package repository is now only ever bind mounted, so it is absent from
# the committed image. Flip it disabled here -- the last step that installs
# anything -- so later dnf calls on the image (the live ISO build's included)
Expand Down
7 changes: 7 additions & 0 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,13 @@ check-parity:
exit 1
fi

# Compare a built image's installed RPMs with Bluefin's, by name. Bluefin's
# inventory is read from its manifest annotation, so nothing is pulled but the
# image under test, which is usually already local.
check-image-parity image bluefin="ghcr.io/ublue-os/bluefin:stable":
podman run --rm --entrypoint /usr/local/libexec/utah-check-image-parity \
"{{image}}" --strict --bluefin-image "{{bluefin}}"

# Re-measure Utah against the published Bluefin and Dakota images: package
# lists and user-visible files from inside each image, and Dakota's SBOM.
# Needs podman and gh. Then review baselines/GAP.md and triage new gaps.
Expand Down
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,26 @@ generated from `packages/bluefin.toml` and `packages/utah.toml`
(`scripts/generate-site-data.py`, `site/data/packages.json`); `just check`
fails if this table drifts from that output (`scripts/check-doc-counts.py`).

That is a list-to-list check, and a list never names what Bluefin's Silverblue
base already carried — which is where the gaps that reached users hid
(`glibc-all-langpacks` #114, `linux-firmware` and the iwlwifi blobs #97). So
the build also compares the two *images*: `scripts/check-image-parity.py` reads
Bluefin's full installed inventory from the `dev.hhd.rechunk.info` annotation on
its manifest (one registry GET, no pull), diffs it against this image's `rpm
-qa` by name, and reports every name Bluefin has that Utah lacks. A gap is
explained if `utah.toml` lists it under `[unavailable]` or
`packages/parity-exceptions.toml` gives a reason; the report prints the reason.

`packages/parity-baseline.txt` is the debt register — the gaps already known,
grouped by where each name could come from today. The report separates a *new*
gap from a known one and names baseline entries that have closed, so the file
gets trimmed. Report-only in the build for now; `--strict` turns a new gap into
a build failure. Both the image's package list
(`/usr/share/utah/packages.txt`) and the report
(`/usr/share/utah/parity-report.txt`) ship in the image, so any published image
can be asked what it has, and `just check-image-parity IMAGE` runs the strict
comparison against a built image locally.



## Known gaps
Expand Down
5 changes: 5 additions & 0 deletions docs/skills/ci-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,11 @@ opaque `exit status 71` from the image build (comment,
and the ban on flavor literals in workflows.
- `just check-parity` -- `packages/bluefin.toml` against Bluefin's upstream
pinned at `packages/.bluefin-parity-ref`.
That is list against list. The image build itself runs
`utah-check-image-parity`, which compares the installed RPM set with the
inventory Bluefin's image publishes in its manifest, and writes the report
to `/usr/share/utah/parity-report.txt` in the image, split into new gaps and
the known ones in `packages/parity-baseline.txt` (report-only today).
- `just check-repos` -- the complete installation transaction against the
digest-pinned base and package repository, including extension build tools.

Expand Down
32 changes: 31 additions & 1 deletion docs/skills/package-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,8 @@ releases or emit missing-module errors with empty kernel names.
is a **build failure**. `just check-repos` reads the base and package-image
digests from `Containerfile`, copies the same repository configuration, and
runs `install-packages.py --resolve` inside that base. This includes the
release-specific Bluefin section, GNOME, services, and extension build tools.
release-specific Bluefin section, GNOME, parity, services, and extension
build tools.
It needs Podman and network access. A name lookup on GitHub Pages is not
evidence that the pinned OCI repository is complete or ABI-compatible.
The factory's leading metadata layer is checked against the pinned manifest
Expand Down Expand Up @@ -186,6 +187,35 @@ assert a package in every section reaches the install set and is counted under
its own heading, so a section wired into one place and not another fails the
suite rather than shipping quietly.

## Image-level parity

`packages/bluefin.toml` is the list Bluefin *asks* for. It never names what
Bluefin's Silverblue base already had before that list was applied, and that
is where the gaps that reached users hid: `glibc-all-langpacks` (#114),
`linux-firmware` and the iwlwifi blobs (#97). A list-to-list diff cannot see
them; only the two images can.

`scripts/check-image-parity.py` runs inside the image build after the last
package step. It reads Bluefin's complete inventory from the
`dev.hhd.rechunk.info` annotation every rechunked Bluefin image carries (one
registry GET, no pull), compares it with `rpm -qa` by name, and reports every
name Bluefin has that this image lacks. A name is explained if `utah.toml`
lists it under `[unavailable]` or `packages/parity-exceptions.toml` gives a
reason for it; the report prints the reason. Both the inventory and the report
ship in the image under `/usr/share/utah/`.

- `packages/parity-baseline.txt` is the debt register: the gaps already
known, grouped by where each name could come from today (Hummingbird's
repository, the pinned factory image, nowhere). The report separates a
**new** gap from a known one and names baseline entries that have closed,
so the file gets trimmed. Report-only in the build today; `--strict` fails
on a new gap and is a one-word change in the Containerfile.
- `just check-image-parity IMAGE` runs the strict comparison against a built
image locally.
- Add to `parity-exceptions.toml` only a difference that is deliberate, with
a reason written for the person reading a build log. A real gap goes to
`utah.toml` with a tracking issue, like every `[unavailable]` entry.

## Verification

```bash
Expand Down
Loading
Loading