Skip to content

refactor: shared sysext packing recipe, arm64 binary pins, base-image cleanups - #279

Merged
jeefy merged 8 commits into
projectbluefin:mainfrom
jeefy:refactor/build-graph-cleanup
Sep 29, 2026
Merged

jeefy merged 8 commits into
projectbluefin:mainfrom
jeefy:refactor/build-graph-cleanup

Conversation

@jeefy

@jeefy jeefy commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

Build-graph cleanup from the image/sysext audit. Each finding was checked first; two turned out to be already done and only needed their stale comments fixed.

1. Kubernetes bits in the base image

  • The KubeStellar console banner already ships in oci/kubestellar-sysext.bst as /usr/lib/issue.d/40-kubestellar.issue. The base banner (30-bluefin.issue) has no KubeStellar text. I checked the built /usr: it has no /etc/issue (factory or otherwise), so agetty falls back to /usr/lib/issue.d, which the merged sysext adds to. The only change here is the misleading comment in os-stack.bst.
  • k0s-first-boot.service stays in /usr. docs/skills/k0s-sysext.md now says why: a sysext can't merge itself, and k0s.raw lives in /var/lib/k0s, which is not an extension directory. It also records that 80-bluefin-opt-in.preset disables both units and no preset enables them (checked with git grep over files/).
  • The kured hook is untouched; feat(update): automatic updates with a Kubernetes interlock, a boot health gate and a diskless update signal #271 reworks it.

3. One packing recipe for the four sysexts

  • New include/sysext.yml defines %{sysext-pack}, which stamps extension-release, runs mkfs.erofs, compresses with zstd -19 and writes SHA256SUMS. Each oci/*-sysext.bst stages ./sysext, sets sysext-{name,image,release,version,architecture} and runs the recipe as its last step. KubeStellar sets sysext-architecture: "" because it had no ARCHITECTURE= before.
  • check-k0s-version.py now reads sysext-image/sysext-version instead of the old FNAME= line and the echo VERSION_ID line. Its tests are updated to match, as are test_signed_lockstep.py and test_kubeadm_sysext.py. The transfer test resolves sysext-image to the emitted .raw name. The unused xz build dependency is gone from the k0s sysext.
  • The refactor alone leaves the outputs unchanged. I extracted every rebuilt image with fsck.erofs --extract --preserve and compared it with upstream main. For all four sysexts the file list, contents, modes, owners and mtimes are identical. The only byte difference is the erofs superblock UUID, which mkfs.erofs randomizes.
  • A follow-up commit, fix(sysext): derive the erofs UUID from the image name, fixes that one difference. With it, two builds (artifacts deleted in between, remote caches ignored) give byte-identical .raw, .raw.zst and SHA256SUMS for all four sysexts.

4. aarch64

  • Removed the ppc64le/riscv64 bootstrap_build_arch branches, which can't be reached because options.arch only allows aarch64 and x86_64.
  • k0s-bin.bst and kubeadm-bin.bst now pick the arm64 sources when arch == "aarch64". Every sha256 comes from the checksum the project publishes next to the asset; the existing amd64 pins match those files too. Installs use a new %{go-arch} (amd64/arm64) from include/arch.yml. The x86_64-only guards are gone, and check-k0s-version.py checks every source URL.
  • The amd64 URLs are unchanged, so the tracker in ci: track patch releases of upstream binaries (Kubernetes, containerd, runc, CNI, k0s, ORAS) #272, which anchors on them, keeps working.
  • bst -o arch aarch64 show resolves all five targets. bst -o arch aarch64 source fetch k0s-bin kubeadm-bin downloads only arm64 assets and every ref checks out. The x86_64 k0s and kubeadm sysexts are unchanged (same files, modes and mtimes).
  • Still blocking an aarch64 image (now listed in architecture-roadmap.md):
    • No cache has aarch64 FSDK artifacts, so it needs a full bootstrap build.
    • There is no aarch64 CI build.
    • scripts/dogfood-*.sh run qemu-system-x86_64 with x64 OVMF.
    • A release feed is single-arch: the transfer MatchPatterns carry no architecture.

8. The base-stack.bst description pointed at elements/oci/base.bst, which doesn't exist. It now describes base-stack as what it is: the build sandbox, which nothing ships.

Verification

  • python3 -m pytest tests/unit -q passes: 372 in a throwaway worktree with no boot-keys.
  • bats tests/unit passes: 0 failures.
  • python3 .github/scripts/docs-checks.py passes.
  • flock … just validate passes.
  • Sysexts before/after: file-list, content, mode and mtime manifests are identical (see 3).
  • Sysext double build is byte-identical after the UUID fix.
  • just export-image then flock … just dogfood-check passes: netboot boot reports os=bluefin-server 26.09.2, failed=0.

Follow-ups

jeefy and others added 8 commits September 29, 2026 04:00
kubestellar, kubeadm and zfs each carried a copy of the same tail: stamp
extension-release, mkfs.erofs, zstd, SHA256SUMS. include/sysext.yml now
holds it as %{sysext-pack}; each element stages ./sysext, names itself with
sysext-* variables and runs the recipe last.

Outputs are unchanged: extracting the rebuilt images gives the same files,
contents, modes, owners and mtimes as before. Only the erofs superblock
UUID differs, which mkfs.erofs randomizes on every build.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The asset name and VERSION_ID now live in sysext-image / sysext-version, so
check-k0s-version.py reads those instead of FNAME= and the echoed
VERSION_ID line, and the transfer test resolves sysext-image to the
emitted .raw name. Drops the unused xz build dependency.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The description pointed at elements/oci/base.bst, which does not exist, and
described base-stack as a shipped distroless runtime. Every reference is a
build-depends; the OS payload is bluefin-server/os-base.bst.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The KubeStellar banner already ships in oci/kubestellar-sysext.bst
(/usr/lib/issue.d, which agetty reads when /etc/issue is absent); the base
image's banner has no KubeStellar text, so fix the os-stack.bst comment
that said otherwise.

k0s-first-boot.service must stay in /usr: a sysext cannot merge itself,
and the k0s image sits in /var/lib/k0s, outside the extension directories.
Record that, and that 80-bluefin-opt-in.preset keeps both units off.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
options.arch in project.conf only allows aarch64 and x86_64, so the
ppc64le and riscv64 bootstrap_build_arch branches could never be taken.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
k0s-bin.bst takes the arm64 asset under arch == aarch64 (sha256 from the
release's sha256sums.txt) and installs the %{go-arch} file, a new amd64 /
arm64 spelling in include/arch.yml. The x86_64-only guards in k0s-bin.bst
and k0s-sysext.bst go; check-k0s-version.py now checks every source URL.

The x86_64 k0s sysext is unchanged (same files, modes and mtimes); a
-o arch aarch64 source fetch downloads and verifies only the arm64 asset.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
arm64 kubelet, kubeadm, kubectl, crictl, containerd static, runc and CNI
plugins, each sha256 taken from the checksum its project publishes next to
the asset (the amd64 pins match theirs too). The x86_64 kubeadm sysext is
unchanged. architecture-roadmap.md lists what still blocks an aarch64
image: no aarch64 FSDK artifacts in any cache, no CI build, and x86-only
dogfood scripts.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
mkfs.erofs gave every build a random filesystem UUID, the only difference
between two builds of the same sysext: file contents, modes and mtimes
already match because BuildStream exports SOURCE_DATE_EPOCH and mkfs.erofs
clamps to it. With the UUID derived from sysext-image, two builds of each
sysext with the artifacts deleted in between (and remote caches ignored)
produce byte-identical .raw, .raw.zst and SHA256SUMS.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
castrojo force-pushed the refactor/build-graph-cleanup branch from 5517e66 to 927b90d Compare September 29, 2026 08:01

@castrojo castrojo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebased branch onto current main to resolve the minor conflict in elements/bluefin-server/os-stack.bst (preserving both the system-container helper from #276 and the updated issue banner comment). All unit tests and docs checks pass locally.

@jeefy
jeefy merged commit 5389d0e into projectbluefin:main Sep 29, 2026
5 checks passed
jeefy added a commit that referenced this pull request Sep 29, 2026
…#288)

#279 added arm64 sources under `arch == "aarch64"` in k0s-bin.bst and
kubeadm-bin.bst after #272's tracker landed. The tracker already treats
every `url:` line carrying a component's marker as a pin, so it rewrites
the arm64 refs too, but nothing held it to that. Behavioural tests (mocked
HTTP) now do: a runc and a k0s bump rewrite both architectures' refs; a
release missing the arm64 asset is not a candidate; a missing arm64 asset
or checksum fails the component and writes nothing; a no-op run leaves
the files byte-identical; the real elements pin arm64 for every amd64
source. The docstring and the tracker docs say so.

Assisted-by: Claude Opus 5.5 via GitHub Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants