Skip to content
37 changes: 19 additions & 18 deletions .github/scripts/check-k0s-version.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@
three must move together or the sysext silently stops updating:

* ``elements/k0s/k0s-bin.bst`` — the upstream download URL.
* ``elements/oci/k0s-sysext.bst`` — the release asset filename, which
* ``elements/oci/k0s-sysext.bst`` — the release asset name
(``sysext-image``, packed by include/sysext.yml), which
``files/os/sysupdate.d/70-k0s.transfer`` reads as the version oracle
through its ``@v`` wildcard.
* ``files/k0s/sysext/extension-release.k0s`` — the ``VERSION_ID=`` reported
Expand Down Expand Up @@ -100,16 +101,17 @@ def main():
"restrict k0s-version to [A-Za-z0-9._~^+-] in include/k0s.yml.",
)

# The upstream download URL must be derived, not restated.
# Every upstream download URL (one per architecture) must be derived.
bin_text = read(K0S_BIN)
url_match = re.search(r"^\s*url:\s*(\S+)\s*$", bin_text, re.MULTILINE)
if not url_match:
urls = re.findall(r"^\s*url:\s*(\S+)\s*$", bin_text, re.MULTILINE)
if not urls:
fail(
"elements/k0s/k0s-bin.bst declares no source 'url:'.",
"restore the pinned upstream k0s release URL.",
)
url = url_match.group(1)
if "%{k0s-upstream-tag}" not in url:
for url in urls:
if "%{k0s-upstream-tag}" in url:
continue
fail(
"elements/k0s/k0s-bin.bst hardcodes the upstream k0s release tag:\n"
f" url: {url}\n"
Expand All @@ -121,36 +123,35 @@ def main():

# The asset filename must be on the k0s axis, not the OS release axis.
sysext_text = read(K0S_SYSEXT)
fname_match = re.search(r'^\s*FNAME="([^"]+)"\s*$', sysext_text, re.MULTILINE)
fname_match = re.search(r'^\s*sysext-image:\s*"([^"]+)"\s*$', sysext_text, re.MULTILINE)
if not fname_match:
fail(
'elements/oci/k0s-sysext.bst declares no FNAME="..." asset name.',
"restore the sysext asset filename assignment.",
'elements/oci/k0s-sysext.bst declares no sysext-image: "..." asset name.',
"restore the sysext-image variable.",
)
fname_expr = fname_match.group(1)
if any(axis in fname_expr for axis in ("%{release-version}", "%{installer-version}", "%{image-version}")):
fail(
"elements/oci/k0s-sysext.bst names the k0s sysext on an OS release\n"
f" axis: FNAME=\"{fname_expr}\"\n"
f" axis: sysext-image: \"{fname_expr}\"\n"
" 70-k0s.transfer reads this filename as the version of the k0s it\n"
" delivers.",
'use FNAME="k0s-%{k0s-version}.raw".',
'use sysext-image: "k0s-%{k0s-version}".',
)
if "%{k0s-version}" not in fname_expr:
fail(
"elements/oci/k0s-sysext.bst does not derive the sysext asset name\n"
f" from the pinned k0s version: FNAME=\"{fname_expr}\"",
'use FNAME="k0s-%{k0s-version}.raw".',
f" from the pinned k0s version: sysext-image: \"{fname_expr}\"",
'use sysext-image: "k0s-%{k0s-version}".',
)

# VERSION_ID must be generated, so sysext status matches the asset.
if 'echo "VERSION_ID=%{k0s-version}"' not in sysext_text:
if 'sysext-version: "%{k0s-version}"' not in sysext_text:
fail(
"elements/oci/k0s-sysext.bst does not generate VERSION_ID= from\n"
" %{k0s-version}, so `systemd-sysext status` can disagree with the\n"
" asset filename an operator sees on disk.",
'append \'VERSION_ID=%{k0s-version}\' to the staged '
"extension-release.k0s.",
'set sysext-version: "%{k0s-version}" in elements/oci/k0s-sysext.bst.',
)

ext_text = read(EXTENSION_RELEASE)
Expand Down Expand Up @@ -184,7 +185,7 @@ def main():
)

# The produced asset name must satisfy the sysupdate transfer pattern.
asset = fname_expr.replace("%{k0s-version}", version) + ".zst"
asset = fname_expr.replace("%{k0s-version}", version) + ".raw.zst"
transfer_text = read(TRANSFER)
pattern_match = re.search(
r"^\s*MatchPattern=(\S*@v\S*\.zst)\s*$", transfer_text, re.MULTILINE
Expand Down Expand Up @@ -214,7 +215,7 @@ def main():
f" k0s-version : {version!r}\n"
" Hosts compare the captured string against the version they have\n"
" installed, so a mismatch means k0s updates silently stop.",
"align FNAME in elements/oci/k0s-sysext.bst with MatchPattern in "
"align sysext-image in elements/oci/k0s-sysext.bst with MatchPattern in "
"files/os/sysupdate.d/70-k0s.transfer.",
)

Expand Down
2 changes: 1 addition & 1 deletion docs/skills/architecture-roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Priorities are derived from [gap-analysis-distros.md](gap-analysis-distros.md).
| # | Item | Rationale / source gap |
|---|------|------------------------|
| 1 | TPM2-sealed /var on installed nodes | Credential sealing exists (`tpm2-credential-sealing.md`); persistent state is not yet bound to the TPM. |
| 2 | aarch64 build axis | `project.conf` and `include/arch.yml` already model it; no CI coverage yet. |
| 2 | aarch64 build axis | `project.conf`, `include/arch.yml` and the k0s/kubeadm arm64 binary pins resolve `-o arch aarch64`. Missing: aarch64 FSDK artifacts in any cache (a full bootstrap build), an aarch64 CI build, and an aarch64 dogfood path (`scripts/dogfood-*.sh` run `qemu-system-x86_64` with x64 OVMF). |
| 3 | Booty merge and its Secure Boot shim story | The Bluefin HTTP-boot support works from [Booty](https://github.com/jeefy/booty) `feat/bluefin-http-boot`; it still needs to merge, and enrollment-free first boots need a shim-signed path. |
| 4 | Credential provisioning smoke tests on real hardware | SSH keys, static network, and firstboot settings are wired through systemd credentials; TPM2-sealed credential decryption still needs hardware proof. |
| 5 | Native reboot coordination for non-Kubernetes and single-node hosts | Kured only covers Kubernetes nodes; no FleetLock/locksmith equivalent. |
Expand Down
10 changes: 10 additions & 0 deletions docs/skills/k0s-sysext.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,16 @@ Design choices:
`systemctl enable --now k0s-first-boot.service`. The unit copies the image to
`/run/extensions/k0s.raw`, runs `systemd-sysext refresh`, and enables the
controller or worker.
- **Why the activation units stay in the base image.** A sysext cannot
activate itself: nothing in it runs until something merges it, and the k0s
image lives in `/var/lib/k0s`, outside the extension directories
`systemd-sysext` scans. `k0s-first-boot.service` (and its fetcher) is that
something, so it ships in `/usr` (`bluefin-server/os-k0s-first-boot.bst`).
A provisioner such as [Booty](https://github.com/jeefy/booty) or an
Ignition config writes `/var/lib/k0s/k0s.raw` and enables the unit. It is
inert otherwise: FSDK has no catch-all `disable *` preset, so
`80-bluefin-opt-in.preset` disables both units explicitly, and no preset in
the image or in any sysext enables them.
- **OTA delivery.** A k0s component `systemd-sysupdate` transfer file
(`70-k0s.transfer`) is installed in the base OS so hosts can pull new k0s
sysext releases from GitHub Releases without updating the root or UKI.
Expand Down
3 changes: 2 additions & 1 deletion docs/skills/kubeadm-sysext.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,8 @@ what democratic-csi's `chroot /host ... iscsiadm` node plugin needs.

## Known gaps

- x86_64 only (amd64 release binaries).
- aarch64: the arm64 release binaries are pinned, but no aarch64 image has
been built or booted.

## Verify

Expand Down
10 changes: 5 additions & 5 deletions elements/base/base-stack.bst
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
kind: stack
description: |
Minimal distroless runtime: glibc + gcc-libs + base files, plus
CA certificates and timezone data. No package manager. In FSDK 26.08,
runtime-gnu provides /bin/sh for ldconfig and ca-certificates
integration scripts; bash is removed by an explicit rm -f step in
elements/oci/base.bst, not by a compose domain.
Build sandbox for this project's manual and script elements: glibc,
runtime-gnu (/bin/sh and coreutils, which the ldconfig and
ca-certificates integration commands also need in FSDK 26.08), CA
certificates and timezone data. Nothing ships it; the OS payload is
bluefin-server/os-base.bst.

depends:
- freedesktop-sdk.bst:public-stacks/runtime-gnu.bst
Expand Down
2 changes: 1 addition & 1 deletion elements/bluefin-server/os-stack.bst
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,5 @@ depends:
# system-container helper (machinectl wrapper)
- bluefin-server/os-system-container.bst

# Console issue banner with KubeStellar dashboard URL
# Console issue banner (the KubeStellar sysext adds its dashboard URL)
- bluefin-server/os-issue.bst
4 changes: 0 additions & 4 deletions elements/freedesktop-sdk.bst
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,6 @@ config:
bootstrap_build_arch: 'x86_64'
- arch == 'aarch64':
bootstrap_build_arch: 'aarch64'
- arch == "ppc64le":
bootstrap_build_arch: "ppc64le"
- arch == "riscv64":
bootstrap_build_arch: "x86_64"

overrides:
# Our public module signing certificate; FSDK's kernel trusts it and
Expand Down
28 changes: 15 additions & 13 deletions elements/k0s/k0s-bin.bst
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
kind: manual
description: |
Import the upstream k0s release binary.
Import the upstream k0s release binary for the build architecture.

Pinned to the latest stable release for x86_64. The release tag comes from
include/k0s.yml (%{k0s-upstream-tag}); only the SHA256 below is restated
here, taken from that release's sha256sums.txt asset.
The release tag comes from include/k0s.yml (%{k0s-upstream-tag}); only the
SHA256 of each architecture's binary is restated here, taken from that
release's sha256sums.txt asset.

# %{k0s-upstream-tag} — the upstream release tag, URL-escaped. Single source
# of truth for the k0s version axis; see include/k0s.yml.
(@): include/k0s.yml
(@):
- include/arch.yml
- include/k0s.yml

build-depends:
# Base runtime providing /bin/sh and install in the sandbox.
Expand All @@ -23,13 +25,13 @@ sources:
url: github:k0sproject/k0s/releases/download/%{k0s-upstream-tag}/k0s-%{k0s-upstream-tag}-amd64
ref: ca1e9e68107335846e8296777fce2ccd654284e6265b4b5d32c34ead872af98f

(?):
- arch == "aarch64":
sources:
- kind: remote
url: github:k0sproject/k0s/releases/download/%{k0s-upstream-tag}/k0s-%{k0s-upstream-tag}-arm64
ref: 6301f64d98f9e4f6fbc2892bc66d137bb4ad5840685300772d8391614d47b8aa

config:
install-commands:
- |
if [ "%{arch}" != "x86_64" ]; then
echo "ERROR: elements/k0s/k0s-bin.bst pins the upstream k0s amd64 binary;" >&2
echo " arch=%{arch} has no pinned source. Add an arch-conditional" >&2
echo " source before building k0s for %{arch}." >&2
exit 1
fi
- install -D -m 0755 k0s-%{k0s-upstream-tag}-amd64 "%{install-root}/usr/bin/k0s"
- install -D -m 0755 k0s-%{k0s-upstream-tag}-%{go-arch} "%{install-root}/usr/bin/k0s"
45 changes: 39 additions & 6 deletions elements/kubeadm/kubeadm-bin.bst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ description: |
sha256 (published alongside the release asset): kubelet, kubeadm and
kubectl from dl.k8s.io, crictl (cri-tools), the containerd static build
(containerd, containerd-shim-runc-v2, ctr), the runc static build, and
the full CNI reference plugin set. Versions come from include/kubeadm.yml.
the full CNI reference plugin set. Versions come from include/kubeadm.yml;
the amd64 or arm64 asset is picked by the build architecture.

Layout: binaries in /usr/bin, CNI plugins in /usr/libexec/cni. Nothing
goes under /opt: /opt/cni/bin stays a writable host directory that the
Expand All @@ -14,6 +15,7 @@ description: |
private location and ships exactly these files.

(@):
- include/arch.yml
- include/kubeadm.yml

build-depends:
Expand Down Expand Up @@ -57,20 +59,51 @@ sources:
base-dir: ''
directory: cni

(?):
- arch == "aarch64":
sources:
- kind: remote
url: k8s_dl:release/v%{kubernetes-version}/bin/linux/arm64/kubelet
ref: 765b740e3ad9c590852652a2623424ec60e2dddce2c6280d7f042f56c8c98619
directory: k8s
- kind: remote
url: k8s_dl:release/v%{kubernetes-version}/bin/linux/arm64/kubeadm
ref: 697cf3aa54f1a5740b883a3b18a5d051b4032fd68ba89af626781a43ec9bccc3
directory: k8s
- kind: remote
url: k8s_dl:release/v%{kubernetes-version}/bin/linux/arm64/kubectl
ref: 46913a7aa0327f6cc2e1cc2775d53c4a2af5e52f7fd8dacbfbfd098e757f19e9
directory: k8s
- kind: tar
url: github:kubernetes-sigs/cri-tools/releases/download/v%{crictl-version}/crictl-v%{crictl-version}-linux-arm64.tar.gz
ref: c31d252e203df5f4cf37f314bd3092eb79087e791631c1e607087c74b6d0423f
base-dir: ''
directory: crictl
- kind: tar
url: github:containerd/containerd/releases/download/v%{containerd-version}/containerd-static-%{containerd-version}-linux-arm64.tar.gz
ref: d1a1e64c4334e17d6f9f40093d5ff9f810b95ac34c7dcba55e7d2226d2a8ab79
base-dir: bin
directory: containerd
- kind: remote
url: github:opencontainers/runc/releases/download/v%{runc-version}/runc.arm64
ref: 3c9a8e9e6dafd00db61f4611692447ebab4a56388bae4f82192aed67b66df712
directory: runc
- kind: tar
url: github:containernetworking/plugins/releases/download/v%{cni-plugins-version}/cni-plugins-linux-arm64-v%{cni-plugins-version}.tgz
ref: 16484966a46b4692028ba32d16afd994e079dc2cc63fbc2191d7bfaf5e11f3dd
base-dir: ''
directory: cni

config:
install-commands:
- |
set -eu
if [ "%{arch}" != "x86_64" ]; then
echo "ERROR: elements/kubeadm/kubeadm-bin.bst pins amd64 release binaries only (arch=%{arch})" >&2
exit 1
fi
bin="%{install-root}%{bindir}"
install -d "${bin}"
for b in kubelet kubeadm kubectl; do install -m 0755 "k8s/${b}" "${bin}/"; done
install -m 0755 crictl/crictl "${bin}/"
for b in containerd containerd-shim-runc-v2 ctr; do install -m 0755 "containerd/${b}" "${bin}/"; done
install -m 0755 runc/runc.amd64 "${bin}/runc"
install -m 0755 "runc/runc.%{go-arch}" "${bin}/runc"
install -d "%{install-root}%{libexecdir}/cni"
find cni -maxdepth 1 -type f -perm -u+x -exec install -m 0755 -t "%{install-root}%{libexecdir}/cni" {} +
for p in loopback bridge host-local portmap; do
Expand Down
43 changes: 10 additions & 33 deletions elements/oci/k0s-sysext.bst
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ description: |
(@):
- include/arch.yml
- include/k0s.yml
- include/sysext.yml

build-depends:
- base/base-stack.bst
- freedesktop-sdk.bst:components/erofs-utils.bst
- freedesktop-sdk.bst:components/xz.bst
- freedesktop-sdk.bst:components/zstd.bst
- filename: k0s/k0s-bin.bst
config:
Expand All @@ -28,43 +28,20 @@ sources:

variables:
strip-binaries: ""
sysext-name: "k0s"
sysext-image: "k0s-%{k0s-version}"
# Merged as /run/extensions/k0s.raw (k0s-first-boot.service), so the
# extension-release name carries no version.
sysext-release: "k0s"
sysext-version: "%{k0s-version}"
sysext-architecture: "%{systemd-arch}"

config:
install-commands:
- |
set -euo pipefail
OUT="%{install-root}"
FNAME="k0s-%{k0s-version}.raw"

if [ "%{arch}" != "x86_64" ]; then
echo "ERROR: elements/oci/k0s-sysext.bst only supports x86_64 (got %{arch})" >&2
exit 1
fi

mkdir -p sysext/usr/bin
mkdir -p sysext/usr/lib/systemd/system
mkdir -p sysext/usr/lib/extension-release.d
mkdir -p "${OUT}"

# Stage k0s binary
mkdir -p sysext/usr/bin sysext/usr/lib/systemd/system
cp -a /usr/bin/k0s sysext/usr/bin/k0s
chmod 0755 sysext/usr/bin/k0s

# Stage systemd service
cp -a sysext-src/k0scontroller.service sysext-src/k0sworker.service sysext/usr/lib/systemd/system/

# Stage extension release metadata
cp -a sysext-src/extension-release.k0s sysext/usr/lib/extension-release.d/
echo "VERSION_ID=%{k0s-version}" >> sysext/usr/lib/extension-release.d/extension-release.k0s
echo "ARCHITECTURE=%{systemd-arch}" >> sysext/usr/lib/extension-release.d/extension-release.k0s

# Build EROFS image
mkfs.erofs -d0 "${OUT}/${FNAME}" sysext

# Compress release asset
zstd -T0 -19 -q "${OUT}/${FNAME}" -o "${OUT}/${FNAME}.zst"

(
cd "${OUT}"
sha256sum --binary "${FNAME}.zst" > SHA256SUMS
)
- "%{sysext-pack}"
20 changes: 7 additions & 13 deletions elements/oci/kubeadm-sysext.bst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ description: |
- include/arch.yml
- include/image.yml
- include/kubeadm.yml
- include/sysext.yml

build-depends:
- base/base-stack.bst
Expand All @@ -33,6 +34,11 @@ sources:

variables:
strip-binaries: ""
sysext-name: "kubeadm"
sysext-image: "kubeadm_%{image-version}"
sysext-release: "kubeadm_%{image-version}"
sysext-version: "%{image-version}"
sysext-architecture: "%{systemd-arch}"

config:
install-commands:
Expand All @@ -59,23 +65,11 @@ config:
*) echo "ERROR: config.toml sandbox image is not %{pause-image} (include/kubeadm.yml)" >&2; exit 1 ;;
esac

rel="${lib}/extension-release.d/extension-release.kubeadm_%{image-version}"
install -D -m 0644 "${src}/extension-release.kubeadm" "${rel}"
echo "VERSION_ID=%{image-version}" >> "${rel}"
echo "ARCHITECTURE=%{systemd-arch}" >> "${rel}"

# A sysext may only carry /usr (and /opt, which must stay a plain
# writable host directory for /opt/cni/bin).
for entry in sysext/* sysext/.[!.]*; do
[ -e "${entry}" ] || [ -L "${entry}" ] || continue
[ "${entry}" = "sysext%{prefix}" ] || { echo "ERROR: sysext ships ${entry#sysext}" >&2; exit 1; }
done

- |
set -euo pipefail
OUT="%{install-root}"
FNAME="kubeadm_%{image-version}.raw"
mkdir -p "${OUT}"
mkfs.erofs -d0 "${OUT}/${FNAME}" sysext
zstd -T0 -19 -q "${OUT}/${FNAME}" -o "${OUT}/${FNAME}.zst"
(cd "${OUT}" && sha256sum --binary "${FNAME}.zst" > SHA256SUMS)
- "%{sysext-pack}"
Loading