Repository navigation
fix(installer): rewrite the network DDI repart override without sed - #250
Merged
Merged
Conversation
This was referenced Sep 26, 2026
Merged
The inst.ddi_url path in the released installer-v26.08.0 PXE initrd cannot complete on any machine: after a successful download, verification and decompression, bluefin-sysinstall runs `sed` to repoint CopyBlocks= in the repart override and dies with /usr/bin/bluefin-sysinstall: line 91: sed: command not found exit 127, FailureAction=poweroff. installer-stack.bst ships uutils coreutils, Bash, curl, zstd and systemd, and grep arrives transitively (all confirmed in the released initrd's file list); sed is a build-depend of the installer element only and never lands in the live initrd. Rewrite the override in plain Bash: copy /usr/lib/repart.d/20-root-a.conf line by line, substituting only the column-0 CopyBlocks= line. Every other line is unchanged apart from normalising a missing final newline. If the recipe carries no CopyBlocks= line the wrapper now errors out before the disk is touched (recipe and wrapper ship from the same repo, so that is a repo bug, not something to paper over by appending a key). No new dependencies. Also split the one `local x="$(...)"` so the extracted wrapper is `shellcheck -S warning` clean. Step 1a of the element now pins every external command the wrapper calls (grep, curl, zstd, modprobe, mount, umount, sha256sum, readlink, ls, cp, mkdir, sleep, systemctl next to the existing five), so a missing tool fails the build instead of the install. tests/unit/conftest.py gains a session-scoped `installer_wrapper` fixture that carves the wrapper out of the heredoc, shared by the installer tests. test_installer_contract.py asserts that no command position names sed, awk or tar and that the build-time check pins the wrapper's tools; two tests run the rewrite block against the shipped 20-root-a.conf (output equals the sed result it replaced, expected path derived from ROOT_COPYBLOCKS_LINE) and against recipes with no or an indented CopyBlocks= line (exit 1 with the wrapper's message, nothing appended). All fail against the previous element. Assisted-by: Claude Fable 5.1 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
State the wrapper's tool inventory next to the PXE flow description and add a Common Rationalizations row: the initrd contains only installer-stack.bst plus its transitive runtime deps, so sed, awk and tar fail with `command not found` (exit 127) after the download. Use Bash builtins, or declare the tool in the stack and add it to the Step 1a build-time tool check that pins every external command bluefin-sysinstall calls. Assisted-by: Claude Fable 5.1 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jeefy
force-pushed
the
fix/installer-sed-missing
branch
from
September 26, 2026 18:18
90bc8fb to
bb87bba
Compare
jeefy
added a commit
that referenced
this pull request
Sep 26, 2026
…254) ## Problem On the installed image `bluefin-firstboot-credentials.service` fails on every boot that carries a `firstboot.*` credential: ```text bluefin-firstboot-credentials.service: Failed at step EXEC spawning systemd-firstboot: No such file or directory ``` so `firstboot.locale`, `firstboot.locale-messages`, `firstboot.keymap`, `firstboot.timezone` and `firstboot.hostname` are silent no-ops — including the ones `systemd-sysinstall` itself propagates from the installer environment (locale/keymap/timezone, see the "Propagates installer environment settings" bullet in `ddi-installer.md`). Observed in QEMU on the installed `installer-v26.08.0` image with credentials staged on the ESP (`Received untrusted credentials: firstboot.hostname, firstboot.locale, firstboot.timezone, …` followed by the EXEC failure). Found while integrating with the [jeefy/booty](https://github.com/jeefy/booty) PXE server, which delivers `firstboot.hostname` per host. ## Root cause The unit runs `ExecStart=systemd-firstboot --force --welcome=no`, written when the OS payload was FSDK's systemd. #132 cut the payload over to Flatcar's `/usr` (`flatcar-usr.bst`), and **Flatcar builds systemd without `systemd-firstboot`**: `flatcar-container.tar.gz` for 4593.2.5 contains neither `/usr/bin/systemd-firstboot` nor `systemd-firstboot.service` (verified by listing the archive; the `ln -sf /dev/null …/systemd-firstboot.service` mask in `bluefin-server-ddi.bst` is therefore a no-op too, harmless). `installer-stack.bst` still has FSDK's systemd with `systemd-firstboot`, which is why the *installer* side works and the *target* side does not. ## Fix Replace the binary with `/usr/libexec/bluefin-firstboot-credentials`, a bash + coreutils helper (both shipped by Flatcar), staged by `os-creds-prov.bst`. It reads the five credentials from `$CREDENTIALS_DIRECTORY`, validates them at least as strictly as systemd's validators (`hostname_is_valid`-style: labels `[A-Za-z0-9]([A-Za-z0-9-]*[A-Za-z0-9])?`, no leading/trailing hyphen, ≤ 64 chars total; timezone must match `[A-Za-z0-9_+-]+(/…)*` **and** exist under `/usr/share/zoneinfo`; locale/keymap restricted to safe characters and < 128 bytes like `locale_is_valid()`/`keymap_is_valid()`; everything ASCII-only under `export LC_ALL=C`, since PID 1 exports the `LANG` this helper itself writes and `[[:alnum:]]` would otherwise widen with it; a NUL byte in a credential is rejected rather than silently dropped by `$(<file)`), and writes the same files `systemd-firstboot` would, in the same format — plus `/etc/hostname`. On `firstboot.hostname`: systemd ≤ v259 (Flatcar 4593.2.5 ships 257.9) has no such credential; systemd v260 added it as a *static* hostname honoured only when none is set yet, whereas this helper applies it unconditionally and additionally sets the live kernel hostname (systemd's own transient hostname credential remains `system.hostname`, consumed by PID 1): | credential | file | |---|---| | `firstboot.locale` / `firstboot.locale-messages` | `/etc/locale.conf` (`LANG=` / `LC_MESSAGES=`) | | `firstboot.keymap` | `/etc/vconsole.conf` (`KEYMAP=`) | | `firstboot.timezone` | `/etc/localtime -> ../usr/share/zoneinfo/<tz>` | | `firstboot.hostname` | `/etc/hostname` + live kernel hostname (`/proc/sys/kernel/hostname`) so the first DHCP request carries it — new relative to `main` (the old `ExecStartPost` set the kernel hostname but never persisted `/etc/hostname`); semantics differ from systemd v260's (always applied, not only when no static hostname exists) | Any invalid value fails the run **before** anything is written (unit fails visibly, stamp file not created; control characters in the rejected value are `%q`-quoted in the log). The helper `sync`s before exiting so the `ExecStartPost` stamp cannot outlive the files on power loss. The unit's ordering, conditions, `ImportCredential=` lines and the `/etc/.bluefin-firstboot-credentials` stamp are unchanged; the old `ExecStartPost` hostname one-liner folds into the helper. Shebang is `/usr/bin/bash` so EXEC does not depend on the tmpfiles-created `/bin` symlink. `--root=DIR` exists for tests only. Known, deliberate differences from `systemd-firstboot` (stated in the helper's header): trailing newlines in a credential are stripped rather than rejected; `LC_MESSAGES=` is written even when it equals `LANG=`; locales/keymaps are shape-checked, not checked against what is installed (Flatcar ships the full zoneinfo, so the timezone existence check is real). ### Why not ship the binary Pulling FSDK's `systemd-firstboot` into the Flatcar `/usr` was the other option and was rejected: it links `libsystemd-shared-<ver>.so` from a different systemd build, so it would drag a second libsystemd-shared into the image and reintroduce exactly the FSDK/Flatcar userspace split #132 removed. The helper is ~100 lines with no dependencies beyond what the payload already has, and follows the precedent of #246's `files/os/libexec/bluefin-root-cred-sync`. ## Verification - `tests/unit/test_credentials_provisioning.py` — unit no longer names `systemd-firstboot`, `ExecStart` points at the helper and precedes the stamp; element stages `files/os/libexec` and installs the helper 0755. - `tests/unit/test_firstboot_credentials_behavior.py` (new, 26 tests) — executes the helper against a scratch `--root`: all five credentials produce the exact files above; a subset writes only its files; `--force`-style overwrite of existing `/etc/hostname` and `/etc/localtime`; interior hyphens/digits in labels accepted; twelve invalid inputs (`bad host`, `-leading`, `trailing-`, `foo-.bar`, `foo..bar`, `hôte`, `host\r`, 65-char hostname, `../../etc/passwd` and unknown timezone, shell metacharacters in locale, `..` in keymap) exit 1 with a named error and write **nothing**; a non-ASCII **keymap/locale** (`[[:alnum:]]` regexes) is rejected with `LC_ALL=C.UTF-8`/`LANG=C.UTF-8` in the environment — this test **fails when `export LC_ALL=C` is removed from the helper** (verified by mutation; the round-1 hostname variant did not, because `[A-Za-z0-9]` does not widen under `C.UTF-8`); 127-byte keymap accepted / 128 rejected; locale-only and messages-only `locale.conf`; invalid `locale-messages`; a NUL byte; the CR case logs `$'host\r'`; empty credential treated as absent; missing and nonexistent `CREDENTIALS_DIRECTORY` are errors; `bash -n`; `shellcheck -S warning` (skipped visibly, not silently passed, when shellcheck is absent — CI's `unit-tests.yml` does not install it). - `python3 -m pytest tests/unit -q` — 371 passed, 1 xfailed (pre-existing). `test_files_payload_reachability` confirms the new `files/os/libexec/` path is staged. - `python3 .github/scripts/docs-checks.py` + the `check-*` scripts — pass. - Not run: `just validate` / `just build-ddi` (no local BuildStream) and a rebuilt-image boot. The helper has only been exercised by the executed unit tests against a scratch `--root`; a first boot with `firstboot.*` credentials on a rebuilt DDI is the remaining proof and I am happy to run it against a CI artefact. Docs: `ddi-installer.md` describes what the unit actually runs and writes (same files as `systemd-firstboot` plus the hostname handling, with the ≤ v259 / v260 distinction); `tpm2-credential-sealing.md` drops `systemd-firstboot` as a consumer of `passwd.hashed-password.root`. No relation to the installer PR stack (#250, #251, #252, #253); no textual or compose overlap with #246 (`bluefin-root-cred-sync` is installed from a separate element).
jeefy
added a commit
that referenced
this pull request
Sep 26, 2026
…e before decompressing (#251) ## Problem The `inst.ddi_url` path staged the download in `/run`. systemd PID 1 mounts `/run` as tmpfs with `size=20%` (**20 % of RAM** ; ≈1.6 GiB on an 8 GiB machine). The 26.08.0 DDI is 365 MiB compressed / 1.58 GiB raw, and `zstd --rm` removes the compressed copy only after a successful decompress, so the staging tmpfs must hold ~1.95 GiB at once. On the released `installer-v26.08.0` PXE initrd that fails on anything below ~10 GiB of RAM (1.95 GiB / 20 %): ```text zstd: error 70 : Write error : cannot write block : No space left on device ERROR: failed to decompress downloaded DDI. ``` Reproduced in QEMU (OVMF, q35/KVM) at 8 GiB with the released initrd; passes at 16 GiB. Found while integrating with the [jeefy/booty](https://github.com/jeefy/booty) PXE server. ## Fix - Stage under `/dev/shm/installer`: PID 1 mounts `/dev/shm` without a `size=` option, so it gets the **kernel tmpfs default of 50 % of RAM**. The wrapper first checks `mountpoint -q /dev/shm` so it never stages on devtmpfs. - Before decompressing, compare the raw size recorded in the zstd frame header (or 5× the compressed size when the header does not carry it — the shipped DDIs compress 2–4.5× — with a `WARN`) against **both** `df -k --output=avail` of the staging directory **and** `MemAvailable` from `/proc/meminfo` minus 256 MiB of headroom — in KiB, rounded up, so MiB truncation cannot wave through an image a few KiB too large — and abort with all the numbers instead of dying mid-write or being OOM-killed: `ERROR: not enough RAM-backed temporary space in /dev/shm/installer to decompress the DDI (need ~N MiB; tmpfs has M MiB free, MemAvailable is K MiB with 256 MiB headroom); add RAM to this machine.` The `MemAvailable` bound matters because the tmpfs cap is not proof of memory: the unpacked initrd is its own tmpfs, so `df` on `/dev/shm` can report space the kernel cannot back. `df` failure or unparsable output fails closed with a message. - Drop the pre-created `/layer/run/installer` (the wrapper creates its own staging dir). `mountpoint`, `stat`, `df`, `tail` join the Step 1a build-time tool check. Docs: `ddi-installer.md` replaces the "~8 GiB guidance" line with a *Memory requirements* section (why `/run` cannot hold the DDI, both copies must fit, the check needs `MemAvailable` after the download ≥ raw DDI + 256 MiB and the `/dev/shm` cap to hold both copies; 8 GiB tested and recommended for the 26.08.0 Flatcar-based DDI, smaller machines not boot-tested — the earlier "computed minimum ≈4 GiB" is gone because the tmpfs-only arithmetic could pass the check and still OOM), adds a Common Rationalizations row and fixes the "2 GiB+" DDI size to ≈1.6 GiB raw. ## Verification - `tests/unit/test_installer_contract.py`: - `test_installer_network_ddi_stages_in_dev_shm_with_space_preflight` — `mountpoint` guard, staging path, no `/run/installer`, preflight ordered between `sha256sum --check` and `zstd -d`, `MemAvailable` read from `DDI_MEMINFO`. - `test_installer_space_preflight_uses_the_zstd_frame_header` — runs the extracted preflight block against a real zstd frame with a stand-in right-aligned `df` and a fake `/proc/meminfo` (the block's `DDI_MEMINFO=/proc/meminfo` is overridden): a 3 MiB + 1 B raw image passes with 3073 KiB free, fails with 3072 KiB, and the message carries `need ~4 MiB; tmpfs has 3 MiB free, MemAvailable is 65536 MiB`. - `test_installer_space_preflight_is_bounded_by_mem_available` — huge tmpfs, `MemAvailable` = headroom + 3073 KiB passes, + 3072 KiB fails; a meminfo without `MemAvailable` fails closed. - `test_installer_space_preflight_falls_back_to_five_times_compressed` — non-zstd payload, 5× rule, one-KiB boundary, `WARN` emitted. - All fail against `main`'s element; the `MemAvailable` test also fails against the round-1 tmpfs-only check (verified by mutation). The tests use the shared `installer_wrapper` fixture from #250. - `python3 -m pytest tests/unit -q` — 352 passed, 1 xfailed (pre-existing). - `bash -n` + `shellcheck -S warning` on the extracted wrapper — clean. - `python3 .github/scripts/docs-checks.py` + the `check-*` scripts — pass. - Not run: `just validate` / `just build-installer` (no local BuildStream). - **QEMU** (OVMF, q35/KVM, 8 GiB, iPXE from a Booty instance): the *combined* wrapper of this stack (sed rewrite + this `/dev/shm` staging + wait-online + `inst.creds_url`) was dropped into the released `installer-v26.08.0` initrd; decompression succeeds and the install completes where the released initrd hit `ENOSPC`. The **preflight abort path was not booted** (no machine small enough was tried) — it is covered by the executed unit tests only, and the KiB-rounding change post-dates the QEMU run.
jeefy
added a commit
that referenced
this pull request
Sep 26, 2026
…#252) ## Problem The `inst.ddi_url` path guards its network wait with ```bash if command -v systemd-networkd-wait-online >/dev/null 2>&1; then systemd-networkd-wait-online --timeout=90 || { ...; exit 1; } fi ``` `systemd-networkd-wait-online` lives in `/usr/lib/systemd/`, not on `PATH`, so the guard never fires, the wait is silently skipped and `curl` races the DHCP lease. With the released `installer-v26.08.0` PXE initrd in QEMU (bridged network, KVM) the download fails ~11 s after boot: ```text curl: (7) Could not connect to server ``` Found while integrating with the [jeefy/booty](https://github.com/jeefy/booty) PXE server; on a slower DHCP server it fails every time. ## Fix Call `/usr/lib/systemd/systemd-networkd-wait-online --any --ipv4 --timeout=90` by absolute path and drop the guard. A missing binary is now a **build** failure — checked in step 1a next to the other critical tools — rather than a skipped wait at install time, so an FSDK relocation surfaces in CI, not on a PXE client. **Why `--any`.** The installer's `80-dhcp.network` matches every `en*`/`eth*`/`ens*`/`enp*` interface. Without `--any`, `systemd-networkd-wait-online` blocks while *any* managed link is still in setup state `configuring` (`wait-online-manager.c` `manager_configured()`: `-EBUSY` makes it return false, so the wait runs to `--timeout` unless `--any`), and an unplugged second NIC sits at `no-carrier configuring` forever. Now that the wait actually runs, that would abort every PXE install on a multi-NIC server with an unplugged port after 90 s. With `--any` the wait returns once one managed link is `configured`. On a single-NIC machine both forms behave identically. **Why `--ipv4`.** networkd marks a link `configured` as soon as *one* dynamic protocol finished (`link_check_ready()`), so on a LAN with router advertisements a SLAAC or DHCPv6 address makes the link configured — and `--any` satisfied — before the DHCPv4 lease, and `curl http://<IPv4>` would race again exactly as before. `--ipv4` (present since v244; FSDK 26.08 ships v261) requires the satisfying link to carry an IPv4 address, which with IPv4LL disabled in the installer's `.network` means the DHCPv4 lease. On IPv4-only LANs it changes nothing. **Why the binary and not `systemd-networkd-wait-online.service`.** Calling it from the wrapper keeps the wait conditional on `inst.ddi_url`/`inst.creds_url` (offline USB installs must not wait); the unit route would need a `ConditionKernelCommandLine=`, an `ExecStart=` drop-in for the flags and `Requires=` to abort the install on failure. ## Verification - `tests/unit/test_installer_contract.py::test_installer_network_ddi_waits_for_networkd_by_absolute_path` — no `command -v` guard, exactly one absolute-path call carrying `--any`, `--ipv4` and a `--timeout=` (matched as arguments, not as one literal), placed **inside** the network-pull block and before the `curl` download, build-time check present. Fails against `main`. Uses the shared `installer_wrapper` fixture from #250. - `python3 -m pytest tests/unit -q` — 353 passed, 1 xfailed (pre-existing). - `bash -n` + `shellcheck -S warning` on the extracted wrapper — clean. - `python3 .github/scripts/docs-checks.py` + the `check-*` scripts — pass. - Not run: `just validate` / `just build-installer` (no local BuildStream). - **QEMU** (OVMF, q35/KVM, 8 GiB, single virtio NIC, iPXE from a Booty instance): the *combined* wrapper of this stack (sed rewrite + `/dev/shm` staging + this wait + `inst.creds_url`) was dropped into the released `installer-v26.08.0` initrd; the wait blocks until the lease arrives and the download succeeds on first try. That run used the plain `--timeout=90` form; `--any` and `--ipv4` were added afterwards from the `wait-online-manager.c`/`networkd-link.c` reading above and have **not** been booted — on one IPv4-only NIC they are provably equivalent, the multi-NIC and dual-stack cases are not covered by a boot. The **timeout/abort path was not booted** either; it is asserted by the unit test only.
jeefy
added a commit
that referenced
this pull request
Sep 26, 2026
…st.creds_sha256) for PXE installs (#253) **Stacked on #252** (wait-online) ← #251 (`/dev/shm` staging) ← #250 (sed rewrite) — only the last four commits (`feat`, `docs`, `test`, `docs`) are new here; the credentials pull shares the same staging directory and network wait, so the diff shrinks to those four as the fixes merge. ## Motivation #229 made PXE installs self-contained by letting the installer pull the DDI over the LAN (`inst.ddi_url` / `inst.ddi_sha256`). Its feature request (#14 §4) left per-host configuration out of scope, naming the natural follow-up: installer-fetched systemd credentials. A PXE fleet still needs a hostname, an operator SSH key and usually a first-boot callback per machine, and the only delivery path today is an operator hand-mounting the ESP after the install. This PR closes that gap with the same shape as the DDI pull: two opt-in kernel parameters, mandatory verification, no new installer logic — the files are handed to the stock systemd credential consumers already documented in `docs/skills/tpm2-credential-sealing.md` (`firstboot.*`, `tmpfiles.extra`, `network.*`, `passwd.*`). Found while integrating with the [jeefy/booty](https://github.com/jeefy/booty) PXE server, which serves `GET /creds/<mac>.tar` per host. ## Behaviour | Parameter | Meaning | |---|---| | `inst.creds_url=<url>` | Download a **flat, uncompressed tar archive** of `systemd-creds encrypt`ed `*.cred` files (any `curl` scheme; the example uses plain `http://`) and place them in the installed system's ESP at `/loader/credentials/`. | | `inst.creds_sha256=<hex>` | **Mandatory** when `inst.creds_url` is set. Verified before anything is unpacked or written. | iPXE example (`${mac}` expanded by iPXE): ```text systemd.unit=system-install.target console=tty0 console=ttyS0,115200 rw unattended inst.ddi_url=${base}/bluefin-server-ddi-<ver>.raw.zst inst.ddi_sha256=<sha256> inst.creds_url=${base}/creds/${mac}.tar inst.creds_sha256=<sha256> ``` Archive contract: an **uncompressed** ustar/pax/gnu tar (the wrapper checks the `ustar` magic at offset 257 and feeds tar on stdin, where GNU tar refuses compressed input instead of transparently decompressing a `.tar.zst` into an unbounded listing) of at most **64** top-level regular files named `<name>.cred`, `<name>` matching `[A-Za-z0-9][A-Za-z0-9._@~-]*` (`~` is systemd's own `systemd.unit-dropin.<unit>~<name>` credential namespace), each name at most **250 bytes**, each file at most **1 MiB**, the archive at most **16 MiB**, and no name repeated even ignoring case (the ESP is vfat, where `a.cred` and `A.cred` are the same file). Any other member (directory, `./` prefix, symlink, hardlink, device, FIFO, path, space/control character/leading dash or dot in the name, non-`.cred` name, over-long or duplicate name) rejects the whole archive with a wrapper-level message before anything is extracted. Build with `tar -cf <mac>.tar -C <dir> a.cred b.cred`. Trust model: `inst.creds_sha256` arrives on the same unauthenticated PXE command line as the URL, so the checksum protects against a compromised or misconfigured *file server*, not against whoever controls DHCP/TFTP — they control both. The archive hardening is defence against operator error and a bad mirror, not a security boundary against a hostile netboot server; secrets belong in TPM2/host-key-sealed credentials. ## Guarantees - **Absent → behaviourally unchanged.** Without `inst.creds_url` the wrapper behaves as before; the one shared change is that the ESP is now located by GPT partition type instead of "first vfat on the target" (design note 1). The commit message and in-script comment now say the same (round 1 said "byte-for-byte", which was wrong). - **Fail closed on missing checksum**, same wording as the DDI check. - **Nothing touches the disk before both downloads verify, and nothing unverified is used.** The archive is fetched right after the DDI (one shared `systemd-networkd-wait-online`), capped at 16 MiB (`curl --max-filesize` — curl ≥ 8.4 aborts a transfer of unknown length once it crosses the cap; re-checked with `stat` after the checksum so the bound does not depend on the curl version), `sha256sum --check`ed, checked for the `ustar` magic, and only then listed — via stdin, so tar cannot auto-decompress. The `tar -tv` listing is validated column by column *before extraction* — type must be a regular file (no directories, symlinks, hardlinks, devices, FIFOs), size numeric and ≤ 1 MiB, date/time columns where GNU tar puts them (`LC_ALL=C`), name matching the strict pattern (which also rejects the backslash sequences `--quoting-style=escape` emits for a smuggled newline or other control character) and ≤ 250 bytes, no duplicate or case-colliding names, ≤ 64 members — and cross-checked against the plain `tar -t` listing so a column shift (e.g. an owner name with a space) is caught. Only the validated names are extracted (`--no-same-owner --no-same-permissions`), pinned to `0644`, and re-checked as regular non-symlink files. Everything lands in `/dev/shm/installer/credentials/` (tmpfs) *before* `systemd-sysinstall` runs. A bad download aborts with the target untouched. - **Placement happens after `systemd-sysinstall`** in the existing post-install ESP pass (`stage_fallback_bootloader` → `finalize_target_esp`): one `mount -t vfat` of the ESP identified by type GUID `c12a7328-f81f-11d2-ba4b-00a0c93ec93b` on the target disk, `mkdir -p loader/credentials`, `cp -- "$CREDS_DIR/$name"` for each validated name (never a glob over the extraction directory), `sync` — **first**, then the best-effort fallback-loader copy on the same mount, so a loader hiccup cannot abort the function with the mandatory step undone — then `umount`. - **Fail closed after install too.** If `inst.creds_url` was given and the ESP cannot be found or mounted, the wrapper exits 1 (→ `FailureAction=poweroff`) instead of letting an unconfigured, unreachable host reboot. - **Credentials must be `systemd-creds` encrypted.** `systemd-stub` stages ESP credentials under `/run/credentials/@encrypted`, so plaintext `*.cred` files are rejected by the consuming services (`status=243/CREDENTIALS`); null-key encrypted files (`systemd-creds --with-key=null encrypt --name=<name> - <name>.cred`) apply. The docs say so and show the commands. - **One new tool**: GNU `tar` (`components/tar.bst`, present at the pinned `freedesktop-sdk-26.08.0`; runtime deps glibc, symlinks, utf-locale) in `installer-stack.bst`, also added to the build-time tool check together with `dd`, `rm`, `chmod`, `sync` (all confirmed in the released initrd). `curl`, `sha256sum`, `stat`, `mount`, `lsblk`, `udevadm` were already there (confirmed against the released 26.08.0 initrd's file list: uutils coreutils 0.11.0, GNU grep, curl, zstd, util-linux; no `sed`/`awk`/`tar`). ## Verification - `tests/unit/test_installer_contract.py::test_installer_network_credentials_pull_contract` — parameter parsing, mandatory-checksum wording, fetch → verify → size cap → `ustar` magic → listing → unpack → `systemd-sysinstall` ordering, tar only ever reading the archive from stdin, the 16 MiB / 64 / 250-byte / 1 MiB caps, mode and name patterns, case-insensitive duplicate check, listing cross-check, 0644 pin and regular-file re-check, ESP lookup by type GUID on a single vfat mount with per-name copies (no glob) written before the loader, fail-closed message, `tar.bst` in the stack and `tar` in the build-time tool check. - `tests/unit/test_installer_creds_pull_behavior.py` (new, 29 tests) — runs the wrapper (shared `installer_wrapper` fixture) under bash: `bash -n`, `shellcheck -S warning` when installed, and the fetch/verify/validate/unpack block against real tar archives with a stand-in `curl`. Accepted: a flat archive lands in tmpfs at `0644` regardless of archive modes/owners and the download is removed; 64 members incl. a 1 MiB member (boundary); a `systemd.unit-dropin.multi-user.target~booty.cred` member; a 250-byte name. Rejected without creating the credentials directory: checksum mismatch, non-`.cred` names, directories, `./` prefixes, symlinks, hardlinks, char/block devices, FIFOs, names with spaces / newlines / colons / leading dash / leading dot, an owner field with a space (column shift), a 1 MiB + 1 B member, 65 members, a 251-byte name, a repeated name, two names differing only by case, a 17 MiB archive, a zstd-compressed tar (rejected at the magic check; GNU tar on stdin is also shown to refuse it), empty and non-tar payloads. **Not executed here**: the curl-side `--max-filesize` (the stand-in curl ignores it; the `stat` re-check is executed) and `finalize_target_esp` (string assertions only) — stated in the module docstring. Tests skip unless GNU tar is present. - `python3 -m pytest tests/unit -q` — 383 passed, 1 xfailed (pre-existing). - `bash -n` + `shellcheck -S warning` on the extracted wrapper — clean. - `python3 .github/scripts/docs-checks.py` + the `check-*` scripts — pass. - **End to end in QEMU** (OVMF, q35/KVM, 8 GiB, 20 GiB virtio disk, single NIC) with the *combined* wrapper of this stack and GNU tar dropped into the released `installer-v26.08.0` initrd, booted via iPXE from a Booty instance serving all four `inst.*` parameters: archive fetched and verified before any disk change, a 7-file `*.cred` archive accepted, `systemd-sysinstall` runs, `finalize_target_esp` stages the fallback loader and writes `/loader/credentials/*.cred` on `/dev/vda1`; on first boot the target logs `Received untrusted credentials: firstboot.hostname, firstboot.locale, firstboot.timezone, tmpfiles.extra, …`, `tmpfiles.extra` is applied and SSH as the provisioned operator works. - **Not booted**: the archive-hardening pass (size caps, column-by-column listing validation, cross-check, `--no-same-permissions`/`0644` pin, per-name copy) and the `--any` on wait-online were added *after* that QEMU run from review; the happy path is behaviourally identical for a well-formed archive (the executed unit tests exercise the exact block that ships), but the current wrapper has not been re-booted. The archive-rejection paths, the ESP-not-found/unmountable failure and the wait-online timeout are unit-tested only. - Not run: `just validate` / `just build-installer` (no local BuildStream); the tested initrd was hand-assembled from the released one. Two target-side observations from that run, both outside this PR: (a) null-key credentials are honoured by `ImportCredential=` consumers (tmpfiles, sysusers, …) but refused by PID 1 and generators (`system.hostname`, `systemd.extra-unit.*`: `Operation not supported`), so anything beyond those consumers needs host/TPM2-key encryption; (b) `bluefin-firstboot-credentials.service` fails on the released image with `Failed at step EXEC spawning systemd-firstboot`, so `firstboot.*` are no-ops there — fixed separately in `fix/firstboot-credentials-binary` (#254). ## Design notes / questions 1. **ESP lookup by partition type.** The fallback-loader pass previously took the first `vfat` partition on the target disk. I switched it to the ESP type GUID so the credentials cannot land on an unrelated vfat data partition on an install-alongside layout. Same partition in every layout the installer produces today; flagging it because it touches the non-PXE path. 2. **Mount reuse vs. separate pass.** I reused the single existing post-install mount rather than mounting twice, so the fallback loader and the credentials share one function. Happy to split it if you prefer the bootloader staging to stay strictly best-effort and isolated. 3. **`sha256sum` vs. `importctl`.** I mirrored the DDI path (`curl` + `sha256sum --check`). If you would rather move both network pulls to `importctl pull-raw --verify=checksum` with a `SHA256SUMS`, I can do that in a follow-up covering both. 4. **Encrypted-only enforcement.** The wrapper checks archive shape, not contents. Should it additionally check for the `systemd-creds` header and refuse plaintext up front? 6. **Caps.** 64 members / 1 MiB each / 16 MiB total are generous for encrypted credentials (a few KiB each) but arbitrary; happy to tune. 5. **`tar` as the only new dep.** GNU tar was the smallest thing that reads the archive; a `cpio` or a DDI-wrapped credentials volume felt heavier for a handful of small files.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Branch:
fix/installer-sed-missing. First of a stack of four installer PRs that touch the sameinst.ddi_urlblock of the wrapper: this one → #251 (/dev/shmstaging) → #252 (wait-online) → #253 (inst.creds_url). Based directly onmain.Problem
Every
inst.ddi_url(PXE) install with the releasedinstaller-v26.08.0initrd dies right after the DDI has been downloaded, verified and decompressed:exit 127 →
FailureAction=poweroff. The wrapper usessedto repointCopyBlocks=in the20-root-a.confoverride it stages under/usr/lib/repart.sysinstall.d/.installer-stack.bstships uutils coreutils, Bash,curl,zstdand systemd, andgreparrives transitively;sedis a build-depend of the installer element only and never lands in the live initrd — a mismatch #162 will make permanent when it drops it from the build too. (Confirmed against the released 26.08.0 initrd's file list:grepis present,sed/awk/tarare not.)Reproduced in QEMU (OVMF, q35/KVM, 8 GiB) booting the released PXE kernel/initrd via iPXE with
inst.ddi_url/inst.ddi_sha256; found while integrating with the jeefy/booty PXE server.Fix
Write the override with plain Bash: copy
/usr/lib/repart.d/20-root-a.confline by line, replace only the column-0CopyBlocks=line; every other line is unchanged apart from normalising a missing final newline. If the recipe carries noCopyBlocks=line the wrapper errors out before the disk is touched (recipe and wrapper ship from the same repo, so that is a repo bug, not something to paper over by appending a key). No new dependencies. Step 1a of the element now pins every external command the wrapper calls (grep curl zstd modprobe mount umount sha256sum readlink ls cp mkdir sleep systemctlnext to the existing five; all confirmed present in the released 26.08.0 initrd), so a missing tool fails the build instead of the install. Also split the onelocal x="$(...)"(SC2155) so the extracted wrapper isshellcheck -S warningclean.Docs:
ddi-installer.mdnow states the live initrd's real inventory (installer-stack.bstplus transitive runtime deps;sed/awk/tarabsent) next to the PXE flow, points at the Step 1a tool check, and adds a Common Rationalizations row.Verification
tests/unit/test_installer_contract.py:test_installer_wrapper_does_not_call_sed_awk_or_tar— asserts no command position in the wrapper namessed/awk/tar(comment stripping only at#after whitespace/BOL, so#in strings does not hide code).test_installer_build_pins_every_external_command_the_wrapper_calls— the Step 1a loop names each external command the wrapper uses.test_installer_wrapper_copyblocks_repoint_matches_the_sed_it_replaced— runs the extracted block against the shippedfiles/installer/repart.d/20-root-a.confand asserts the output equals thesedresult it replaced (expectedCopyBlocks=path derived from the wrapper'sROOT_COPYBLOCKS_LINE, so the stacked/dev/shmPR does not have to touch this test).test_installer_wrapper_copyblocks_repoint_fails_closed_without_a_copyblocks_line— a recipe with noCopyBlocks=line, and one with an indentedCopyBlocks=(deliberately not matched), both exit 1 with the wrapper's message and nothing appended.tests/unit/conftest.pygains the shared, session-scopedinstaller_wrapperfixture (carves the heredoc out of the element) that the later PRs in the stack reuse.main's element (and the fails-closed test also fails against the round-1 append fallback), pass with this branch.python3 -m pytest tests/unit -q— 348 passed, 1 xfailed (pre-existing).bash -n+shellcheck -S warningon the extracted wrapper — clean.python3 .github/scripts/docs-checks.py+ thecheck-*scripts — pass.just validate/just build-installer(no local BuildStream)./dev/shmstaging + wait-online +inst.creds_url) was dropped into the releasedinstaller-v26.08.0initrd; the install completes and the target boots. This Bash rewrite was not booted in isolation.Review round 2
Applied from the line-by-line review:
installer-stack.bstplus transitive runtime deps (the wrapper itself usesmodprobe,lsblk,mount,umount,sha256sum,readlink;greparrives transitively) — "anything else fails" dropped. The Step 1a build-time check now pins every external command the wrapper calls, so the docs' "checked at build time" is true.ROOT_COPYBLOCKS_SEEN=0append fallback replaced by an explicitERROR … has no CopyBlocks= line to repoint+exit 1, with an executed test for the missing-line and indented-key cases (indented keys are documented as not matched and hit the same error).(sed|awk|tar)regex with[\s;|&)>]|$lookahead and a whitespace-anchored comment stripper; the three literal-implementation asserts dropped;shlex.quote()on the substituted paths; session-scoped fixture; a comment on the block-slice coupling; commit body no longer saysgrepis declared in the stack or that every line is byte-identical.build(run 36259014114) passed; the new build is running.