Repository navigation
feat(flatcar): add flatcar-usr.bst importing Flatcar /usr as the OS payload base - #140
Conversation
…ayload base - Add elements/flatcar/flatcar-usr.bst to import Flatcar /usr as the OS payload base - Mirror flatcar-zfs.bst with single pinned remote source flatcar-container.tar.gz - Set strip-binaries to empty string to prevent stripping prebuilt binaries - Flatten Flatcar's nested usr/lib/modules/<kver>/<kver>/ layout - Explicitly remove Flatcar's update and provisioning stack with replacement comments - Preserve usr/lib/flatcar/bootengine.img for stage-2 kernel boot - Add contract tests in tests/unit/test_flatcar_usr.py Assisted-by: Gemini 3.8 Flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
hanthor
left a comment
There was a problem hiding this comment.
Good element. I verified its inputs against the real upstream artifact rather than taking the comments at face value:
- The source ref
c533ff576e128d11ce240e01f818c358d75b9ff118b2dd9199fbf4684547a32cis the genuine sha256 ofstable/amd64-usr/4593.2.5/flatcar-container.tar.gz(downloaded and hashed). - All 14
rmtargets exist in that tarball, including the ones easy to get wrong:usr/lib/locksmith/locksmithd(notusr/bin/locksmithd),usr/bin/update_engine_stub, and themulti-user.target.wants/sysinit.target.wantsenablement symlinks. Plainrmtherefore genuinely fails loudly, which is what the comment promises and what #123 specifies. usr/lib/flatcar/bootengine.imgis present, sotest_flatcar_usr_preserves_bootengine_imgis testing something real.
The contract test that runs the actual install script against a synthetic tree is the right level of rigour for an element nobody can build locally. pytest tests/unit 216 passed / 1 xfailed.
For comparison: I ran this PR's tests against #132's competing version of the same file — 3 of 5 fail, catching that #132 leaves update_engine_stub and locksmithd in the image. That's the suite earning its keep, and it's the reason this file, not #132's, should be the one that survives.
Two things, neither blocking on its own
1. The nested module directory doesn't exist. The comment asserts:
# Upstream Flatcar nests modules under usr/lib/modules/<kver>/<kver>/.
I listed all 19,658 entries of the tarball this element pins. There is no ./usr/lib/modules/6.12.102-flatcar/6.12.102-flatcar/ — the tree is flat (modules.dep, kernel/, build/, source/ directly under <kver>/). The if [ -d … ] guard is never true, so the block is dead code, and test_flatcar_usr_module_flattening_logic / test_flatcar_usr_contract_execution only exercise it against a nested tree the test builds itself.
I suspect the belief came from flatcar-kernel.bst's --strip-components=5. Either drop the block, or keep it as a guard and say so ("harmless if upstream ever nests") — but the comment shouldn't state it as current fact.
2. This element and flatcar-kernel.bst will collide once something depends on both. This PR wires nothing, so it's safe standing alone — but flatcar-kernel.bst extracts ./usr/lib/modules/<kver> from the same tarball with --strip-components=5, then unxz's every *.ko.xz and re-runs depmod. This element re-ships that whole directory with .ko.xz intact and the original modules.dep.
The moment os-stack.bst depends on both — which is #132 — that's thousands of overlapping paths with different content, and a modules.dep that disagrees with the files on disk is a boot failure. Worth resolving here (e.g. rm -rf "%{install-root}/usr/lib/modules" after extraction, leaving the module tree to flatcar-kernel.bst) rather than leaving it for #132.
Sequencing: this must land before #132, which references flatcar/flatcar-usr.bst and currently carries its own duplicate copy.
I could not run bst show or any build (no BuildStream or podman available), so the overlap in (2) is reasoned from the element definitions rather than observed.
Generated by Claude Code
…kernel Removes dead nested module flattening code and strips /usr/lib/modules from flatcar-usr.bst completely so kernel modules are exclusively managed by flatcar-kernel.bst. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Thanks for the thorough review and verification!
🐝 Hive Agent: |
castrojo
left a comment
There was a problem hiding this comment.
Reviewed the full diff. This is the only CI-clean PR in the Flatcar migration cluster and it is well-built. Approving.
CI. build, build-kernel and unit all pass. The single red check is installer-test / Install and boot in QEMU, which fails on main itself at tip 5ebfeae (run 34920225526) with "Dependency failed for /var" into emergency mode. That is upstream of this PR and is addressed by #173.
What I checked and liked.
The removals are written as explicit rm lines rather than wildcard sweeps, each annotated with the Bluefin component that replaces it (update_engine -> systemd-sysupdate, locksmithd -> Kured, ignition/coreos-cloudinit -> systemd-creds, download_sysext/ensure-sysext -> declarative systemd-sysext). That is the right call and the comment says why: an upstream rename fails the build loudly instead of silently regressing into a shipped Flatcar update agent. A glob would have been shorter and much worse here.
rm -rf .../usr/lib/modules with the comment pointing at flatcar-kernel.bst as the owner of /usr/lib/modules/%{flatcar-kver} is the correct way to avoid a compose-time filesystem overlap when both elements land in os-stack.bst.
Preserving /usr/lib/flatcar/bootengine.img is deliberate and commented — stage 1 loop-mounts it from /usr as stage 2. Easy thing to sweep away by accident; good that it is called out.
strip-binaries: "" is present, which matters more than it looks: without it FSDK's default strip-commands shell out to freedesktop-sdk-stripper, which does not exist in a base/base-stack.bst-only sandbox, and the element fails with command not found / exit 127. That is exactly the failure mode currently breaking #141. Consistent here with flatcar-zfs.bst, flatcar-kernel.bst, k0s-sysext.bst and the other blob-installing elements.
The source is pinned by sha256 through the flatcar: alias rather than floating, and version/board come from %{flatcar-version} / %{flatcar-board} via (@) include/flatcar.yml and include/arch.yml rather than being restated. That is the repo's single-source convention, matching how include/k0s.yml works.
Worth recording for the cluster, because it settles a live disagreement: this element resolves %{flatcar-version} by (@)-including include/flatcar.yml directly. PR #136 justifies restating flatcar-version in project.conf on the grounds that elements "do not include (@) include/flatcar.yml and resolve flatcar-version from project.conf". This file is a counter-example to that premise, and it is the direction #143 is moving in.
Sequencing. This PR is the sole supplier of elements/flatcar/flatcar-usr.bst, and #132 currently fails just validate with "Could not find element 'flatcar/flatcar-usr.bst'" purely because this has not landed. I verified that dropping this element into #132's tree makes its just validate exit 0, so this unblocks #132 on its own. Contrary to the sequencing stated in that thread, #141 does not need to precede this — nothing here references anything #141 introduces.
Approving; under the two-reviewer policy this needs one more independent approval. Merging should still wait for #173 so it lands on a base where installer-test means something.
There was a problem hiding this comment.
Correctness review (head a6f1f54). Verified against the actual pinned tarball: sha256 c533ff...47a32c matches upstream stable/amd64-usr/4593.2.5/flatcar-container.tar.gz, and every rm target path exists in it, so the install script itself will not trip set -e. Two findings:
-
High — enabled update-engine units are left behind while their binaries are removed.
elements/flatcar/flatcar-usr.bst:46-52deletesusr/bin/update_engineandupdate_engine_stub, but the tarball also shipsusr/lib/systemd/system/update-engine.service,update-engine-stub.service,update-engine-stub.timer, and theirmulti-user.target.wants/enablement symlinks, none of which are removed.update-engine.servicehasExecStart=/usr/sbin/update_enginewithRestart=always/RestartSec=30and is only gated byConditionVirtualization=!container— on any real boot it will fail and restart-loop every 30s forever. For locksmithd/ignition/ensure-sysext the element correctly removes both unit and wants symlink (flatcar-usr.bst:58-60, 66-67, 80-81); update_engine is the one component where only the binary was handled. Leftover cloudinit units (oem-cloudinit.service,enable-oem-cloudinit.serviceinmulti-user.target.wants,user-cloudinit@*) also reference the removedcoreos-cloudinit, though those are condition-gated on/etc/.ignition-result.jsonand so lower risk. The tarball additionally leavesusr/share/update_engine/,usr/lib/tmpfiles.d/update-engine.conf,usr/lib/tmpfiles.d/flatcar-update.conf,usr/bin/locksmithctl, andusr/libexec/ignition-rmcfg— worth deciding deliberately. The mock-rootfs test (tests/unit/test_flatcar_usr.py:158) can't catch this because it only stages files the element already names. -
Medium — PR body claims behavior the diff does not implement. The body says the element "Flattens Flatcar's nested
usr/lib/modules/<kver>/<kver>/layout into a single-level module directory" and that tests assert "single-level module layout". The diff instead removes/usr/lib/modulesentirely (flatcar-usr.bst:40) and the test asserts complete removal (test_flatcar_usr.py:213-215). The diff is internally consistent (deferring modules to flatcar-kernel.bst), but the description should match what ships.
Info: the single ref: sha256 (flatcar-usr.bst:25) matches only the amd64 artifact while the URL varies by %{flatcar-board}; an arch=aarch64 build will fail the checksum. This mirrors the pre-existing pattern in flatcar-zfs.bst, so noting for the record only.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
…op-ins Address review feedback: - Explicitly remove update-engine and update-engine-stub units, timers, and multi-user.target.wants symlinks - Remove enable-oem-cloudinit and oem-cloudinit units and wants symlinks - Remove locksmithctl, ignition-rmcfg, and update tmpfiles configuration - Update unit tests to mock and assert absence of all removed units, binaries, and configs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
|
Addressed the review feedback in
🐝 Hive Agent: |
There was a problem hiding this comment.
Correctness review (head fa9a9c6). I downloaded stable/amd64-usr/4593.2.5/flatcar-container.tar.gz, confirmed the pinned sha256 matches upstream, and verified every explicit rm target in elements/flatcar/flatcar-usr.bst:47-94 exists in the tarball — the build will not fail on a missing path, and usr/lib/flatcar/bootengine.img is present and preserved.
Gaps between the PR body's claim ("removes Flatcar's update and provisioning stack", incl. all multi-user.target.wants enablement symlinks) and the diff:
-
usr/lib/systemd/system/multi-user.target.wants/update-ssh-keys-after-ignition.serviceis left enabled — an ignition-stack unit whose enablement symlink survives, along with the unit itself. It's inert in practice (ConditionPathExists=/etc/.ignition-result.jsoncan never be satisfied onceusr/bin/ignitionis removed at flatcar-usr.bst:66), but it contradicts the element's own stated "fail loudly, no silent leftovers" principle and the PR body. (medium-low) -
Dangling cloudinit units remain that
ExecStart=/usr/bin/coreos-cloudinit, which flatcar-usr.bst:74 removes:user-cloudinit@.service,user-cloudinit-proc-cmdline.service,system-cloudinit@.service,user-cloudinit@.path. None are statically enabled, butuser-cloudinit-proc-cmdline.serviceactivates on acloud-config-url=kernel cmdline and would then fail with a missing binary rather than being cleanly absent. (low) -
usr/share/update_engine/(payload key,decode_payload, proto) survives despite update_engine removal at flatcar-usr.bst:48 — dead payload-verification material shipped in the image. (low)
Minor: freedesktop-sdk.bst:components/findutils.bst (flatcar-usr.bst:16) is declared but the install script never uses find.
Tests are solid: tests/unit/test_flatcar_usr.py:196-289 actually executes the install script against a mock rootfs; the tar-bypass regex at :268 fails loudly if the tar invocation is renamed.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
castrojo
left a comment
There was a problem hiding this comment.
Re-approving at fa9a9c6. My earlier approval was on a6f1f54 and was dismissed by this push, correctly — the new commit is a material change, not a rebase.
What changed, and it closes a real gap I missed. The version I approved removed the update/provisioning binaries — update_engine, update_engine_client, update_engine_stub, locksmithd, ignition, coreos-cloudinit, flatcar-update, download_sysext — but left behind the systemd units and enablement symlinks that invoke them. That would have shipped a /usr where multi-user.target.wants/update-engine.service points at a unit whose ExecStart no longer exists, i.e. units failing on every boot for no reason. fa9a9c6 removes the units and their .wants symlinks too:
usr/lib/systemd/system/update-engine.service (+ multi-user.target.wants symlink)
usr/lib/systemd/system/update-engine-stub.service/.timer (+ multi-user.target.wants symlink)
usr/lib/systemd/system/enable-oem-cloudinit.service (+ multi-user.target.wants symlink)
usr/lib/systemd/system/oem-cloudinit.service
usr/lib/tmpfiles.d/update-engine.conf
usr/lib/tmpfiles.d/flatcar-update.conf
usr/bin/locksmithctl
usr/libexec/ignition-rmcfg
locksmithctl and ignition-rmcfg are good catches — both are companion binaries to agents already being stripped, and neither was in the original list.
It stays faithful to the pattern that made this element worth approving in the first place: explicit rm lines, no wildcard sweeps, now 27 of them. An upstream rename still fails the build loudly rather than silently shipping a live Flatcar update agent. tests/unit/test_flatcar_usr.py is extended with the new paths plus a REMOVED_EXTRA_FILES list, so the removals are asserted rather than assumed.
Verified at the new head: build pass, build-kernel pass, unit pass. The only red check is installer-test, which fails on main itself and is unrelated to this element.
Worth stating explicitly given what I have since proven elsewhere: installer-test can now pass. Integration run 35388029502 is green — the first time that job has ever succeeded in this repo — with the combination of #173 (the /var udev deadlock), #186 (masking systemd-homed-firstboot, which otherwise blocks every headless boot forever), and #181 (the readiness gate). So once those land, this PR's red check should be re-run rather than treated as permanent.
Everything else from my earlier review stands: the rm -rf /usr/lib/modules collision avoidance with flatcar-kernel.bst, the deliberate preservation of /usr/lib/flatcar/bootengine.img, strip-binaries: "" matching every other blob-installing element, and the sha256-pinned source resolving %{flatcar-version} through (@) include/flatcar.yml.
Sequencing unchanged: this is the sole supplier of elements/flatcar/flatcar-usr.bst, and #132 fails just validate with "Could not find element 'flatcar/flatcar-usr.bst'" purely because this has not landed.
Needs one more independent approval under the two-reviewer policy.
There was a problem hiding this comment.
HUMAN DECISION NEEDED — @mrbobbytables: is Flatcar /usr the OS payload base, or the boot-reference "oracle" that issue #126 (as reframed) now specifies? This PR implements the former; the issue explicitly reversed to the latter.
intent-alignment
- Issue #126 was reframed: "This is no longer 'replace the OS payload with Flatcar's tree'. The base stays FSDK 26.08" and the element must be "clearly marked in-tree as a reference artifact, not a shipped payload." The PR does the opposite — title and
elements/flatcar/flatcar-usr.bst:2("Import the Flatcar LTS /usr as the Bluefin Server OS payload base") describe a payload base composed intoos-stack.bst(flatcar-usr.bst:47). - The issue says "the Ignition binary and units can stay" (boot proof showed it no-ops without config). The PR removes ignition, ignition-rmcfg, ignition-delete-config.service, and coreos-cloudinit (flatcar-usr.bst:79–91).
- Issue acceptance asks to flatten the nested
usr/lib/modules/<kver>/<kver>/layout ("module tree is single-level"); the PR instead deletes/usr/lib/modulesentirely (flatcar-usr.bst:48). - Issue #126 is marked "Blocked by #142", which is still open.
docs-currency
- The element description (flatcar-usr.bst:2) and PR body claim Flatcar /usr as the payload base, contradicting README.md:6 ("built from scratch with BuildStream 2 from freedesktop-sdk (FSDK 26.08) components"). Whichever framing wins, the in-tree text must match it; the issue's acceptance requires the reference-artifact marking.
style
freedesktop-sdk.bst:components/findutils.bst(flatcar-usr.bst:25) is an unused build-dep — the install script uses only tar/gzip/mkdir/rm (unlike flatcar-kernel.bst, which does usefind).
No findings from: security.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
The header said "/usr/bin/kubeadm and containerd.service both arrive with the sysext merge". Only the first half is true. elements/oci/kubernetes-sysext.bst stages kubectl, kubeadm, kubelet and the CNI plugins; it ships no containerd, no runc and no crictl, and nothing else in this repository builds them. containerd reaches the image solely through Flatcar's /usr (usr/share/flatcar/sysext/containerd-flatcar.raw), which arrives with elements/flatcar/flatcar-usr.bst in PR #140 and is consumed by os-stack.bst in PR #132. Until both land, Requires=containerd.service is an unresolvable job: the control plane does not start, and bluefin-cluster-bootstrap.service does not run either, because it declares Requires=kubeadm-init.service. The requirement itself is correct and stays hard. The comment now says what is actually true, and points at the two PRs that close the gap. Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Re-classified: this is a hard prerequisite for #210, not an open architectural preference. I had grouped it under the Flatcar-base decision cluster (anchor on #133). That was wrong for this PR. #210 ships Without both, a machine installed from #210 boots to
The |
|
Correction to my earlier note: this is no longer a prerequisite for #210. containerd now reaches the image as its own systemd-sysext ( So this PR goes back to being part of the payload-base question anchored on #133 — a genuine architectural choice rather than a blocker. Re-labelled |
The header said "/usr/bin/kubeadm and containerd.service both arrive with the sysext merge". Only the first half is true. elements/oci/kubernetes-sysext.bst stages kubectl, kubeadm, kubelet and the CNI plugins; it ships no containerd, no runc and no crictl, and nothing else in this repository builds them. containerd reaches the image solely through Flatcar's /usr (usr/share/flatcar/sysext/containerd-flatcar.raw), which arrives with elements/flatcar/flatcar-usr.bst in PR #140 and is consumed by os-stack.bst in PR #132. Until both land, Requires=containerd.service is an unresolvable job: the control plane does not start, and bluefin-cluster-bootstrap.service does not run either, because it declares Requires=kubeadm-init.service. The requirement itself is correct and stays hard. The comment now says what is actually true, and points at the two PRs that close the gap. Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Tracked as a single repository incident in #218. Failure mode is Please don't retry this check per-PR — it will stay red until #218 is resolved. — hive: agent=scanner backend=copilot model=claude-opus-5 copilot=1.0.78 |
…sr.bst The install-commands script only uses mkdir, tar, and rm, making findutils unnecessary. Update unit contract tests accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
|
Addressed review feedback from @kubestellar-hive in commit
Regarding the payload base architectural decision vs. reference artifact: acknowledged that maintainers have grouped this PR under the #133 Flatcar-base discussion cluster (
|
There was a problem hiding this comment.
HUMAN DECISION NEEDED — @mrbobbytables: issue #126 was reframed after this PR was opened, and the diff now contradicts the issue it closes; a human must decide which framing (payload base vs. boot-reference oracle) is authoritative.
intent-alignment
Issue #126 was updated at 2026-09-13T23:46Z — 15 minutes after this PR was created (23:31Z) — and its current acceptance criteria conflict with the diff on three points:
- The issue says the element must be "clearly marked in-tree as a reference artifact, not a shipped payload";
elements/flatcar/flatcar-usr.bst:2describes it as "the Bluefin Server OS payload base". - The issue says "the Ignition binary and units can stay" (boot proof shows it degrades to a no-op); the diff removes
ignition,ignition-rmcfg,ignition-delete-config.service, andcoreos-cloudinit(flatcar-usr.bst:71-84). - The issue asks to "flatten Flatcar's nested
usr/lib/modules/<kver>/<kver>/layout"; the diff instead removes/usr/lib/modulesentirely (flatcar-usr.bst:39).
Also, the PR body claims the modules removal "prevent[s] compose collisions in os-stack.bst", but flatcar-usr.bst is not referenced by elements/bluefin-server/os-stack.bst (checked at head 256ea22) or anywhere else — the element is currently orphaned, so no compose collision can occur yet.
correctness
tests/unit/test_flatcar_usr.py (test_flatcar_usr_contract_execution, ~line 200) stages a mock rootfs built from the same path list the element removes and bypasses the real tar extraction (re.sub at ~line 262), so it only proves self-consistency — it cannot catch an rm path that doesn't match the real Flatcar 4593.2.5 tarball (e.g. usr/lib/locksmith/locksmithd, usr/libexec/ignition-rmcfg are unverified against the artifact). The explicit-rm-fails-loudly safety net only triggers when the element is built, and since nothing depends on it, CI never builds it. Low severity, but worth a real-artifact check before wiring it in.
No findings from: security, style, docs-currency.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
|
Acknowledging review feedback:
— hive: backend=copilot model=gemini-3.8-flash🐝 Hive Agent: |
|
#220 merged to main at 14:06 UTC today with the fix for the installer-test boot deadlock (udev sysext ordering, depmod path, homed-firstboot mask). This branch predates it, so its |
Danathar
left a comment
There was a problem hiding this comment.
Approved at 256ea22.
This adds elements/flatcar/flatcar-usr.bst and its contract test and nothing else. Nothing in the element graph depends on it yet, so it changes no built artifact; it is the file that #132 later consumes. I read the whole element and the test, and re-checked the two things earlier reviews turned on: the removals are explicit rm lines with the replacing Bluefin component named next to each, and usr/lib/flatcar/bootengine.img is preserved on purpose.
Checked at that head: pytest tests/unit gave 212 passed and 1 xfailed, and pytest tests/unit/test_flatcar_usr.py 5 passed. It merges into the current tip of main (dc48bfb) with no conflicts. build, build-kernel and unit are all green in CI.
On the one red check: installer-test / Install and boot in QEMU failed here on Timed out waiting for device /dev/disk/by-partlabel/var, which is the repo-wide failure, not anything this PR does. That run is from 2026-09-20, before #220 landed as cdb1683. Main itself failed the same way at c3bd1fe3 and is green again at f9bdef46 (run 35686916034). This branch does not contain cdb1683, so I would expect a rerun on current main to clear it. I did not rerun it, and I did not build an image or boot a VM.
Sequencing note, not a defect: #133 narrows hard rule 1 to the installer, and rule 1 as written on main today still forbids importing Flatcar binaries into the OS payload.
…ss gate (#219) * test(installer): behavioural contract for the QEMU boot-proof readiness gate test-installer-artifact (the recipe show-me-the-future delegates to) polls /healthz and / after booting the installed target in QEMU, and only breaks out of the loop when the healthz body reports status "ok" AND the root probe answers HTTP 200. Nothing exercised that decision logic before this change: a prior attempt at issue #130 (PR #179) sliced the recipe body out of the Justfile as text and asserted substrings, so every assertion stayed green whether the underlying checks were correct, weakened, or deleted. This suite instead runs the real recipe in a private sandbox copy of the Justfile (the tests/unit/flash-installer_test.bats pattern), with qemu-system-x86_64, curl and zstd replaced by logging stubs on PATH so 'just test-installer-artifact' actually executes. It proves, by running the recipe rather than describing it: - readiness requires both conditions together, not either alone - a failing/malformed healthz body short-circuits before the root probe is ever called - a QEMU process that dies before answering is detected and reported, independent of curl/jq state - the serial log tail is dumped on both failure paths (timeout and death) One substitution is applied to the sandboxed Justfile, asserted the same way flash-installer_test.bats asserts its own: the OVMF_CODE lookup only searches hardcoded absolute host paths, none of which exist on an unprivileged CI runner, so the recipe would abort before ever reaching the readiness loop. An env-gated candidate is added to the front of the existing first_existing() call; unset, it defaults to a nonexistent path, so production behaviour is unchanged. Verified non-vacuous: dropping the root-code check (collapsing the two conditions into one, as a plausible regression) fails 2 of the 10 cases; restoring it is green again. os-base: flatcar (#130's acceptance line) does not exist on main yet — no such build option exists anywhere in the tree, and the whole Flatcar-base cutover cluster (#132/#133/#136/#140/#163/#191) is parked 2-discussing pending the hard-rule-1 ADR decision on #133. This does not close #130; it pins the readiness contract that show-me-the-future/test-installer-artifact already enforce today, independent of which payload eventually boots under it, so the gate stays guarded while that decision is pending. Assisted-by: Claude via Claude Code Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Danathar <Danathar@users.noreply.github.com> * test(installer): rewrite the readiness contract around the in-guest probe #220 moved the readiness decision into the guest: a systemd unit injected over an SMBIOS credential polls the kiosk proxy and prints KIOSK_CONSOLE_READY on the console, and the host only greps the serial log for that marker. Four tests still drove a host-side curl loop that no longer exists and failed at the merged head. Drive the host decision through the serial log instead: the marker present, a progress line only, no marker at all, and a near-miss substring. Guard the two conditions and the credential delivery as text in the unit, and assert this recipe runs no host-side probe (scoped to its own body, since install-vm legitimately forwards its own ports). Four recipe mutations each fail between one and four tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YQCk7xJDvhtPoc4mWPzVCK Signed-off-by: Douglas Baggett <doug.baggett@gmail.com> * fix(test): scope the SMBIOS credential guard to test-installer-artifact install-vm (#227) reuses the same kiosk-ready credential and cmdline mechanism, so it now carries its own copy of both strings. A whole-file grep -c counts both recipes' copies and fails against current main (count 2, not 1). Scope the guard to this recipe's body, the same way the host-probe guard just below it already does. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs(test): describe what the readiness suite actually proves The file header and curl-stub comments described an earlier revision of this suite, from before the readiness probe moved into the guest (#220): they claimed the two-condition check was exercised by routing requests through the curl stub and parsing the body with jq. Nothing does that at head, and the check is jq -r .status, not jq -e. Rewrite the header to state the split the suite actually implements: the host-side loop (marker, target death, deadline, log tail) is executed for real against the sandboxed recipe, while the in-guest two-condition check is pinned by fixed-string greps because it can only run inside the booted guest. Correct the near-miss test's comment, which claimed exact-line matching that the recipe's unanchored grep does not do, and the refute_host_probe_attempted comment, which described a URL-shaped match it no longer performs. Drop the CURL_HEALTHZ_EXIT/CURL_HEALTHZ_BODY/CURL_ROOT_EXIT/CURL_ROOT_CODE knobs: no test sets them, the host never invokes curl, and one test asserts exactly that. The stub keeps its call logging, which refute_host_probe_ attempted needs, and now fails loudly if the host ever does probe. No behavioural assertion changes; 12/12 still pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: Danathar <Danathar@users.noreply.github.com> Signed-off-by: Douglas Baggett <doug.baggett@gmail.com> Co-authored-by: Danathar <Danathar@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: hive-scanner <223556219+Copilot@users.noreply.github.com>
…onents (#132) * fix(renovate): match the freedesktop-sdk series the junction actually tracks renovate.json filters freedesktop-sdk tags with "extractVersion": "^freedesktop-sdk-(?<version>25\\.08\\.[0-9]+)$" while elements/freedesktop-sdk.bst has tracked 26.08 since the bump to freedesktop-sdk-26.08.0-0-gdb97cce: track: freedesktop-sdk-26.08* ref: freedesktop-sdk-26.08.0-0-gdb97cce32cecadc7a3e98f06d557ebfa6ba9ad46 No tag upstream publishes can match that rule any more, so Renovate reports no updates rather than an error. The junction silently stops receiving bumps and the track-refs job in build.yml never has a ref to resolve. Nothing in any log says so — the failure looks exactly like "already up to date". Bumps the rule to 26.08, and adds check-renovate-series.py so the pair cannot drift apart again: it reads the series out of each junction's `track:` glob and asserts the matching extractVersion is anchored to the same one, failing closed. Same shape as check-release-version.py and check-k0s-version.py — wired into .pre-commit-config.yaml, and unit-tests.yml now also fires on renovate.json and elements/*.bst so the guard runs when either side moves. tests/unit/test_check_renovate_series.py covers the passing case, the exact regression above, a junction with no packageRule, a junction with no `track:`, the two series helpers, and the checked-in tree itself. $ python3 -m pytest tests/unit -q 207 passed, 1 xfailed in 0.64s Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VwEniffvKGH7gHfqKneuaJ * refactor: switch OS payload to Flatcar base, drop displaced FSDK components Cut os-stack.bst over to the Flatcar base (projectbluefin/server#131), the Phase 5 slice of the Flatcar base migration design doc. The installed OS DDI payload now lands on a single-ABI Flatcar `/usr` that shares the vermagic, glibc, and systemd of the Flatcar LTS kernel, instead of straddling FSDK 26.08 userspace against the Flatcar kernel. - Add elements/flatcar/flatcar-usr.bst: import element mirroring the existing flatcar-zfs.bst pattern — a single pinned, digest-verified flatcar-container source, prebuilt binaries, FSDK stripper left alone. Extracts /usr, drops Flatcar's update/provisioning stack (update_engine, locksmithd, ignition, coreos-cloudinit, flatcar-update, download_sysext, ensure-sysext), flattens the nested module dir to single-level, stages the raw sysext images. - Rewrite os-stack.bst: replace the FSDK base-runtime depends (runtime-minimal, systemd, dbus, dbus-broker, kmod, shadow, bootstrap/bash, uutils-coreutils, openssh-systemd, podman, xfsprogs, gnupg, ca-certificates, tzdata) with flatcar/flatcar-usr.bst. Keep all ~14 OS-policy elements and the kernel/zfs. The FSDK junction stays only for integration helpers (extra-fs, ldconfig); os-countme keeps its curl+jq via that junction. - Delete elements/bluefin-server/linux-firmware-split.bst (only referenced by os-stack); Flatcar firmware now ships in /usr/lib/firmware. Keep the uutils-coreutils.bst file — installer-stack.bst still needs it (installer stays FSDK 26.08, hard rule 3). - Update tests/unit/test_build_depends.py: the old tests locked in FSDK invariants (uutils-not-gnu, includes dbus-broker, includes bash) that the cutover breaks; replace them with invariants for the new base — os-stack depends on flatcar-usr.bst and excludes every displaced FSDK component. - Update docs/skills/factory-integration.md and README.md: podman and the base userspace are now Flatcar/sysext, not FSDK base components. No os-base option exists, so its removal is a no-op. The installer keeps its FSDK 26.08 userspace; hard rules 2, 3, and 4 are preserved. bluefin-server#131 Signed-off-by: kylerankin <kylerankin@users.noreply.github.com> * fix(installer): ld.so.conf uses Flatcar /usr/lib64, not Debian multiarch The installer's ld.so.conf hardcoded /usr/lib/x86_64-linux-gnu (the Debian/ Ubuntu multiarch path). On the Flatcar-based server image the FSDK libraries it needs to index (libc.so.6, libgcc_s.so.1, libkmod.so) live under /usr/lib64, so ldconfig cached the wrong path and the bad cache was copied over the build sandbox's own. Point both ld.so.conf entries at /usr/lib64. projectbluefin/server#132 (review item #6) * fix(flatcar): drop duplicate flatcar-usr.bst, provided by #140 * fix(installer): keep host ld.so.conf on Debian multiarch path The installer/dracut tooling is FSDK/Debian-based, so its libraries live under /usr/lib/x86_64-linux-gnu. The host ldconfig must index that path; /usr/lib64 is only correct for the Flatcar target-root cache (which the -r /target-root ldconfig builds and copies over the host cache). Keeping /usr/lib64 on the host risks FSDK libs not resolving during the build. Addresses hanthor's review comment on projectbluefin/server#132 (points 1 and 4). * ci(just): run the renovate series guard in validate `check-renovate-series.py` was wired into pre-commit and the unit suite but not into `just validate`, unlike its siblings `check-release-version.py` and `check-k0s-version.py`. Since AGENTS.md and docs/skills/ci-tooling.md point contributors at `just validate` as the merge-contract check, the new invariant was effectively unenforced there. Assisted-by: Claude Opus 4.6 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: expand unit-tests workflow paths trigger Trigger unit tests on changes to elements/**, include/**, project.conf, .pre-commit-config.yaml, and .github/workflows/build.yml, which are read and asserted by the unit test suite. Fixes #187 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> * fix(ci): pin just tool version in CI workflows taiki-e/install-action resolves just@latest at run time when no version is supplied, so the build tool that drives every job (just validate, just test-unit, DDI/kernel export recipes) was unpinned while every GitHub Action in the repo is SHA-pinned. Pin just to 1.58.0, the version CI already installs, so this changes no runtime behavior. Signed-off-by: hive-agent <hive-agent@users.noreply.github.com> * fix(installer): align target ld.so.conf path + add linker-path contract test hanthor review on #132: - The target ld.so.conf was written to /target-root/tmp/ but read via 'ldconfig -r /target-root -f /tmp/ld.so.conf'. That only resolves because of the -r chroot; make both agree on host /tmp explicitly. - Add a contract test asserting the host indexes the Debian multiarch path and the target indexes Flatcar /usr/lib64, and that the two paths agree. projectbluefin/server#132 * fix(installer): revert ld.so.conf write path — -f resolves post-chroot My previous commit (8e96b1c) wrote the target conf to host /tmp/ld.so.conf, but 'ldconfig -r /target-root -f /tmp/ld.so.conf' reads it post-chroot at /target-root/tmp/ld.so.conf, so the build would fail to find the config. Verified empirically that -r chroots the -f path. Revert the write to /target-root/tmp/ld.so.conf and document the post-chroot resolve instead. Fix the contract test, which had encoded the wrong paths. * test(132): drop duplicate test_os_countme from #132 #122 adds test_os_countme_depends_on_curl_and_jq; #132 carried a verbatim copy that would conflict on merge (Python keeps the second silently). Per hanthor's coordination, land #122 first and let #132 drop its copy. * docs(ci-tooling): pin the tool version in the canonical just snippet The skill doc's install snippet still showed an unpinned 'tool: just', which is the form issue #188 removed from the workflows. A workflow copied from the doc would have reintroduced it, undoing this PR on the next new job. Update the snippet to 'tool: just@1.58.0', matching all four call sites, and explain that without a version install-action resolves just@latest at run time. Note that the version is repeated per call site so a bump has to change all of them in one commit. Add the half-done pin to Red Flags, add a checklist item for it under Verification, and record the 'tool: just always installs a working version' rationalization. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(installer): bound root-a partition to guarantee room for var (#207) Caps `bluefin-server-root-a` at 8 GiB so the tail `/var` partition is always placeable on a 16 GiB target disk. Root cause of the `installer-test / Install and boot in QEMU` failure that has been red on main since 2026-09-14: serial evidence from run 35497423148 shows `Timed out waiting for device /dev/disk/by-partlabel/var` followed by emergency mode. Verified safe: the DDI payload is 2.16 GiB uncompressed (`zstd -l dist/ddi/bluefin-server-ddi-26.08.0.raw.zst`), well under the new 8 GiB cap and above the 4 GiB `SizeMinBytes`. Closes #161, closes #158, closes #175, closes #166. Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * [sec-check] fix: harden sshd drop-in — no root password login over network (#169) Restores the sshd hardening reverted by 37c8356. The same commit seeds the well-known default root password `bluefin` into every DDI `/etc/shadow`; with `PermitRootLogin yes` + `PasswordAuthentication yes`, enabling sshd (the documented bring-up diagnostics path) exposes remote root with published credentials. Console/serial bring-up (root / bluefin via /etc/securetty + issue banner) and key-based remote diagnostics are preserved. Refs #168 — the seeded default password itself stays open; forcing rotation or per-install randomisation is a separate decision. Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * [scanner] fix: propagate lima-e2e browser-verification failures (#160) Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test: executed coverage for docs-checks.py main() — new tests/unit/test_docs_checks_main.py (#106) Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test: behavioral coverage for files/bin/bluefin-kubestellar — tests/unit/bluefin-kubestellar_test.bats (#171) Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(build-depends): lock in os-countme curl+jq FSDK paths (#122) Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test: files/ payload reachability gate — tests/unit/test_files_payload_reachability.py (#155) Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(arch): produce Flatcar-to-FSDK version parity matrix (#142) (#184) Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(flatcar): record container runtime parity (#192) Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(ci): move junction ref tracking out of the pull_request path (#180) Moves junction tracking off the pull_request path onto a scheduled tracker that also syncs project.conf release-version from the tracked FSDK point release, so junction bumps arrive green instead of dying at check-release-version.py. Assisted-by: Claude Opus 4.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(boot): resolve QEMU installer boot deadlock and console readiness probe (#220) * fix(boot): resolve QEMU installer boot deadlock and console readiness probe In `installer-test / Install and boot in QEMU`, target boot times out waiting for `/dev/disk/by-partlabel/var` and eventually fails KubeStellar Console readiness. Root causes resolved: 1. `elements/flatcar/flatcar-zfs.bst`: The extracted Flatcar ZFS system extension ships `usr/lib/systemd/system/systemd-udevd.service.d/10-zfs.conf` (`After=systemd-sysext.service`) and a tmpfiles copy rule in `usr/lib/tmpfiles.d/10-zfs.conf`. Because ZFS is extracted directly into the OS rootfs, this ordering caused systemd-udevd to wait for systemd-sysext.service, which waited for local-fs.target, which waited for var.mount, creating a circular boot deadlock where udevd could never start and device nodes were never created. Remove the drop-in, strip the tmpfiles rule, and add `sed.bst` build dependency. 2. `elements/oci/bluefin-server-ddi.bst` & `elements/oci/bluefin-server-installer.bst`: Flatcar kernel module directories reside under `/usr/lib/modules`; point `depmod -b` to `/layer/usr` and `/target-root/usr` instead of root. 3. `elements/oci/bluefin-server-ddi.bst` & `Justfile`: Mask `systemd-homed-firstboot.service` alongside `systemd-firstboot.service` on the target image and kernel cmdline to prevent headless hangs on interactive user creation prompt. 4. `Justfile` & `41-kubestellar-kiosk-proxy.yaml`: Move console readiness probe inside the guest using an injected test unit over TLS to correctly reach loopback-bound kiosk proxy and announce readiness on serial console. 5. `tests/unit/test_installer_contract.py`: Add unit contract assertions for the fixes. Fixes #218 Assisted-by: Gemini 3.8 Flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> * fix(boot): drop dead host-side kiosk probe and rebuild cleaned zfs sysext Address review findings on #220: 1. `Justfile`: remove the host-side poll of `http://127.0.0.1:8080/healthz` from the `installer-test` smoke recipe. The PR removed the QEMU hostfwd and the kiosk proxy terminates TLS (`files/k0s/kiosk/nginx.conf`, `listen 8080 ssl`), so this probe can never reach the guest. It could only ever hit an unrelated host process on port 8080 and false-pass the smoke test. The in-guest serial `KIOSK_CONSOLE_READY` marker is now the sole readiness check. 2. `elements/flatcar/flatcar-zfs.bst`: the preserved `/usr/lib/sysexts/zfs.raw` was the untouched upstream image and still carried the `After=systemd-sysext.service` udevd drop-in this PR removes from the rootfs. Inert today, but the deadlock ordering would return if that raw were ever merged for its stated runtime sysext usage. Rebuild the raw with mksquashfs from the cleaned install root after the drop-in and tmpfiles cleanup. 3. `tests/unit/test_installer_contract.py`: assert the host-side probe is gone from the smoke recipe and that the preserved raw is rebuilt post-cleanup. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Co-authored-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(installer): hard preflight abort when bluefin-installer-data is absent (#224) Add a hard preflight check before any installation begins: if /dev/disk/by-partlabel/bluefin-installer-data is absent or not a block device, abort with diagnostic lsblk and /dev/disk/by-partlabel output. This prevents unattended disk discovery from falling back to an empty installer partition and potentially selecting and erasing its own installation media on bare-metal USB drives. Assisted-by: github-copilot/gemini-3.8-flash via GitHub Copilot Co-authored-by: Jorge Castro <castrojo@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(justfile): probe install-vm kiosk readiness from inside the guest (rebased) (#227) * fix(justfile): probe install-vm kiosk readiness from inside the guest The kiosk proxy publishes hostPort 8080 on the guest's loopback only (files/k0s/manifests/kubestellar/41-kubestellar-kiosk-proxy.yaml) and terminates TLS (files/k0s/kiosk/nginx.conf, `listen 8080 ssl`). This broke install-vm's readiness wait and advertised access URLs the same way #218/#220 diagnosed for the installer-test smoke recipe: 1. A QEMU/libslirp hostfwd with an unspecified guest address is delivered to the guest's DHCP address, never to 127.0.0.1, so hostfwd=tcp::8080-:8080 could never reach the loopback-bound proxy. 2. The host-side poll spoke plain HTTP against a TLS listener. 3. The printed LAN/local access URLs and xdg-open call were therefore unreachable. Probe readiness from inside the guest instead: inject a oneshot systemd unit via an smbios(11) credential that polls the proxy over TLS from inside the guest and echoes a marker on the system console, which is captured to a host-side serial log file the host polls instead of curl. This is the same in-guest-probe approach proposed for test-installer-artifact in #220 (not yet merged); test-installer- artifact itself still probes from the host on this branch. Order ttyS0 last on the kernel cmdline so /dev/console resolves to the serial device the host captures, matching the ordering test-installer-artifact already uses for the same reason. The in-guest probe deliberately does not parse the /healthz response body: curl --fail already fails on a non-2xx response, which is exactly "healthz ok and / returns 200", and it avoids embedding quotes inside a systemd ExecStart= line, whose escaping rules differ from a plain shell. Drop the non-working direct 8080 hostfwd and the LAN/local URLs and xdg-open call that depended on it; print instructions for reaching the console from the host over an SSH tunnel through the existing hostfwd=tcp::2222-:22 forward instead, per the issue's suggested direction. Preserve the unrelated hostfwd=tcp::6443-:6443 (Kubernetes API) forward. Correct the kiosk proxy manifest comment, which still claimed the VM flow reaches the proxy via the existing hostfwd. Fixes #222 Assisted-by: Claude (Anthropic) via Hive Signed-off-by: Danathar <Danathar@users.noreply.github.com> * fix(os): correct login banner and de-brittle the jq test assertion files/os/issue.d/40-kubestellar.issue still advertised 'KubeStellar Console: http://\4:8080/ (or http://127.0.0.1:8080 from host)', which is doubly wrong after this PR: the proxy binds guest loopback and terminates TLS, and the direct 8080 hostfwd is gone. Fix in-scope rather than deferring, since the fix is one line and the PR already changed the reachability story the banner describes: point at https://127.0.0.1:8080/ and name the SSH tunnel the Justfile recipe now prints instead. Anchor test_install_vm_kiosk_readiness.py's jq-absence check to the ExecStart= line instead of the whole recipe body, so a future comment or variable elsewhere in the recipe containing 'jq' cannot fail it for the wrong reason, and it stays anchored to the actual contract: the ExecStart command line has no jq in it. Verified test_install_vm_kiosk_readiness.py's four module-level tests directly (no pytest in this environment): all pass, including the updated assertion. test_console_issue.py needs PyYAML, unavailable here; its one-line assertion update is a straight text match against the corrected banner and needs a real run in CI before merge. Signed-off-by: Danathar <Danathar@users.noreply.github.com> * fix(os): reformat login banner for TTY legibility The banner line grew from 71 to ~152 chars and would wrap badly on an 80-column console -- the entire point of /etc/issue.d is legibility at a getty prompt. Split it across two lines and dropped the markdown backticks, which render literally there: KubeStellar Console: https://127.0.0.1:8080/ (loopback, TLS only) Remote access: ssh -L 8080:127.0.0.1:8080 <user>@<host> Added an assertion for the tunnel line to tests/unit/test_console_issue.py. Mutation-tested the ExecStart jq guard against the real Justfile instead of an in-memory copy: piped the echo through 'jq -R .', confirmed test_install_vm_probes_readiness_inside_the_guest_over_tls fails, reverted. Inverse check: added a harmless '# jq' comment elsewhere in the install-vm recipe, confirmed the test still passes -- proving the anchor to the ExecStart= line, not just narrowing the match by luck. Signed-off-by: Danathar <Danathar@users.noreply.github.com> * fix(justfile): verify target partition layout before marking install complete Assert that bluefin-server-root-a and var partitions exist on TARGET_RAW before touching INSTALL_COMPLETE, preventing failed QEMU installer runs from falsely marking installation as finished. Assisted-by: github-copilot/gemini-3.8-flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(justfile): use sfdisk --json to parse regular target disk partition table in install-vm lsblk does not inspect partitions inside a regular file image; use sfdisk --json and jq to parse partitiontable.partitions.name instead. Assisted-by: github-copilot/gemini-3.8-flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(justfile): mount /var and bound readiness wait in install-vm The installed target carries no fstab entry for /var and gpt-auto cannot mount it, so the install-vm boot needs the same SMBIOS fstab.extra credential test-installer-artifact passes. Without it, k0s state (and the kiosk proxy it serves) is missing and the readiness marker never appears. Bound the readiness wait with INSTALL_VM_READY_DEADLINE (default 600s) and dump the serial log tail on timeout or premature QEMU exit, so a broken guest boot fails loudly instead of hanging forever. Also document why the target partition layout is verified before marking the install complete, and tell the operator to re-run the installer when an interactive install was aborted. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: Danathar <Danathar@users.noreply.github.com> Co-authored-by: Danathar <Danathar@users.noreply.github.com> Co-authored-by: Jorge Castro <castrojo@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> * fix: generate SSH host keys on device, not in the published DDI (#216) Drops build-time ssh-keygen calls from elements/oci/bluefin-server-ddi.bst so the published DDI carries no private host keys. Adds ssh-host-keygen.service and sshd.service drop-in to generate host keys on first boot before sshd starts. Reviewed-by: Danathar Assisted-by: claude-3-5-sonnet via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: include scripts/** in unit-tests workflow paths filter (#234) Include scripts/** in pull_request and push paths filters for unit-tests workflow. Reviewed-by: castrojo Assisted-by: github-copilot/gemini-3.8-flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: appliance identity, mDNS, and Bash prompt design (#208) Appliance identity, mDNS, and Bash prompt design spec. Reviewed-by: castrojo Assisted-by: claude-3-5-sonnet via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore(renovate): disable the homebrew manager for binary-asset formulae (#211) Formula/kc-agent.rb and Formula/bluefin-kubestellar.rb pin per-architecture release binaries with one sha256 each. Renovate's built-in homebrew manager only models source-archive formulae: it rewrites the url to https://github.com/<owner>/<repo>/archive/refs/tags/<tag>.tar.gz and carries over a single checksum. The result leaves `bin.install "kc-agent"` with no binary to install, and the other three architectures unresolvable. That is not hypothetical — it is exactly what #209 proposed: - url ".../releases/download/v0.3.41/kc-agent_0.3.41_darwin_amd64.tar.gz" + url ".../archive/refs/tags/v0.3.42.tar.gz" Bumping these formulae means fetching four assets and four checksums, so it stays a manual change until a generator exists. Disabling the manager stops the bot from filing PRs that cannot be merged. Validated with renovate-config-validator 44.103.6, the version the repository bot runs. Assisted-by: Claude Opus 4.5 via GitHub Copilot Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> Co-authored-by: castrojo <castrojo@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> * fix(installer): harden flash-installer with refusal guards, write verification, and GPT backup relocation (#225) * fix(installer): constrain flash image lookup to top-level and verify PARTLABEL Constrain flash-installer image lookup to dist/ maxdepth 1 to prevent flashing stale nested checkout artifacts. After dd, trigger and settle udev and verify the bluefin-installer-data PARTLABEL is present on the target device. Add bats unit test coverage. Assisted-by: github-copilot/gemini-3.8-flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(installer): make missing PARTLABEL fatal after flashing Ensure flash-installer exits nonzero and suppresses the success message when bluefin-installer-data PARTLABEL is not observed after writing. Add positive and negative Bats contract tests. Assisted-by: github-copilot/gemini-3.8-flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(installer): reread partition table via blockdev/partprobe before verifying partlabel Invoke blockdev --rereadpt / partprobe after dd so kernel partition tables are updated before udev triggers and the label verification runs. Assisted-by: github-copilot/gemini-3.8-flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(installer): abort if partition table reread fails after flashing Ensure flash-installer aborts with an error if both blockdev --rereadpt and partprobe fail to update the kernel partition table, preventing stale partition label checks. Add Bats unit test. Assisted-by: github-copilot/gemini-3.8-flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(installer): harden flash-installer safety, write verification, and partition checks - Reject disks backing the running system (/ and /sysroot). - Reject disks with mounted partitions. - Verify archive integrity with zstd -t before prompting. - Run dd pipeline with pipefail. - Flush buffers and verify payload bytes and sha256 read back from device. - Relocate GPT backup header and verify with sfdisk --verify. - Require exact match on bluefin-installer-data PARTLABEL with grep -Fxq. - Add full unit test coverage in tests/unit/flash-installer_test.bats. Assisted-by: github-copilot/gemini-3.8-flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(installer): add pipefail, digest mismatch, and sfdisk verify Bats tests Model dd of= by consuming stdin to EOF without masking SIGPIPE, test failing zstd -dc pipeline abort under pipefail, test corrupt readback digest mismatch abort, and distinguish sfdisk --relocate from --verify. Assisted-by: github-copilot/gemini-3.8-flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(installer): fail closed on mount checks and stop interpolating image path Address review findings on the flash-installer hardening: - The mounted-partitions guard used lsblk's MOUNTPOINTS column, which needs util-linux >= 2.37. On older hosts lsblk failed, the failure was swallowed, and the guard passed silently. Fall back to the MOUNTPOINT column and refuse the device outright when neither column can be read. - findmnt reports btrfs roots as /dev/sdXn[/subvol], which lsblk cannot resolve, so the running-system-disk refusal was skipped. Strip the subvolume suffix before resolving the parent disk. - The image path was interpolated into a root shell command. Pass the image and the device as positional arguments instead. - Refresh the bats header comment to describe the current guards and stubs, and document the added guards in docs/skills/ddi-installer-build.md. - Add coverage for the btrfs subvolume source, the MOUNTPOINT fallback, the unreadable-mount-state refusal, the multi-image refusal, and the absence of image interpolation in the root shell. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(installer): resolve flash target to its parent disk and hash in one pass The running-system guard compared the target's KNAME against the parent disk that / resolves to via PKNAME. Naming a partition node made the two sides disagree by construction: targeting /dev/sda3 while / lives on /dev/sda2 compared sda3 against sda, so the guard passed and only the mount check stood in the way. Resolve the target through PKNAME first, falling back to KNAME for whole-disk nodes, so both sides are parent disks. Covered by a new Bats case that fails without the fix. Also collapse the two pre-write decompressions into one: the digest and the byte count now come from a single `zstd -dc` piped through `tee` into a FIFO, with an explicit `wait` on the hashing subshell so the digest is fully written before it is read. This roughly halves pre-write time on multi-GB images. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Jorge Castro <castrojo@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(kiosk): executed coverage for kiosk-gate.js — tests/unit/test_kiosk_gate_behavior.py (#232) kiosk-gate.js gates the kiosk console while a session exists but the kc-agent is unreachable. It had no executed coverage: the existing suite only greps the source for literals, so inverting the session check, dropping the dedupe guard, trapping keydown with no gate up, ignoring a non-ok health response, or sending credentials to the agent all keep it green. Run the real script in node:vm against a stub DOM and assert the behaviour instead. Closes #231 Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> Co-authored-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> * revert: "docs: appliance identity, mDNS, and Bash prompt design (#208)" (#235) This reverts commit 3f7959217ab396bb30576f4fd013f2202c166707. The design specification contains an unviable mechanism: systemd-hostnamed writes /etc/hostname using WRITE_STRING_FILE_ATOMIC (open temporary file and renameat), which destroys the /etc/hostname symlink pointing to /var/lib/bluefin-server/hostname upon static hostname mutation. Reverting until an architecturally sound mechanism is reviewed independently. Assisted-by: github-copilot/gemini-3.8-flash via GitHub Copilot Co-authored-by: Jorge Castro <castrojo@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(creds): extend systemd-creds provisioning to SSH keys, network, firstboot, and TPM2 (#221) Wire systemd credential provisioning for network configuration, first-boot locale/timezone/hostname settings, and SSH key provisioning: - bluefin-firstboot-credentials.service: runs systemd-firstboot non-interactively when firstboot.* credentials are present, setting the live hostname before networkd starts and stamping /etc/.bluefin-firstboot-credentials so it runs once. - 10-bluefin-credentials.conf: drop-in for systemd-network-generator importing network.conf.*, network.link.*, network.netdev.*, and network.network.* credentials under /run/systemd/network/, falling back to DHCP when absent. - Presets enabling bluefin-firstboot-credentials.service and systemd-network-generator.service. - os-creds-prov.bst: converted to manual element to stage sysusers credentials and systemd credential units, with strip-binaries disabled. - Document TPM2 sealing, ESP and hypervisor delivery for bare metal and VMs, and tmpfiles.extra payloads for SSH key provisioning. - Contract unit tests in test_credentials_provisioning.py. Fixes #138 Supersedes #144 Assisted-by: Gemini 3.8 Flash via GitHub Copilot Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Co-authored-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * arch: make include/flatcar.yml the single source of truth for Flatcar pins (#143) * arch: make include/flatcar.yml the single source of truth for Flatcar pins include/flatcar.yml now declares every Flatcar upstream pin for the target release - the version, kernel vermagic, and the vmlinuz / container / zfs source sha256s - and every Flatcar element reads those values through %{...} variable substitution instead of restating a literal. - flatcar-kernel.bst and flatcar-zfs.bst reference %{flatcar-kernel-vmlinuz-sha256}, %{flatcar-kernel-container-sha256} and %{flatcar-zfs-sha256} instead of hard-coded refs. - Add .github/scripts/check-flatcar-version.py, a grep-based gate (in the spirit of check-release-version.py) that fails the build when any pinned literal appears outside include/flatcar.yml in elements/ or files/. - Wire the gate into 'just validate'. - Add unit tests covering declared_pins(), the scan, and the live-tree invariant. Signed-off-by: kylerankin <kylerankin@users.noreply.github.com> * fix(flatcar): quote variable-substituted source refs A bare `%` is a reserved YAML indicator and cannot start a plain scalar, so `ref: %{flatcar-...-sha256}` made elements/flatcar/flatcar-kernel.bst and flatcar-zfs.bst unparseable. Both bst and the pytest suite failed to load the elements, breaking the unit, build and build-kernel jobs. Quote the refs so they parse; BuildStream expands %{...} in source configs, so the pins still resolve from include/flatcar.yml. Assisted-by: Claude Opus 4.6 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: kylerankin <kylerankin@users.noreply.github.com> Co-authored-by: castrojo <castrojo@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(kiosk): console shows live local cluster state on first boot (#214) * fix(kiosk): console shows live local cluster state on first boot On a freshly installed single-node k0s cluster no client kc-agent is connected yet, so the KubeStellar console fell back to mock/demo cards instead of the real cluster. - Set NO_LOCAL_AGENT so the console reads the local embedded k0s API directly from this pod's ServiceAccount instead of waiting for a connected kc-agent. - Set SKIP_ONBOARDING so the kiosk shows the dashboard immediately. - Add a read-only ClusterRole bound to the console ServiceAccount so the in-cluster reads for nodes, namespaces and pods succeed. Assisted-by: lemonade/Ornith-1.5-35B-A3B-GGUF-Q6_K via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: kylerankin <kylerankin@users.noreply.github.com> * fix(kiosk): correct limitranges RBAC resource and SKIP_ONBOARDING doc - 42-kubestellar-console-rbac.yaml: limitquotas is not a Kubernetes resource; the intended grant was limitranges. Misspelled resources are silently ignored in RBAC rules, so LimitRange reads were forbidden. - SKIP_ONBOARDING comment: it skips the onboarding questionnaire, not sign-in (sign-in is bypassed by DEV_MODE/DEV_USER_LOGIN). - NO_LOCAL_AGENT comment: the console forces this in-cluster regardless; the real fix for #193 is the ServiceAccount + read RBAC. Addresses kubestellar-hive review findings on #214. * fix(kiosk): scope console RBAC to real resources and correct comments Follow-up to the #214 review findings. - 42-kubestellar-console-rbac.yaml: replicationcontrollers is a core ("") resource, not an apps resource. Listed under apps it was silently ignored in the RBAC rule, exactly like the limitquotas typo, so the grant never applied. Move it to the core group. - Document why the ClusterRole reads configmaps, events, limitranges and resourcequotas: the console dashboard renders them (namespace overview and ConfigMap cards). Record that verbs stay read-only, secrets are excluded, and the console is only reachable via the 127.0.0.1-bound kiosk proxy. - 40-kubestellar-console.yaml: attribute the fix to the ServiceAccount plus read RBAC. The console backend already forces NO_LOCAL_AGENT on in-cluster, so the env var only records intent. Note that SKIP_ONBOARDING skips the questionnaire and DEV_MODE/DEV_USER_LOGIN bypasses sign-in. - tests: pin every ClusterRole resource name to the real resource for its apiGroup and assert secrets are never granted, so a misspelled or misgrouped resource fails the suite instead of silently dropping the grant. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: kylerankin <kylerankin@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(k0s): pin argocd image by digest and upgrade past EOL v2.12.3 (#212) * fix(k0s): pin argocd image by digest and upgrade past EOL v2.12.3 argocd-server ran quay.io/argoproj/argocd:v2.12.3 (the EOL 2.12 line with unbackported CVEs) by mutable tag, so a re-pointed or compromised tag could silently change what runs. Upgrade to the latest supported v2.14.9 and pin by digest so the running image is integrity-checked. --insecure serves plaintext HTTP; keep it only because the argocd-server Service is ClusterIP-only, and document that it must stay internal (terminate TLS at a proxy, then drop the flag) rather than silently disabling it and breaking the deployment. sec-check #74 Signed-off-by: kylerankin <kylerankin@users.noreply.github.com> * fix(k0s): repin argocd to the live v2.14.9 manifest digest The previously pinned digest sha256:af3dfddc… is a legacy schema-v1 manifest that returns 404 on digest lookup; kubelet pulls tag@digest with manifest-list accept types, so argocd-server lands in ImagePullBackOff. Verified against quay.io: af3dfddc… 404s on every content type, while the v2.14.9 index sha256:f5885af0… resolves for all types and carries the amd64 child 78e45540…. Note: the 2.14 line is itself past EOL (3.x is current); the digest fix removes the hard regression but a maintainer may prefer a supported 3.x release. --------- Signed-off-by: kylerankin <kylerankin@users.noreply.github.com> * test(installer): behavioural contract for the QEMU boot-proof readiness gate (#219) * test(installer): behavioural contract for the QEMU boot-proof readiness gate test-installer-artifact (the recipe show-me-the-future delegates to) polls /healthz and / after booting the installed target in QEMU, and only breaks out of the loop when the healthz body reports status "ok" AND the root probe answers HTTP 200. Nothing exercised that decision logic before this change: a prior attempt at issue #130 (PR #179) sliced the recipe body out of the Justfile as text and asserted substrings, so every assertion stayed green whether the underlying checks were correct, weakened, or deleted. This suite instead runs the real recipe in a private sandbox copy of the Justfile (the tests/unit/flash-installer_test.bats pattern), with qemu-system-x86_64, curl and zstd replaced by logging stubs on PATH so 'just test-installer-artifact' actually executes. It proves, by running the recipe rather than describing it: - readiness requires both conditions together, not either alone - a failing/malformed healthz body short-circuits before the root probe is ever called - a QEMU process that dies before answering is detected and reported, independent of curl/jq state - the serial log tail is dumped on both failure paths (timeout and death) One substitution is applied to the sandboxed Justfile, asserted the same way flash-installer_test.bats asserts its own: the OVMF_CODE lookup only searches hardcoded absolute host paths, none of which exist on an unprivileged CI runner, so the recipe would abort before ever reaching the readiness loop. An env-gated candidate is added to the front of the existing first_existing() call; unset, it defaults to a nonexistent path, so production behaviour is unchanged. Verified non-vacuous: dropping the root-code check (collapsing the two conditions into one, as a plausible regression) fails 2 of the 10 cases; restoring it is green again. os-base: flatcar (#130's acceptance line) does not exist on main yet — no such build option exists anywhere in the tree, and the whole Flatcar-base cutover cluster (#132/#133/#136/#140/#163/#191) is parked 2-discussing pending the hard-rule-1 ADR decision on #133. This does not close #130; it pins the readiness contract that show-me-the-future/test-installer-artifact already enforce today, independent of which payload eventually boots under it, so the gate stays guarded while that decision is pending. Assisted-by: Claude via Claude Code Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Danathar <Danathar@users.noreply.github.com> * test(installer): rewrite the readiness contract around the in-guest probe #220 moved the readiness decision into the guest: a systemd unit injected over an SMBIOS credential polls the kiosk proxy and prints KIOSK_CONSOLE_READY on the console, and the host only greps the serial log for that marker. Four tests still drove a host-side curl loop that no longer exists and failed at the merged head. Drive the host decision through the serial log instead: the marker present, a progress line only, no marker at all, and a near-miss substring. Guard the two conditions and the credential delivery as text in the unit, and assert this recipe runs no host-side probe (scoped to its own body, since install-vm legitimately forwards its own ports). Four recipe mutations each fail between one and four tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YQCk7xJDvhtPoc4mWPzVCK Signed-off-by: Douglas Baggett <doug.baggett@gmail.com> * fix(test): scope the SMBIOS credential guard to test-installer-artifact install-vm (#227) reuses the same kiosk-ready credential and cmdline mechanism, so it now carries its own copy of both strings. A whole-file grep -c counts both recipes' copies and fails against current main (count 2, not 1). Scope the guard to this recipe's body, the same way the host-probe guard just below it already does. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs(test): describe what the readiness suite actually proves The file header and curl-stub comments described an earlier revision of this suite, from before the readiness probe moved into the guest (#220): they claimed the two-condition check was exercised by routing requests through the curl stub and parsing the body with jq. Nothing does that at head, and the check is jq -r .status, not jq -e. Rewrite the header to state the split the suite actually implements: the host-side loop (marker, target death, deadline, log tail) is executed for real against the sandboxed recipe, while the in-guest two-condition check is pinned by fixed-string greps because it can only run inside the booted guest. Correct the near-miss test's comment, which claimed exact-line matching that the recipe's unanchored grep does not do, and the refute_host_probe_attempted comment, which described a URL-shaped match it no longer performs. Drop the CURL_HEALTHZ_EXIT/CURL_HEALTHZ_BODY/CURL_ROOT_EXIT/CURL_ROOT_CODE knobs: no test sets them, the host never invokes curl, and one test asserts exactly that. The stub keeps its call logging, which refute_host_probe_ attempted needs, and now fails loudly if the host ever does probe. No behavioural assertion changes; 12/12 still pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: Danathar <Danathar@users.noreply.github.com> Signed-off-by: Douglas Baggett <doug.baggett@gmail.com> Co-authored-by: Danathar <Danathar@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: hive-scanner <223556219+Copilot@users.noreply.github.com> * feat(installer): add opt-in PXE network DDI pull (#229) Extend the installer wrapper (bluefin-sysinstall) to honor inst.* kernel cmdline parameters so PXE/netboot clients can download the OS DDI over the LAN instead of reading the embedded bluefin-installer-data partition: - inst.ddi_url=<url> fetches the zstd-compressed DDI via curl after systemd-networkd-wait-online brings up DHCP. - inst.ddi_sha256=<hex> is mandatory when inst.ddi_url is set; the download is verified before the target disk is touched (fail closed). - inst.target_disk=/dev/... overrides first-writable-disk auto-detection. The downloaded DDI is stream-decompressed with zstd and fed into the native systemd-sysinstall flow by staging a temporary /usr/lib/repart.sysinstall.d/ override whose 20-root-a.conf CopyBlocks= points at the downloaded image — no custom installer logic. Without inst.ddi_url the embedded path is byte-for-byte unchanged, and the hard preflight for the embedded partition is skipped under PXE. Add curl + zstd to installer-stack.bst. Part 1 (standalone PXE kernel/initrd export) already landed in main; this completes issue #14 Part 2. Assisted-by: lemonade/Ornith-1.5-35B-A3B-GGUF-Q6_K via GitHub Copilot Signed-off-by: kylerankin <kylerankin@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * build: os-base option to compare FSDK payload against Flatcar reference (#163) * build: add os-base option to build-ddi for FSDK/Flatcar parity harness projectbluefin/server#129 wants an os-base: fsdk | flatcar-reference option so the FSDK-composed payload and the imported Flatcar reference can be built from one tree and diffed. build-ddi now takes an OS_BASE parameter (default fsdk) and routes to the matching element: fsdk -> oci/bluefin-server-ddi.bst (shipped payload) flatcar-reference -> oci/bluefin-server-ddi-flatcar-reference.bst, added alongside the #126 imported Flatcar tree The flatcar-reference route refuses with a pointer to #126 until that element exists, and an unknown base is rejected. The default path is unchanged, so CI's 'just build-ddi' keeps building the FSDK payload. Adds a bats test covering the routing and guards. Assisted-by: lemonade/Ornith-1.5-35B-A3B-GGUF via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: kylerankin <kylerankin@users.noreply.github.com> * fix: resolve flatcar-reference element against elements/ in build-ddi The build-ddi os-base=flatcar-reference guard tested `-f "oci/bluefin-server-ddi-flatcar-reference.bst"`, but project.conf declares `element-path: elements`, so the element file lives at `elements/oci/bluefin-server-ddi-flatcar-reference.bst`. The guard could therefore never be satisfied and the flatcar-reference route was permanently unreachable, matching the Justfile's existing convention of referencing element files as `elements/...` on disk. The bats test asserting the refusal passed for the wrong reason: it ran against the repo root where no `oci/` directory exists at all. It now runs against a sandbox that holds a decoy at the bare `oci/` path and nothing under `elements/oci/`, so it fails if the guard resolves the wrong path and stays a genuine refusal test once the #126 element lands. Assisted-by: Claude Opus 4.6 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(build): stop interpolating os-base into the build-ddi recipe body just substitutes {{...}} textually into the recipe body before bash parses it, so `case "{{OS_BASE}}"` made the argument executable: $ just build-ddi '$(echo PWNED >&2)' PWNED PWNED ERROR: unknown os-base '' (expected 'fsdk' or 'flatcar-reference') It ran twice -- once at the case, once in the error echo -- so even the rejection path executed it, and the message that was supposed to name the bad value printed empty because the substitution had consumed it. Exporting the parameter (`build-ddi $OS_BASE="fsdk"`) hands it to the recipe as an environment variable, so bash only ever sees "$OS_BASE" and never expands the caller's text. Same invocation now prints the payload verbatim and exits 1. Only the local invoker could reach this, and it matches the existing {{ARGS}} pattern in the bst recipe, but os-base is a fixed enum of two values and has no reason to be interpolated at all. Adds two regression tests: one asserts no {{OS_BASE}} remains in the recipe body, the other passes $(touch <marker>) and asserts the marker is absent. The marker matters -- the payload's own text necessarily appears in the rejection message, so a string assertion could not distinguish echoed from executed. Both fail against the previous Justfile, along with the parameter-shape guard. Also documents os-base in docs/skills/ddi-installer-build.md, which still showed a bare `just build-ddi` and is what operators drive the #129 parity harness from: a Payload base section covering both values, the #126 refusal before the reference element exists, and the fact that export-ddi does not forward os-base, so the reference tree stays a comparison input and never becomes a release artifact. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> --------- Signed-off-by: kylerankin <kylerankin@users.noreply.github.com> Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: castrojo <castrojo@users.noreply.github.com> Co-authored-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> * feat(flatcar): add flatcar-usr.bst importing Flatcar /usr as the OS payload base (#140) * feat(flatcar): add flatcar-usr.bst importing Flatcar /usr as the OS payload base - Add elements/flatcar/flatcar-usr.bst to import Flatcar /usr as the OS payload base - Mirror flatcar-zfs.bst with single pinned remote source flatcar-container.tar.gz - Set strip-binaries to empty string to prevent stripping prebuilt binaries - Flatten Flatcar's nested usr/lib/modules/<kver>/<kver>/ layout - Explicitly remove Flatcar's update and provisioning stack with replacement comments - Preserve usr/lib/flatcar/bootengine.img for stage-2 kernel boot - Add contract tests in tests/unit/test_flatcar_usr.py Assisted-by: Gemini 3.8 Flash via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> * fix(flatcar): strip /usr/lib/modules to avoid collision with flatcar-kernel Removes dead nested module flattening code and strips /usr/lib/modules from flatcar-usr.bst completely so kernel modules are exclusively managed by flatcar-kernel.bst. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(flatcar): remove update-engine and cloudinit systemd units and drop-ins Address review feedback: - Explicitly remove update-engine and update-engine-stub units, timers, and multi-user.target.wants symlinks - Remove enable-oem-cloudinit and oem-cloudinit units and wants symlinks - Remove locksmithctl, ignition-rmcfg, and update tmpfiles configuration - Update unit tests to mock and assert absence of all removed units, binaries, and configs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> * style(flatcar): drop unused findutils build-dependency from flatcar-usr.bst The install-commands script only uses mkdir, tar, and rm, making findutils unnecessary. Update unit contract tests accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> --------- Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Co-authored-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: castrojo <castrojo@users.noreply.github.com> * fix(flatcar): use flatcar-kernel-container-sha256 variable in flatcar-usr.bst (#236) Co-authored-by: castrojo <castrojo@users.noreply.github.com> * refactor(arch): split release-version into installer and Flatcar axes (#136) * refactor(arch): split release-version into installer and Flatcar axes Split the single release-version axis in project.conf into two independent axes: - installer-version: FSDK point release describing the installer - flatcar-version: Flatcar LTS release describing the OS payload Update .github/scripts/check-release-version.py to independently validate both axes against their respective pins (freedesktop-sdk.bst junction ref for FSDK, include/flatcar.yml for Flatcar). Update unit tests in tests/unit/test_release_version.py to cover the two-axis contract, and update element asset naming and os-release metadata. Closes #125 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> * docs: update stale release-version references and clarify flatcar-version declaration Updates avoid-over-engineering.md and ddi-installer-build.md to reference the split axes (installer-version and flatcar-version), noting that release tags track the installer axis while the DDI and UKI track Flatcar. Also adds a comment in project.conf explaining why flatcar-version is deliberately declared there in addition to include/flatcar.yml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(arch): declare flatcar-version only in include/flatcar.yml project.conf restated flatcar-version alongside include/flatcar.yml, whose own header declares it the single source of truth for the Flatcar axis. Add include/flatcar.yml to project.conf's existing (@) include list instead, so %{flatcar-version} and %{flatcar-kver} resolve project-wide from one declaration, matching the k0s axis precedent. check-release-version.py no longer compares two copies of the same fact: it validates project.conf's installer-version against the freedesktop-sdk junction ref and that include/flatcar.yml declares a well-formed flatcar-version. Stale standing fact in docs/skills/index.md replaced with the two-axis description this refactor introduces. Assisted-by: Claude Opus 4.6 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(arch): update check-k0s-version guard and k0s.yml comment for split axes - Guard check-k0s-version against %{installer-version} and %{flatcar-version} in addition to %{release-version} - Update header comment in include/k0s.yml referencing split axes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> * fix(test): update test_main_rejects_fname_on_the_os_release_axis assertion string --------- Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Co-authored-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Jorge Castro <jorge.castro@gmail.com> Co-authored-by: castrojo <castrojo@users.noreply.github.com> --------- Signed-off-by: kylerankin <kylerankin@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Signed-off-by: hive-agent <hive-agent@users.noreply.github.com> Signed-off-by: Danathar <Danathar@users.noreply.github.com> Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> Signed-off-by: Douglas Baggett <doug.baggett@gmail.com> Co-authored-by: James Reilly <jreilly1821@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: castrojo <castrojo@users.noreply.github.com> Co-authored-by: Jorge Castro <jorge.castro@gmail.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Co-authored-by: hive-agent <hive-agent@users.noreply.github.com> Co-authored-by: ci-maintainer <ci-maintainer@users.noreply.github.com> Co-authored-by: Bob Killen <killen.bob@gmail.com> Co-authored-by: kubestellar-hive[bot] <280983584+kubestellar-hive[bot]@users.noreply.github.com> Co-authored-by: Danathar <Danathar@users.noreply.github.com> Co-authored-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> Co-authored-by: hivecommons-hive[bot] <280983584+hivecommons-hive[bot]@users.noreply.github.com> Co-authored-by: mendezr <18744521+mendezr@users.noreply.github.com> Co-authored-by: Danathar <doug.baggett@gmail.com>
Summary
Adds
elements/flatcar/flatcar-usr.bstto import Flatcar's/usrtree as the base for the Bluefin Server OS payload (#126, Phase 2 of the Flatcar base migration design in PR #123).Changes
elements/flatcar/flatcar-usr.bst:kind: manual, mirroringflatcar-zfs.bst.kind: remotesource forflatcar-container.tar.gzpinned by sha256 via theflatcar:alias.strip-binaries: ""to preserve prebuilt binaries without running the FSDK stripper../usrinto%{install-root}/usr./usr/lib/modulescompletely to defer kernel module handling toflatcar-kernel.bstand prevent compose collisions inos-stack.bst.rmlines (no wildcards):update_engine,update_engine_client,update_engine_stubbinaries,update-engine.service,update-engine-stub.service,update-engine-stub.timer, andmulti-user.target.wantsenablement symlinks.locksmithd,locksmithctl,locksmithd.service, andmulti-user.target.wants/locksmithd.service.ignition,ignition-rmcfg,ignition-delete-config.service, andsysinit.target.wantssymlink.coreos-cloudinit,enable-oem-cloudinit.service,oem-cloudinit.service, andmulti-user.target.wantssymlink.flatcar-updatebinary andtmpfiles.dconfigs (update-engine.conf,flatcar-update.conf).download_sysextandensure-sysext.service+sysinit.target.wantssymlink./usr/lib/flatcar/bootengine.imgneeded for stage-2 initramfs booting (feat(installer): delete the dracut path; the Flatcar kernel already carries its initramfs #128).tests/unit/test_flatcar_usr.py:bootengine.img, complete removal of/usr/lib/modules, and execution verification against a staged mock rootfs covering all removed binaries, units, symlinks, and tmpfiles configs.Verification
pytest tests/unitpasses (212 passed, 1 xfailed).bats tests/unitpasses (62 passed).python3 .github/scripts/docs-checks.pypasses.Closes #126
— hive: backend=copilot model=gemini-3.8-flash