Skip to content

feat(bootc-iso): experimental bootc-installer ISO variant (resurrected from #61) - #71

Draft
hanthor wants to merge 4 commits into
mainfrom
claude/resurrect-bootc-iso
Draft

hanthor wants to merge 4 commits into
mainfrom
claude/resurrect-bootc-iso

Conversation

@hanthor

@hanthor hanthor commented Aug 7, 2026 •

Copy link
Copy Markdown
Member

Resurrects the bootc-installer (fisherman) live ISO builder originally merged in #61 (credit: @hanthor) and reverted in #66 — now framed as an experimental variant to be merged and published alongside the Anaconda ISOs for a comparison period.

What this does

  • Purely additive: the entire builder lives under bootc-iso/ (justfile, 3-stage Containerfile, bluefin/ + bluefin-lts/ variant configs, src/, scripts/, HANDOFF.md). The current Anaconda/titanoboa system at the repo root is untouched.
  • .github/workflows/build-iso-bootc-experimental.yml (dispatch-only): builds the fisherman/bootcDirect ISOs and uploads them as workflow artifacts with an explicit -experimental-bootc suffix (e.g. bluefin-lts-experimental-bootc-x86_64.iso). Deliberately not wired into the scheduled/production publish workflows or releases — clearly labeled artifacts only, so both installer paths can be compared for a while.
  • .github/workflows/test-luks-install-bootc.yml (dispatch-only, no pull_request/schedule triggers): the LUKS install E2E from feat: Bluefin bootc-ISO builder (bootcDirect + containers-storage, 8 GB) #61, paths pointed at the bootc-iso/ subtree, matrix trimmed to target: [bluefin, bluefin-lts] x installer_channel: [stable].
  • bootc-iso/README.md documents the history, blockers, and fix direction.

Fisherman fork dependency: dropped

#61 depended on the fork branch projectbluefin/fisherman @ fix/overlay-driver-for-ostree-bootc-install (which still exists, unmerged). Verified against current fisherman main (Aug 2026): direct mode now natively emits bootc install --source-imgref containers-storage:<ref> to read the payload from the ISO-embedded store — the same mechanism bootc-image-builder uses upstream (fisherman/internal/install/bootc.go, BuildBootcArgs). The E2E workflow therefore builds fisherman from main; the fork is no longer referenced.

Why it was reverted (maintainer-confirmed) — still open

  1. ISO size blowup from no dedupe between the live squashfs rootfs and the embedded container image: the vfs driver stores flat copies, and the overlay driver fought with ostree-based bootc install (originally patched in the fisherman fork, now superseded upstream — but the duplication itself remains).
  2. bluefin-lts never built due to a dracut failure — bluefin-lts: dracut initramfs build fails with 'Failed to find module ntfs3' #63.

Intended endgame

Single-copy composefs/erofs design: mount the live rootfs directly from the embedded container store so the installer payload and the live environment share one copy of the OS content, eliminating the duplication structurally.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KSjBPVDcnK71WveusaK7Nc

claude added 2 commits August 7, 2026 20:09
Restore the fisherman/bootc-installer live ISO builder merged in #61 and
reverted in #66, additively under bootc-iso/ so the Anaconda/titanoboa
system at the repo root is untouched. The LUKS E2E workflow returns as
.github/workflows/test-luks-install-bootc.yml, workflow_dispatch-only,
with the matrix trimmed to stable x {bluefin, bluefin-lts} and paths
pointed at the bootc-iso/ subtree. bootc-iso/README.md documents the
known blockers (ISO size/dedupe, vfs-vs-overlay, LTS dracut failure #63)
and the intended single-copy composefs/erofs fix direction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KSjBPVDcnK71WveusaK7Nc
…migration

Per maintainer direction: add a dispatch-only build+publish workflow
(build-iso-bootc-experimental.yml) that uploads the fisherman/bootcDirect
ISOs as workflow artifacts with an explicit -experimental-bootc suffix,
deliberately not wired into scheduled/production publishing. Drop the
dependency on the fisherman fix/overlay-driver-for-ostree-bootc-install
fork branch, since upstream fisherman main now natively supports
direct-mode bootc install --source-imgref containers-storage:, and
record the maintainer-confirmed revert reasons in the README.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KSjBPVDcnK71WveusaK7Nc
@hanthor hanthor changed the title feat(bootc-iso): resurrect bootc-installer ISO builder (from #61) feat(bootc-iso): experimental bootc-installer ISO variant (resurrected from #61) Aug 7, 2026
workflow_dispatch cannot resolve workflows that do not exist on the
default branch; a branch-scoped push trigger lets the experimental
builds run before merge. Remove before merging.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KSjBPVDcnK71WveusaK7Nc
@hanthor

hanthor commented Aug 7, 2026

Copy link
Copy Markdown
Member Author

@castrojo — green LUKS E2E proof, including bluefin-lts 🎉

https://github.com/projectbluefin/iso/actions/runs/31218241709

Both matrix jobs passed on the resurrected bootc-installer path:

  • LUKS E2E (bluefin-lts, stable) ✅ — debug ISO built, LUKS-encrypted install via fisherman/bootcDirect, unlock verified at boot, screenshots published to ci-screenshots
  • LUKS E2E (bluefin, stable) ✅ — same

Two notable firsts vs the original #61 (which never had LTS green — it was blocked on the dracut failure):

  1. LTS passes end-to-end.
  2. This runs on stock fisherman main — upstream now emits bootc install --source-imgref containers-storage:<ref> natively, so the fix/overlay-driver-for-ostree-bootc-install fork dependency is gone.

Per the maintainer plan, this branch also adds build-iso-bootc-experimental.yml publishing clearly-labeled -experimental-bootc ISO artifacts alongside the Anaconda ISOs (nothing wired into production publishing). The known #61 revert issues (ISO size / no rootfs-vs-store dedupe, vfs-vs-overlay) are documented in bootc-iso/README.md with the composefs single-copy design as the fix direction.

The Anaconda-path LUKS test (PR #70, E2E_LUKS=1 on the production LTS ISO) is still being driven to green in parallel.


Generated by Claude Code

@hanthor

hanthor commented Aug 8, 2026

Copy link
Copy Markdown
Member Author

Experimental build results are in — and the size regression that killed #61 appears to be gone:

ISO (x86_64, artifact size) bootc/fisherman (this PR) Anaconda (run 31180530212)
bluefin (stable) 4.67 GB 5.35 GB
bluefin-lts 4.19 GB 4.59 GB

Both experimental ISOs built green in ~22 min each (run 31218241802), uploaded as bluefin-experimental-bootc-x86_64 / bluefin-lts-experimental-bootc-x86_64 artifacts (14-day retention). The likely reason the blowup is gone: stock fisherman's overlay-driver path now works for the ostree bootc install, so the embedded store no longer needs vfs's flat per-layer copies.

Combined with the green LUKS E2E above (both variants), this looks viable as the experimental side-by-side variant. The composefs single-copy design remains the endgame for structural dedupe, tracked in bootc-iso/README.md.


Generated by Claude Code

Weekly Monday 04:00 UTC run (the original #61 cadence) so the LUKS
install+unlock coverage for bluefin and bluefin-lts keeps running
without manual dispatch. Note: schedule triggers only fire from the
default branch, so this activates on merge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KSjBPVDcnK71WveusaK7Nc

@hanthor hanthor left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of this draft. Framing is sound — additive subtree, dispatch-only workflows, nothing wired into production publish — so the risk of merging is genuinely low. Three things to fix before this comes out of draft, and one note on what the LUKS test does and does not prove.

What I verified

  • Merge: clean onto main. git diff --name-only --diff-filter=U is empty. (I unshallowed the clone first and merged with -c commit.gpgsign=false --no-gpg-sign to rule out the usual false-conflict traps.) 2966 insertions, 0 deletions — the "purely additive" claim holds; nothing at the repo root is touched.
  • CI never ran. get_status returns total_count: 0 for 00515466. That is by design here — both new workflows are workflow_dispatch-only, and pull-request.yml triggers on paths that this PR does not touch. So nothing in these 2966 lines has been executed by CI. Treat it as unverified rather than as passing.
  • The fisherman claim checks out. I cloned projectbluefin/fisherman and confirmed on main: fisherman/internal/install/bootc.go:243 does args = append(args, "--source-imgref", "containers-storage:"+bareImageRef(resolvedTargetImgref)), with BuildBootcArgs at line 217. The fork branch fix/overlay-driver-for-ostree-bootc-install still exists at dfc4abd2010932d4a2f701191af0bb108096e030 and is not merged into main (9c0f3b8fcc9a3e570ac2c0d4eee8055cccef4ca8) — exactly as the body describes. Also confirming FISHER_REPO: /tmp/fisherman/fisherman is correct, not a typo: fisherman/ really is a subdirectory of that repo.

1. The fisherman checkout is unpinned, and it is still the fork

git clone https://github.com/projectbluefin/fisherman.git \
  --branch main --depth 1 /tmp/fisherman

The body says "the fork is no longer referenced". That is not accurate — the clone URL is still projectbluefin/fisherman. What changed is the branch, from the feature branch to the fork's main. Worth restating precisely, since "we dropped the fork dependency" and "we moved from the fork's feature branch to the fork's main branch" are different claims.

More importantly, --branch main is a moving target, for the component that decides how bits get written to a user's disk. main is 9c0f3b8f… today; a force-push or any merge upstream silently changes what the installer does, with no diff in this repo. Pin the SHA:

git clone https://github.com/projectbluefin/fisherman.git /tmp/fisherman
git -C /tmp/fisherman checkout 9c0f3b8fcc9a3e570ac2c0d4eee8055cccef4ca8

(--depth 1 and an explicit SHA need --filter / fetch-by-sha; simplest is a normal clone plus checkout, as the root reusable-build-iso-anaconda.yml already does for titanoboa.)

2. The LUKS test cannot fail

- name: Run LUKS end-to-end test
  id: luks_test
  run: just luks-test-qemu ${{ matrix.target }}
  continue-on-error: true

continue-on-error: true means the only test in this PR that actually installs to a disk can never fail the job. The later step writes RESULT="${{ steps.luks_test.outcome }}" into the summary, but nothing exits non-zero on it. So the workflow is green whether the LUKS install works, silently produces an unencrypted disk, or never boots.

This is the dakota-iso#137 shape: a test exists, is visible, and is structurally incapable of gating. For a dispatch-only experimental workflow that is defensible if deliberate — but it should be stated in README.md/HANDOFF.md, and it must not stay this way if this ever gets wired into anything scheduled. At minimum, add a final step that fails the job when steps.luks_test.outcome == 'failure', matching how iso-smoke-test.yml already handles its own continue-on-error step at the repo root.

3. Disk-destructive behaviour and what the LUKS test actually covers

This PR adds a second installer that partitions, encrypts, and writes a bootloader, and bootc-iso/src/etc/bootc-installer/recipe.json exposes disk and encryption steps — i.e. install-time disk selection is a user-facing feature here. Stating coverage plainly:

Covered: a UEFI install. Credit where due — bootc-iso/justfile locates OVMF_CODE.fd / OVMF_VARS.fd and passes a proper -drive if=pflash pair (lines ~309-310), with a per-run writable OVMF_VARS copy from mktemp. That is strictly better than the repo-root Anaconda harness, which runs SeaBIOS and has no EFI variable store at all. A real NVRAM write path exists in this VM.

Not covered:

  • Dirty disks. qemu-img create -f qcow2 "{{ luks-qemu-disk }}" 64G — blank image. No pre-existing partition table, no foreign ESP, no second OS. The interesting failure — a leftover Windows EFI partition confusing the install — cannot occur.
  • Disk selection. DISK="/dev/vda" is hardcoded in the recipe generation. The disk step in recipe.json is never exercised; there is only ever one candidate disk. A multi-disk VM (-drive ×2, one pre-populated) is what would actually test that step.
  • NVRAM collisions. The pflash vars file is fresh per run, so there are never pre-existing BootXXXX entries to collide with, duplicate, or be ordered behind. Having OVMF present makes this testable — it just isn't tested yet.

One concrete nit that interacts with the above:

[[ -f "{{ luks-qemu-disk }}" ]] || qemu-img create -f qcow2 "{{ luks-qemu-disk }}" 64G

The disk is reused if it already exists. In CI the runner is fresh so this is always a new disk, but locally a second just luks-test-qemu run installs over the previous install. That means the one place a dirty disk can arise is accidental and nondeterministic rather than a controlled fixture — the worst of both worlds. Either force a fresh image per run, or make dirty-disk an explicit, seeded mode.

Given that this PR is already standing up an OVMF-based install VM, it is by far the cheapest place in the org to close the dirty-disk gap: pre-seed a GPT + FAT32 ESP (and ideally a stale BootXXXX entry) into the qcow2 before install, as a second matrix leg. That would make this variant test something the Anaconda path structurally cannot.

Minor

  • actions/checkout@v4 (line 41) is an unpinned tag, while the repo root pins checkout by SHA (de0fac2e… in iso-smoke-test.yml). Inconsistent; #67 just moved the repo toward pinned v7.
  • jlumbroso/free-disk-space@54081f13… is SHA-pinned but carries no # vN comment — fine, and honestly preferable to #69's incorrect # v10.
  • Body says the E2E matrix is trimmed to target: [bluefin, bluefin-lts] x installer_channel: [stable]; worth noting bluefin-lts is the leg that #63 says never builds, so that leg is expected red from day one. Say so in the body so nobody reads the first failure as a regression.

None of this blocks the draft. Items 1 and 2 should be resolved before it leaves draft.


Generated by Claude Code

@Danathar

Copy link
Copy Markdown
Contributor

@hanthor, is this draft still live? It has had no commits since 08-08.

Nothing on main has replaced it: main has no bootc-iso/ directory. The six red ISO builds are not caused by this PR. They are the two failures #70 fixed on main (the empty "Rename ISO" path, and no arm64 image in the manifest). This head is older than #70, so it has not run since that fix.

Two things look out of date:

  • Issue bluefin-lts: dracut initramfs build fails with 'Failed to find module ntfs3' #63 is still open, but your comments here say bluefin-lts now builds and passes the LUKS test.
  • test-luks-install-bootc.yml says at the top that it "must never run on pull_request or a schedule", but it has a weekly cron (0 4 * * 1). Both workflow files also still carry the temporary push trigger for claude/resurrect-bootc-iso.

If you plan to continue: a rebase, the two items from the 09-12 review (pin the fisherman clone, drop continue-on-error), and a body update would get it moving. If not, closing it and keeping the branch would clear it from the queue. I did not run any builds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants