Repository navigation
test(installer): assert offline payload and default Flatpaks survive encrypted install - #215
Conversation
…encrypted install iso/scripts/luks-e2e.sh already installs to a LUKS2-encrypted disk, unlocks it, and boots to a GNOME session with no network route -- but nothing asserted that the offline embedded payload and the default Flatpak set actually persisted through that install, and a failed run never kept even non-sensitive metadata about the disk that failed. Add two checks on the installed, still network-isolated system: - bootc status confirms the booted image is exactly the offline payload fisherman's recipe named (targetImgref), not a substitute -- checked right after the graphical target, before login. (/usr/share/utah/contract.txt, tried in an earlier revision of this change, is the resolved *RPM* contract scripts/install-packages.py writes at image-build time; it says nothing about which OCI image or digest actually got deployed.) - Every default Flatpak in the Brewfile contract, using the same Brewfile-to-list conversion iso/live/Containerfile already uses to seed the ISO's own repo. This runs last, with a short retry: the 2026-09-19 verified run (docs/verification) shows the full default set only appears on the installed disk after roughly two minutes of uptime, so checking right after login would misreport apps that simply have not deployed yet. UTAH_E2E_FLATPAKS overrides the expected set; empty skips the check. A failure now also captures qemu-img metadata (size, format) for the disposable disk -- never the disk itself, which carries test credentials and stays excluded from every CI upload per existing policy. The capture moved into the EXIT trap rather than fail()'s body: a bare command failing under set -e (the installer step, for one) aborts straight to the trap without ever calling fail(). Renumber the script's phases from six to seven to name the booted-image check, and update docs/skills/local-testing.md and the embedded verification-README template to match. docs/skills/ci-workflows.md is left alone: open PR #198 already edits that file in the same area. New coverage in tests/test_iso_ci.py extracts the Flatpak set-difference block and runs it under bash with stubbed ssh_target/sleep, rather than only asserting strings are present in the script. This is a hard gate inside the luks job, which promote-to-testing and documentation depend on for every flavor. It has not been exercised against a real installed system -- only bash -n, the script-syntax gate, and the stubbed unit test above ran here. The first post-testing-e2e run is what proves or disproves whether fisherman actually carries the Brewfile-derived Flatpak set across the encrypted install; if it does not, UTAH_E2E_FLATPAKS is the escape hatch to unblock promotion while that gets investigated. Addresses #15. Signed-off-by: Danathar <Danathar@users.noreply.github.com>
ca66a6c to
feddac6
Compare
castrojo
left a comment
There was a problem hiding this comment.
Reviewed diff and test execution: asserts offline payload and default Flatpaks survive encrypted install in installer tests. Contract gate rerun passed cleanly after transient network EOF. Awaiting review from an independent maintainer to satisfy policy.
There was a problem hiding this comment.
correctness
preserve_failed_disk_diagnosticsis called fromcleanup()before the QEMU PIDs are killed (iso/scripts/luks-e2e.sh:201runs before the kill loop at :202-204). In the common failure case a QEMU process still holds the qcow2's write lock, andqemu-img infowithout-U/--force-sharerefuses locked images — so the "failed disk diagnostics" log will usually contain only a lock error instead of the size/format metadata it exists to capture. Useqemu-img info -U "${INSTALL_DISK}"(read-only metadata, safe here) or capture after the kill loop.- Minor:
bootc status --jsonruns as the unprivileged${TEST_USER}(iso/scripts/luks-e2e.sh:539,ssh_targetat :118-121). If the installed bootc requires root forstatus, the check fails closed with'<unreadable>'and — unlike the Flatpak check — has no env escape hatch. The PR body already flags the gate as unexercised; just noting the failure mode is a hard block onpromote-to-testingwith no override.
docs-currency
UTAH_E2E_FLATPAKS— which the PR body calls "the immediate escape hatch to unblock promotion" — is documented only in a script comment (iso/scripts/luks-e2e.sh:731).docs/skills/local-testing.mddocuments the other harness knobs (UTAH_E2E_REQUIRE_FASTFETCH,UTAH_E2E_REQUIRE_SCREENSHOTSat :160-161,UTAH_E2E_RAM/UTAH_E2E_CPUSat :182-183) but not this one; the new paragraph added at :147-152 describes the Flatpak check without naming the override. An operator needing the escape hatch under CI pressure won't find it in the docs.
No findings from: security, intent-alignment, style.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
…cument hatches - qemu-img info runs from the EXIT trap while QEMU still holds the write lock, so the retained log recorded a lock error instead of the size and format it exists to capture; read with -U. - bootc status ran only as the unprivileged test user, hard-failing the luks job as '<unreadable>' if that version needs root. Retry under sudo and add UTAH_E2E_PAYLOAD_CHECK="" to skip, matching UTAH_E2E_FLATPAKS. - Document both escape hatches in docs/skills/local-testing.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
security (low)
iso/scripts/luks-e2e.sh:557— the sudo fallback interpolates${TEST_PASSWORD}into a single-quoted remote command (printf '%s\n' '${TEST_PASSWORD}' | sudo -S …). IfUTAH_E2E_PASSWORDcontains a single quote the remote command breaks and the gate fails as<unreadable>instead of naming the real cause. It also puts the password on the guest's command line (visible in guest ps/audit logs) — contained, since the disk is never uploaded, but consider feeding sudo via stdin without embedding the literal (e.g.printf %q, or reuse sshpass's stdin). Same latent pattern as the pre-existing recipe heredoc (line 341), so non-blocking.
No findings from: correctness, intent-alignment, style, docs-currency.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
castrojo
left a comment
There was a problem hiding this comment.
Reviewed diff and CI: adds integration test in tests/test_encrypted_offline_flatpaks.py validating default Flatpaks survive encrypted installs. All 14 checks pass. Outstanding: two independent maintainer reviews required (+2 policy).
hanthor
left a comment
There was a problem hiding this comment.
Reviewed: checks green, change matches its description. Merging per maintainer triage.
Summary
Progresses #15. The existing
iso/scripts/luks-e2e.shharness (from #41/#84,already on
main) installs Utah to a LUKS2-encrypted disk, unlocks it, andboots to a GNOME session with no network route out -- covering two of #15's
four acceptance criteria already. This PR closes the remaining gaps that
don't require the still-open, still-blocked production-ISO pipeline (#14,
blocked by #13):
bootc status. Right after the installedsystem reaches the graphical target (before login), the harness confirms
bootc status --jsonreports the booted image as exactly the offlinepayload fisherman's recipe named (
targetImgref), not a substitute. Anearlier revision of this change checked for
/usr/share/utah/contract.txt, but that file is the resolved RPMcontract
scripts/install-packages.pywrites at image-build time -- itsays nothing about which OCI image/digest actually got deployed.
/usr/share/ublue-os/homebrew/system-flatpaks.Brewfile(the sameBrewfile-to-list conversion
iso/live/Containerfilealready uses to seedthe ISO's own repo) must be present and listed by
flatpakon theinstalled, network-isolated system. This runs last, after login and the
fastfetch/extension checks, with a short retry: the 2026-09-19 verified
run in
docs/verificationshows the full default set (45 Flatpaks) onlyappears roughly two minutes after boot, so checking immediately after SSH
becomes available -- this PR's first attempt -- would misreport apps that
simply have not finished deploying yet as missing.
UTAH_E2E_FLATPAKSoverrides the expected set directly; empty skips it.failure captures
qemu-img infometadata (virtual/actual size, format)for the disposable install disk into a
*.logfile the CI workflow'sexisting artifact glob already picks up. This intentionally never uploads
the disk itself --
post-testing-e2e.ymlexplicitly forbids that becausethe disk carries the test LUKS passphrase, test account password, and host
keys, and
tests/test_iso_ci.pyalready asserts no upload path containsqcow2. The capture lives in thecleanup()EXIT trap rather than onlyin
fail()'s body: a bare command failing underset -e(the installerstep in phase 3, for one) aborts straight to the trap without ever calling
fail().Not addressed here: "using the production ISO" (#15's third criterion). The
harness drives the installer over SSH, which only a debug ISO enables; a
production (no-SSH) ISO installer gate is the explicit scope of #14, which is
itself blocked by #13. Open PR #198 already covers that criterion (composes
a
DEBUG=0ISO + checksum after the LUKS matrix passes) and touches.github/workflows/post-testing-e2e.yml/docs/skills/ci-workflows.mdinthe same area, so this PR leaves both of those files alone and stays
entirely inside
iso/scripts/luks-e2e.sh(plusdocs/skills/local-testing.md, which #198 does not touch).The script's six phases are renumbered to seven to name the new booted-image
check; the Flatpak check runs after phase 7 and isn't itself numbered, since
it deliberately waits past login. The embedded verification-README template
is updated to match.
Risk
This is a hard gate inside the
luksjob, andpromote-to-testing/documentationdepend on it for every flavor. It has not been exercisedagainst a real installed system -- there is no QEMU/KVM host available
here, so only
bash -n, the repo's script-syntax gate, and a stubbed unittest (below) ran. The first
post-testing-e2erun after this merges is whatactually proves or disproves whether fisherman carries the Brewfile-derived
Flatpak set across an encrypted install for every configured flavor; if it
does not,
UTAH_E2E_FLATPAKS=""is the immediate escape hatch to unblockpromotion while that gets investigated, without reverting the
bootc statuscheck.
Validation
bash -n iso/scripts/luks-e2e.sh-- syntax OK.python3 scripts/check-script-syntax.py-- all 56 tracked scripts pass.python3 -m unittest tests.test_iso_ci -v-- new tests pass, includingtest_default_flatpak_check_executes_against_stubbed_ssh, which extractsthe real set-difference block from the script and runs it under
bashwith
ssh_target/sleepstubbed (complete set passes, a missing app failsnaming it, an empty override skips), not just string assertions.
python3 -m unittest discover -s tests-- same 8 failures / 48 errorsbefore and after this change (all
ModuleNotFoundError: No module named 'yaml', a pre-existing sandbox gap with nopip/dnfavailable here --confirmed identical via
git stash && python3 -m unittest discover -s tests 2>&1 | tail -3 && git stash pop).git diff --check-- 0 whitespace errors.— hive: backend=omp model=anthropic/claude-sonnet-5 effort=medium