Repository navigation
ci: retain gated production ISO artifacts - #198
Conversation
…encrypted install iso/scripts/luks-e2e.sh already installs to a LUKS2-encrypted disk, unlocks it, and boots to a GNOME session with no network route -- but nothing asserted that the offline embedded payload and the default Flatpak set actually persisted through that install, and a failed run never kept even non-sensitive metadata about the disk that failed. Add two checks on the installed, still network-isolated system: - bootc status confirms the booted image is exactly the offline payload fisherman's recipe named (targetImgref), not a substitute -- checked right after the graphical target, before login. (/usr/share/utah/contract.txt, tried in an earlier revision of this change, is the resolved *RPM* contract scripts/install-packages.py writes at image-build time; it says nothing about which OCI image or digest actually got deployed.) - Every default Flatpak in the Brewfile contract, using the same Brewfile-to-list conversion iso/live/Containerfile already uses to seed the ISO's own repo. This runs last, with a short retry: the 2026-09-19 verified run (docs/verification) shows the full default set only appears on the installed disk after roughly two minutes of uptime, so checking right after login would misreport apps that simply have not deployed yet. UTAH_E2E_FLATPAKS overrides the expected set; empty skips the check. A failure now also captures qemu-img metadata (size, format) for the disposable disk -- never the disk itself, which carries test credentials and stays excluded from every CI upload per existing policy. The capture moved into the EXIT trap rather than fail()'s body: a bare command failing under set -e (the installer step, for one) aborts straight to the trap without ever calling fail(). Renumber the script's phases from six to seven to name the booted-image check, and update docs/skills/local-testing.md and the embedded verification-README template to match. docs/skills/ci-workflows.md is left alone: open PR #198 already edits that file in the same area. New coverage in tests/test_iso_ci.py extracts the Flatpak set-difference block and runs it under bash with stubbed ssh_target/sleep, rather than only asserting strings are present in the script. This is a hard gate inside the luks job, which promote-to-testing and documentation depend on for every flavor. It has not been exercised against a real installed system -- only bash -n, the script-syntax gate, and the stubbed unit test above ran here. The first post-testing-e2e run is what proves or disproves whether fisherman actually carries the Brewfile-derived Flatpak set across the encrypted install; if it does not, UTAH_E2E_FLATPAKS is the escape hatch to unblock promotion while that gets investigated. Addresses #15. Signed-off-by: Danathar <Danathar@users.noreply.github.com>
There was a problem hiding this comment.
correctness
The retained checksum can't be verified as shipped. sha256sum output/utah-live.iso > output/utah-live.iso.sha256 (.github/workflows/post-testing-e2e.yml:186) embeds the output/ path prefix, but the upload-artifact step roots the artifact at the files' common ancestor output/ (lines 192–195), so the downloaded artifact contains utah-live.iso and utah-live.iso.sha256 side by side and sha256sum -c utah-live.iso.sha256 fails with "No such file or directory: output/utah-live.iso". Since verifiable checksums are the point of retaining this artifact, generate the checksum with a bare filename, e.g. (cd output && sha256sum utah-live.iso > utah-live.iso.sha256).
No findings from: security, intent-alignment, style, docs-currency.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
|
Confirmed and fixed in This environment still has no PyYAML (the pre-existing gap the PR body already documents), so I couldn't run
|
There was a problem hiding this comment.
intent-alignment
- (low) The PR body scopes the gate to testing-tag promotion, but the diff also gates the
documentationjob onproduction-iso(.github/workflows/post-testing-e2e.yml:226). A production-ISO failure now blocks the README screenshot-refresh PR, which only concerns LUKS results. Intentional-looking (the test attests/test_iso_ci.py:170asserts it), but it's unstated in the body and only partially reflected indocs/skills/ci-workflows.md:181("promotion prerequisite").
correctness
- (low) Each retained artifact is a ~8 GB ISO (size budget in
iso/scripts/build-iso.sh:24) per flavor per run at 30-day retention (.github/workflows/post-testing-e2e.yml:196-204). That's a substantial Actions-storage bill per dispatch, and the 90-minute job timeout must cover both the compose and the multi-GB upload — the LUKS job that proves the compose time budget has 180. Worth confirming one real run fits.
No findings from: security, style, docs-currency.
— hive: agent=reviewer backend=copilot model=claude-fable-5
|
Addressing the remaining intent-alignment findings from the second review (the first, correctness, was already fixed in
|
There was a problem hiding this comment.
security
- (low) The retained "production" ISO boots with
enforcing=0 console=ttyS0,115200n8— the same kernel cmdline as the disposable debug ISO (iso/scripts/build-iso.sh:149and:165, shared by both DEBUG values). That cmdline was fine while ISOs were never retained; this PR now ships it in a 30-day artifact labeled production media (.github/workflows/post-testing-e2e.yml,Retain production ISOstep). Not a blocker for an internal artifact, but SELinux-permissive live boot is a decision that should be made explicitly in #186 before any of these ISOs reach users. Verified that DEBUG=0 does exclude the test credentials/sshd path (iso/live/src/configure-live.sh:195-205), so no secrets are retained.
No findings from: correctness, intent-alignment, style, docs-currency.
— hive: agent=reviewer backend=copilot model=claude-fable-5
castrojo
left a comment
There was a problem hiding this comment.
Reviewed diff and CI runs: retains gated production ISO artifacts in post-testing-e2e. All CI checks pass. Note: checksum verification path should be verified on download. Awaiting review from an independent maintainer to satisfy policy.
|
Thanks — going through the four findings across these reviews, since they're at different stages. 1. Checksum path (correctness, resolved). Right, and already fixed at this head: One note while I was in there: 2. Retained "production" ISO boots permissive (security). Confirmed, and addressed in 5c6d04c — by documenting, not by changing the cmdline. Flipping it here would break the E2E path and pre-empt a product decision, so the cmdline stays and the note now sits in all three places someone would look — the cmdline itself, the 3. Storage and timeout (correctness, low). Recorded in the same commit: multi-GB per flavor per dispatch at 30-day retention, with a note to revisit the window before widening the matrix. I've deliberately not asserted a measured size — the 8 GB in 4. 241 tests pass, YAML parses,
|
Compose DEBUG=0 ISO artifacts only after every flavor passes the offline LUKS matrix, retain checksums, and block testing promotion on production ISO composition. Assisted-by: GPT-5.6 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Douglas Baggett <doug.baggett@gmail.com>
sha256sum output/utah-live.iso > output/utah-live.iso.sha256 embedded the output/ path prefix inside the checksum file. upload-artifact roots the artifact at output/, so the downloaded artifact holds utah-live.iso and utah-live.iso.sha256 side by side and sha256sum -c utah-live.iso.sha256 fails with 'No such file or directory: output/utah-live.iso'. Generate it with cd output so the recorded filename is bare and verifiable as shipped. Updated the matching assertion in tests/test_iso_ci.py. This environment has no PyYAML (documented pre-existing gap in the PR body), so full-suite execution is deferred to CI; verified by grep that the workflow literal and the test's assertIn string match byte-for-byte. Signed-off-by: Danathar <Danathar@users.noreply.github.com>
The intent-alignment review noted the PR body scopes the gate to testing-tag promotion, but post-testing-e2e.yml also adds production-iso to the documentation job's needs (asserted intentionally by tests/test_iso_ci.py:167-169). Neither the body nor the doc mentioned this, so a maintainer reading only the doc would be surprised the screenshot-refresh PR now waits on the whole production ISO matrix.
The retained DEBUG=0 ISO boots `enforcing=0 console=ttyS0,115200n8` -- both boot paths in build-iso.sh apply that cmdline for every DEBUG value, because the E2E harness drives the guest over serial and the live root has no policy relabel pass. That was unremarkable while every ISO was disposable. This PR keeps them for 30 days under the name "production-iso-*", which makes the name a claim the artifact does not support. No behaviour change: the cmdline and the retention stay as they are. Whether the live environment ships SELinux-permissive is a product decision, and it belongs to #186 with signing and Secure Boot rather than to a CI change. What was missing was saying so where someone would look -- the cmdline itself, the retain step, and the CI skill doc -- so a green production-iso job is not mistaken for release-ready media. Also records the storage shape the same review raised: multi-GB per flavor per dispatch at 30-day retention, revisit the window before widening the matrix. DEBUG=0 does exclude the test credentials and sshd path (iso/live/src/configure-live.sh), so nothing secret is retained. 241 tests pass, YAML parses, build-iso.sh passes bash -n. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
5c6d04c to
75f6fc2
Compare
|
Follow-up on my previous comment: that commit put the branch into conflict with The collision was instructive, and it corrected two things I'd written:
Substance is unchanged: the retained Re-verified after the rebase:
|
castrojo
left a comment
There was a problem hiding this comment.
Reviewed diff and CI: retains production ISO artifacts in post-testing-e2e. All 14 checks pass. Outstanding: two independent maintainer reviews required (+2 policy).
…encrypted install (#215) * test(installer): assert offline payload and default Flatpaks survive encrypted install iso/scripts/luks-e2e.sh already installs to a LUKS2-encrypted disk, unlocks it, and boots to a GNOME session with no network route -- but nothing asserted that the offline embedded payload and the default Flatpak set actually persisted through that install, and a failed run never kept even non-sensitive metadata about the disk that failed. Add two checks on the installed, still network-isolated system: - bootc status confirms the booted image is exactly the offline payload fisherman's recipe named (targetImgref), not a substitute -- checked right after the graphical target, before login. (/usr/share/utah/contract.txt, tried in an earlier revision of this change, is the resolved *RPM* contract scripts/install-packages.py writes at image-build time; it says nothing about which OCI image or digest actually got deployed.) - Every default Flatpak in the Brewfile contract, using the same Brewfile-to-list conversion iso/live/Containerfile already uses to seed the ISO's own repo. This runs last, with a short retry: the 2026-09-19 verified run (docs/verification) shows the full default set only appears on the installed disk after roughly two minutes of uptime, so checking right after login would misreport apps that simply have not deployed yet. UTAH_E2E_FLATPAKS overrides the expected set; empty skips the check. A failure now also captures qemu-img metadata (size, format) for the disposable disk -- never the disk itself, which carries test credentials and stays excluded from every CI upload per existing policy. The capture moved into the EXIT trap rather than fail()'s body: a bare command failing under set -e (the installer step, for one) aborts straight to the trap without ever calling fail(). Renumber the script's phases from six to seven to name the booted-image check, and update docs/skills/local-testing.md and the embedded verification-README template to match. docs/skills/ci-workflows.md is left alone: open PR #198 already edits that file in the same area. New coverage in tests/test_iso_ci.py extracts the Flatpak set-difference block and runs it under bash with stubbed ssh_target/sleep, rather than only asserting strings are present in the script. This is a hard gate inside the luks job, which promote-to-testing and documentation depend on for every flavor. It has not been exercised against a real installed system -- only bash -n, the script-syntax gate, and the stubbed unit test above ran here. The first post-testing-e2e run is what proves or disproves whether fisherman actually carries the Brewfile-derived Flatpak set across the encrypted install; if it does not, UTAH_E2E_FLATPAKS is the escape hatch to unblock promotion while that gets investigated. Addresses #15. Signed-off-by: Danathar <Danathar@users.noreply.github.com> * fix(e2e): unlock disk diagnostics, sudo fallback for bootc status, document hatches - qemu-img info runs from the EXIT trap while QEMU still holds the write lock, so the retained log recorded a lock error instead of the size and format it exists to capture; read with -U. - bootc status ran only as the unprivileged test user, hard-failing the luks job as '<unreadable>' if that version needs root. Retry under sudo and add UTAH_E2E_PAYLOAD_CHECK="" to skip, matching UTAH_E2E_FLATPAKS. - Document both escape hatches in docs/skills/local-testing.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: Danathar <Danathar@users.noreply.github.com> Co-authored-by: Danathar <Danathar@users.noreply.github.com> Co-authored-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
castrojo
left a comment
There was a problem hiding this comment.
LGTM: retains gated production ISO artifacts in release workflow. All checks green.
Problem
The LUKS matrix verifies disposable DEBUG=1 media, but it discards every safe production ISO.
Decision
After every configured digest passes the offline LUKS matrix, compose a DEBUG=0 ISO and SHA-256 checksum per flavor, retain them as 30-day Actions artifacts, and require those compositions before testing-tag promotion. This intentionally does not choose a release host, naming policy, flavor policy, signing strategy, or Secure Boot policy; those remain in #186.
Check
python3 -m unittest tests.test_iso_ci -v— 23 tests passed.just check— reached the host-suite phase, then failed 4 existingtest_live_kernelpath-alias assertions (/homevs/var/hometemporary-root paths), unrelated to this workflow change.pre-commitnoractionlintis installed on this host.— hive: backend=omp