Skip to content

test(iso): add artifact-backed stable installer E2E - #68

Merged
castrojo merged 11 commits into
mainfrom
fix/installer-slitherer-webui-engine
Jul 20, 2026
Merged

castrojo merged 11 commits into
mainfrom
fix/installer-slitherer-webui-engine

Conversation

@castrojo

@castrojo castrojo commented Jul 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • preserve Titanoboa installer-hook and Flatpak inputs with a compatible build path
  • make ISO artifacts available to downstream E2E jobs
  • implement source workflow-run artifact download in the ISO E2E workflow
  • run stable main and NVIDIA installer E2E jobs on pull requests

Validation

  • pre-commit run --all-files
  • actionlint .github/workflows/*.yml
  • just check
  • python3 hack/check-docs.py

Stable build validation: https://github.com/projectbluefin/iso/actions/runs/29781623548

Summary by CodeRabbit

  • New Features

    • Added automated ISO smoke and end-to-end testing, including downloadable validation evidence.
    • Added a unified ISO test command for local validation.
    • Added documentation checks and expanded build, QA, release, and contribution guidance.
    • Pull request ISO builds now run stable image installation tests.
  • Bug Fixes

    • ISO promotion now requires a successful smoke-test validation before proceeding.
    • LTS ISO builds are manual-only until an upstream image issue is resolved.
  • Improvements

    • Build outputs now include manifests and checksums for easier verification.

@castrojo
castrojo requested a review from a team as a code owner July 20, 2026 21:49
@coderabbitai

coderabbitai Bot commented Jul 20, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@castrojo, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 41 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2db28f6f-2561-4aeb-8c75-0a43b2b3181e

📥 Commits

Reviewing files that changed from the base of the PR and between 15ac7dd and 8e786a6.

📒 Files selected for processing (19)
  • .github/workflows/iso-e2e-test.yml
  • .github/workflows/iso-smoke-test.yml
  • .github/workflows/iso-testsuite.yml
  • .github/workflows/pull-request.yml
  • .github/workflows/reusable-build-iso-anaconda.yml
  • .gitignore
  • Justfile
  • docs/architecture.md
  • docs/qa.md
  • docs/skills/build-iso/SKILL.md
  • docs/skills/test-iso/SKILL.md
  • hack/iso-e2e-test.sh
  • hack/iso-smoke-test.sh
  • hack/iso-test.sh
  • hack/local-iso-build.sh
  • tests/iso/data/bluefin-unattended.ks.tmpl
  • tests/iso/e2e.sh
  • tests/iso/smoke.sh
  • tests/iso/suite.sh
📝 Walkthrough

Walkthrough

This change adds ISO smoke and installer E2E validation, artifact manifests, smoke-gated promotion, pinned local ISO building, and a structured documentation and skill system with automated Markdown checks.

Changes

ISO build and validation pipeline

Layer / File(s) Summary
Build artifact production
.github/workflows/reusable-build-iso-anaconda.yml, .github/workflows/pull-request.yml, hack/local-iso-build.sh, iso_files/*
Builds use a pinned Titanoboa revision, publish ISO manifests and artifacts, expose release metadata, dispatch candidates for lab validation, and enable stable E2E checks.
Validation workflow orchestration
.github/workflows/iso-smoke-test.yml, .github/workflows/iso-e2e-test.yml, .github/workflows/iso-testsuite.yml, tests/iso/suite.sh, hack/iso-*-test.sh
Workflows resolve ISO URLs, paths, or artifacts, run validation harnesses, and upload smoke or E2E evidence.
QEMU validation harnesses
tests/iso/smoke.sh, tests/iso/e2e.sh, tests/iso/data/bluefin-unattended.ks.tmpl
Smoke validation checks boot evidence and screenshots; E2E validation performs unattended installation and post-install boot checks.
Smoke-gated promotion
.github/workflows/promote-iso.yml
Promotion now requires a successful smoke run matching the resolved source tag and passes gate outputs into preflight.

Documentation and repository controls

Layer / File(s) Summary
Canonical repository guidance
AGENTS.md, CONTRIBUTING.md, README.md, docs/architecture.md, docs/contributing.md, docs/qa.md, docs/release.md
Repository ownership, validation, release, workflow, and documentation procedures are consolidated into canonical references.
Task-specific skill guides
docs/skills/*
Build, test, release, documentation-maintenance, and skill-improvement procedures are added and routed through the skills index.
Documentation validation
hack/check-docs.py, .github/workflows/documentation-checks.yml, Justfile
Documentation budgets, skill metadata, local links, and scoped Justfile syntax checks are automated.
Repository hygiene adjustments
.gitignore, .github/workflows/validate-renovate.yml
Local worktrees and ISO test outputs are ignored, and the Renovate install argument is quoted.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

Suggested labels: enhancement

Poem

I’m a rabbit with ISOs tucked tight,
QEMU boots through the moonlit night.
Smoke proofs sparkle, releases gate,
Docs guide paws to the proper state.
Build, test, and hop along—
Every artifact sings its song.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding artifact-backed stable installer end-to-end testing.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/installer-slitherer-webui-engine

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

castrojo and others added 11 commits July 20, 2026 19:11
Assisted-by: GPT-5.6 Terra via GitHub Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: GPT-5 via pi
Move ISO validation ownership out of hack/ and add the testsuite reusable workflow caller.

Assisted-by: GPT-5 via pi
Keep factory builds unchanged while making local build paths configurable.

Assisted-by: GPT-5 via pi
@castrojo
castrojo force-pushed the fix/installer-slitherer-webui-engine branch from 28a4998 to 8e786a6 Compare July 20, 2026 23:12
@dosubot dosubot Bot added size:L and removed size:XXL labels Jul 20, 2026
@castrojo
castrojo merged commit efa1235 into main Jul 20, 2026
4 of 12 checks passed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 18

🧹 Nitpick comments (5)
.github/workflows/promote-iso.yml (1)

70-96: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Eliminate redundant API call and simplify JSON parsing.

The script makes a second API call to fetch the run details and checks CONCLUSION, but this is unnecessary because:

  1. The initial jq query already filters by select(.conclusion == "success"), so success is guaranteed.
  2. The workflow_runs objects returned by the first API call already include the .html_url field.

Extracting .html_url in the first pass allows you to delete the second gh api call and the subsequent state checks, improving reliability and reducing rate-limit usage.

(Note: If iso-smoke-test.yml runs frequently on pull requests, the per_page=100 limit in the initial query might not look back far enough to find the release run. If release smoke tests use a distinct trigger, consider appending &event=repository_dispatch (or the relevant event) to the API URL to filter out PR noise.)

♻️ Proposed refactor
-          RUN_ID=$(printf '%s' "${WORKFLOW_JSON}" | jq -r --arg smoke_title "${SMOKE_TITLE}" '
-            .workflow_runs
-            | map(select(.conclusion == "success" and .display_title == $smoke_title))
-            | sort_by(.created_at)
-            | last
-            | .id // empty
-          ')
-
-          if [ -z "${RUN_ID}" ] || [ "${RUN_ID}" = "null" ]; then
-            echo "No successful ISO smoke validation found for source prerelease '${SOURCE_TAG}'"
-            exit 1
-          fi
-
-          RUN_JSON=$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs/${RUN_ID}" 2>/dev/null || true)
-          if [ -z "${RUN_JSON}" ]; then
-            echo "Unable to inspect ISO smoke validation run ${RUN_ID}"
-            exit 1
-          fi
-
-          CONCLUSION=$(printf '%s' "${RUN_JSON}" | jq -r '.conclusion // empty')
-          HTML_URL=$(printf '%s' "${RUN_JSON}" | jq -r '.html_url // empty')
-
-          if [ "${CONCLUSION}" != "success" ]; then
-            echo "ISO smoke validation run ${RUN_ID} did not succeed (conclusion: ${CONCLUSION:-unknown})"
-            exit 1
-          fi
+          HTML_URL=$(printf '%s' "${WORKFLOW_JSON}" | jq -r --arg smoke_title "${SMOKE_TITLE}" '
+            .workflow_runs
+            | map(select(.conclusion == "success" and .display_title == $smoke_title))
+            | sort_by(.created_at)
+            | last
+            | .html_url // empty
+          ')
+
+          if [ -z "${HTML_URL}" ]; then
+            echo "No successful ISO smoke validation found for source prerelease '${SOURCE_TAG}'"
+            exit 1
+          fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/promote-iso.yml around lines 70 - 96, Update the initial
jq extraction that selects the successful workflow run by capturing both its id
and html_url, then remove the RUN_JSON gh api call, CONCLUSION parsing, and
subsequent validation block. Preserve the no-run failure check and use the
extracted html_url for the remaining promotion flow; if applicable, narrow the
workflow-runs API query to the release smoke-test event to avoid PR-run noise.
.github/workflows/documentation-checks.yml (1)

31-32: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

Set persist-credentials: false in checkout step.

To adhere to security best practices and prevent credential persistence in the local git repository (which could potentially be read by subsequent steps), consider explicitly disabling credential persistence.

🛡️ Proposed fix to disable credential persistence
       - name: Checkout
         uses: actions/checkout@v4
+        with:
+          persist-credentials: false
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/documentation-checks.yml around lines 31 - 32, Update the
“Checkout” step using actions/checkout@v4 to explicitly set persist-credentials
to false, preventing checkout credentials from being retained for subsequent
workflow steps.

Source: Linters/SAST tools

hack/check-docs.py (1)

77-88: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Enhance link resolution robustness.

The current link checker might produce false positive failures for a few valid Markdown link formats:

  1. Root-relative links: A path like /docs/architecture.md when appended to path.parent will resolve to the root of the file system rather than the repository root.
  2. Link titles: A link with a title like [text](target.md "Title") will include the title in target, causing the file existence check to fail.
  3. URL encoding: Spaces encoded as %20 won't map to local file paths without unquoting.

Consider refining the extraction and resolution logic to gracefully handle these cases.

🛠️ Proposed refactor

First, add import urllib.parse at the top of the file (e.g., near import sys). Then, update the link checking logic:

 def check_links(errors: list[str]) -> None:
     for path in markdown_files():
         for target in LINK_RE.findall(path.read_text(encoding="utf-8")):
+            # Strip optional Markdown link title (e.g., `link "Title"`)
+            target = target.split()[0]
             if target.startswith(("http://", "https://", "#", "mailto:")):
                 continue
-            target_path = target.split("#", 1)[0]
+            # Decode URL-encoded characters (like %20)
+            target_path = urllib.parse.unquote(target.split("#", 1)[0])
             if not target_path:
                 continue
-            resolved = (path.parent / target_path).resolve()
+            
+            # Resolve relative to repository root if absolute, else relative to current file
+            if target_path.startswith("/"):
+                resolved = (ROOT / target_path.lstrip("/")).resolve()
+            else:
+                resolved = (path.parent / target_path).resolve()
+
             if not resolved.is_file():
                 fail(errors, f"{path.relative_to(ROOT)}: missing link target {target}")
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/check-docs.py` around lines 77 - 88, Update check_links to normalize
local Markdown targets before resolving them: strip optional link titles,
URL-decode paths with urllib.parse, and interpret leading-slash targets relative
to ROOT rather than the filesystem root. Preserve existing handling for
external, anchor, mailto, and empty targets, then validate the normalized path
with the current missing-link error flow.
tests/iso/e2e.sh (1)

91-92: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Bind the kickstart HTTP server to loopback instead of all interfaces.

QEMU user-mode networking (slirp) only requires the host service to be reachable from the guest via the emulated NAT; binding to 0.0.0.0 exposes the server on every host interface unnecessarily.

🔧 Proposed fix
-python3 -m http.server "$http_port" --bind 0.0.0.0 --directory "$http_root" >/dev/null 2>&1 &
+python3 -m http.server "$http_port" --bind 127.0.0.1 --directory "$http_root" >/dev/null 2>&1 &
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/iso/e2e.sh` around lines 91 - 92, Update the HTTP server startup
command in the e2e test to bind to loopback only instead of 0.0.0.0, while
preserving the existing port, directory, background execution, and output
redirection behavior.
.github/workflows/iso-e2e-test.yml (1)

110-111: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Prefer find for robust recursive file discovery.

Using **/*.iso without first setting shopt -s globstar causes bash to treat ** as just *, which matches exactly one subdirectory level (e2e-input/*/*.iso). If an artifact unzips into a different directory depth, this will silently fail to find the ISO.

Using find (as seen in iso-smoke-test.yml) ensures reliable recursive discovery.

♻️ Proposed refactor
-            shopt -s nullglob
-            matches=(e2e-input/**/*.iso e2e-input/*.iso)
+            mapfile -d '' matches < <(find e2e-input -type f -name '*.iso' -print0 | sort -z)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/iso-e2e-test.yml around lines 110 - 111, Replace the
matches assignment in the ISO discovery step with a find-based recursive search
rooted at e2e-input, preserving the .iso extension filter and array-based
handling used by the surrounding workflow. Remove the glob-based discovery and
its nullglob setup if no longer needed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/iso-e2e-test.yml:
- Around line 85-104: Remove direct GitHub expression interpolation of
artifact_name from the bash script. Add INPUT_ARTIFACT_NAME and
DISPATCH_ARTIFACT_NAME to the step’s env block, then derive artifact_name from
those environment variables using the same dispatch-over-input precedence as
iso_url, source_run_id, and iso_path.

In @.github/workflows/pull-request.yml:
- Around line 30-47: Scope permissions and secrets for both test-stable-main and
test-stable-nvidia jobs. Add an explicit minimal permissions block, such as
contents: read, and replace secrets: inherit with only the specific secrets
required by the reusable iso-e2e-test.yml workflow, preserving the existing
inputs and job dependencies.

In @.github/workflows/reusable-build-iso-anaconda.yml:
- Around line 216-230: Align candidate_id generation across the manifest and
lab-dispatch steps. Keep the manifest candidate_id built from GITHUB_RUN_ID and
OUTPUT_NAME, and update the dispatch step at
.github/workflows/reusable-build-iso-anaconda.yml lines 538-554 to strip the
.iso suffix from iso_filename before constructing candidate_id so both records
use the same value.
- Around line 209-230: Update the manifest-generation step around the heredoc
and the `inputs.image_tag || matrix.image_version` expression so the resolved
image tag is passed through the step’s `env` mapping and referenced only as a
shell environment variable inside the script. Remove the direct GitHub Actions
expression from the heredoc while preserving the existing fallback and manifest
value.
- Around line 518-522: Move the dispatch guard from the “Dispatch exact
candidates to lab” step to its containing job scope, defining a job-level flag
based on secrets.LAB_DISPATCH_TOKEN. Update the step’s if condition to use that
job-level flag while retaining the existing LAB_DISPATCH_TOKEN and GH_TOKEN
environment variables.

In @.github/workflows/validate-renovate.yml:
- Line 31: Pin the Renovate CLI dependency in the workflow step using an exact
version instead of the mutable RENOVATE_VERSION value set to latest; update the
npm install command while preserving the existing global installation behavior.

In @.gitignore:
- Around line 31-35: Correct the E2E test artifact entry in .gitignore by
removing the leading dot from .e2e-output/, so it matches the generated
e2e-output/ directory referenced by the workflow. Leave the existing
smoke-output/ and iso-test-output/ entries unchanged.

In `@AGENTS.md`:
- Around line 41-49: Unify the local-command ownership guidance across all three
sites: in AGENTS.md lines 41-49, add tests/iso/ as the canonical owner for local
test commands and mark hack/ as compatibility-only; update CONTRIBUTING.md lines
14-23 to match; and revise docs/contributing.md lines 7-18 so its owner table
assigns smoke/E2E behavior and local test code to tests/iso/ while treating
hack/ as compatibility-only.
- Around line 62-80: Centralize validation instructions in docs/qa.md by
replacing the duplicated command lists with links to the canonical procedure:
update AGENTS.md lines 62-80, CONTRIBUTING.md lines 25-41, README.md lines
34-40, and docs/contributing.md lines 28-36, using a concise QA reference in
README.md and preserving any document-specific context needed.

In `@docs/release.md`:
- Around line 33-39: Update the `lts` production-files filter documentation to
explicitly exclude `*-lts-hwe-*.iso*` from the broader `*-lts-*.iso*` match,
using the same exclusion semantics as the promotion workflow. Verify the
documented storage and release-asset filters remain aligned and keep the
`lts-hwe` selection unchanged.

In `@docs/skills/build-iso/SKILL.md`:
- Around line 31-32: Renumber the duplicated procedure step in the build-ISO
instructions so the architecture-documentation step and every subsequent step
follow one sequential Markdown numbering order, without changing their wording
or order.

In `@docs/skills/improve-skill/SKILL.md`:
- Line 27: Update the verification step in the skill instructions to make
“repository preflight” actionable by linking to the canonical validation
document or naming the exact preflight command. Keep hack/check-docs.py and
avoid duplicating the repository’s existing command list.

In `@docs/skills/maintain-docs/SKILL.md`:
- Around line 40-47: Centralize the shared validation commands in
docs/skills/maintain-docs/SKILL.md lines 40-47 as the canonical procedure, or
link directly to the existing canonical owner there. Replace the duplicated
validation blocks in docs/skills/build-iso/SKILL.md lines 38-43 and
docs/skills/test-iso/SKILL.md lines 29-34 with links to that procedure,
retaining only their build-specific and ISO-test-specific commands.

In `@tests/iso/e2e.sh`:
- Around line 108-113: Make the kickstart endpoint readiness loop fail
explicitly when curl never succeeds after all 20 attempts. Track whether the
request succeeded in the loop around curl, then exit with a clear error after
the loop if it remained unreachable, matching the fail-fast behavior used by the
post-install QMP-socket wait.
- Around line 208-237: Separate the QMP screendump block from the ffmpeg
availability check in the e2e capture flow. Always run the Python screendump
command to produce final_screen_ppm, then conditionally run only the PPM-to-PNG
conversion when ffmpeg is available, preserving the existing best-effort
conversion behavior.

In `@tests/iso/smoke.sh`:
- Around line 222-247: Change the no-evidence branch after the boot-monitor loop
so that when boot_evidence remains "none" at timeout, the smoke test fails
instead of assigning "qemu-running" and continuing to pass. Preserve the
existing success path when serial boot evidence is detected and retain the
current QEMU liveness checks.
- Around line 195-208: Quote the combined machine and accelerator argument in
the QEMU invocation, updating the -machine option to use
"${qemu_machine},accel=${qemu_accel}" and preserve the existing command
behavior.

In `@tests/iso/suite.sh`:
- Around line 22-28: Update the script invocations in the suite harness to call
the local smoke.sh and e2e.sh files instead of iso-smoke-test.sh and
iso-e2e-test.sh, preserving the existing iso_path, output arguments, and status
handling.

---

Nitpick comments:
In @.github/workflows/documentation-checks.yml:
- Around line 31-32: Update the “Checkout” step using actions/checkout@v4 to
explicitly set persist-credentials to false, preventing checkout credentials
from being retained for subsequent workflow steps.

In @.github/workflows/iso-e2e-test.yml:
- Around line 110-111: Replace the matches assignment in the ISO discovery step
with a find-based recursive search rooted at e2e-input, preserving the .iso
extension filter and array-based handling used by the surrounding workflow.
Remove the glob-based discovery and its nullglob setup if no longer needed.

In @.github/workflows/promote-iso.yml:
- Around line 70-96: Update the initial jq extraction that selects the
successful workflow run by capturing both its id and html_url, then remove the
RUN_JSON gh api call, CONCLUSION parsing, and subsequent validation block.
Preserve the no-run failure check and use the extracted html_url for the
remaining promotion flow; if applicable, narrow the workflow-runs API query to
the release smoke-test event to avoid PR-run noise.

In `@hack/check-docs.py`:
- Around line 77-88: Update check_links to normalize local Markdown targets
before resolving them: strip optional link titles, URL-decode paths with
urllib.parse, and interpret leading-slash targets relative to ROOT rather than
the filesystem root. Preserve existing handling for external, anchor, mailto,
and empty targets, then validate the normalized path with the current
missing-link error flow.

In `@tests/iso/e2e.sh`:
- Around line 91-92: Update the HTTP server startup command in the e2e test to
bind to loopback only instead of 0.0.0.0, while preserving the existing port,
directory, background execution, and output redirection behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 75b27907-e9ea-4baf-8eaa-894023ed5769

📥 Commits

Reviewing files that changed from the base of the PR and between c78ba85 and 15ac7dd.

📒 Files selected for processing (36)
  • .github/workflows/build-iso-lts.yml
  • .github/workflows/documentation-checks.yml
  • .github/workflows/iso-e2e-test.yml
  • .github/workflows/iso-smoke-test.yml
  • .github/workflows/iso-testsuite.yml
  • .github/workflows/promote-iso.yml
  • .github/workflows/pull-request.yml
  • .github/workflows/reusable-build-iso-anaconda.yml
  • .github/workflows/validate-renovate.yml
  • .gitignore
  • AGENTS.md
  • CONTRIBUTING.md
  • Justfile
  • README.md
  • docs/architecture.md
  • docs/contributing.md
  • docs/qa.md
  • docs/release.md
  • docs/skills/build-iso/SKILL.md
  • docs/skills/improve-skill/SKILL.md
  • docs/skills/improve-skill/references/skill-review-checklist.md
  • docs/skills/index.md
  • docs/skills/maintain-docs/SKILL.md
  • docs/skills/release-iso/SKILL.md
  • docs/skills/test-iso/SKILL.md
  • hack/check-docs.py
  • hack/iso-e2e-test.sh
  • hack/iso-smoke-test.sh
  • hack/iso-test.sh
  • hack/local-iso-build.sh
  • iso_files/configure_iso_anaconda.sh
  • iso_files/configure_lts_iso_anaconda.sh
  • tests/iso/data/bluefin-unattended.ks.tmpl
  • tests/iso/e2e.sh
  • tests/iso/smoke.sh
  • tests/iso/suite.sh
💤 Files with no reviewable changes (2)
  • iso_files/configure_lts_iso_anaconda.sh
  • iso_files/configure_iso_anaconda.sh

Comment on lines +85 to +104
env:
INPUT_ISO_URL: ${{ inputs.iso_url }}
INPUT_SOURCE_RUN_ID: ${{ inputs.source_run_id }}
INPUT_ISO_PATH: ${{ inputs.iso_path }}
DISPATCH_ISO_URL: ${{ github.event.client_payload.iso_url }}
DISPATCH_SOURCE_RUN_ID: ${{ github.event.client_payload.source_run_id }}
DISPATCH_ISO_PATH: ${{ github.event.client_payload.iso_path }}
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
iso_url="${DISPATCH_ISO_URL:-${INPUT_ISO_URL}}"
source_run_id="${DISPATCH_SOURCE_RUN_ID:-${INPUT_SOURCE_RUN_ID}}"
iso_path="${DISPATCH_ISO_PATH:-${INPUT_ISO_PATH}}"
if [[ -n "$iso_url" ]]; then
mkdir -p e2e-input
curl --fail --location --retry 3 --retry-all-errors --output e2e-input/source.iso "$iso_url"
iso_path="$PWD/e2e-input/source.iso"
elif [[ -n "$source_run_id" ]]; then
mkdir -p e2e-input
artifact_name="${{ github.event.client_payload.artifact_name || inputs.artifact_name || '' }}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🔴 Critical | ⚡ Quick win

Fix critical script injection vulnerability by passing artifact_name via environment variables.

Interpolating the user-controllable input inputs.artifact_name directly into the bash script using ${{ ... }} exposes the workflow to remote code execution (RCE) via script injection.

Add INPUT_ARTIFACT_NAME and DISPATCH_ARTIFACT_NAME to the env block, and retrieve the value safely via environment variables, matching the pattern used for other inputs.

🔒️ Proposed fix for secure input handling
       - name: Resolve ISO input
         id: context
         env:
           INPUT_ISO_URL: ${{ inputs.iso_url }}
           INPUT_SOURCE_RUN_ID: ${{ inputs.source_run_id }}
+          INPUT_ARTIFACT_NAME: ${{ inputs.artifact_name }}
           INPUT_ISO_PATH: ${{ inputs.iso_path }}
           DISPATCH_ISO_URL: ${{ github.event.client_payload.iso_url }}
           DISPATCH_SOURCE_RUN_ID: ${{ github.event.client_payload.source_run_id }}
+          DISPATCH_ARTIFACT_NAME: ${{ github.event.client_payload.artifact_name }}
           DISPATCH_ISO_PATH: ${{ github.event.client_payload.iso_path }}
           GH_TOKEN: ${{ github.token }}
         run: |
           set -euo pipefail
           iso_url="${DISPATCH_ISO_URL:-${INPUT_ISO_URL}}"
           source_run_id="${DISPATCH_SOURCE_RUN_ID:-${INPUT_SOURCE_RUN_ID}}"
           iso_path="${DISPATCH_ISO_PATH:-${INPUT_ISO_PATH}}"
           if [[ -n "$iso_url" ]]; then
             mkdir -p e2e-input
             curl --fail --location --retry 3 --retry-all-errors --output e2e-input/source.iso "$iso_url"
             iso_path="$PWD/e2e-input/source.iso"
           elif [[ -n "$source_run_id" ]]; then
             mkdir -p e2e-input
-            artifact_name="${{ github.event.client_payload.artifact_name || inputs.artifact_name || '' }}"
+            artifact_name="${DISPATCH_ARTIFACT_NAME:-${INPUT_ARTIFACT_NAME}}"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
env:
INPUT_ISO_URL: ${{ inputs.iso_url }}
INPUT_SOURCE_RUN_ID: ${{ inputs.source_run_id }}
INPUT_ISO_PATH: ${{ inputs.iso_path }}
DISPATCH_ISO_URL: ${{ github.event.client_payload.iso_url }}
DISPATCH_SOURCE_RUN_ID: ${{ github.event.client_payload.source_run_id }}
DISPATCH_ISO_PATH: ${{ github.event.client_payload.iso_path }}
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
iso_url="${DISPATCH_ISO_URL:-${INPUT_ISO_URL}}"
source_run_id="${DISPATCH_SOURCE_RUN_ID:-${INPUT_SOURCE_RUN_ID}}"
iso_path="${DISPATCH_ISO_PATH:-${INPUT_ISO_PATH}}"
if [[ -n "$iso_url" ]]; then
mkdir -p e2e-input
curl --fail --location --retry 3 --retry-all-errors --output e2e-input/source.iso "$iso_url"
iso_path="$PWD/e2e-input/source.iso"
elif [[ -n "$source_run_id" ]]; then
mkdir -p e2e-input
artifact_name="${{ github.event.client_payload.artifact_name || inputs.artifact_name || '' }}"
env:
INPUT_ISO_URL: ${{ inputs.iso_url }}
INPUT_SOURCE_RUN_ID: ${{ inputs.source_run_id }}
INPUT_ARTIFACT_NAME: ${{ inputs.artifact_name }}
INPUT_ISO_PATH: ${{ inputs.iso_path }}
DISPATCH_ISO_URL: ${{ github.event.client_payload.iso_url }}
DISPATCH_SOURCE_RUN_ID: ${{ github.event.client_payload.source_run_id }}
DISPATCH_ARTIFACT_NAME: ${{ github.event.client_payload.artifact_name }}
DISPATCH_ISO_PATH: ${{ github.event.client_payload.iso_path }}
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
iso_url="${DISPATCH_ISO_URL:-${INPUT_ISO_URL}}"
source_run_id="${DISPATCH_SOURCE_RUN_ID:-${INPUT_SOURCE_RUN_ID}}"
iso_path="${DISPATCH_ISO_PATH:-${INPUT_ISO_PATH}}"
if [[ -n "$iso_url" ]]; then
mkdir -p e2e-input
curl --fail --location --retry 3 --retry-all-errors --output e2e-input/source.iso "$iso_url"
iso_path="$PWD/e2e-input/source.iso"
elif [[ -n "$source_run_id" ]]; then
mkdir -p e2e-input
artifact_name="${DISPATCH_ARTIFACT_NAME:-${INPUT_ARTIFACT_NAME}}"
🧰 Tools
🪛 zizmor (1.26.1)

[error] 104-104: code injection via template expansion (template-injection): may expand into attacker-controllable code

(template-injection)


[error] 104-104: code injection via template expansion (template-injection): may expand into attacker-controllable code

(template-injection)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/iso-e2e-test.yml around lines 85 - 104, Remove direct
GitHub expression interpolation of artifact_name from the bash script. Add
INPUT_ARTIFACT_NAME and DISPATCH_ARTIFACT_NAME to the step’s env block, then
derive artifact_name from those environment variables using the same
dispatch-over-input precedence as iso_url, source_run_id, and iso_path.

Source: Linters/SAST tools

Comment thread .github/workflows/pull-request.yml Resolved
Comment on lines +216 to +230
cat > "${OUTPUT_DIRECTORY}/${OUTPUT_NAME}.manifest.json" <<EOF
{
"candidate_id": "${GITHUB_RUN_ID}-${OUTPUT_NAME}",
"source_repo": "${GITHUB_REPOSITORY}",
"source_sha": "${GITHUB_SHA}",
"source_run_id": "${GITHUB_RUN_ID}",
"variant": "${IMAGE_VERSION}",
"flavor": "${FLAVOR}",
"platform": "${PLATFORM:-$(uname -m)}",
"image_ref": "${IMAGE_REF:-${IMAGE_REGISTRY}/${IMAGE_NAME}}",
"image_tag": "${{ inputs.image_tag || matrix.image_version }}",
"iso_filename": "${OUTPUT_NAME}.iso",
"iso_sha256": "${iso_sha256}"
}
EOF

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

candidate_id format diverges between manifest.json and the lab-dispatch payload for the same artifact. Both are computed independently from ${GITHUB_RUN_ID}/${SOURCE_RUN_ID} plus a filename, but one includes the .iso suffix and the other doesn't, so the two records for the same build won't correlate by candidate_id.

  • .github/workflows/reusable-build-iso-anaconda.yml#L216-L230: manifest.json's candidate_id is "${GITHUB_RUN_ID}-${OUTPUT_NAME}" (no .iso); align this with the dispatch format or vice versa.
  • .github/workflows/reusable-build-iso-anaconda.yml#L538-L554: dispatch step derives candidate_id from iso_filename ("${OUTPUT_NAME}.iso"), producing a trailing .iso not present in the manifest's value — strip the .iso suffix (e.g. iso_basename="${iso_filename%.iso}") before building candidate_id so it matches the manifest.
🧰 Tools
🪛 zizmor (1.26.1)

[error] 226-226: code injection via template expansion (template-injection): may expand into attacker-controllable code

(template-injection)


[warning] 226-226: code injection via template expansion (template-injection): may expand into attacker-controllable code

(template-injection)

📍 Affects 1 file
  • .github/workflows/reusable-build-iso-anaconda.yml#L216-L230 (this comment)
  • .github/workflows/reusable-build-iso-anaconda.yml#L538-L554
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/reusable-build-iso-anaconda.yml around lines 216 - 230,
Align candidate_id generation across the manifest and lab-dispatch steps. Keep
the manifest candidate_id built from GITHUB_RUN_ID and OUTPUT_NAME, and update
the dispatch step at .github/workflows/reusable-build-iso-anaconda.yml lines
538-554 to strip the .iso suffix from iso_filename before constructing
candidate_id so both records use the same value.

Comment on lines +518 to +522
- name: Dispatch exact candidates to lab
if: env.LAB_DISPATCH_TOKEN != ''
env:
LAB_DISPATCH_TOKEN: ${{ secrets.LAB_DISPATCH_TOKEN }}
GH_TOKEN: ${{ secrets.LAB_DISPATCH_TOKEN }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

Can GitHub Actions step if:condition referencesecretscontext directly, e.g.if: secrets.MY_SECRET != ''?

💡 Result:

No, you cannot directly reference the secrets context in a GitHub Actions if: conditional [1][2][3]. According to official GitHub documentation, secrets cannot be used directly in if: expressions [1][2][4]. If you attempt to use them, they will not behave as expected. The recommended workaround is to map the secret to an environment variable at the job or step level, and then reference that environment variable within your if: condition [1][3][5]. Example of the recommended approach: jobs: example-job: runs-on: ubuntu-latest env: # Map the secret to an env variable MY_SECRET_SET: ${{ secrets.MY_SECRET!= '' }} steps: - name: Run conditional step # Reference the env variable instead if: env.MY_SECRET_SET == 'true' run: echo "The secret is set!" When a secret has not been set or does not exist, expressions referencing it will evaluate to an empty string [1][2][3]. Note that because of this behavior, typos in secret names will fail silently (evaluating to an empty string) rather than throwing an error [5].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

file=".github/workflows/reusable-build-iso-anaconda.yml"

echo "== Matching lines =="
rg -n "LAB_DISPATCH_TOKEN|Dispatch exact candidates to lab" "$file"

echo
echo "== Surrounding context =="
sed -n '480,545p' "$file"

Repository: projectbluefin/iso

Length of output: 3647


🌐 Web query:

GitHub Actions step if:evaluation order relative to the same step'senv: block

💡 Result:

In GitHub Actions, the if conditional is evaluated by the GitHub Actions service runner before the job or step is sent to the virtual machine runner [1][2]. Because the if conditional is processed during this initial phase, it acts as a gatekeeper that determines whether the step will be executed at all [1][3]. In contrast, expressions used within an env block—or other parts of a step definition like run, with, or name—are generally evaluated as part of the step's configuration setup [4][5]. A critical detail of this architecture is that GitHub Actions often performs these interpolations and evaluations upfront to fully define the step's configuration before checking the if condition [4]. Consequently, if you use an expression in your env block that results in an error (such as referencing a context that doesn't exist or using invalid syntax), the step may fail during this configuration/interpolation phase regardless of whether the if conditional would have caused the step to be skipped [4]. Because of this behavior, you should not rely on the if conditional to "short-circuit" or protect against invalid expressions within the same step's env block [4]. If a step depends on conditional logic to determine whether certain environment variables should be set or accessed, it is safer to: 1. Use safe, resilient expressions (e.g., handling null or empty values gracefully) [4]. 2. Break the logic into separate steps if the environment variable setup itself must be conditional [6]. Summary of evaluation: - The if conditional is processed by the GitHub Actions service [1][2]. - Step configuration, including env blocks and other interpolations, is evaluated as the step is being prepared for execution [4][5]. - Interpolation errors within a step definition can cause workflow failure before the if condition is even processed or respected [4].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

file=".github/workflows/reusable-build-iso-anaconda.yml"

sed -n '1,45p' "$file"

Repository: projectbluefin/iso

Length of output: 1315


Move the dispatch guard to job scope if: env.LAB_DISPATCH_TOKEN != '' reads a step-local value that isn't available when the condition is evaluated, so this dispatch step will never run. Define a job-level flag from secrets.LAB_DISPATCH_TOKEN and gate the step on that instead.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/reusable-build-iso-anaconda.yml around lines 518 - 522,
Move the dispatch guard from the “Dispatch exact candidates to lab” step to its
containing job scope, defining a job-level flag based on
secrets.LAB_DISPATCH_TOKEN. Update the step’s if condition to use that job-level
flag while retaining the existing LAB_DISPATCH_TOKEN and GH_TOKEN environment
variables.

Comment thread tests/iso/e2e.sh
Comment on lines +108 to +113
for _ in $(seq 1 20); do
if curl --silent --show-error --fail "http://127.0.0.1:${http_port}/kickstart.ks" >/dev/null 2>&1; then
break
fi
sleep 1
done

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

No fail-fast if the kickstart HTTP endpoint never becomes reachable.

Unlike the post-install QMP-socket wait (Lines 199-206), which explicitly fails if the socket never appears, this loop silently continues even if curl never succeeds, letting the installer boot proceed and only fail after the full install_timeout.

🔧 Proposed fix
+ready=""
 for _ in $(seq 1 20); do
     if curl --silent --show-error --fail "http://127.0.0.1:${http_port}/kickstart.ks" >/dev/null 2>&1; then
+        ready=1
         break
     fi
     sleep 1
 done
+if [[ -z "$ready" ]]; then
+    echo "Kickstart HTTP server did not become reachable" >&2
+    exit 1
+fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for _ in $(seq 1 20); do
if curl --silent --show-error --fail "http://127.0.0.1:${http_port}/kickstart.ks" >/dev/null 2>&1; then
break
fi
sleep 1
done
ready=""
for _ in $(seq 1 20); do
if curl --silent --show-error --fail "http://127.0.0.1:${http_port}/kickstart.ks" >/dev/null 2>&1; then
ready=1
break
fi
sleep 1
done
if [[ -z "$ready" ]]; then
echo "Kickstart HTTP server did not become reachable" >&2
exit 1
fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/iso/e2e.sh` around lines 108 - 113, Make the kickstart endpoint
readiness loop fail explicitly when curl never succeeds after all 20 attempts.
Track whether the request succeeded in the loop around curl, then exit with a
clear error after the loop if it remained unreachable, matching the fail-fast
behavior used by the post-install QMP-socket wait.

Comment thread tests/iso/e2e.sh
Comment on lines +208 to +237
if command -v ffmpeg >/dev/null 2>&1; then
python3 - "$post_qmp_socket" "$final_screen_ppm" <<'PY'
import json
import socket
import sys

sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.settimeout(10)
sock.connect(sys.argv[1])

def read_message():
data = b''
while b'\r\n' not in data:
chunk = sock.recv(4096)
if not chunk:
raise RuntimeError('QMP socket closed')
data += chunk
return json.loads(data.split(b'\r\n', 1)[0])

read_message()
sock.sendall(b'{"execute":"qmp_capabilities"}\r\n')
read_message()
command = {"execute": "screendump", "arguments": {"filename": sys.argv[2]}}
sock.sendall((json.dumps(command) + "\r\n").encode())
response = read_message()
if 'error' in response:
raise RuntimeError(response['error']['desc'])
PY
ffmpeg -y -loglevel error -i "$final_screen_ppm" "$final_screen_png" >/dev/null 2>&1 || true
fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Screendump capture unnecessarily gated on ffmpeg.

The QMP screendump call itself doesn't require ffmpeg — only the PPM→PNG conversion does. Wrapping both in if command -v ffmpeg means the raw screen evidence is skipped entirely when ffmpeg isn't installed, unlike smoke.sh's convert_screen_to_png, which always captures the raw .ppm and only conditionally converts it.

🔧 Proposed fix
-if command -v ffmpeg >/dev/null 2>&1; then
-    python3 - "$post_qmp_socket" "$final_screen_ppm" <<'PY'
+python3 - "$post_qmp_socket" "$final_screen_ppm" <<'PY'
 ...
 PY
+if command -v ffmpeg >/dev/null 2>&1; then
     ffmpeg -y -loglevel error -i "$final_screen_ppm" "$final_screen_png" >/dev/null 2>&1 || true
 fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if command -v ffmpeg >/dev/null 2>&1; then
python3 - "$post_qmp_socket" "$final_screen_ppm" <<'PY'
import json
import socket
import sys
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.settimeout(10)
sock.connect(sys.argv[1])
def read_message():
data = b''
while b'\r\n' not in data:
chunk = sock.recv(4096)
if not chunk:
raise RuntimeError('QMP socket closed')
data += chunk
return json.loads(data.split(b'\r\n', 1)[0])
read_message()
sock.sendall(b'{"execute":"qmp_capabilities"}\r\n')
read_message()
command = {"execute": "screendump", "arguments": {"filename": sys.argv[2]}}
sock.sendall((json.dumps(command) + "\r\n").encode())
response = read_message()
if 'error' in response:
raise RuntimeError(response['error']['desc'])
PY
ffmpeg -y -loglevel error -i "$final_screen_ppm" "$final_screen_png" >/dev/null 2>&1 || true
fi
python3 - "$post_qmp_socket" "$final_screen_ppm" <<'PY'
import json
import socket
import sys
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.settimeout(10)
sock.connect(sys.argv[1])
def read_message():
data = b''
while b'\r\n' not in data:
chunk = sock.recv(4096)
if not chunk:
raise RuntimeError('QMP socket closed')
data += chunk
return json.loads(data.split(b'\r\n', 1)[0])
read_message()
sock.sendall(b'{"execute":"qmp_capabilities"}\r\n')
read_message()
command = {"execute": "screendump", "arguments": {"filename": sys.argv[2]}}
sock.sendall((json.dumps(command) + "\r\n").encode())
response = read_message()
if 'error' in response:
raise RuntimeError(response['error']['desc'])
PY
if command -v ffmpeg >/dev/null 2>&1; then
ffmpeg -y -loglevel error -i "$final_screen_ppm" "$final_screen_png" >/dev/null 2>&1 || true
fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/iso/e2e.sh` around lines 208 - 237, Separate the QMP screendump block
from the ffmpeg availability check in the e2e capture flow. Always run the
Python screendump command to produce final_screen_ppm, then conditionally run
only the PPM-to-PNG conversion when ffmpeg is available, preserving the existing
best-effort conversion behavior.

Comment thread tests/iso/smoke.sh Resolved
Comment thread tests/iso/smoke.sh
Comment on lines +222 to +247
deadline=$((SECONDS + timeout_seconds))
started_at=$SECONDS
while (( SECONDS < deadline )); do
if ! kill -0 "$qemu_pid" 2>/dev/null; then
wait "$qemu_pid" || true
fail "QEMU exited before the ISO completed its boot smoke window"
fi

qemu_status="$(qmp_status)" || fail "Unable to query QEMU monitor status"
[[ "$qemu_status" == "running" ]] || fail "QEMU is not running (status: $qemu_status)"

if grep -Eqi 'Linux version|dracut|anaconda|Starting ' "$serial_file" 2>/dev/null; then
boot_evidence="serial-console"
break
fi

sleep 5
done

if [[ "$boot_evidence" == "none" ]]; then
boot_evidence="qemu-running"
fi

capture_screen || fail "Unable to capture the QEMU display"
[[ -s "$screen_file" ]] || fail "QEMU display capture is empty"
result_message="QEMU ran for $((SECONDS - started_at)) seconds with ${boot_evidence} evidence"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

Boot-evidence fallback can produce a false "passed" smoke result.

If none of Linux version|dracut|anaconda|Starting appear in the serial log before timeout_seconds elapses, boot_evidence falls back to "qemu-running" (Line 241-243) and the script proceeds to a passing result — as long as QEMU itself hasn't crashed. An ISO that hangs at the boot menu or panics silently (no matching serial output, but the QEMU process/monitor stays alive) would still report result: passed, undermining the smoke test's purpose of verifying the ISO actually boots.

Consider treating the "no evidence within timeout" case as a failure (or a distinctly weaker/failing state) rather than a silent pass.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/iso/smoke.sh` around lines 222 - 247, Change the no-evidence branch
after the boot-monitor loop so that when boot_evidence remains "none" at
timeout, the smoke test fails instead of assigning "qemu-running" and continuing
to pass. Preserve the existing success path when serial boot evidence is
detected and retain the current QEMU liveness checks.

Comment thread tests/iso/suite.sh
Comment on lines +22 to +28
if ! bash "$(dirname "$0")/iso-smoke-test.sh" "$iso_path" "$smoke_output"; then
status=1
fi

if ! bash "$(dirname "$0")/iso-e2e-test.sh" "$iso_path" "$e2e_output"; then
status=1
fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
echo "--- files matching iso-smoke-test.sh / iso-e2e-test.sh ---"
fd -a 'iso-smoke-test.sh|iso-e2e-test.sh'
echo "--- contents of tests/iso directory ---"
fd . tests/iso --max-depth 1

Repository: projectbluefin/iso

Length of output: 408


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "--- tests/iso/smoke.sh ---"
cat -n tests/iso/smoke.sh

echo "--- tests/iso/e2e.sh ---"
cat -n tests/iso/e2e.sh

Repository: projectbluefin/iso

Length of output: 19435


Point this harness at the local ISO test scripts
tests/iso/suite.sh is invoking tests/iso/iso-smoke-test.sh and tests/iso/iso-e2e-test.sh, but the scripts in this directory are smoke.sh and e2e.sh. Update those paths or the suite will fail immediately.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/iso/suite.sh` around lines 22 - 28, Update the script invocations in
the suite harness to call the local smoke.sh and e2e.sh files instead of
iso-smoke-test.sh and iso-e2e-test.sh, preserving the existing iso_path, output
arguments, and status handling.

@castrojo castrojo added 4-review A pull request is awaiting review. and removed enhancement labels Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4-review A pull request is awaiting review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant