OSAC-3354/3355/3356: E2E validation workflows for snapshot flavors

[omer-vishlitzky]

Summary

• Add E2E validation gate workflows that boot a cluster from a snapshot
  flavor, run the matching test suite, and report pass/fail
• A shared reusable workflow (validate-snapshot.yaml) handles the common
  logic: pull flavor, boot clone, refresh OSAC deployment, build test
  container from osac-test-infra, run pytest, upload results, and destroy
  the clone
• Three thin callers for each flavor: validate-vmaas.yaml (OSAC-3354),
  validate-caas.yaml (OSAC-3355), validate-bmaas.yaml (OSAC-3356)
• Each supports both workflow_dispatch (manual) and workflow_call
  (called by build workflows to gate publishing)

Jira: OSAC-3354, OSAC-3355, OSAC-3356

Test plan

• Trigger validate-vmaas via workflow_dispatch with a known-good snapshot tag
• Verify cluster boots, E2E tests run, artifacts are uploaded, and clone is destroyed
• Trigger validate-caas and validate-bmaas similarly
• Verify concurrency groups prevent duplicate runs for the same tag
• Verify validation_passed output is true on success, false on failure
• Wire a build workflow to call validate-vmaas via workflow_call and confirm the output gates publishing

Summary by CodeRabbit

• New Features
  • Added reusable and manually triggerable validation workflows for BMaaS, CaaS, and VMaaS snapshots.
  • Supports snapshot tags and optional flavor images as validation inputs.
  • Added a shared snapshot validation process that provisions an isolated test environment, runs the selected end-to-end suite, publishes test artifacts, and cleans up resources.
  • Validation results are exposed for integration with other automation.
  • Prevents superseded validations for the same snapshot from continuing unnecessarily.

[coderabbitai]

Warning

Review limit reached

@omer-vishlitzky, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 15 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Repository: osac-project/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 38cbd577-700d-4c88-876e-80878e61ed07

📥 Commits

Reviewing files that changed from the base of the PR and between ade62ac and caeb51e.

📒 Files selected for processing (4)

• .github/workflows/validate-bmaas.yaml
• .github/workflows/validate-caas.yaml
• .github/workflows/validate-snapshot.yaml
• .github/workflows/validate-vmaas.yaml

Walkthrough

Added reusable and manually dispatchable workflows for BMaaS, CaaS, and VMaaS snapshot validation. A shared workflow resolves images, prepares a cluster, refreshes OSAC, runs E2E tests, uploads artifacts, reports status, and cleans resources.

Changes

Snapshot validation

Layer / File(s)	Summary
Validation entrypoints .github/workflows/validate-bmaas.yaml, .github/workflows/validate-caas.yaml, .github/workflows/validate-vmaas.yaml	The workflows accept snapshot and optional flavor image inputs, support manual and reusable calls, cancel superseded runs per snapshot, and delegate to the shared workflow.
Infrastructure preparation .github/workflows/validate-snapshot.yaml	The shared workflow resolves images, writes pull credentials, cleans stale resources, boots the validation cluster, prepares kubeconfig, and refreshes OSAC.
Test execution and cleanup .github/workflows/validate-snapshot.yaml	The workflow builds the test image, runs the selected E2E suite, uploads results for 14 days, exposes the test outcome, and always removes clusters, containers, credentials, and temporary files.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant WorkflowCaller
  participant ValidateSnapshot
  participant ValidationCluster
  participant E2ESuite
  participant ArtifactStorage

  WorkflowCaller->>ValidateSnapshot: submit snapshot validation inputs
  ValidateSnapshot->>ValidationCluster: boot and refresh validation cluster
  ValidateSnapshot->>E2ESuite: run selected pytest suite
  E2ESuite-->>ValidateSnapshot: return test outcome and output
  ValidateSnapshot->>ArtifactStorage: upload E2E output
  ValidateSnapshot-->>WorkflowCaller: return validation_passed
  ValidateSnapshot->>ValidationCluster: destroy cluster and clean resources

Loading

---

Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

• Ignore

❌ Failed checks (1 error, 1 inconclusive)

Check name	Status	Explanation	Resolution
Container-Privileges	❌ Error	The validation container runs with Podman --network=host at validate-snapshot.yaml:205, which grants host-network access and matches the prohibited hostNetwork condition.	Remove --network=host and use an isolated container network with only the required connectivity.
No-Sensitive-Data-In-Logs	❓ Inconclusive	placeholder	placeholder

✅ Passed checks (9 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the addition of E2E validation workflows for VMaaS, CaaS, and BMaaS snapshot flavors.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
No-Hardcoded-Secrets	✅ Passed	Added workflows contain no hardcoded credential values; PULL_SECRET is only referenced through GitHub secrets, and scans found no vendor tokens, private keys, embedded credentials, or long blobs.
No-Weak-Crypto	✅ Passed	The added validation workflows contain no MD5, SHA1, DES, 3DES, RC4, Blowfish, ECB, custom crypto, or secret/token comparisons.
No-Injection-Vectors	✅ Passed	The four added workflows contain no SQL concatenation, eval/exec, pickle or unsafe YAML loading, os.system, dangerouslySetInnerHTML, or shell=True; input values are passed through quoted shell argu...
Ai-Attribution	✅ Passed	AI use is identified by Assisted-by: Claude Code on both PR commits; no Co-authored-by trailer appears.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[omer-vishlitzky]

Code Review Findings

MAJOR — actions/checkout SHA mismatch

.github/workflows/validate-snapshot.yaml:74 — SHA 34e114876b0b11c390a56381ad16ebd13914f8d5 is from the v6 dev line (commit message: "Cleanup actions/checkout@v6 auth style (#2305)"). Comment says # v4. Real v4.4.0 SHA is 11d5960a326750d5838078e36cf38b85af677262.

Fix: Use correct v4 SHA, or update comment to v6 if that's intentional.

MAJOR — cleanup-haproxy is not a valid cluster-tool subcommand

.github/workflows/validate-snapshot.yaml:116 — sudo python3 ./cluster-tool cleanup-haproxy doesn't exist. Valid subcommands: snapshot, boot, list, flavors, verify, destroy, push, pull, setup, connect, servers, use. The call silently fails via 2>/dev/null || true, so orphaned HAProxy entries are never cleaned up on shared runners.

Fix: Either add a cleanup-haproxy subcommand to cluster-tool, or remove this line if destroy already handles HAProxy cleanup internally (it calls remove_haproxy_clone).

MAJOR — Must use OSAC baremetal infrastructure

These validation workflows boot clusters from snapshot flavors and run E2E tests — this requires libvirt, cluster-tool, and bare metal hardware. OSAC already has self-hosted GitHub Actions runners on baremetal machines registered at the org level with the osac-ci label. These machines have everything pre-installed via machine-init.sh (libvirt, qemu-kvm, podman, haproxy, cluster-tool, dnsmasq, Vault CLI, etc.) and are connected to Vault for secrets (pull-secret, registry auth).

The workflows should use runs-on: [self-hosted, osac-ci] to target these machines, matching the convention used by all existing E2E workflows in osac-test-infra. If a different label like baremetal is intended, it needs to be registered on the runners via action-runners-setup.sh. The workflows should also follow the existing secrets pattern — fetch pull-secret and registry auth from Vault via AppRole (see the fetch-secrets composite action in osac-test-infra) rather than using GitHub Actions secrets directly.

MINOR — No timeout-minutes on validate job

.github/workflows/validate-snapshot.yaml:63 — Boots a real cluster, runs E2E tests, cleans up. Any step could hang. Self-hosted baremetal runners don't auto-terminate, default GHA timeout is 6 hours.

Fix: Add timeout-minutes: 60.

MINOR — Unpinned curl | bash for helm install

.github/workflows/validate-snapshot.yaml:163 — curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash on a self-hosted runner with PULL_SECRET access. Supply chain risk. Helm is already available on the OSAC baremetal runners (or can be added to machine-init.sh).

Fix: Pre-install helm on the runners and remove this fallback.

NIT — Flavor grep fragile for long names

.github/workflows/validate-snapshot.yaml:121 — grep -q "^${FLAVOR_NAME} " relies on {name:<20s} padding. Names >= 20 chars won't match. Consider grep -qE "^${FLAVOR_NAME}( |$)".

---

Overall design is sound — reusable workflow + thin callers, concurrency groups, safe secret handling, unconditional cleanup via if: always().

[omer-vishlitzky]

Superseded by #38 (unified snapshot pipeline PR)