fix(black-box): prefer runner Docker over installing Podman on GHA

[gciavarrini]

Summary

Black-box jobs were hanging on compose up with depends_on: service_healthy.
Containers stayed starting with Log: null until the 25m timeout.

Podman runs healthchecks via systemd user timers. On GitHub Actions those often never fire, even after PR #32.
Docker's daemon runs healthchecks itself.

This PR stops installing Podman in the workflow and uses the runner engine (prefer Docker).
Pull and diagnostics follow CONTAINER_ENGINE.

Podman Upstream related issues

• Healthcheck always shows "starting" podman-container-tools/podman#19326
• healthcheck for non-systemd podman podman-container-tools/podman#27033
• Healthchecks not working when running Podman without systemd podman-container-tools/podman#28192

Fixes

dcm-project/control-plane#37

[qodo-code-review]

PR Summary by Qodo

Fix black-box GHA hangs by preferring runner Docker engine over Podman

🐞 Bug fix ⚙️ Configuration changes 🕐 10-20 Minutes

AI Description

• Stop installing Podman in black-box GitHub Actions jobs to avoid healthcheck hangs.
• Detect and use the runner’s container engine (prefer Docker; fallback to Podman).
• Make image pulls and failure diagnostics follow CONTAINER_ENGINE.

Diagram

graph TD
  A["GHA black-box job"] --> B["Detect container engine"] --> C{"Docker present?"}
  C -->|"yes"| D["Set CONTAINER_ENGINE=docker"] --> H["Pull images & compose up"] --> J["Diagnostics (ps/health/logs)"]
  C -->|"no"| E{"Podman present?"}
  E -->|"yes"| F["Set CONTAINER_ENGINE=podman"] --> H
  E -->|"no"| G["Fail: no engine"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Force Docker-only on GHA

• ➕ Eliminates fallback behavior and reduces variability across runs
• ➕ Aligns with the root cause (Podman healthchecks via systemd timers on GHA)
• ➖ Less portable if some runners/environments intentionally rely on Podman
• ➖ Harder to reuse the workflow outside GitHub-hosted runners

2. Keep Podman but add systemd timer/workaround

• ➕ Maintains Podman-based parity with local dev environments
• ➕ Avoids relying on Docker availability
• ➖ Upstream-dependent and brittle; the underlying GHA timer behavior can still fail
• ➖ More complexity (lingering/systemd user services) for CI-only benefit

Recommendation: Proceed with the current approach: prefer the runner’s Docker engine and gate all pulls/diagnostics through CONTAINER_ENGINE. It directly addresses the observed hang by relying on dockerd-managed healthchecks, while still preserving a Podman fallback when Docker isn’t present.

Files changed (1) +28 / -16

Bug fix (1) +28 / -16

black-box.yamlUse runner container engine (prefer Docker) for black-box workflow steps +28/-16

Use runner container engine (prefer Docker) for black-box workflow steps

• Removes Podman installation/setup and adds a detection step that exports CONTAINER_ENGINE (docker preferred, podman fallback). Updates image pre-pull and diagnostic commands (ps/inspect/logs) to run via the selected engine, improving reliability of compose healthcheck gating on GitHub Actions.

.github/workflows/black-box.yaml

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

1. Podman fallback unprepared 🐞 Bug ☼ Reliability

Description

The workflow can fall back to Podman when Docker is missing, but it no longer sets up a
Podman-capable runtime environment before later steps use $CONTAINER_ENGINE, so Podman-only runners
can fail when pulling images or collecting diagnostics. Additionally, the diagnostics step defaults
to docker when CONTAINER_ENGINE is unset, which can yield misleading or empty output in
non-Docker environments.

Code

.github/workflows/black-box.yaml[R44-47]

+          if command -v docker >/dev/null 2>&1; then
+            engine=docker
+          elif command -v podman >/dev/null 2>&1; then
+            engine=podman

Evidence

The engine-detection logic can choose engine=podman when Docker is absent, and later workflow
steps rely on CONTAINER_ENGINE to perform container operations, but there is no Podman-specific
preparation performed elsewhere to ensure that choice is actually usable. In the diagnostics section
specifically, the script sets engine="${CONTAINER_ENGINE:-docker}", meaning it will silently
select Docker if CONTAINER_ENGINE was never populated, and then runs ${engine} ps/inspect/logs,
which explains why diagnostics can be wrong or empty on runners where Docker isn’t present or
intended.

.github/workflows/black-box.yaml[42-66]
.github/workflows/black-box.yaml[56-66]
.github/workflows/black-box.yaml[74-83]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The workflow contains a Podman fallback path, but it no longer performs the necessary Podman preparation/validation before subsequent steps invoke the selected container engine (e.g., pulls and diagnostics). Separately, the diagnostics step silently defaults to `docker` when `CONTAINER_ENGINE` is unset, which can produce misleading/empty diagnostics on non-Docker runners.

## Issue Context
This reusable workflow runs on `ubuntu-latest` and includes engine-detection logic that can select Podman when Docker is missing. If the Podman branch is kept, the workflow should either ensure Podman works equivalently to the Docker path (including any required setup/validation) or fail fast with a clear error; diagnostics should not independently default to Docker when the engine is unknown, and should instead reuse detection or require a previously-determined engine.

## Fix Focus Areas
- .github/workflows/black-box.yaml[42-66]
- .github/workflows/black-box.yaml[74-99]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Tip of the day

💡 Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[qodo-code-review]

1. Podman fallback unprepared 🐞 Bug ☼ Reliability

The workflow can fall back to Podman when Docker is missing, but it no longer sets up a
Podman-capable runtime environment before later steps use $CONTAINER_ENGINE, so Podman-only runners
can fail when pulling images or collecting diagnostics. Additionally, the diagnostics step defaults
to docker when CONTAINER_ENGINE is unset, which can yield misleading or empty output in
non-Docker environments.

Agent Prompt

## Issue description
The workflow contains a Podman fallback path, but it no longer performs the necessary Podman preparation/validation before subsequent steps invoke the selected container engine (e.g., pulls and diagnostics). Separately, the diagnostics step silently defaults to `docker` when `CONTAINER_ENGINE` is unset, which can produce misleading/empty diagnostics on non-Docker runners.

## Issue Context
This reusable workflow runs on `ubuntu-latest` and includes engine-detection logic that can select Podman when Docker is missing. If the Podman branch is kept, the workflow should either ensure Podman works equivalently to the Docker path (including any required setup/validation) or fail fast with a clear error; diagnostics should not independently default to Docker when the engine is unknown, and should instead reuse detection or require a previously-determined engine.

## Fix Focus Areas
- .github/workflows/black-box.yaml[42-66]
- .github/workflows/black-box.yaml[74-99]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

[jordigilh]

lgtm

[chadcrum]

Good temp workaround, thanks!