[architect] unit-test and shellcheck catalogs restated 3x with drift: 12 bats suites ungated, just test red on main

[kubestellar-hive]

Architecture Finding

Type: tech-debt / unenforced-contract (hand-maintained manifests)
Affected area: .github/workflows/unit-tests.yml, Justfile (test recipe), tests/

The repo's test/lint catalog is stated three times — CI workflow, just test recipe, and the files on disk — with no derivation and no drift gate. All three copies disagree.

Measured drift (main @ 56d81ef)

bats — 33 suites on disk:

• unit-tests.yml runs 22 files across 19 hand-written steps. 12 suites are never run in CI: test_image_repo, test_shared_just, test_system_just, test_default_just, test_ujust, test_brew_tap_trust, test_hardware_hooks, test_theming_hook, test_nvidia_flatpak_sync, test_clean_system_podman_path, test_bling_preexec_rearm (+ test_libvirt_helper, deliberately excluded — needs libvirtd).
• Justfile test runs 30 — missing test_image_repo.bats and test_shared_just.bats (the regression suite for the refactor: single source of truth for image-name → upstream repo routing #1045 routing refactor is on disk but in no executed list).

pytest — 6 modules on disk; workflow runs 2 (test_hooks, test_chairlift_config); Justfile runs 6.

shellcheck extensionless scripts — 18 extensionless bash-shebang scripts under system_files/; the workflow hand-lists 9. Unlinted: bonedigger-report, ujust, brew-preinstall (bin + libexec), ublue-image-repo, bootc-update-stage, rechunker-group-fix, ublue-bling-fastfetch, ensure-libvirt-session-config, ublue-nvidia-flatpak-runtime-sync.

Proof of rot

Ungated suites decay silently. just test is red on main today:

• tests/test_ujust.bats: 5 failures. The mock brew in setup() exits 1 on any subcommand outside install/shellenv — bundle was never handled, so the install-ai-tools and bluespeed tests were broken at birth (fix: add Bluespeed setup recipe #1023, 2026-09-08).
• tests/test_shared_just.bats: 2 failures. Two tests assert powerwash delegates to bctl powerwash; that delegation was removed by fix: restore native recipes without bluefinctl #1083 (2026-09-05) — five days before the tests were written (test(shared.just): BATS coverage for powerwash, toggle-tpm2 #1034, 2026-09-10). Broken at birth; merged green because CI never executes the suite.

Impact

Every new test suite requires a human to remember two hand-maintained lists; the failure mode is silent (suite never runs) rather than loud. Two of the last three suite additions landed broken and nobody noticed. Coverage numbers are fiction for 12 of 33 suites.

Recommendation

Derive the catalog from disk in the one place that gates merges (unit-tests.yml):

1. bats: run tests/*.bats with a single explicit skip list (test_libvirt_helper, with the reason inline).
2. pytest: glob tests/test_*.py.
3. shellcheck: derive the extensionless set from shebangs instead of a path list.
4. Fix the two broken-at-birth suites so the derived gate lands green.

Justfile test enumeration is left in place for now — common#1064 is actively editing that recipe; deduplicating it is a follow-up once that PR lands.

---

Filed by architect agent (ACMM L5 — hold-gated mode)

🐝 Hive Agent: architect | Instance: hosted-projectbluefin-knuckle-gjvq | SHA: unknown

— hive: agent=architect backend=copilot model=kimi-k3 copilot=1.0.78

[Danathar]

Folding #968 into this issue — its list was a subset of the drift documented here. Carrying over the practical note from #968: the Hive App token has no workflows permission, so the unit-tests.yml edit has to be pushed by a human, not an agent PR.