test: gate the Justfile test recipe against tests/ — tests/test_suite_registration.bats

[hivecommons-hive]

Test Improvement

Claimed ground: Justfile (the test recipe and its comment block only) and the new tests/test_suite_registration.bats. No other file is touched.

The hole

tests/test_image_repo.bats — 93 lines, 12 assertions over system_files/shared/usr/libexec/ublue-image-repo, the single source of truth for image-name/tag → upstream GitHub repository routing consumed by bonedigger-report — is named in neither the Justfile test recipe nor any workflow. No runner has ever executed it. It is green; it just never runs.

The recipe is maintained by hand and nothing compares it against tests/, so the next suite can be orphaned the same way.

The change

• Register tests/test_image_repo.bats in the test recipe.
• Add tests/test_suite_registration.bats (7 assertions), which fails when:
  • a tests/test_*.bats or tests/test_*.py is neither registered in the recipe nor declared excluded,
  • the recipe names a suite that does not exist on disk,
  • an exclusion comment names a suite that no longer exists,
  • an exclusion states no reason,
  • the gate or test_image_repo.bats falls out of the recipe.
• Move the two intentional omissions into that machine-checkable form, # <file> is excluded — <reason>: test_libvirt_helper.bats (requires a running libvirtd session) and test_shared_just.bats (its bctl delegation cases are red).
• Register the gate itself.

Verification (at 44b4b9f, bats 1.14.0)

$ bats tests/test_suite_registration.bats tests/test_image_repo.bats
1..19
ok 1 registration: the Justfile test recipe is parseable and non-empty
...
ok 19 image-repo: consumers do not hardcode upstream repo routing

Each drift mode was confirmed to actually fail the gate, not merely pass vacuously:

injected drift	result
drop test_image_repo.bats from the recipe	not ok 2 — suite never runs — add it to the Justfile test recipe or declare it excluded: test_image_repo.bats
add bats tests/test_nonexistent.bats	not ok 3 — Justfile test recipe names a suite that does not exist
strip the reason from an exclusion	not ok 5 — exclusion without a reason

just check passes, and just --list still shows the test recipe's original summary line.

Deliberately out of scope

• tests/test_shared_just.bats is not repaired here. Its two red bctl delegation cases are test: repair broken-at-birth bats suites (test_ujust mock brew, powerwash bctl) #1106's ground; this PR only records the omission with a reason so the gate is green without silently hiding it. When test: repair broken-at-birth bats suites (test_ujust mock brew, powerwash bctl) #1106 lands, the exclusion can be swapped for a registration and the gate keeps passing either way.
• CI registration. .github/workflows/unit-tests.yml runs 27 of the 40 suites in tests/ and never invokes just test, so 13 maintained suites never execute in CI. That fix lives inside .github/workflows/, which this agent cannot push, and is filed separately.

Related Issue

Closes #1165

---

Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5

[hivecommons-hive]

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will automatically remove the hold label once current policy no longer requires a level hold for "quality". If this is an outreach PR, a human must review it and remove the label.

[Danathar]

Reviewed at head 7ed073d. This adds a bats gate that checks every tests/test_.bats and tests/test_.py file is either named in the Justfile test recipe or declared excluded with a reason, and it registers the two suites that were missing (test_image_repo.bats and the gate itself). I diffed the tests/ directory against the recipe on main and got exactly the four unregistered files the PR accounts for: test_image_repo.bats and test_suite_registration.bats are now in the recipe, test_libvirt_helper.bats and test_shared_just.bats are excluded with reasons. I confirmed the test_shared_just.bats reason is true: at this head, bats tests/test_shared_just.bats fails its two bctl delegation cases (tests 2 and 3) and passes the rest. I ran bats tests/test_suite_registration.bats (7/7 pass) and bats tests/test_image_repo.bats (12/12 pass), and tried two negative cases: an unregistered tests/test_zzz_fake.bats makes test 2 fail with the file named, and stripping the reason from the exclusion comment makes test 5 fail. Every other bats line in the recipe also passes locally at this head; the only red one is test_setup_scripts.bats, which fails identically on main in my environment and passes in CI, and this PR does not touch it. One note for later, not a blocker: .github/workflows/unit-tests.yml lists suites by hand and does not call this gate, so it only protects just test until CI is switched to it (issue #1105 covers that drift).

[Danathar]

Hold removed: this was the Hive's ACMM level gate, which I operate, and the PR has a human approval with CI green. Ready for a maintainer to merge.

[hivecommons-hive]

style

• recipe_block (tests/test_suite_registration.bats:20-28) collects every top-of-column # comment that appears anywhere above test: in the Justfile, not just the contiguous block over the recipe. It works today only because test: is the first recipe (Justfile:8); if a documented recipe or file-header comment mentioning test_*.bats is excluded is ever added above it, that text silently feeds the exclusion list. Anchoring the awk to the contiguous comment block directly above test: would remove the trap. (low)
• present_suites (tests/test_suite_registration.bats:38-40) uses GNU find -printf, so the gate errors on BSD/macOS find. Repo tooling is Linux-oriented, so info only.

Verified at head 7ed073d: the gate passes 7/7 and the newly-registered tests/test_image_repo.bats — never previously executed by any runner — passes 12/12 with bats 1.12, so registering it does not break just test. All 41 tests/test_* files are accounted for (39 registered, 2 excluded with reasons, Justfile:3-4). The # Run unit tests… docstring stays immediately above test: (Justfile:7), preserving just --list output.

No findings from: correctness, security, intent-alignment, docs-currency.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78

[castrojo]

LGTM. Rebased cleanly onto main, registered test_shared_just.bats alongside test_image_repo.bats and verified test_suite_registration.bats passes 7/7.