test(apps.just): BATS coverage for install-opentabletdriver and cncf - #1064
Conversation
hanthor
left a comment
There was a problem hiding this comment.
New tests/test_apps_just.bats only, scope matches the claim, exercises real control flow with mocked binaries, and correctly pins the existing blacklist uninstall mismatch as a regression guard rather than silently "fixing" it. CI test+E2E green; validate fails only on title-lint (bracket prefix), not a code issue.
Generated by Claude Code
hanthor
left a comment
There was a problem hiding this comment.
Tests verified green and non-vacuous; two things to fix before merge
I merged this into current main, resolved the Justfile conflict, and ran it:
$ bats tests/test_apps_just.bats
1..21
ok 1 install-opentabletdriver recipe body is extractable and non-empty
...
ok 21 cncf: a failing brew bundle does not stop the recipe (no set -e)
Full suite after merge is identical to the main baseline — same three pre-existing failure groups (test_motd_integration 1/2/4, test_shared_just 2/3, test_ujust 7-10/12), no new failures. These 21 are pure addition.
I spot-checked the one test that looked like it could pass vacuously — "install removes the temp extraction directory" guards its assertion behind if [ -n "${tmpdir}" ], which is the shape that usually hides a no-op. It is genuinely wired up. Deleting the code under test makes it fail:
$ sed -i 's|^ rm -rf "${OTD_TMPDIR}"$| : rm-removed-for-mutation-test|' .../apps.just
$ bats tests/test_apps_just.bats -f "removes the temp extraction directory"
not ok 1 install-opentabletdriver: install removes the temp extraction directory
# (in test file tests/test_apps_just.bats, line 233)
# `[ ! -d "${tmpdir}" ]' failed
The bug you found is real, confirmed on main:
$ grep -n "modprobe.d" system_files/shared/usr/share/ublue-os/just/apps.just
35: echo -ne "blacklist hid_uclogic\nblacklist wacom\n" | sudo tee /etc/modprobe.d/blacklist-opentabletdriver.conf
47: sudo rm -f /etc/modprobe.d/blacklist-opentabletdriver.rules /etc/udev/rules.d/71-opentabletdriver.rules
Writes .conf, removes .rules. Uninstall leaves hid_uclogic and wacom blacklisted forever.
1. Please don't pin the bug as the contract
Test 16, install-opentabletdriver: uninstall targets the wrong modprobe filename (regression guard), asserts that the stale blacklist file survives uninstall. That encodes a user-facing breakage (built-in tablet/Wacom drivers stay disabled after removal) as the expected behaviour, and it means the eventual one-character fix arrives as a red CI run that a future reader has to decide is "expected red". The comment says to flip it, but comments don't run.
Two better options: either (a) drop test 16 and land the one-line fix (.rules → .conf) in this PR — the risk is a sudo rm -f of a file this recipe itself owns, and you already have the coverage to prove it; or (b) keep the test but invert it to the correct assertion and mark it skip "blocked on <issue>", so it flips to green when the fix lands instead of flipping to red.
2. The scope-boundary rationale is stale
The header comment and PR body say install-jetbrains-toolbox / install-asus are excluded because they hold brew tap --trust lines contested by open PRs #978/#941. #941 already merged:
$ git log --oneline -3 -- tests/test_brew_tap_trust.bats
2281f4f fix(brew): replace invalid `brew tap --trust` with `brew tap` + `brew trust` (#941)
$ sed -n '10,14p' system_files/shared/usr/share/ublue-os/just/apps.just
install-jetbrains-toolbox:
#!/usr/bin/env bash
brew tap ublue-os/tap 2>/dev/null || true
brew trust ublue-os/tap 2>/dev/null || true
main has no --trust left in system_files/, and tests/test_brew_tap_trust.bats passes on main. So the carve-out no longer protects anything, and test 3 (apps.just recipes covered here do not call brew tap) is now guarding a boundary that doesn't exist. Worth refreshing the comment and, if you want, extending coverage to those two recipes in a follow-up.
3. Mechanics
mergeable_stateisdirty, but only trivially — theJustfileconflict is your addedbats tests/test_apps_just.batsline againstmain'stest_system_just/test_brew_tap_trustlines. All three belong; keep all three.- The missing
.github/workflows/unit-tests.ymlstep matters more than the PR body implies: without it these 21 tests run only underjust testlocally and never in CI. Please get that one-line step in (a maintainer can push it to this branch) rather than merging a suite CI does not execute.
Generated by Claude Code
… and cncf
Adds tests/test_apps_just.bats — 21 cases covering the two apps.just
recipes that do not touch brew tap:
install-opentabletdriver (gum confirm dispatch, asset selection,
udev rule rename, modprobe blacklist,
tmpdir cleanup, flatpak install/remove,
user service unit, Ctrl-C exit 130)
cncf (brew bundle Brewfile path, non-tty
ujust --choose skip, failure tolerance)
The install-jetbrains-toolbox and install-asus recipes in the same file
are deliberately left uncovered: their brew tap --trust lines are being
changed by other open PRs.
Wires the file into the Justfile test recipe.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
install-opentabletdriver writes /etc/modprobe.d/blacklist-opentabletdriver.conf on install but the uninstall branch removed a .rules name, so hid_uclogic and wacom stayed blacklisted after removal. Point the rm at the .conf file, and turn the test that pinned the old behaviour into the correct assertion. The suite fails on the old recipe and passes on the fixed one. Also, from hanthor's review on #1064: - refresh the scope comment: the brew tap --trust carve-out no longer exists (#941 merged), so drop the test that guarded that boundary - wire the suite into .github/workflows/unit-tests.yml so CI runs it Fixes #1065 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YQCk7xJDvhtPoc4mWPzVCK
a8d1b2d to
5a035e8
Compare
|
Rebased onto current
Local run at |
Danathar
left a comment
There was a problem hiding this comment.
What I checked at 5a035e8: read the whole diff (new tests/test_apps_just.bats, the Justfile and unit-tests.yml wiring, and the one-line apps.just uninstall fix). The suite extracts the two recipe bodies and runs them against mocked gum, curl, sudo, flatpak, systemctl, brew and ujust with /etc and $HOME redirected into a per-test tree, so nothing touches the host. Ran bats tests/test_apps_just.bats locally, 20 ok; reverting the apps.just fix makes the uninstall test fail, so it is not vacuous. tests/test_brew_tap_trust.bats and tests/test_native_recipes.bats still pass. CI at this head is all green, including the test job that now runs the new step and the composed E2E suite. I pushed the rebase and the follow-up commit that addresses hanthor's three points myself, so a second maintainer look at 5a035e8 is fair; second approval needed from castrojo or hanthor.
…ojectbluefin#1198) # bluefin-common PR ## What does this change? `ujust install-opentabletdriver` now pins its upstream release tag and sha256-verifies the tarball with `sha256sum -c -` before extracting anything (including the root-installed udev rule), fetches the flathub `opentabletdriver.service` unit from a pinned commit (also sha256-verified) instead of the moving `refs/heads/master` branch, and runs the install branch with `set -euo pipefail` so any failed fetch aborts before anything is copied as root or enabled. A Renovate custom regex manager keeps the pinned tag current. ## Why? Closes projectbluefin#1170. The recipe previously streamed an unverified "latest" release tarball through `curl -s` into a root `sudo cp` of udev rules, and installed an auto-enabled user service unit from a moving branch ref — both fetched with `curl -s` (no `-f`), so an HTTP error page would have been written verbatim into system/user files. A compromised upstream asset or flathub packaging repo meant root-installed udev rules (`RUN+=` code execution) and session-level code execution. **Behavior note:** the recipe now installs the pinned `v0.6.7` release rather than always tracking latest. Updates arrive via Renovate PRs (tag only) — the two sha256 pins are **not** Renovate-managed and must be bumped manually in the same PR as documented in `docs/skills/ci-tooling/references/renovate-and-tools.md`; the checksum gate fails closed until they are. Also verified the rewritten recipe end-to-end against the real upstream assets (tarball and unit both verify `: OK`, exit 0). ## Test coverage `tests/test_apps_just.bats` extends the existing `install-opentabletdriver` suite (projectbluefin#1064) with: - pinned-tarball-URL and pinned-commit-unit-URL assertions (no `api.github.com` latest lookup, no `refs/heads/` fetch) - sha256 gate ordering: tarball verified before extraction, unit verified before `systemctl enable` - tampered-payload run: fails closed with nothing copied as root, no flatpak install, no unit written - `curl` HTTP-error run: fails closed before anything is installed (proves the `-f` flag is load-bearing) - the curl mock now honours `-f`/`-o`; uninstall-branch and `cncf` assertions unchanged ## PR pipeline ``` opened ──▶ 4-review ──▶ approved ──▶ merged ``` > A maintainer reviews and approves; merge goes through the merge queue. > Select `blocked` or `hold` to pause the work. ## Checklist - [x] PR title follows Conventional Commits (`fix:`, `feat:`, `docs:`, `ci:`, `refactor:`, etc.) - [x] `just check` passes - [x] pre-commit-equivalent hygiene passes (`pre-commit` unavailable in this environment; ran end-of-file/trailing-whitespace/doc-links/skill-index/frontmatter/skill-catalog checks individually — all green) - [x] Skill doc updated if the change affects agent-facing conventions or behavior (see `docs/skills/skill-improvement.md`) — `docs/skills/ci-tooling/references/renovate-and-tools.md` + `docs/TESTING.md` - [x] `AGENTS.md` / `docs/SKILL.md` / `docs/skills/` links remain valid - [ ] CI is green after push: `gh run list --repo projectbluefin/common --limit 5` (checked post-push) ## AI attribution AI-authored commit carries an `Assisted-by:` trailer (model noted honestly; the Copilot co-author pair from the template would be a false attribution for this run). — hive: backend=pi model=openrouter/z-ai/glm-5.3-flash --- 🐝 **Hive Agent**: `contributor` | **SHA:** `a0d64d7` Signed-off-by: mendezr <mendezr@users.noreply.github.com> Co-authored-by: mendezr <mendezr@users.noreply.github.com>
Claimed ground
File under test:
system_files/shared/usr/share/ublue-os/just/apps.justRecipes claimed:
install-opentabletdriver,cncf— only these two.New file:
tests/test_apps_just.bats(21 cases)Also touched:
Justfile(one added line in thetestrecipe)Disjointness check against open PRs
apps.justis partially occupied ground, so this PR deliberately claims onlythe recipes no open PR touches:
apps.just[scanner] fix: use valid brew tap trust commandsbrew tap --trustlines ininstall-jetbrains-toolbox,install-asusfix(brew): replace invalid brew tap --trust(+tests/test_brew_tap_trust.bats)brew tap --trustlines[quality] test(default.just)default.just/tests/test_default_just.bats[quality] test(shared.just)shared.just/tests/test_shared_just.bats[quality] test(system.just)system.just/tests/test_system_just.bats[architect] setup hook dispatchublue-*-setup,hookrunner.sh[architect] image-name → upstream routingbonedigger-report,changelog.justinstall-jetbrains-toolboxandinstall-asusare not covered by this PR.(The earlier scope-boundary test about
brew tapwas dropped in the2026-09-21 revision: #941 has merged and #978 is closed, so that carve-out no
longer exists.)
Justfileis also edited by #1048 and #1031, but each adds its own line to thesame
testrecipe list — a trivial textual conflict at worst, no semanticoverlap.
What this adds
apps.just(76 lines) had zero test coverage: no file undertests/referenced it, and neither the
Justfiletestrecipe nor.github/workflows/unit-tests.ymlexecuted anything against it.tests/test_apps_just.batsextracts each recipe body into a standalone scriptand runs it against a sandboxed
PATHof mocks (gum,curl,sudo,flatpak,systemctl,brew,ujust) with/etc/udev/rules.d,/etc/modprobe.d,/usr/share/ublue-os/homebrewand$HOMEredirected into aper-test temp tree. This follows the existing pattern in
tests/test_update_just.bats. No network, no root, no host mutation.install-opentabletdrivergum confirmexit 0 → install branch; exit 1 → uninstall branch; exit 130(Ctrl-C) → neither branch runs and no
flatpakcall is madejqasset filter picks the.tar.gzrelease asset and not the.debor
.rpmdecoys70-opentabletdriver.rules→71-opentabletdriver.rulesblacklist hid_uclogic/blacklist wacomland inblacklist-opentabletdriver.confmktemp -dextraction directory is removedflatpak --system install/flatpak --system removeare issued on thecorrect branches
$HOME/.config/systemd/user/and enabledcncfbrew bundle --file=.../cncf.Brewfileis invoked with the curated Brewfile pathujust --chooseis skipped when stdin is not a tty, and the recipe stillexits 0 (the
|| truetail)brew bundledoes not abort the recipe (noset -ein this body)Bug found while writing these tests
install-opentabletdriverwrites the modprobe blacklist to/etc/modprobe.d/blacklist-opentabletdriver.confon install, but the uninstallbranch removes
/etc/modprobe.d/blacklist-opentabletdriver.rules— a differentfilename. Uninstalling leaves
hid_uclogicandwacompermanentlyblacklisted, so the built-in tablet/Wacom drivers stay disabled after the
user has removed OpenTabletDriver.
Per hold-gated quality scope this PR does not fix the recipe. The test
install-opentabletdriver: uninstall targets the wrong modprobe filename (regression guard)pins the current (buggy) behaviour and carries a commentsaying to flip the assertion when the recipe is corrected. Filed separately as
an issue.
Not included
The matching step in
.github/workflows/unit-tests.ymlwas prepared but had tobe dropped: the hive GitHub App token has no
workflowspermission, so pushinga workflow edit is rejected. The suite is wired into the
Justfiletestrecipe only. A one-line CI step is needed as a follow-up:
Verification
Revision of 2026-09-21 (rebased onto main, hanthor's review addressed)
install-opentabletdriveruninstall path now removesblacklist-opentabletdriver.conf, the file the install path writes, insteadof a
.rulesfile that never existed. The test that documented the oldbehaviour as a "regression guard" is now the correct assertion: it fails on
the old recipe and passes on the fixed one. This is hanthor's option (a).
brew tapscope comment and its guard test are gone (see above)..github/workflows/unit-tests.ymlruns the new suite as its own step.Fixes #1065
Filed by quality agent (hold-gated mode). Human review required — do not merge without review, and do not remove the
holdlabel.— hive: agent=quality backend=copilot model=claude-opus-5