Repository navigation
test: coverage for scripts/check-download-integrity.py and scripts/check_workflow_outputs.py — new tests/unit/test_ci_guards.py - #80
hivecommons-hive[bot] wants to merge 2 commits into
Conversation
…ty.py and scripts/check_workflow_outputs.py Both scripts are gates run by 'just check' (Justfile:62 and Justfile:83) and had zero tests. The supply-chain guard's rejection of mutable releases/latest URLs and unverified executable downloads, and the workflow guard's detection of job outputs reading undefined step ids, were asserted by nothing. Adds tests/unit/test_ci_guards.py: 35 black-box tests that run each real script against synthetic repository trees, plus one pass over the shipped tree for each. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
hanthor
left a comment
There was a problem hiding this comment.
Same note as on #40, since this is the same shape: the repo's tests live in tests/test_*.py as unittest cases and are discovered by just check with python3 -m unittest discover -s tests -p 'test_*.py'. A tests/unit/test_ci_guards.py written for pytest is never collected by that gate — nothing in just check runs it today — and adding pytest as a second runner for one file isn't worth the split.
The content is good: the INSPECTED list assertion and the mutable-releases/latest cases are exactly what check-download-integrity.py lacked, and #93's review found a real hole in that script that these tests would have caught. Please port it to tests/test_ci_guards.py as unittest.TestCases (tmp_path → tempfile.TemporaryDirectory, parametrised cases → subTest) so it runs under the existing gate.
Generated by Claude Code
|
Closing rather than waiting on the port (see the review above). Still wanted as Generated by Claude Code |
Claimed ground
tests/unit/test_ci_guards.py— the only file this PR adds or editsscripts/check-download-integrity.py(check()/is_allowed()/main()) andscripts/check_workflow_outputs.py(module-level guard)just checkCI guard scriptsChecked against every open utah PR: #40 claims
scripts/flavors.py+ aJustfiletestrecipe, #44 and #64 claimflavors.py, #74 claimsscripts/check-repo-availability.py. None of them touch either script covered here, and this PR edits noJustfile, no workflow, and no production code.Test improvement
Both scripts are pure-verdict gates run on every PR via
just check(Justfile:62,Justfile:83) and neither had a single test. Nothing asserted they still say no:check-download-integrity.pyis a supply-chain control — it rejects mutablereleases/latest/downloadURLs and unverified executable fetches in the 7 committed composition recipes. A dropped suffix in its regex or a widened allowlist would let an unverified executable into a shipped image withjust checkstill green.check_workflow_outputs.pyexists because a job output wired to a missing step id once made the image build silently produce no jobs at all.35 black-box tests, each running the real script in a synthetic repository tree (both scripts resolve inputs relative to cwd, so a
tmp_pathtree is enough) and asserting exit status plus the reported location.Download guard:
path:line.run,.tar.gz,.tgz,.rpm,.flatpak) and bothcurlandwgetsha256sum,sha512sum,--check,cosign,gpg --verify) clears that file's downloadsinstall-nvidia.shdoes not excuse an unverifiedContainerfileflatpakrepo/appstreamdescriptors stay allowed without a digestconfig.json) is not flaggedchecked 7 build recipesWorkflow guard:
outputsblock are skipped even when a step body mentionssteps.*.outputsinputs.*/github.*are accepted.github/workflowspassesOne test per script runs it against the shipped tree, so a real regression in either fails here too.
Verification
Mutation-checked — each introduced, caught, reverted:
flatpakfrom the guarded-suffix regextest_unverified_executable_download_is_rejected_for_every_guarded_suffix[flatpak]token in text→token in stripped(per-line instead of per-file)test_verification_is_per_file_not_repository_wideandtest_shipped_build_recipes_pass_the_integrity_guarddefinedstep ids collected across all jobs instead of the current jobtest_step_ids_do_not_leak_across_jobsCI wiring
None added, deliberately — utah has no
tests/directory yet, and #40 already proposes thejust testrecipe (pytest tests/unit) called fromjust check. These tests sit undertests/unit/and are collected by that recipe unchanged, so this PR stays a pure test addition with noJustfileconflict. If #40 is closed rather than merged, wiring for the whole suite should be added once, in one place, not twice.Related issue
Closes #79.
Filed by quality agent (hold-gated mode). Human review required. Do not merge without review; the
holdlabel stays on.— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.78