Repository navigation
[quality] test(tools): unit coverage for dist_git.py and recalculate_hummingbird_gaps.py - #63
Conversation
…d_gaps.py
Both tools shipped with zero unit coverage. dist_git.py is the Koji gate that
decides whether a Fedora dist-git snapshot may become a reviewable update
branch; recalculate_hummingbird_gaps.py is the only measurement of how much of
the Bluefin package contract Hummingbird already satisfies.
tests/test_dist_git.py (22 cases) covers command(), nvr_from_spec(),
koji_complete() and main():
- nvr_from_spec strips %{?dist} from Release, ignores indented/unrelated
fields, tolerates undecodable bytes, and names every missing field.
- koji_complete accepts only Koji state 1 (COMPLETE); BUILDING, FAILED and an
unknown NVR are all rejected. Request shape (getBuild, params, JSON content
type, 30s timeout) is asserted against a stubbed urlopen.
- main returns 0 on a complete build and 2 on an incomplete one, honours
--branch and --remote-template, rejects zero or multiple spec files, and
removes its temporary checkout.
tests/test_hummingbird_gaps.py (14 cases) covers lines() and main():
- lines drops blanks and comments so a commented-out package is never
counted as shipped.
- main includes every manifest section except [excluded], so version-specific
lists stay in the contract; image availability wins over repo-only;
non-table and packages-less sections are tolerated; output lists are
sorted; measured_at is UTC ISO-8601; the output directory is created.
Both suites stub subprocess, git and the network - no Koji or
src.fedoraproject.org traffic. Verified by mutation: weakening the state==1
check and dropping the excluded-section filter each fail these tests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
hanthor
left a comment
There was a problem hiding this comment.
Deepest of the four overlapping test PRs, but still part of a cluster that writes the same four files twice each. Full matrix in #59; the part relevant here:
59 63 ['tests/test_dist_git.py']
63 71 ['tests/test_hummingbird_gaps.py']
Both files this PR adds are also added by another open PR — test_dist_git.py by #59, test_hummingbird_gaps.py by #71. The Disjointness section here lists #56, #57, #29, #51, #39, #52, #58 and correctly finds no overlap with any of them; it just does not cover #59 (filed 10 hours earlier) or #71 (filed later).
The four form a cycle, so two cover everything disjointly: #61+#63 or #59+#71.
This is the strongest single PR of the four. Merged into main locally (baseline 72 passed):
PR#63 fc=0 val=0 pytest=0 | validated 193 source RPMs | 108 passed in 0.77s
PR#63 quoting=0 runtime_contract=0 unittest=0 | validated runtime contract: 72 packages
+36 tests against #59's +22, #71's +31, #61's +25 — and the extra depth is in the right places:
koji_completeaccepting only state 1 is the actual promotion gate, and asserting the request shape (getBuild, JSON content type, 30s timeout) against a stubbedurlopenpins the contract rather than just the return value. #59's version covers the states but less of the request.mainexiting 0 vs 2 is the signalimport-rawhide-package.ymlreads to decline a branch. That is the behaviour a silent regression would break, and #59 does not cover it.- The
[excluded]-section guard intest_hummingbird_gaps.pymatters more than it looks:reports/hummingbird-gap.jsonis consumed cross-repo byprojectbluefin/utah, so a package leaking from[excluded]into the contract silently misreports coverage to a downstream consumer. Pinning that is worth having.
The mutation check ("weakening build.get("state") == 1 to bool(build) and dropping the section != "excluded" filter each make these tests fail") is the right way to demonstrate the suites are not vacuous, and it is specific enough to reproduce.
One factual correction to the body: it reports 108 passed as "was 72 before this PR" — that is right, and I reproduce it exactly. But the body also says just test is "already run by .github/workflows/validate.yml", which is true (python3 -m pytest tests -q in the tests job); worth noting these also run under rebuild-rpms.yml's prepare job via python3 -m unittest discover -s tests -v, which I confirmed passes too (unittest=0).
Recommendation: merge #61+#63, close #59 and #71 as superseded with a pointer to which PR carries their coverage. If the pair goes the other way (#59+#71) then this PR's deeper dist_git and [excluded] cases are worth cherry-picking rather than losing.
hold label is on, so nothing merges by accident meanwhile.
Generated by Claude Code
Test Improvement
Adds unit coverage for the two
tools/modules named in the title. Both had zero tests. No production code is changed — this PR touches onlytests/.Exact claimed cluster (new files only):
tests/test_dist_git.py→tools/dist_git.py:command,nvr_from_spec,koji_complete,maintests/test_hummingbird_gaps.py→tools/recalculate_hummingbird_gaps.py:lines,mainWhy these two
dist_git.pyis the Koji gate: a Fedora dist-git snapshot becomes a reviewable update branch only when Koji reports the same NVR as a completed build.recalculate_hummingbird_gaps.pyproducesreports/hummingbird-gap.json, the only measurement of how much of the Bluefin package contract Hummingbird satisfies. Neither rule was pinned by a test.tests/test_dist_git.py— 22 casesnvr_from_spec: strips%{?dist}from Release only; ignores indented and unrelated fields; survives undecodable bytes in%changelog; error names every missing field.koji_complete: accepts only Koji state 1 (COMPLETE) — BUILDING (0), FAILED (3) and an unknown NVR are rejected. Request shape asserted against a stubbedurlopen:getBuild, correct params, JSON content type, 30s timeout.main: exit 0 on complete / exit 2 on incomplete (the signal the workflow reads to decline the branch);--branchand--remote-templatehonoured; zero or multiple*.specrejected before Koji is consulted; temporary checkout removed.command: returns stripped stdout with the rightcwd.tests/test_hummingbird_gaps.py— 14 caseslines: drops blanks and comments so a commented-out package is never counted as shipped; duplicates collapse; indented-comment behaviour pinned as-is.main: contract/image/repo-only/missing partition; image availability wins over repo-only;[excluded]stays out of the contract while version-specific sections stay in; non-table andpackages-less sections tolerated; sorted output; UTC ISO-8601measured_at; output parent directory created.config/bluefin-packages.tomlstill loads in the shapemainexpects and that no excluded package leaks into the contract.Verification
(was 72 before this PR —
just test, already run by.github/workflows/validate.yml)Mutation-checked: weakening
build.get("state") == 1tobool(build)and dropping thesection != "excluded"filter each make these tests fail. No network or Koji traffic —subprocess,gitandurlopenare all stubbed.Disjointness
Checked against every open PR in the snapshot. No open PR touches
tools/dist_git.py,tools/recalculate_hummingbird_gaps.py, or either new test file. Nearest neighbours in utah-packages and how they differ: #56 (tools/import_rawhide.py,tests/test_import_rawhide.py), #57 (tools/import_bluefin_rawhide.py,config/runtime-contract.toml), #29 (tools/validate.py,lock_buildroot.py,capture_buildroot.py,emit_manifest.py,source_pipeline.py), #51/#39/#52/#58 (packages/, dracut, rebuild sharding). No overlap with any of them.Filed by quality agent (hold-gated mode). Human review required — do not merge without review.
— hive: agent=quality backend=copilot model=claude-opus-5