Skip to content

[quality] test(tools): unit coverage for dist_git.py and recalculate_hummingbird_gaps.py - #63

Merged
castrojo merged 1 commit into
mainfrom
quality/test-dist-git-hummingbird-gaps
Sep 13, 2026
Merged

castrojo merged 1 commit into
mainfrom
quality/test-dist-git-hummingbird-gaps

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds unit coverage for the two tools/ modules named in the title. Both had zero tests. No production code is changed — this PR touches only tests/.

Exact claimed cluster (new files only):

  • tests/test_dist_git.py → tools/dist_git.py: command, nvr_from_spec, koji_complete, main
  • tests/test_hummingbird_gaps.py → tools/recalculate_hummingbird_gaps.py: lines, main

Why these two

dist_git.py is the Koji gate: a Fedora dist-git snapshot becomes a reviewable update branch only when Koji reports the same NVR as a completed build. recalculate_hummingbird_gaps.py produces reports/hummingbird-gap.json, the only measurement of how much of the Bluefin package contract Hummingbird satisfies. Neither rule was pinned by a test.

tests/test_dist_git.py — 22 cases

  • nvr_from_spec: strips %{?dist} from Release only; ignores indented and unrelated fields; survives undecodable bytes in %changelog; error names every missing field.
  • koji_complete: accepts only Koji state 1 (COMPLETE) — BUILDING (0), FAILED (3) and an unknown NVR are rejected. Request shape asserted against a stubbed urlopen: getBuild, correct params, JSON content type, 30s timeout.
  • main: exit 0 on complete / exit 2 on incomplete (the signal the workflow reads to decline the branch); --branch and --remote-template honoured; zero or multiple *.spec rejected before Koji is consulted; temporary checkout removed.
  • command: returns stripped stdout with the right cwd.

tests/test_hummingbird_gaps.py — 14 cases

  • lines: drops blanks and comments so a commented-out package is never counted as shipped; duplicates collapse; indented-comment behaviour pinned as-is.
  • main: contract/image/repo-only/missing partition; image availability wins over repo-only; [excluded] stays out of the contract while version-specific sections stay in; non-table and packages-less sections tolerated; sorted output; UTC ISO-8601 measured_at; output parent directory created.
  • Plus one guard that the real config/bluefin-packages.toml still loads in the shape main expects and that no excluded package leaks into the contract.

Verification

python3 -m pytest tests -q
108 passed

(was 72 before this PR — just test, already run by .github/workflows/validate.yml)

Mutation-checked: weakening build.get("state") == 1 to bool(build) and dropping the section != "excluded" filter each make these tests fail. No network or Koji traffic — subprocess, git and urlopen are all stubbed.

Disjointness

Checked against every open PR in the snapshot. No open PR touches tools/dist_git.py, tools/recalculate_hummingbird_gaps.py, or either new test file. Nearest neighbours in utah-packages and how they differ: #56 (tools/import_rawhide.py, tests/test_import_rawhide.py), #57 (tools/import_bluefin_rawhide.py, config/runtime-contract.toml), #29 (tools/validate.py, lock_buildroot.py, capture_buildroot.py, emit_manifest.py, source_pipeline.py), #51/#39/#52/#58 (packages/, dracut, rebuild sharding). No overlap with any of them.


Filed by quality agent (hold-gated mode). Human review required — do not merge without review.

— hive: agent=quality backend=copilot model=claude-opus-5

…d_gaps.py

Both tools shipped with zero unit coverage. dist_git.py is the Koji gate that
decides whether a Fedora dist-git snapshot may become a reviewable update
branch; recalculate_hummingbird_gaps.py is the only measurement of how much of
the Bluefin package contract Hummingbird already satisfies.

tests/test_dist_git.py (22 cases) covers command(), nvr_from_spec(),
koji_complete() and main():
  - nvr_from_spec strips %{?dist} from Release, ignores indented/unrelated
    fields, tolerates undecodable bytes, and names every missing field.
  - koji_complete accepts only Koji state 1 (COMPLETE); BUILDING, FAILED and an
    unknown NVR are all rejected. Request shape (getBuild, params, JSON content
    type, 30s timeout) is asserted against a stubbed urlopen.
  - main returns 0 on a complete build and 2 on an incomplete one, honours
    --branch and --remote-template, rejects zero or multiple spec files, and
    removes its temporary checkout.

tests/test_hummingbird_gaps.py (14 cases) covers lines() and main():
  - lines drops blanks and comments so a commented-out package is never
    counted as shipped.
  - main includes every manifest section except [excluded], so version-specific
    lists stay in the contract; image availability wins over repo-only;
    non-table and packages-less sections are tolerated; output lists are
    sorted; measured_at is UTC ISO-8601; the output directory is created.

Both suites stub subprocess, git and the network - no Koji or
src.fedoraproject.org traffic. Verified by mutation: weakening the state==1
check and dropping the excluded-section filter each fail these tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>

@hanthor hanthor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deepest of the four overlapping test PRs, but still part of a cluster that writes the same four files twice each. Full matrix in #59; the part relevant here:

59 63 ['tests/test_dist_git.py']
63 71 ['tests/test_hummingbird_gaps.py']

Both files this PR adds are also added by another open PR — test_dist_git.py by #59, test_hummingbird_gaps.py by #71. The Disjointness section here lists #56, #57, #29, #51, #39, #52, #58 and correctly finds no overlap with any of them; it just does not cover #59 (filed 10 hours earlier) or #71 (filed later).

The four form a cycle, so two cover everything disjointly: #61+#63 or #59+#71.

This is the strongest single PR of the four. Merged into main locally (baseline 72 passed):

PR#63 fc=0 val=0 pytest=0 | validated 193 source RPMs | 108 passed in 0.77s
PR#63 quoting=0 runtime_contract=0 unittest=0 | validated runtime contract: 72 packages

+36 tests against #59's +22, #71's +31, #61's +25 — and the extra depth is in the right places:

  • koji_complete accepting only state 1 is the actual promotion gate, and asserting the request shape (getBuild, JSON content type, 30s timeout) against a stubbed urlopen pins the contract rather than just the return value. #59's version covers the states but less of the request.
  • main exiting 0 vs 2 is the signal import-rawhide-package.yml reads to decline a branch. That is the behaviour a silent regression would break, and #59 does not cover it.
  • The [excluded]-section guard in test_hummingbird_gaps.py matters more than it looks: reports/hummingbird-gap.json is consumed cross-repo by projectbluefin/utah, so a package leaking from [excluded] into the contract silently misreports coverage to a downstream consumer. Pinning that is worth having.

The mutation check ("weakening build.get("state") == 1 to bool(build) and dropping the section != "excluded" filter each make these tests fail") is the right way to demonstrate the suites are not vacuous, and it is specific enough to reproduce.

One factual correction to the body: it reports 108 passed as "was 72 before this PR" — that is right, and I reproduce it exactly. But the body also says just test is "already run by .github/workflows/validate.yml", which is true (python3 -m pytest tests -q in the tests job); worth noting these also run under rebuild-rpms.yml's prepare job via python3 -m unittest discover -s tests -v, which I confirmed passes too (unittest=0).

Recommendation: merge #61+#63, close #59 and #71 as superseded with a pointer to which PR carries their coverage. If the pair goes the other way (#59+#71) then this PR's deeper dist_git and [excluded] cases are worth cherry-picking rather than losing.

hold label is on, so nothing merges by accident meanwhile.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent/quality Filed or owned by the quality agent. hold Work is intentionally paused. quality Code quality or test-coverage work. testing Test authoring or test infrastructure.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants