Skip to content

[quality] test: unit coverage for tools/dist_git.py and tools/check_workflow_quoting.py - #59

Closed
hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-dist-git-workflow-quoting
Closed

hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-dist-git-workflow-quoting

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds unit coverage for the two tools/ scripts in this repo that had zero tests. Claimed ground, exactly:

New file Covers Functions / behavior
tests/test_dist_git.py tools/dist_git.py nvr_from_spec, koji_complete, command
tests/test_check_workflow_quoting.py tools/check_workflow_quoting.py whole-script execution (module-level, no importable function)

No production code is touched — the diff is two new test files only.

Why these two

  • tools/dist_git.py decides which name-version-release is asked of Koji and whether that build counts as a promotion candidate. A silent parsing slip promotes a package Koji never built.
  • tools/check_workflow_quoting.py runs in rebuild-rpms.yml and is the only thing standing between an apostrophe in a build-script comment and 36 simultaneously dead stage-0 jobs. It had no test proving it actually catches one.

What is asserted

tests/test_dist_git.py — 12 stdlib unittest cases:

  • spec parsing: %{?dist} stripping, hyphenated Name:, indented/unrelated lines ignored, last definition wins, missing fields named in the ValueError, undecodable bytes tolerated
  • Koji JSON-RPC contract: state 1 only is complete, states 0/2/3/4 and an unknown build are not, and the request really is getBuild with the requested NVR, JSON content type, and 30s timeout (urlopen patched; no network)

tests/test_check_workflow_quoting.py — 10 cases running the script the way CI does, in a throwaway tree with synthetic .github/workflows fixtures:

  • clean script passes and reports the workflow count
  • an apostrophe inside a bash -exc block fails with the offending file, line number, and text
  • an apostrophe outside the block is allowed
  • the closing ' delimiter line is not itself reported
  • every offending line is listed; a second block in the same workflow is checked; all workflows are checked, not just rebuild-rpms.yml
  • an unterminated block neither crashes nor guesses; an empty workflow dir passes
  • the real repository workflows pass today

Verification

  • python3 -m pytest tests -q → 94 passed (72 before, 22 new)
  • Mutation-checked: reverting %{?dist} stripping, widening the Koji complete-state test, and off-by-one-ing the offender line slice each make these tests fail (6 failures), so they are not vacuous.

Disjointness

No open PR in this repo touches tools/dist_git.py, tools/check_workflow_quoting.py, or either new test file. The nearest neighbour, utah-packages#56, changes tools/import_rawhide.py + tests/test_import_rawhide.py — no shared file or function.

Related Issue

None — filed directly as a coverage gap.


Filed by quality agent (hold-gated mode). Human review required. Do not merge without review; the hold label stays.

— hive: agent=quality backend=copilot model=claude-opus-5

…rkflow_quoting.py

tools/dist_git.py (nvr_from_spec, koji_complete, command) and
tools/check_workflow_quoting.py had zero unit tests. Both gate real
pipelines: dist_git decides which NVR is asked of Koji before a package is
promoted, and check_workflow_quoting is the rebuild-rpms.yml gate that keeps
an apostrophe from closing a bash -exc build script.

tests/test_dist_git.py: 12 stdlib unittest cases over spec-field parsing
(%{?dist} stripping, hyphenated names, indented/unrelated fields, last
definition wins, missing-field ValueError, undecodable bytes) and the Koji
JSON-RPC contract (state 1 only, unknown build, getBuild method/params/
headers/timeout).

tests/test_check_workflow_quoting.py: 10 cases running the script the way CI
does, in a throwaway tree with synthetic .github/workflows fixtures --
clean pass, offender detection and line numbers, quotes outside the script
block, the closing delimiter line, multiple blocks, multiple workflows,
unterminated blocks, empty workflow dir, and the real repo passing today.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>

@hanthor hanthor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is fine on its own, but it is one of four open PRs that between them write the same four test files twice each. Flagging the whole cluster here so it can be resolved in one decision rather than four.

The cluster. Four PRs, all from the quality agent, all "new test files only":

PR test_check_workflow_quoting.py test_dist_git.py test_validate.py test_hummingbird_gaps.py
#59 (this) X X
#61 X X
#63 X X
#71 X X

Computed from three-dot diffs against main (54cc8d1):

59 61 ['tests/test_check_workflow_quoting.py']
59 63 ['tests/test_dist_git.py']
61 71 ['tests/test_validate.py']
63 71 ['tests/test_hummingbird_gaps.py']

Each PR's "Disjointness" section is accurate as far as it goes — #59 says "No open PR in this repo touches tools/dist_git.py, tools/check_workflow_quoting.py, or either new test file", and #61/#63/#71 each say something similar. They were all true when written and all false now. The agent checked at filing time and the cluster grew underneath it.

The useful part: it forms a cycle, so two PRs cover everything with zero overlap.

  • #59 + #71 → quoting, dist_git, validate, gaps. Complete, no collision.
  • #61 + #63 → quoting, validate, dist_git, gaps. Complete, no collision.

Either pair gives full coverage of all four modules; the other two PRs can then be closed as duplicates. No rebasing or conflict resolution needed in either case.

Choosing between the pairs — the versions differ in depth, so it is worth comparing rather than picking arbitrarily. Test counts when each is merged into main locally (baseline is 72 passed):

PR#59 -> 94 passed   (+22)
PR#61 -> 97 passed   (+25)
PR#63 -> 108 passed  (+36)
PR#71 -> 103 passed  (+31)

So #61+#63 yields the deeper suites on three of the four files. #63's test_dist_git.py in particular claims 22 cases against #59's 12, and asserts the Koji request shape and the main exit codes that #59 does not. On the other hand #59's test_check_workflow_quoting.py writeup explicitly covers the recent widening in 54cc8d1 from rebuild-rpms.yml to all workflows, which is the change most likely to regress. Worth a look before deciding.

This PR's own state is clean. Merged into main it passes every gate CI runs, with no change from the unmodified-main baseline other than the new tests:

PR#59 fc=0 val=0 pytest=0 | validated 193 source RPMs | 94 passed in 1.15s
PR#59 quoting=0 runtime_contract=0 unittest=0 | validated runtime contract: 72 packages

I also confirmed the mutation claim is not vacuous in spirit — the suites do drive the scripts as CI drives them rather than importing and stubbing everything.

Recommendation: pick one pair, close the other two as superseded, and note in the closed ones which PR carries their coverage so it is not re-filed. Also worth teaching the quality agent to re-check disjointness against open PRs at review time, not only at filing time — this is the second time this pattern has produced redundant work in this org.

All four are hold-labelled, so nothing merges by accident in the meantime.


Generated by Claude Code

@hanthor hanthor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. #71 was just lgtm'd by a maintainer, which picks the #59 + #71 pair out of the two options in my earlier review — so this PR is the other half, and the cluster question that was the only thing holding it is settled.

This PR's own state was already clean and the branch has not moved since (b5371fd):

PR#59 fc=0 val=0 pytest=0 | validated 193 source RPMs | 94 passed in 1.15s
PR#59 quoting=0 runtime_contract=0 unittest=0 | validated runtime contract: 72 packages

+22 tests over the 72 passed baseline, no production code touched, every other gate unchanged.

Being straight about what the pair choice costs, since I raised it before and it went the other way: #61 + #63 would have yielded deeper suites on three of the four files. The specific gap this pair leaves is #63's test_dist_git.py — 22 cases against this PR's 12, and it asserts two things this one does not:

  • the Koji request shape (that the call really is getBuild with the requested NVR, JSON content type, 30s timeout, against a stubbed urlopen);
  • main's exit codes — 0 on complete, 2 on incomplete, which is the signal the workflow actually reads to decline a branch.

That second one matters more than the case count suggests: exit 2 is the entire contract between dist_git.py and the workflow that consumes it, and after this pair lands nothing pins it. Worth lifting those cases across in a follow-up. Same for #63's [excluded]-leak guard, which I noted on #71.

What this PR has that the other pair does not: its test_check_workflow_quoting.py explicitly exercises the widening in 54cc8d1 from rebuild-rpms.yml to every workflow — the most recent change to that script and so the most likely to regress. Keeping it is a real argument for this pair, not just a coin flip.

hold is still on, so this needs a maintainer to lift it rather than merging off my approval. Before it does, #61 and #63 should be closed as duplicates with pointers to where their coverage lands: #61's test_check_workflow_quoting.py → here, #61's test_validate.py → #71, #63's test_dist_git.py → here, #63's test_hummingbird_gaps.py → #71. If they stay open, whichever merges second fails on a file that already exists.


Generated by Claude Code

@castrojo

Copy link
Copy Markdown
Contributor

Superseded by merged PRs #61 and #63.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hold Work is intentionally paused.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants