Skip to content

test: executed coverage for tools/runtime_contract.py main() — tests/test_runtime_contract_main.py - #209

Merged
castrojo merged 1 commit into
mainfrom
quality/test-runtime-contract-main
Sep 21, 2026
Merged

castrojo merged 1 commit into
mainfrom
quality/test-runtime-contract-main

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds executed coverage for main() in tools/runtime_contract.py. Touches exactly one file: the new tests/test_runtime_contract_main.py. No production code changes.

tests/test_runtime_contract.py imports only resolve() and base_image(), so lines 62-89 — argument parsing, all four output modes, the refusal path and the empty-contract refusal — were never executed by any suite.

That CLI is on the atomic-publish path. .github/workflows/rebuild-rpms.yml shells out to it three times:

  • :98 gates the whole rebuild on --check before any wave starts
  • :740 captures default stdout into $CONTRACT
  • :742 captures --base-image into $BASE_IMAGE, which is then the image argument to docker run

The latter two run inside the publish job's "Validate Hummingbird-only consumer transaction" step, which tools/publish_gate.py:106 requires to succeed before ghcr.io/.../utah-packages:latest may move.

Approach

Each case calls main() in-process with a patched sys.argv, so the assertions land on the real statements rather than on a subprocess the coverage report cannot see. stdout and stderr are captured separately, because that split is load-bearing: the workflow assigns stdout to a shell variable, so a refusal that printed to stdout would silently become $CONTRACT or $BASE_IMAGE.

19 cases in three groups:

  • OutputModeTests — default, --json, --check, --base-image, plus both precedence orderings of the if/elif chain (the flags are not declared mutually exclusive, so that order is the contract a caller passing two flags gets). --base-image is asserted to print the reference alone.
  • RefusalTests — missing policy file, missing manifest, unparseable TOML, unpinned [base].image, an [unavailable] exception nobody requested, a missing [bluefin] section, an empty sections list, a non-list sections, a non-string packages entry, and a contract resolving to nothing. Each asserts exit 1, an ERROR: prefix on stderr, and empty stdout.
  • CommittedConfigurationTests — the real config/bluefin-packages.toml + config/runtime-contract.toml against the exact invocations rebuild-rpms.yml makes.

No existing test was modified, skipped or weakened.

Verification

$ python -m pytest tests/test_runtime_contract_main.py -q
19 passed

$ just test                       # python3 -m pytest tests -q
421 passed, 2126 subtests passed          (was 402 + 2126)

$ python3 -m unittest discover -s tests   # what rebuild-rpms.yml:100 runs
Ran 421 tests ... OK

$ just validate
validated 342 source RPMs

$ just factory-check
factory contract satisfied (2409 tracked files checked)

Coverage for the module, before and after:

before:  tools/runtime_contract.py   68 stmts   29 miss   57%   missing 17, 39, 41, 62-89, 93
after:   tools/runtime_contract.py   68 stmts    1 miss   99%   missing 93

Line 93 is the if __name__ == "__main__" guard.

Disjointness

Claims tools/runtime_contract.py::main and the new file tests/test_runtime_contract_main.py only. Disjoint from the other open quality PRs in this repository: #192 claims tools/rebuild_matrix.py / tests/test_rebuild_matrix.py, and #200 claims tools/packit_workflow.py package_chunks/packit_workflow. No file, function or test file overlaps.

Related Issue

Closes #208


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.78

main() had zero executed coverage: tests/test_runtime_contract.py imports
only resolve() and base_image(), leaving lines 62-89 unexecuted.

rebuild-rpms.yml shells out to the CLI three times -- line 98 gates the
rebuild on --check, line 740 captures default stdout into $CONTRACT, and
line 742 captures --base-image into $BASE_IMAGE, which is then the image
argument to docker run inside the publish job's Hummingbird-only consumer
transaction step that publish_gate.py requires before the consumer OCI tag
may move.

Drives main() in-process with a patched sys.argv so the assertions land on
the real statements rather than on a subprocess the coverage report cannot
see, and captures stdout and stderr separately because the workflow assigns
stdout to a shell variable.

Covers all four output modes, both precedence orderings between them, every
refusal branch, and the committed config against the exact invocations the
workflow makes.

tools/runtime_contract.py: 57% -> 99% (only the __main__ guard remains).

Closes #208

Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
@hivecommons-hive hivecommons-hive Bot added the hold Work is intentionally paused. label Sep 20, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will automatically remove the hold label once current policy no longer requires a level hold for "quality". If this is an outreach PR, a human must review it and remove the label.

@castrojo castrojo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed: Adds thorough executed coverage for tools/runtime_contract.py CLI entry point main(), covering output modes, error exit handling, and committed configuration paths. Clean trial merge and passes all tests. Carries maintainer 'hold' label; awaiting maintainer removal of hold and second independent review to satisfy merge policy.

@Danathar Danathar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 226dd60. The only change is the new tests/test_runtime_contract_main.py; tools/runtime_contract.py and the existing tests are untouched. The 19 cases call main() in-process with sys.argv patched and stdout and stderr captured separately, and pytest --cov shows 99% of the module executed (only the if name guard is missed). Mutation check on my side: sending the refusal to stdout instead of stderr, dropping the empty-contract refusal, swapping the --base-image/--check precedence, and making --json print lines each break at least one test, so the tests hold the behaviour rebuild-rpms.yml depends on rather than restating it. Full suite passes at this head (421 passed) and after a trial merge of current main (430 passed); validate.py, factory_contract.py, runtime_contract.py --check and check_workflow_quoting.py pass. The hive hold label still needs a maintainer to remove it.

@castrojo castrojo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved: Adds complete test coverage for tools/runtime_contract.py main() CLI entry point and all output/refusal modes in tests/test_runtime_contract_main.py. CI is green (3/3 checks passing) and git merge-tree confirms clean merge onto origin/main.

@castrojo
castrojo merged commit cd1d664 into main Sep 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hold Work is intentionally paused.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] coverage-gap: tools/runtime_contract.py main() is unexecuted — the CLI the rebuild gate and publish transaction shell out to

2 participants