Add measured, reported, floor-gated code coverage - #5
Merged
Conversation
Configures coverage.py in pyproject.toml: [tool.coverage.run] scopes
measurement to source = ["bits_for_gaps"] (the shipped package only --
examples/, paper/, tests/ excluded) with branch = true; [tool.coverage.
report] adds the honest exclude_lines (pragma: no cover, TYPE_CHECKING,
NotImplementedError, __repr__, __main__ guard) and fail_under = 99 (1
point of slack below the 100% this commit actually reaches, for minor
cross-Python-version branch-counting variation -- not to paper over a
regression). Deliberately NOT wired into default `pytest -q` addopts,
so a contributor's plain test run stays exactly as fast/unchanged as
before; coverage is opt-in via `pytest --cov=bits_for_gaps
--cov-report=term-missing` (CI wires it into one job separately).
Triaged the 18 statements missed on the established 97% baseline
(608 stmts) one by one -- 15 got a real test, 3 got a `# pragma: no
cover` with a reason:
Tested (real behavior, not line-execution-only):
- __init__.py:39-40,45 -- new tests/unit/test_init.py exercises the PEP
562 __getattr__ lazy-resolution path (bits_for_gaps.AnisotropicSE is
the same object as the direct import; from-import works too) and
__dir__ (lists both lazy and eager names, no duplicates).
- design.py:75-76 -- full_factorial_design's grid-overshoot trim
branch; the existing test picked a perfect-square case that never
needed trimming. New test uses n_train=10 (d=2 -> a 16-point grid)
and confirms no duplicate points and seed-dependent selection.
- entropy.py:134-136 -- cholesky()'s Cholesky-based matrix inverse.
Found while triaging: this function is NOT actually called by
second_order_entropy's multivariate branch (which uses np.linalg.inv
directly) despite its own docstring's claim -- dead code, not a bug.
Docstring corrected; function kept (public, documented, correct) and
given a real correctness test (matches np.linalg.inv; C @ C^-1 == I).
- sampler.py:199-201, 224 -- adaptiveEntropy.sample_gp_posterior_mixture/
entropy_objective, thin instance-method wrappers over mixture.py/
acquisition.py that nothing else in the codebase calls (run() and
every other test reach the module-level functions directly instead).
entropy_objective's wrapper is deterministic (predict_f, not
predict_f_samples) so it's compared value-for-value against the
module function. sample_gp_posterior_mixture draws from TF's
ambient RNG (can't compare values across calls), so its test
monkeypatches the delegated call to assert seed/size are forwarded
correctly instead.
- sampler.py:380-381 -- the showLMLres=True diagnostic-printing branch
(initalLML=True alone, already tested, doesn't set this).
- sampler.py:431-432 -- run_model(), the deprecated disk-based
zero-argument entry point (read_data + run(checkpoint_dir=self.path));
read_data alone was already tested, but never chained through
run_model itself.
- sampler.py:462->464 (a branch, not a statement, found once branch
coverage was enabled) -- _write_checkpoint's `if entropy_field is not
None` False path, for non-2-D runs. New test in test_nd_synthetic.py
runs a 1-D/3-D case with checkpoint_dir set and confirms entropy_{it}
is correctly NOT written.
Pragma'd with a reason (genuinely unreachable or untraceable, not
faked):
- design.py:72-73 -- the "grid too small" ValueError. Mathematically
unreachable for any (bounds, n_train, n_test): levels =
ceil(n_total**(1/d)) guarantees levels**d >= n_total. Already
documented as such in tests/unit/test_design.py's NOTE (kept, with
its stale line-number reference fixed).
- gp.py:177 -- the `return tfp.mcmc.sample_chain(...)` inside
run_mcmc's @tf.function-decorated run_chain_fn. Confirmed genuinely
exercised (every integration test that calls run() hits it,
repeatedly) by running the integration suite with --cov=bits_for_gaps
and observing it stay "missing" regardless -- AutoGraph compiles this
body into a TF graph that executes outside CPython's per-line trace
hooks, which is what coverage.py's line tracker relies on. A tooling
blind spot, not an untested path.
Result: 605 statements / 134 branches, 100% both, 218 passed (was 204)
+ 2 deselected. Verified: pytest -q (fast, no --cov, unchanged output),
pytest -m vle (2 passed), ruff clean, sphinx-build -W clean, lazy-import
contract intact. No dependency changed; no tolerance, reference file,
or algorithm touched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
.github/workflows/ci.yml: the 3.12 matrix leg (only -- avoids four duplicate uploads) runs the suite with `--cov=bits_for_gaps --cov-report=xml --cov-report=term-missing` and uploads to Codecov (codecov/codecov-action@v5, no token needed for this public repo, fail_ci_if_error: false since the real coverage gate is pytest-cov's own fail_under, not the upload succeeding). Chose Codecov over the job-summary+artifact alternative: public-repo tokenless upload has zero setup burden for the maintainer, and a repo-topline badge is more discoverable than an artifact buried in a workflow run -- see the PR body for the full justification. .gitignore gains coverage.xml (the other coverage artifacts were already ignored). README.md: adds the Codecov badge alongside CI/PyPI/Docs, and a "Quick test" note on measuring coverage locally. docs/installation.md: a "Coverage" admonition with the same local command, what CI does, and where the floor is configured; also fixed a stale test count (204 -> 218). CHANGELOG.md: [Unreleased] gains an entry for the coverage work (config, floor, CI reporting, README badge, and a summary of the 18-gap triage from the previous commit). Verified: pytest -q (218 passed, 2 deselected), ruff clean, sphinx-build -W clean, lazy-import contract intact. `coverage.xml` confirmed produced by the exact command CI runs, then removed (gitignored, never committed). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds measured, reported, and floor-gated code coverage for
src/bits_for_gaps(theshipped package only). This was not a "coverage is bad" task — the established
baseline was already 97% (608 stmts, 18 missed). The work here is: configure
coverage properly, triage every one of those 18 gaps individually (test it for real,
or pragma it with a documented reason), enable branch coverage (which surfaced one
more gap statement coverage couldn't see), add a regression floor, and wire up CI
reporting.
Before / after
--cov-fail-under(605 vs 608 statements:
branch=true+ the exclude_lines change how a couple oflines are counted; not a scope change —
source = ["bits_for_gaps"]is unchanged.)Per-gap triage
Tested (15 statements — real behavior, not line-execution-only):
__init__.py:39-40__getattr__'s lazy-resolution branchtests/unit/test_init.py:bits_for_gaps.AnisotropicSE(etc.) resolves to the same object as the direct submodule import;from bits_for_gaps import BitsForGapsworks too__init__.py:45__dir__dir(bits_for_gaps)lists both lazy and eager names, no duplicatesdesign.py:75-76full_factorial_design's grid-overshoot trim (rng.choice)n_train=10(d=2 → a 16-point grid) and checks no duplicate points + seed-dependent selectionentropy.py:134-136cholesky()'s Cholesky-based matrix inversesecond_order_entropy's multivariate branch, but that branch usesnp.linalg.invdirectly. Docstring corrected; function kept (public, correct, documented) with a real correctness test (cholesky(C) == np.linalg.inv(C),C @ cholesky(C) == I)sampler.py:199-201adaptiveEntropy.sample_gp_posterior_mixture(instance wrapper)run()and every other test reachmixture.sample_gp_posterior_mixturedirectly. It draws from TF's ambient RNG so two calls can't be compared by value; test monkeypatches the delegated call and assertsseed/sizeare forwarded correctlysampler.py:224adaptiveEntropy.entropy_objective(instance wrapper)predict_f, notpredict_f_samples) — compared value-for-value against callingacquisition.entropy_objectivedirectly with the same configsampler.py:380-381showLMLres=True's diagnostic-printing branchinitalLML=Truetest doesn't also setshowLMLres. New test sets both and checkscapsysoutputsampler.py:431-432run_model(), the deprecated zero-argument entry pointread_dataalone was tested; nothing chained it throughrun_model(read_data+run(checkpoint_dir=...)) end to end. New test writes a fakeactivity_data_1, callsrun_model(), checks the returned history and the checkpoint writtensampler.py:462→464(a branch, found oncebranch=truewas enabled — statement coverage couldn't see this)_write_checkpoint'sif entropy_field is not NoneFalse path (non-2-D runs)entropy_fieldis always non-Nonethere. New test intest_nd_synthetic.pyruns the existing 1-D/3-D cases withcheckpoint_dirset and confirmsentropy_{it}is correctly not writtenPragma'd (3 statements — genuinely unreachable or untraceable, not faked):
design.py:72-73ValueError. Mathematically unreachable for any(bounds, n_train, n_test):levels = ceil(n_total**(1/d))guaranteeslevels**d >= n_total. Already documented as such intests/unit/test_design.py's NOTE (kept, with its stale line-number reference fixed)gp.py:177return tfp.mcmc.sample_chain(...)insiderun_mcmc's@tf.function-decorated closure. Confirmed genuinely exercised, not skipped: ran the integration suite (which callsrun(), which calls this, repeatedly) with--cov=bits_for_gapsand the line stayed "missing" regardless —@tf.function's AutoGraph compiles this body into a TF graph that executes outside CPython's per-line trace hooks coverage.py relies on. A tooling blind spot, not an untested pathNo
paper/data/,paper/reference/, tolerance, or algorithm was touched anywhere in this triage. The one non-test source change beyond pragmas/comments isentropy.py'scholesky()docstring correction (removing the false "used by second_order_entropy" claim) — a documentation fix, not a behavior change.The floor:
fail_under = 99Set in
pyproject.toml's[tool.coverage.report]. Currently at 100%; the 1-pointslack is for legitimate small variation (e.g. branch-counting differences coverage.py
might show across the four supported Python versions), not to paper over a real
regression — the codebase is small enough (605 statements) that any meaningfully
untested new function will drop well below 99%, not hover just under it.
pytest -q(no flags) is not affected — it stays exactly as fast and unchangedas before. Coverage is opt-in via
pytest --cov=bits_for_gaps --cov-report=term-missing(now documented in
README.mdanddocs/installation.md); CI wires that exactinvocation into one job (see below), where
fail_underactually gates something.CI reporting: Codecov (not the artifact alternative)
Chose Codecov (
codecov/codecov-action@v5) over the job-summary + HTML-artifactalternative:
burden — no secret to create or rotate, nothing for the maintainer to configure
beyond the (optional, cosmetic) act of connecting the repo on codecov.io.
buried inside a specific workflow run.
pytest-cov's ownfail_under(enforced locally in thesame step, on every matrix leg) — Codecov's upload is reporting/visibility on top
of that, not a second gate, so
fail_ci_if_error: falsemeans a transientcodecov.io outage can't fail CI.
Uploads from the 3.12 matrix leg only (
if: matrix.python-version == '3.12') —uploading from all four would just repeat the same numbers four times.
Hard rules confirmed
git diff main -- pyproject.tomltouches only[tool.coverage.*]/[tool.pytest.ini_options]comments;dependencies/optional-dependenciesare untouched (pytest-covwas already present).git diff main -- tests/integration/data paper/data paper/referenceis empty. Every new/changed testpassed against existing behavior; nothing needed re-pinning.
pytest -q(218 passed, 2 deselected — up from 204 becauseof the new gap-closing tests, not because anything was loosened),
pytest -m vle(2 passed),
ruff check .(clean),sphinx-build -W docs docs/_build/html(clean),and the lazy-import contract (
import bits_for_gapsloads neitherjuliacallnortensorflow).origin/main(after PRs docs: move the VLE physics equations to the example page; fix a grammar slip #3 and docs: add the paper's graphical abstract to the README and docs landing page #4 merged) — noconflicts; branch is left in a clean, mergeable state.
🤖 Generated with Claude Code