Skip to content

Add measured, reported, floor-gated code coverage - #5

Merged
adowling2 merged 2 commits into
mainfrom
coverage-reporting
Aug 7, 2026
Merged

Add measured, reported, floor-gated code coverage#5
adowling2 merged 2 commits into
mainfrom
coverage-reporting

Conversation

@adowling2

Copy link
Copy Markdown
Contributor

Summary

Adds measured, reported, and floor-gated code coverage for src/bits_for_gaps (the
shipped package only). This was not a "coverage is bad" task — the established
baseline was already 97% (608 stmts, 18 missed). The work here is: configure
coverage properly, triage every one of those 18 gaps individually (test it for real,
or pragma it with a documented reason), enable branch coverage (which surfaced one
more gap statement coverage couldn't see), add a regression floor, and wire up CI
reporting.

Before / after

Before After
Statement coverage 97% (608 stmts, 18 missed) 100% (605 stmts, 0 missed)
Branch coverage not measured 100% (134 branches, 0 partial)
Tests 204 passed, 2 deselected 218 passed, 2 deselected
--cov-fail-under none 99

(605 vs 608 statements: branch=true + the exclude_lines change how a couple of
lines are counted; not a scope change — source = ["bits_for_gaps"] is unchanged.)

Per-gap triage

Tested (15 statements — real behavior, not line-execution-only):

Location What was untested Test added
__init__.py:39-40 PEP 562 __getattr__'s lazy-resolution branch tests/unit/test_init.py: bits_for_gaps.AnisotropicSE (etc.) resolves to the same object as the direct submodule import; from bits_for_gaps import BitsForGaps works too
__init__.py:45 __dir__ same file: dir(bits_for_gaps) lists both lazy and eager names, no duplicates
design.py:75-76 full_factorial_design's grid-overshoot trim (rng.choice) The existing test picked a perfect-square case (9 = 3×3, no trim needed). New test uses n_train=10 (d=2 → a 16-point grid) and checks no duplicate points + seed-dependent selection
entropy.py:134-136 cholesky()'s Cholesky-based matrix inverse Finding, not a bug: this function is never actually called elsewhere — its own docstring claimed it was used by second_order_entropy's multivariate branch, but that branch uses np.linalg.inv directly. Docstring corrected; function kept (public, correct, documented) with a real correctness test (cholesky(C) == np.linalg.inv(C), C @ cholesky(C) == I)
sampler.py:199-201 adaptiveEntropy.sample_gp_posterior_mixture (instance wrapper) Nothing in the codebase calls this — run() and every other test reach mixture.sample_gp_posterior_mixture directly. It draws from TF's ambient RNG so two calls can't be compared by value; test monkeypatches the delegated call and asserts seed/size are forwarded correctly
sampler.py:224 adaptiveEntropy.entropy_objective (instance wrapper) Same situation, but this one is deterministic (predict_f, not predict_f_samples) — compared value-for-value against calling acquisition.entropy_objective directly with the same config
sampler.py:380-381 showLMLres=True's diagnostic-printing branch The existing initalLML=True test doesn't also set showLMLres. New test sets both and checks capsys output
sampler.py:431-432 run_model(), the deprecated zero-argument entry point read_data alone was tested; nothing chained it through run_model (read_data + run(checkpoint_dir=...)) end to end. New test writes a fake activity_data_1, calls run_model(), checks the returned history and the checkpoint written
sampler.py:462→464 (a branch, found once branch=true was enabled — statement coverage couldn't see this) _write_checkpoint's if entropy_field is not None False path (non-2-D runs) Every existing checkpoint test uses 2-D bounds, so entropy_field is always non-None there. New test in test_nd_synthetic.py runs the existing 1-D/3-D cases with checkpoint_dir set and confirms entropy_{it} is correctly not written

Pragma'd (3 statements — genuinely unreachable or untraceable, not faked):

Location Reason
design.py:72-73 The "grid too small" ValueError. Mathematically unreachable for any (bounds, n_train, n_test): levels = ceil(n_total**(1/d)) guarantees levels**d >= n_total. Already documented as such in tests/unit/test_design.py's NOTE (kept, with its stale line-number reference fixed)
gp.py:177 return tfp.mcmc.sample_chain(...) inside run_mcmc's @tf.function-decorated closure. Confirmed genuinely exercised, not skipped: ran the integration suite (which calls run(), which calls this, repeatedly) with --cov=bits_for_gaps and the line stayed "missing" regardless — @tf.function's AutoGraph compiles this body into a TF graph that executes outside CPython's per-line trace hooks coverage.py relies on. A tooling blind spot, not an untested path

No paper/data/, paper/reference/, tolerance, or algorithm was touched anywhere in this triage. The one non-test source change beyond pragmas/comments is entropy.py's cholesky() docstring correction (removing the false "used by second_order_entropy" claim) — a documentation fix, not a behavior change.

The floor: fail_under = 99

Set in pyproject.toml's [tool.coverage.report]. Currently at 100%; the 1-point
slack is for legitimate small variation (e.g. branch-counting differences coverage.py
might show across the four supported Python versions), not to paper over a real
regression — the codebase is small enough (605 statements) that any meaningfully
untested new function will drop well below 99%, not hover just under it.

pytest -q (no flags) is not affected — it stays exactly as fast and unchanged
as before. Coverage is opt-in via pytest --cov=bits_for_gaps --cov-report=term-missing
(now documented in README.md and docs/installation.md); CI wires that exact
invocation into one job (see below), where fail_under actually gates something.

CI reporting: Codecov (not the artifact alternative)

Chose Codecov (codecov/codecov-action@v5) over the job-summary + HTML-artifact
alternative:

  • This is a public repo, so Codecov's tokenless upload has genuinely zero setup
    burden — no secret to create or rotate, nothing for the maintainer to configure
    beyond the (optional, cosmetic) act of connecting the repo on codecov.io.
  • A repo-topline badge is more discoverable to a visitor or reviewer than an artifact
    buried inside a specific workflow run.
  • The real coverage gate is pytest-cov's own fail_under (enforced locally in the
    same step, on every matrix leg) — Codecov's upload is reporting/visibility on top
    of that, not a second gate, so fail_ci_if_error: false means a transient
    codecov.io outage can't fail CI.

Uploads from the 3.12 matrix leg only (if: matrix.python-version == '3.12') —
uploading from all four would just repeat the same numbers four times.

Hard rules confirmed

  • No dependency version changed. git diff main -- pyproject.toml touches only
    [tool.coverage.*]/[tool.pytest.ini_options] comments; dependencies/
    optional-dependencies are untouched (pytest-cov was already present).
  • No tolerance, reference/baseline file, or algorithm touched. git diff main -- tests/integration/data paper/data paper/reference is empty. Every new/changed test
    passed against existing behavior; nothing needed re-pinning.
  • Green at every commit: pytest -q (218 passed, 2 deselected — up from 204 because
    of the new gap-closing tests, not because anything was loosened), pytest -m vle
    (2 passed), ruff check . (clean), sphinx-build -W docs docs/_build/html (clean),
    and the lazy-import contract (import bits_for_gaps loads neither juliacall nor
    tensorflow).
  • Rebased cleanly onto the current origin/main (after PRs docs: move the VLE physics equations to the example page; fix a grammar slip #3 and docs: add the paper's graphical abstract to the README and docs landing page #4 merged) — no
    conflicts; branch is left in a clean, mergeable state.

🤖 Generated with Claude Code

adowling2 and others added 2 commits August 6, 2026 21:41
Configures coverage.py in pyproject.toml: [tool.coverage.run] scopes
measurement to source = ["bits_for_gaps"] (the shipped package only --
examples/, paper/, tests/ excluded) with branch = true; [tool.coverage.
report] adds the honest exclude_lines (pragma: no cover, TYPE_CHECKING,
NotImplementedError, __repr__, __main__ guard) and fail_under = 99 (1
point of slack below the 100% this commit actually reaches, for minor
cross-Python-version branch-counting variation -- not to paper over a
regression). Deliberately NOT wired into default `pytest -q` addopts,
so a contributor's plain test run stays exactly as fast/unchanged as
before; coverage is opt-in via `pytest --cov=bits_for_gaps
--cov-report=term-missing` (CI wires it into one job separately).

Triaged the 18 statements missed on the established 97% baseline
(608 stmts) one by one -- 15 got a real test, 3 got a `# pragma: no
cover` with a reason:

Tested (real behavior, not line-execution-only):
- __init__.py:39-40,45 -- new tests/unit/test_init.py exercises the PEP
  562 __getattr__ lazy-resolution path (bits_for_gaps.AnisotropicSE is
  the same object as the direct import; from-import works too) and
  __dir__ (lists both lazy and eager names, no duplicates).
- design.py:75-76 -- full_factorial_design's grid-overshoot trim
  branch; the existing test picked a perfect-square case that never
  needed trimming. New test uses n_train=10 (d=2 -> a 16-point grid)
  and confirms no duplicate points and seed-dependent selection.
- entropy.py:134-136 -- cholesky()'s Cholesky-based matrix inverse.
  Found while triaging: this function is NOT actually called by
  second_order_entropy's multivariate branch (which uses np.linalg.inv
  directly) despite its own docstring's claim -- dead code, not a bug.
  Docstring corrected; function kept (public, documented, correct) and
  given a real correctness test (matches np.linalg.inv; C @ C^-1 == I).
- sampler.py:199-201, 224 -- adaptiveEntropy.sample_gp_posterior_mixture/
  entropy_objective, thin instance-method wrappers over mixture.py/
  acquisition.py that nothing else in the codebase calls (run() and
  every other test reach the module-level functions directly instead).
  entropy_objective's wrapper is deterministic (predict_f, not
  predict_f_samples) so it's compared value-for-value against the
  module function. sample_gp_posterior_mixture draws from TF's
  ambient RNG (can't compare values across calls), so its test
  monkeypatches the delegated call to assert seed/size are forwarded
  correctly instead.
- sampler.py:380-381 -- the showLMLres=True diagnostic-printing branch
  (initalLML=True alone, already tested, doesn't set this).
- sampler.py:431-432 -- run_model(), the deprecated disk-based
  zero-argument entry point (read_data + run(checkpoint_dir=self.path));
  read_data alone was already tested, but never chained through
  run_model itself.
- sampler.py:462->464 (a branch, not a statement, found once branch
  coverage was enabled) -- _write_checkpoint's `if entropy_field is not
  None` False path, for non-2-D runs. New test in test_nd_synthetic.py
  runs a 1-D/3-D case with checkpoint_dir set and confirms entropy_{it}
  is correctly NOT written.

Pragma'd with a reason (genuinely unreachable or untraceable, not
faked):
- design.py:72-73 -- the "grid too small" ValueError. Mathematically
  unreachable for any (bounds, n_train, n_test): levels =
  ceil(n_total**(1/d)) guarantees levels**d >= n_total. Already
  documented as such in tests/unit/test_design.py's NOTE (kept, with
  its stale line-number reference fixed).
- gp.py:177 -- the `return tfp.mcmc.sample_chain(...)` inside
  run_mcmc's @tf.function-decorated run_chain_fn. Confirmed genuinely
  exercised (every integration test that calls run() hits it,
  repeatedly) by running the integration suite with --cov=bits_for_gaps
  and observing it stay "missing" regardless -- AutoGraph compiles this
  body into a TF graph that executes outside CPython's per-line trace
  hooks, which is what coverage.py's line tracker relies on. A tooling
  blind spot, not an untested path.

Result: 605 statements / 134 branches, 100% both, 218 passed (was 204)
+ 2 deselected. Verified: pytest -q (fast, no --cov, unchanged output),
pytest -m vle (2 passed), ruff clean, sphinx-build -W clean, lazy-import
contract intact. No dependency changed; no tolerance, reference file,
or algorithm touched.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
.github/workflows/ci.yml: the 3.12 matrix leg (only -- avoids four
duplicate uploads) runs the suite with `--cov=bits_for_gaps
--cov-report=xml --cov-report=term-missing` and uploads to Codecov
(codecov/codecov-action@v5, no token needed for this public repo,
fail_ci_if_error: false since the real coverage gate is pytest-cov's
own fail_under, not the upload succeeding). Chose Codecov over the
job-summary+artifact alternative: public-repo tokenless upload has
zero setup burden for the maintainer, and a repo-topline badge is more
discoverable than an artifact buried in a workflow run -- see the PR
body for the full justification. .gitignore gains coverage.xml (the
other coverage artifacts were already ignored).

README.md: adds the Codecov badge alongside CI/PyPI/Docs, and a
"Quick test" note on measuring coverage locally.

docs/installation.md: a "Coverage" admonition with the same local
command, what CI does, and where the floor is configured; also fixed
a stale test count (204 -> 218).

CHANGELOG.md: [Unreleased] gains an entry for the coverage work
(config, floor, CI reporting, README badge, and a summary of the
18-gap triage from the previous commit).

Verified: pytest -q (218 passed, 2 deselected), ruff clean,
sphinx-build -W clean, lazy-import contract intact. `coverage.xml`
confirmed produced by the exact command CI runs, then removed
(gitignored, never committed).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@adowling2
adowling2 merged commit f981d55 into main Aug 7, 2026
5 checks passed
@adowling2
adowling2 deleted the coverage-reporting branch August 7, 2026 02:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant