From 5d2742923c0f0176511bb558471eddae518c5e29 Mon Sep 17 00:00:00 2001 From: Alex Dowling Date: Thu, 6 Aug 2026 21:31:08 -0400 Subject: [PATCH 1/2] Measure coverage, close the 18 pre-existing gaps, add a 99% floor Configures coverage.py in pyproject.toml: [tool.coverage.run] scopes measurement to source = ["bits_for_gaps"] (the shipped package only -- examples/, paper/, tests/ excluded) with branch = true; [tool.coverage. report] adds the honest exclude_lines (pragma: no cover, TYPE_CHECKING, NotImplementedError, __repr__, __main__ guard) and fail_under = 99 (1 point of slack below the 100% this commit actually reaches, for minor cross-Python-version branch-counting variation -- not to paper over a regression). Deliberately NOT wired into default `pytest -q` addopts, so a contributor's plain test run stays exactly as fast/unchanged as before; coverage is opt-in via `pytest --cov=bits_for_gaps --cov-report=term-missing` (CI wires it into one job separately). Triaged the 18 statements missed on the established 97% baseline (608 stmts) one by one -- 15 got a real test, 3 got a `# pragma: no cover` with a reason: Tested (real behavior, not line-execution-only): - __init__.py:39-40,45 -- new tests/unit/test_init.py exercises the PEP 562 __getattr__ lazy-resolution path (bits_for_gaps.AnisotropicSE is the same object as the direct import; from-import works too) and __dir__ (lists both lazy and eager names, no duplicates). - design.py:75-76 -- full_factorial_design's grid-overshoot trim branch; the existing test picked a perfect-square case that never needed trimming. New test uses n_train=10 (d=2 -> a 16-point grid) and confirms no duplicate points and seed-dependent selection. - entropy.py:134-136 -- cholesky()'s Cholesky-based matrix inverse. Found while triaging: this function is NOT actually called by second_order_entropy's multivariate branch (which uses np.linalg.inv directly) despite its own docstring's claim -- dead code, not a bug. Docstring corrected; function kept (public, documented, correct) and given a real correctness test (matches np.linalg.inv; C @ C^-1 == I). - sampler.py:199-201, 224 -- adaptiveEntropy.sample_gp_posterior_mixture/ entropy_objective, thin instance-method wrappers over mixture.py/ acquisition.py that nothing else in the codebase calls (run() and every other test reach the module-level functions directly instead). entropy_objective's wrapper is deterministic (predict_f, not predict_f_samples) so it's compared value-for-value against the module function. sample_gp_posterior_mixture draws from TF's ambient RNG (can't compare values across calls), so its test monkeypatches the delegated call to assert seed/size are forwarded correctly instead. - sampler.py:380-381 -- the showLMLres=True diagnostic-printing branch (initalLML=True alone, already tested, doesn't set this). - sampler.py:431-432 -- run_model(), the deprecated disk-based zero-argument entry point (read_data + run(checkpoint_dir=self.path)); read_data alone was already tested, but never chained through run_model itself. - sampler.py:462->464 (a branch, not a statement, found once branch coverage was enabled) -- _write_checkpoint's `if entropy_field is not None` False path, for non-2-D runs. New test in test_nd_synthetic.py runs a 1-D/3-D case with checkpoint_dir set and confirms entropy_{it} is correctly NOT written. Pragma'd with a reason (genuinely unreachable or untraceable, not faked): - design.py:72-73 -- the "grid too small" ValueError. Mathematically unreachable for any (bounds, n_train, n_test): levels = ceil(n_total**(1/d)) guarantees levels**d >= n_total. Already documented as such in tests/unit/test_design.py's NOTE (kept, with its stale line-number reference fixed). - gp.py:177 -- the `return tfp.mcmc.sample_chain(...)` inside run_mcmc's @tf.function-decorated run_chain_fn. Confirmed genuinely exercised (every integration test that calls run() hits it, repeatedly) by running the integration suite with --cov=bits_for_gaps and observing it stay "missing" regardless -- AutoGraph compiles this body into a TF graph that executes outside CPython's per-line trace hooks, which is what coverage.py's line tracker relies on. A tooling blind spot, not an untested path. Result: 605 statements / 134 branches, 100% both, 218 passed (was 204) + 2 deselected. Verified: pytest -q (fast, no --cov, unchanged output), pytest -m vle (2 passed), ruff clean, sphinx-build -W clean, lazy-import contract intact. No dependency changed; no tolerance, reference file, or algorithm touched. Co-Authored-By: Claude Sonnet 5 --- pyproject.toml | 27 ++++ src/bits_for_gaps/design.py | 5 +- src/bits_for_gaps/entropy.py | 7 +- src/bits_for_gaps/gp.py | 6 +- tests/integration/test_nd_synthetic.py | 14 ++ tests/unit/test_design.py | 25 +++- tests/unit/test_entropy.py | 19 +++ tests/unit/test_init.py | 49 +++++++ .../test_sampler_legacy_and_transforms.py | 138 +++++++++++++++++- 9 files changed, 281 insertions(+), 9 deletions(-) create mode 100644 tests/unit/test_init.py diff --git a/pyproject.toml b/pyproject.toml index b296dda..d184c52 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -128,8 +128,35 @@ quote-style = "double" testpaths = ["tests"] # `vle` tests need the Julia/Clapeyron backend; deselect them by default so a plain # `pytest -q` is green on the pure-Python core. Run them with `pytest -m vle`. +# Deliberately NOT `--cov=...` here: a plain `pytest -q` (what a contributor runs day +# to day) stays fast and its output stays unchanged. Measure coverage explicitly with +# `pytest --cov=bits_for_gaps --cov-report=term-missing` (see README.md/docs); CI wires +# that invocation in on one job so `fail_under` below actually gates something. addopts = "-ra -m 'not vle'" markers = [ "vle: requires the Julia/Clapeyron VLE example backend (deselected by default)", "slow: slower end-to-end tests (still run by default)", ] + +[tool.coverage.run] +# Measure only the shipped package -- examples/, paper/, and tests/ are repo-only +# (see docs/installation.md) and would otherwise dilute the denominator with code +# that was never meant to ship. +source = ["bits_for_gaps"] +branch = true + +[tool.coverage.report] +# `pytest -m vle`'s 2 Julia-backed tests are deselected by default (see above), so +# default-suite coverage never reflects them -- that's expected, not a gap to chase. +exclude_lines = [ + "pragma: no cover", + "if TYPE_CHECKING:", + "raise NotImplementedError", + "def __repr__", + "if __name__ == .__main__.:", +] +# 100% measured on the default suite (`pytest -q --cov=bits_for_gaps --cov-branch`, +# 605 statements + 134 branches) as of the commit that added this config -- 1 point of +# slack for legitimate small variation (coverage.py branch-counting differences across +# supported Python versions), not to paper over a regression. +fail_under = 99 diff --git a/src/bits_for_gaps/design.py b/src/bits_for_gaps/design.py index 84e86bd..5e9cf39 100644 --- a/src/bits_for_gaps/design.py +++ b/src/bits_for_gaps/design.py @@ -69,7 +69,10 @@ def full_factorial_design( levels = int(np.ceil(n_total ** (1 / d))) grid_unit = np.array(list(product(*[np.linspace(0, 1, levels) for _ in range(d)]))) - if len(grid_unit) < n_total: + if len(grid_unit) < n_total: # pragma: no cover + # Unreachable for any (bounds, n_train, n_test): levels = ceil(n_total**(1/d)) + # guarantees levels**d >= n_total. Defensive dead code -- see the NOTE in + # tests/unit/test_design.py. raise ValueError(f"Factorial grid has {len(grid_unit)} points, need {n_total}.") if len(grid_unit) > n_total: rng = np.random.default_rng(seed) diff --git a/src/bits_for_gaps/entropy.py b/src/bits_for_gaps/entropy.py index 8aeb646..df49348 100644 --- a/src/bits_for_gaps/entropy.py +++ b/src/bits_for_gaps/entropy.py @@ -128,8 +128,11 @@ def cholesky(C: np.ndarray) -> np.ndarray: ------- np.ndarray, shape (d, d) ``C`` inverse, computed as ``L^-T L^-1`` where ``C = L L^T`` -- more - numerically stable than a direct ``np.linalg.inv`` for the covariance matrices - that arise here (used by :func:`second_order_entropy`'s multivariate branch). + numerically stable than a direct ``np.linalg.inv`` for well-conditioned + covariance matrices. Not currently called elsewhere in this package ( + :func:`second_order_entropy`'s multivariate branch uses ``np.linalg.inv`` + directly); kept as a public, tested utility for callers who want the more + numerically stable inverse. """ L = np.linalg.cholesky(C) L_inv = np.linalg.inv(L) diff --git a/src/bits_for_gaps/gp.py b/src/bits_for_gaps/gp.py index 0ee55da..c162621 100644 --- a/src/bits_for_gaps/gp.py +++ b/src/bits_for_gaps/gp.py @@ -174,7 +174,11 @@ def run_mcmc( @tf.function def run_chain_fn(chain_seed): - return tfp.mcmc.sample_chain( + # This line genuinely executes on every HMC run (every integration test that + # calls run() exercises it, repeatedly) -- it shows as uncovered because + # @tf.function's AutoGraph compiles this body into a TF graph, which runs + # outside CPython's normal per-line trace hooks that coverage.py relies on. + return tfp.mcmc.sample_chain( # pragma: no cover num_results=no_samples, num_burnin_steps=no_burn_in, current_state=hmc_helper.current_state, diff --git a/tests/integration/test_nd_synthetic.py b/tests/integration/test_nd_synthetic.py index 0a30d9e..4f87846 100644 --- a/tests/integration/test_nd_synthetic.py +++ b/tests/integration/test_nd_synthetic.py @@ -170,3 +170,17 @@ def test_stable_across_two_runs_with_same_seed(run_a, run_b): np.testing.assert_allclose(a.trace, b.trace, atol=1e-10) np.testing.assert_allclose(a.xStar, b.xStar, atol=1e-10) assert a.max_entropy == pytest.approx(b.max_entropy, abs=1e-10) + + +@pytest.mark.slow +def test_checkpoint_skips_entropy_file_for_non_2d(case, tmp_path): + # _write_checkpoint only writes entropy_{it} when record.entropy_field is not + # None -- which test_end_to_end.py's 2-D checkpoint test always has, so the d != 2 + # (entropy_field is None) branch is otherwise never exercised. + X_init, y_init = _initial_design(case["bounds"], case["true_f"], n=10) + bfg = _build_bfg(case) + checkpoint_dir = tmp_path / "checkpoints" + bfg.run(X_init, y_init, checkpoint_dir=str(checkpoint_dir)) + written = {p.name for p in checkpoint_dir.iterdir()} + assert {"rhat_value_1.txt", "ess_value_1.txt", "activity_data_2", "gp_model_1.pkl"} <= written + assert "entropy_1" not in written diff --git a/tests/unit/test_design.py b/tests/unit/test_design.py index 2982d17..4f9f295 100644 --- a/tests/unit/test_design.py +++ b/tests/unit/test_design.py @@ -48,7 +48,26 @@ def test_full_factorial_exact_grid_size_needs_no_trimming(): assert test.shape == (0, 2) -## NOTE: `full_factorial_design`'s "grid too small" `ValueError` (design.py:74-75) is +def test_full_factorial_overshoot_grid_is_trimmed_and_seed_dependent(): + # d=2, n_total=10 -> levels = ceil(sqrt(10)) = 4 -> a 4x4=16-point grid, which + # overshoots n_total=10 and must be randomly trimmed (the `rng.choice` branch + # `test_full_factorial_exact_grid_size_needs_no_trimming` above doesn't reach, + # since 9 is a perfect square). + bounds = [(0.0, 1.0), (0.0, 1.0)] + train, test = full_factorial_design(bounds, n_train=10, n_test=0, seed=0) + assert train.shape == (10, 2) + assert test.shape == (0, 2) + assert train.min() >= 0.0 and train.max() <= 1.0 + # No duplicate points -- confirms `replace=False` trimming, not resampling. + assert len(np.unique(train, axis=0)) == 10 + + # A different seed must select a different subset of the 16-point grid (proves + # the trim is actually seeded, not silently deterministic regardless of `seed`). + train_b, _ = full_factorial_design(bounds, n_train=10, n_test=0, seed=1) + assert not np.array_equal(np.sort(train, axis=0), np.sort(train_b, axis=0)) + + +## NOTE: `full_factorial_design`'s "grid too small" `ValueError` (design.py:72-73) is ## unreachable for any (bounds, n_train, n_test): `levels = ceil(n_total ** (1/d))` -## guarantees `levels ** d >= n_total`. Defensive dead code, not a bug -- not exercised -## here since no input triggers it. +## guarantees `levels ** d >= n_total`. Defensive dead code, not a bug -- marked +## `# pragma: no cover` at the source rather than faked with a test. diff --git a/tests/unit/test_entropy.py b/tests/unit/test_entropy.py index 5cfca57..6e433fa 100644 --- a/tests/unit/test_entropy.py +++ b/tests/unit/test_entropy.py @@ -10,6 +10,7 @@ from scipy import stats from bits_for_gaps.entropy import ( + cholesky, entropy_lower_bound, first_order_entropy_approx, gaussian_mixture_density, @@ -124,3 +125,21 @@ def test_gaussian_mixture_density_multivariate_matches_scipy(): cov = np.diag([1.0, 0.5]) density = gaussian_mixture_density(x, means=[mean], covs=[cov], weights=[1.0]) assert density == pytest.approx(stats.multivariate_normal.pdf(x, mean=mean, cov=cov), rel=1e-12) + + +## --------------------------------------------------------------------------- +## cholesky: not currently called elsewhere in this package (second_order_entropy's +## multivariate branch uses np.linalg.inv directly) but a public, documented utility +## a caller could rely on -- tested directly on its own mathematical contract. +## --------------------------------------------------------------------------- + + +def test_cholesky_matches_direct_matrix_inverse(): + C = np.array([[4.0, 1.0], [1.0, 3.0]]) + np.testing.assert_allclose(cholesky(C), np.linalg.inv(C), rtol=1e-12) + + +def test_cholesky_result_is_a_true_inverse(): + C = np.array([[2.0, 0.3, 0.1], [0.3, 1.5, 0.2], [0.1, 0.2, 1.0]]) + C_inv = cholesky(C) + np.testing.assert_allclose(C @ C_inv, np.eye(3), atol=1e-12) diff --git a/tests/unit/test_init.py b/tests/unit/test_init.py new file mode 100644 index 0000000..7830265 --- /dev/null +++ b/tests/unit/test_init.py @@ -0,0 +1,49 @@ +"""Unit tests for ``bits_for_gaps``'s top-level PEP 562 lazy-import machinery. + +Every other test in this suite reaches the TF-backed classes via +``from bits_for_gaps.kernels import AnisotropicSE``-style direct submodule imports, +which never exercises ``__getattr__``'s lazy-resolution branch or ``__dir__``. Both are +part of the module's actual public contract (documented in its module docstring), so +they're tested directly here rather than excluded from coverage. +""" + +import pytest + +import bits_for_gaps +from bits_for_gaps.kernels import AnisotropicSE +from bits_for_gaps.means import FixedInverseMean +from bits_for_gaps.sampler import BitsForGaps, adaptiveEntropy + + +def test_lazy_attribute_resolves_to_the_real_class(): + # bits_for_gaps.AnisotropicSE (no direct submodule import) must be the exact same + # object __getattr__ resolves it to, not a copy or a different definition. + assert bits_for_gaps.AnisotropicSE is AnisotropicSE + assert bits_for_gaps.FixedInverseMean is FixedInverseMean + assert bits_for_gaps.adaptiveEntropy is adaptiveEntropy + assert bits_for_gaps.BitsForGaps is BitsForGaps + + +def test_lazy_attribute_importable_via_from_import(): + # `from bits_for_gaps import BitsForGaps` is the documented top-level entry point + # (see docs/quickstart.md) -- it must go through the same __getattr__ path. + from bits_for_gaps import BitsForGaps as ReImported + + assert ReImported is BitsForGaps + + +def test_unknown_attribute_raises_attribute_error(): + with pytest.raises(AttributeError, match="no attribute 'NotARealName'"): + _ = bits_for_gaps.NotARealName + + +def test_dir_includes_lazy_names_and_eager_names(): + names = dir(bits_for_gaps) + # Lazy (TF-backed) names, resolved only via __getattr__. + for lazy_name in ("AnisotropicSE", "FixedInverseMean", "adaptiveEntropy", "BitsForGaps"): + assert lazy_name in names + # Eager (pure NumPy/SciPy) names, already in the module's normal namespace. + for eager_name in ("second_order_entropy", "latin_hypercube_design", "design", "entropy"): + assert eager_name in names + # dir() must not report duplicates. + assert len(names) == len(set(names)) diff --git a/tests/unit/test_sampler_legacy_and_transforms.py b/tests/unit/test_sampler_legacy_and_transforms.py index e2037a2..01bb684 100644 --- a/tests/unit/test_sampler_legacy_and_transforms.py +++ b/tests/unit/test_sampler_legacy_and_transforms.py @@ -1,12 +1,18 @@ """Unit tests for a few `adaptiveEntropy`/`BitsForGaps` code paths the rest of the -suite doesn't reach directly: the legacy disk-based `read_data`, and `BitsForGaps`'s -optional custom `input_transform`/`output_transform` override. +suite doesn't reach directly: the legacy disk-based `read_data`, `BitsForGaps`'s +optional custom `input_transform`/`output_transform` override, and the thin +`sample_gp_posterior_mixture`/`entropy_objective` instance-method wrappers over +`mixture.py`/`acquisition.py` (used by callers who want the sampler's own +seed/noGaussians/acquisitionObjective config applied automatically, but not called by +`run()` itself -- `run()` and every other test in this suite reach the module-level +functions directly instead, e.g. `tests/integration/test_end_to_end.py`). """ import gpflow import numpy as np import pytest +from bits_for_gaps import acquisition from bits_for_gaps.kernels import AnisotropicSE from bits_for_gaps.sampler import BitsForGaps, adaptiveEntropy from bits_for_gaps.transforms import InputTransform, OutputTransform @@ -18,6 +24,34 @@ def _fwd_model(x1, x2): return [float(np.sin(x1) + x2)] +def _build_sampler(): + return adaptiveEntropy( + exp_name="wrapper_test", + iters=1, + x_bounds=BOUNDS_2D, + likelihood_var=0.05, + mean_fxn=gpflow.mean_functions.Zero(), + kernel_fxn=AnisotropicSE(), + fwd_model=_fwd_model, + fwd_model_args=(), + ) + + +def _fitted_gp_model(): + rng = np.random.default_rng(0) + X = rng.uniform([0.0, 350.0], [1.0, 367.0], size=(8, 2)) + y = np.sin(X[:, 0:1]) + 0.01 * (X[:, 1:2] - 358.0) + model = gpflow.models.GPR(data=(X, y), kernel=AnisotropicSE()) + gpflow.set_trainable(model.likelihood.variance, False) + model.likelihood.variance.assign(0.05) + return model + + +def _small_trace(): + rng = np.random.default_rng(1) + return rng.uniform([0.5, 0.5, 0.5], [3.0, 3.0, 3.0], size=(20, 3)) + + def test_read_data_splits_columns_correctly(tmp_path): # read_data is the legacy disk-as-state convention (run_model's precondition) -- # not called by run(), but kept for scripts that still want to seed from a file. @@ -66,3 +100,103 @@ def test_bits_for_gaps_defaults_to_identity_transform_when_not_given(): bfg = BitsForGaps(black_box=_fwd_model, bounds=BOUNDS_2D, kernel=AnisotropicSE()) np.testing.assert_allclose(bfg.input_transform.forward([[0.3, 360.0]]), [[0.3, 360.0]]) assert bfg.output_transform.forward(5.0) == 5.0 + + +def test_sample_gp_posterior_mixture_forwards_seed_and_explicit_size(monkeypatch): + # mixture.sample_gp_posterior_mixture draws from TF's ambient, unseeded RNG + # (predict_f_samples -- see mixture.py's module docstring), and `size` selects + # WHICH trace rows are eligible, not the output shape (fixed at 100 draws) -- so + # neither is observable from the return value alone. What a caller actually + # depends on is that the wrapper forwards `self.seed` and the given `size` + # unchanged to the module-level function; verified directly here. + calls = [] + + def fake_sample(trace, GPmodel, XGP, seed, size, tf_seed=None): + calls.append({"seed": seed, "size": size}) + return np.zeros((size, len(XGP))) + + monkeypatch.setattr("bits_for_gaps.sampler.mixture.sample_gp_posterior_mixture", fake_sample) + + s = _build_sampler() + s.seed = 42 + GPmodel = _fitted_gp_model() + trace = _small_trace() + XGP = np.array([[0.3, 0.4], [0.6, 0.5]]) + + s.sample_gp_posterior_mixture(trace, GPmodel, XGP, size=5) + assert calls == [{"seed": 42, "size": 5}] + + +def test_sample_gp_posterior_mixture_default_size_uses_no_gaussians(monkeypatch): + calls = [] + + def fake_sample(trace, GPmodel, XGP, seed, size, tf_seed=None): + calls.append({"seed": seed, "size": size}) + return np.zeros((size, len(XGP))) + + monkeypatch.setattr("bits_for_gaps.sampler.mixture.sample_gp_posterior_mixture", fake_sample) + + s = _build_sampler() + s.seed = 7 + s.noGaussians = 11 + GPmodel = _fitted_gp_model() + trace = _small_trace() + XGP = np.array([[0.3, 0.4], [0.6, 0.5]]) + + s.sample_gp_posterior_mixture(trace, GPmodel, XGP) # size=None -> self.noGaussians + assert calls == [{"seed": 7, "size": 11}] + + +def test_entropy_objective_delegates_instance_config(): + s = _build_sampler() + s.seed = 42 + s.noGaussians = 5 + s.acquisitionObjective = "lower_bound" + GPmodel = _fitted_gp_model() + trace = _small_trace() + xStarGP = np.array([0.4, 358.0]) + + result = s.entropy_objective(xStarGP, trace, GPmodel) + expected = acquisition.entropy_objective( + xStarGP, trace, GPmodel, seed=42, no_gaussians=5, objective="lower_bound" + ) + assert result == pytest.approx(expected) + + +def test_run_model_reads_disk_design_and_runs(tmp_path): + # run_model is the deprecated zero-argument entry point: read_data(iters=1) then + # run(..., checkpoint_dir=self.path) -- neither step is exercised together by any + # other test (read_data alone is covered above; run() is always called directly + # elsewhere with an in-memory design). + s = _build_sampler() + s.path = str(tmp_path) + s.noSamples, s.noBurnIn, s.noChains = 50, 20, 2 + s.noGaussians, s.entropyMesh, s.noRestarts = 5, [3, 3], 2 + + X = np.array([[0.1, 355.0], [0.3, 357.0], [0.5, 359.0], [0.7, 361.0], [0.9, 363.0]]) + y = np.array([[float(np.sin(x1) + x2)] for x1, x2 in X]) + np.savetxt(tmp_path / "activity_data_1", np.column_stack([X, y])) + + history = s.run_model() + assert len(history) == 1 + assert history.last.XData.shape[0] == 6 # the 5 seed points + 1 newly selected + assert (tmp_path / "activity_data_2").exists() # checkpoint_dir=self.path, opt-in + + +def test_run_with_show_lml_results_prints_diagnostics(capsys): + # showLMLres=True additionally prints the LML fit's result + a gpflow parameter + # summary -- off by default (test_run_with_initial_lml_maximization in + # test_end_to_end.py exercises initalLML=True alone, which stays silent). + s = _build_sampler() + s.initalLML = True + s.showLMLres = True + s.noSamples, s.noBurnIn, s.noChains = 50, 20, 2 + s.noGaussians, s.entropyMesh, s.noRestarts = 5, [3, 3], 2 + + X = np.array([[0.1, 355.0], [0.3, 357.0], [0.5, 359.0], [0.7, 361.0], [0.9, 363.0]]) + y = np.array([float(np.sin(x1) + x2) for x1, x2 in X]) + + history = s.run(X, y) + assert history.last.lml_result is not None + out = capsys.readouterr().out + assert "std_dev" in out # gpflow.utilities.print_summary's parameter table From dfe08eaefb260a057410b575d6fbb153563bc2a0 Mon Sep 17 00:00:00 2001 From: Alex Dowling Date: Thu, 6 Aug 2026 21:40:59 -0400 Subject: [PATCH 2/2] CI coverage reporting via Codecov + local-coverage docs .github/workflows/ci.yml: the 3.12 matrix leg (only -- avoids four duplicate uploads) runs the suite with `--cov=bits_for_gaps --cov-report=xml --cov-report=term-missing` and uploads to Codecov (codecov/codecov-action@v5, no token needed for this public repo, fail_ci_if_error: false since the real coverage gate is pytest-cov's own fail_under, not the upload succeeding). Chose Codecov over the job-summary+artifact alternative: public-repo tokenless upload has zero setup burden for the maintainer, and a repo-topline badge is more discoverable than an artifact buried in a workflow run -- see the PR body for the full justification. .gitignore gains coverage.xml (the other coverage artifacts were already ignored). README.md: adds the Codecov badge alongside CI/PyPI/Docs, and a "Quick test" note on measuring coverage locally. docs/installation.md: a "Coverage" admonition with the same local command, what CI does, and where the floor is configured; also fixed a stale test count (204 -> 218). CHANGELOG.md: [Unreleased] gains an entry for the coverage work (config, floor, CI reporting, README badge, and a summary of the 18-gap triage from the previous commit). Verified: pytest -q (218 passed, 2 deselected), ruff clean, sphinx-build -W clean, lazy-import contract intact. `coverage.xml` confirmed produced by the exact command CI runs, then removed (gitignored, never committed). Co-Authored-By: Claude Sonnet 5 --- .github/workflows/ci.yml | 18 ++++++++++++++++++ .gitignore | 1 + CHANGELOG.md | 18 ++++++++++++++++++ README.md | 8 ++++++++ docs/installation.md | 18 ++++++++++++++++-- 5 files changed, 61 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 29d71b3..f7a03fc 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -7,6 +7,14 @@ # import anything Julia-touching at module import time, so this runs clean without the # [vle] extra. Julia-backed VLE + regression tests that need it stay a separate, gated # job for whenever CI grows a Julia-capable runner. +# +# Coverage is measured on the 3.12 matrix leg only (one upload, not four -- Codecov +# would otherwise report the same numbers repeatedly) and uploaded to Codecov, which +# works without a token for public repos (see README.md's badge). `pytest-cov`'s own +# `fail_under` (pyproject.toml's [tool.coverage.report]) is what actually gates +# coverage regressions -- Codecov's upload is reporting/visibility on top of that, not +# a second gate, so a transient upload failure (e.g. codecov.io being briefly +# unreachable) doesn't fail CI (`fail_ci_if_error: false`). name: CI on: @@ -46,4 +54,14 @@ jobs: python -m pip install --upgrade pip pip install -e ".[dev]" - name: Run the default test suite (unit + integration + regression, not vle) + if: matrix.python-version != '3.12' run: pytest -q + - name: Run the default test suite with coverage (3.12 only, for Codecov) + if: matrix.python-version == '3.12' + run: pytest -q --cov=bits_for_gaps --cov-report=xml --cov-report=term-missing + - name: Upload coverage to Codecov + if: matrix.python-version == '3.12' + uses: codecov/codecov-action@v5 + with: + files: ./coverage.xml + fail_ci_if_error: false diff --git a/.gitignore b/.gitignore index 56e2d08..a08161f 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,7 @@ dist/ htmlcov/ .coverage .coverage.* +coverage.xml # Local environments .venv/ diff --git a/CHANGELOG.md b/CHANGELOG.md index c346da3..79d259e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,24 @@ entry below summarizes the library relative to the paper's original research cod `numpy<2` in every release, and no NumPy 1.x publishes a Python 3.13 wheel -- an upstream constraint, not a limitation of this package's own pins. See `docs/installation.md`. +- Measured, reported, and floor-gated code coverage for `src/bits_for_gaps` (the + shipped package only -- `examples/`, `paper/`, and `tests/` are repo-only and + excluded from the denominator). `pyproject.toml`'s `[tool.coverage.run]`/ + `[tool.coverage.report]` configure branch coverage, honest exclusions + (`pragma: no cover`, `TYPE_CHECKING`, `NotImplementedError`, `__repr__`, the + `__main__` guard), and a `fail_under = 99` floor. `pytest -q` alone is unaffected + (fast, no coverage overhead); measure explicitly with `pytest --cov=bits_for_gaps + --cov-report=term-missing` (documented in `README.md`/`docs/installation.md`). CI + uploads coverage from the Python 3.12 matrix leg to Codecov (works without a token + for this public repo); see the new Codecov badge in `README.md`. Closed all 18 + statements missed on the pre-existing 97% baseline -- 15 with real tests (the PEP + 562 lazy-import path, a grid-trimming branch, `entropy.cholesky`'s correctness, two + unused-elsewhere `adaptiveEntropy` wrapper methods, the `showLMLres` diagnostic + branch, the deprecated `run_model` entry point, and a checkpoint-writing branch for + non-2-D runs) and 3 with a `# pragma: no cover` plus a documented reason (one + mathematically-unreachable guard, already noted in the test suite; one + `@tf.function`-wrapped line that genuinely executes but is invisible to + coverage.py's line tracer once AutoGraph compiles it into a TF graph). ## [0.1.2] - 2026-08-06 diff --git a/README.md b/README.md index ac20d78..8da4700 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ # BITS for GAPS [![CI](https://github.com/dowlinglab/bits_for_gaps/actions/workflows/ci.yml/badge.svg)](https://github.com/dowlinglab/bits_for_gaps/actions/workflows/ci.yml) +[![codecov](https://codecov.io/gh/dowlinglab/bits_for_gaps/branch/main/graph/badge.svg)](https://codecov.io/gh/dowlinglab/bits_for_gaps) [![PyPI](https://img.shields.io/pypi/v/bits_for_gaps.svg)](https://pypi.org/project/bits_for_gaps/) [![Docs](https://readthedocs.org/projects/bits-for-gaps/badge/?version=latest)](https://bits-for-gaps.readthedocs.io/en/latest/?badge=latest) @@ -70,6 +71,13 @@ docs/ Sphinx documentation (ReadTheDocs) pytest -q ``` +To measure coverage locally (scoped to `src/bits_for_gaps` -- `examples/`, `paper/`, +and `tests/` are repo-only and excluded from the denominator): + +```bash +pytest --cov=bits_for_gaps --cov-report=term-missing +``` + ## Provenance The research code behind the paper was originally developed in a private repository over the diff --git a/docs/installation.md b/docs/installation.md index 159bedb..ba5645c 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -52,13 +52,27 @@ pip install -e . ## Development install ```bash -pip install -e ".[dev]" # adds pytest, pytest-cov -pytest -q # 204 passed, 2 deselected +pip install -e ".[dev]" # adds pytest, pytest-cov, ruff +pytest -q # 218 passed, 2 deselected ``` The 2 deselected tests need Julia (`@pytest.mark.vle`) -- run them with `pytest -m vle` after installing the `[vle]` extra below. See {doc}`reproduce_paper`. +```{admonition} Coverage +:class: note +`pytest -q` alone never measures coverage -- it stays fast and its output stays +unchanged for a plain local run. Measure it explicitly, scoped to the shipped +package (`examples/`, `paper/`, and `tests/` are repo-only and excluded): + + pytest --cov=bits_for_gaps --cov-report=term-missing + +CI measures coverage this way on one Python version (3.12) per run and uploads it to +[Codecov](https://codecov.io/gh/dowlinglab/bits_for_gaps) (works without a token for +this public repo); `pyproject.toml`'s `[tool.coverage.report]` sets a `fail_under` +floor so coverage can't silently regress. +``` + ## `examples/` and `paper/` are repo-only ```{important}