Pin test GRCh38 resolution to Ensembl 81 (10.11.2)

CHANGELOG.md

@@ -1,5 +1,12 @@
 # Change Log
 
+## [v10.11.2](https://github.com/openvax/varcode/tree/v10.11.2) (2026-10-01)
+
+- Pin bare GRCh38 references during pytest to the existing Ensembl 81 fixture
+  (#493). Remove the installed-release scan that could download missing data
+  while probing `EnsemblRelease.db`. Test reference selection leaves the cache
+  untouched and preserves explicit release choices; library behavior is unchanged.
+
 ## [v10.11.1](https://github.com/openvax/varcode/tree/v10.11.1) (2026-09-30)
 
 - Compose mixed inherited/somatic cis groups in the experimental transcript

CONTRIBUTING.md

@@ -40,9 +40,10 @@ pyensembl install --release 95 --species human --custom-mirror https://github.co
 pyensembl install --release 95 --species mouse --custom-mirror https://github.com/openvax/ensembl-data/releases/download/GRCm38.95/
 ```
 
-The test reference resolver currently probes uninstalled releases in a way
-that can trigger downloads; see [#493](https://github.com/openvax/varcode/issues/493).
-The bundled Osteosarc snapshot checks themselves open their data offline.
+During pytest, bare GRCh38 references use Ensembl 81, matching the shared
+fixtures. Reference selection does not scan or populate the user cache;
+install the data above before running annotation tests. Explicit release
+choices are preserved. The bundled Osteosarc snapshot checks open their data offline.
 See [tests/README.md](tests/README.md) for fixture provenance and regeneration.
 
 ## Make and verify a change

docs/development_audit.md

@@ -39,7 +39,7 @@ domain-specific reviews and independent scientific oracles.
 | High | VCF export derives header sample order from a set but writes row values in dictionary order. A two-row reproduction assigns tumor GT/DP to normal and emits `44:0/1` under `GT:DP`. Differing sample sets also discard sample data; headers and allele-indexed metadata lack a complete preservation contract. | Filed [#502](https://github.com/openvax/varcode/issues/502); resolved in 10.1.3 with explicit [export guarantees and limits](csv.md#vcf-export). |
 | High | The phase cap returns an all-cis placeholder that the consumer classifies as a precise effect. The existing CFTR pair yields cis/trans alternatives at cap 8 but only `p.S159T` at cap 1, with no molecular evidence resolving phase. Four unphased germline alleles exceed the default cap. | Filed [#503](https://github.com/openvax/varcode/issues/503); documented in [germline annotation](germline.md). |
 | High | `deploy.sh` lacks the branch/clean-tree guards, version handling, tagging, and pushing described in AGENTS.md. It can upload from the wrong checkout if invoked without independent checks. | Existing [#414](https://github.com/openvax/varcode/issues/414); release instructions now describe the actual script and required manual checks. |
-| Medium | The test cache probe accesses `EnsemblRelease.db`, which can download missing reference data while supposedly checking local installation. | Existing [#493](https://github.com/openvax/varcode/issues/493); contributor instructions distinguish reference provisioning from offline snapshot tests. |
+| Medium | The test cache probe accesses `EnsemblRelease.db`, which can download missing reference data while supposedly checking local installation. | Resolved [#493](https://github.com/openvax/varcode/issues/493) in 10.11.2 by pinning bare GRCh38 test references to the existing Ensembl 81 fixture and removing the cache probe. |
 | Medium | Some tests cannot detect failures, including the fast-path self-comparison and protocol/attribute surveys. VCF round-trip checks omitted ALT and complete record counts. | Existing [#479](https://github.com/openvax/varcode/issues/479); this change strengthens the VCF checks. The broader test review remains open. |
 | Medium | Reference mismatch fields contain transcript-oriented bases but the message labels them as genome bases; minus-strand errors can be read backwards. | Existing [#434](https://github.com/openvax/varcode/issues/434); troubleshooting now explains the current orientation. |
 | Medium | Historical changelog coverage is incomplete. | Existing [#411](https://github.com/openvax/varcode/issues/411); no invented reconstruction of missing release history. |
@@ -83,8 +83,8 @@ files or rewrite all experimental-annotator documentation.
 ## Follow-up sequence
 
 Sample identity/FORMAT export integrity (#502) was fixed in 10.1.3.
-Phase-cap uncertainty (#503) is the next scientific correctness priority. Then make release guards executable (#414) and make reference
-cache checks truly local (#493). Broader uncertainty composition (#421/#423),
+Phase-cap uncertainty (#503) is the next scientific correctness priority. Then make release guards executable (#414).
+Test reference selection no longer probes the cache (#493, fixed in 10.11.2). Broader uncertainty composition (#421/#423),
 shared RNA fixture adoption (#464), and test-oracle cleanup (#479) remain
 foundational work; they should be reviewed as their own changes.
 

tests/README.md

@@ -9,9 +9,11 @@ requirement. RNA fixture harmonization is tracked in
 [#464](https://github.com/openvax/varcode/issues/464).
 
 See [CONTRIBUTING.md](../CONTRIBUTING.md) for environment and reference setup.
-The reference-cache probe has a separate download issue
-([#493](https://github.com/openvax/varcode/issues/493)); the offline guarantees
-below apply to the bundled Osteosarc snapshot checks.
+During pytest, bare GRCh38 references (including aliases) use the suite's
+Ensembl 81 fixture. Selection does not download data, probe newer releases,
+or modify the reference cache. Install the documented datasets before
+annotation tests; explicit release choices remain available. The offline
+guarantees below apply to the bundled Osteosarc snapshot checks.
 
 ## Osteosarc test variants
 

tests/conftest.py

@@ -24,65 +24,27 @@
 expected to satisfy tests for unsupported inputs (for example SVs).
 """
 
-import os
-
 import pytest
 
 import varcode
 import varcode.reference as _reference
-from pyensembl import EnsemblRelease as _EnsemblRelease
 from pyensembl import cached_release as _cached_release
 
 
-# --- Cap bare-"GRCh38" resolution at the newest INSTALLED release ------------
-#
-# A bare reference name like "GRCh38" (e.g. ``Variant("7", ..., "GRCh38")`` or
-# ``from_csv(genome="GRCh38")``) resolves, via pyensembl, to the latest release
-# pyensembl KNOWS about -- currently 115 -- which is not necessarily one that's
-# been downloaded. CI installs GRCh38 only up to release 95 (the
-# openvax/ensembl-data mirror tops out there), so those tests would try to
-# auto-fetch release 115 from Ensembl's FTP at *runtime*. That's slow and
-# flaky, and is the source of intermittent, whole-job
-# "GTF database needs to be created, run: pyensembl install --release 115"
-# failures scattered across the suite whenever the FTP is unreachable.
-#
-# During the test session -- and only during the test session; the library is
-# untouched -- transparently cap GRCh38 string resolution at the newest release
-# whose GTF index actually exists on disk. Effect annotations are stable across
-# recent GRCh38 releases (verified: the full suite passes identically under 95
-# and 115), so this removes the network dependency without changing any
-# expected value. GRCh38 begins at Ensembl release 76, so the walk floors there
-# and never crosses into a GRCh37 release.
-_GRCH38_FIRST_RELEASE = 76
-
-
-def _release_index_exists(release_number):
-    """True if the sqlite GTF index for a human release is already built."""
-    try:
-        return os.path.exists(_EnsemblRelease(release_number).db.local_db_path)
-    except Exception:
-        return False
-
-
+# Match the suite's shared GRCh38 fixtures in tests/data.py. Choosing a test
+# reference must not inspect .db (which can download missing files) or depend
+# on unrelated installations in the user's cache (#493). CI and CONTRIBUTING
+# explicitly install release 81; tests needing another release select it.
 _original_reference_resolver = _reference.get_genome_for_ensembl_reference_name
 
 
-def _resolve_grch38_to_installed_release(reference_name):
-    genome = _original_reference_resolver(reference_name)
-    if getattr(genome, "reference_name", None) != "GRCh38":
-        return genome
-    resolved_release = getattr(genome, "release", 0)
-    if _release_index_exists(resolved_release):
-        return genome
-    for candidate in range(resolved_release - 1, _GRCH38_FIRST_RELEASE - 1, -1):
-        if _release_index_exists(candidate):
-            return _cached_release(candidate)
-    # Nothing installed: leave the original genome so pyensembl's usual
-    # "please install release N" error still surfaces with a clear message.
-    return genome
+def _resolve_test_reference(reference_name):
+    if reference_name == "GRCh38":
+        return _cached_release(81)
+    return _original_reference_resolver(reference_name)
 
 
-_reference.get_genome_for_ensembl_reference_name = _resolve_grch38_to_installed_release
+_reference.get_genome_for_ensembl_reference_name = _resolve_test_reference
 
 
 def pytest_addoption(parser):

tests/test_reference_fixture.py

@@ -0,0 +1,68 @@
+"""Reference selection must not provision data or depend on the user cache."""
+
+import os
+from pathlib import Path
+import subprocess
+import sys
+
+import pytest
+
+
+@pytest.mark.parametrize("cache_state", ["empty", "partial", "stale_index"])
+def test_grch38_fixture_resolution_never_downloads_or_changes_cache(tmp_path, cache_state):
+    # A fresh process avoids PyEnsembl/Varcode memoization and reads the cache
+    # environment before either library is imported. The only intercepted
+    # operation is the source-acquisition hook implicated in #493.
+    script = r'''
+import os
+from pathlib import Path
+from unittest.mock import patch
+
+from pyensembl import EnsemblRelease, Genome, cached_release
+
+cache = Path(os.environ["PYENSEMBL_CACHE_DIR"])
+state = os.environ["VARCODE_TEST_CACHE_STATE"]
+if state != "empty":
+    newer = EnsemblRelease(95)
+    gtf = Path(newer.download_cache.cached_path(newer.gtf_url))
+    gtf.parent.mkdir(parents=True, exist_ok=True)
+    gtf.write_bytes(b"incomplete GTF download")
+    if state == "stale_index":
+        # A database path alone does not establish a usable installation.
+        gtf.with_suffix(".db").write_bytes(b"incomplete database")
+
+def snapshot():
+    return {str(p.relative_to(cache)): p.read_bytes() if p.is_file() else None
+            for p in cache.rglob("*")}
+
+before = snapshot()
+attempts = []
+
+def forbid_source_acquisition(*args, **kwargs):
+    attempts.append((args, kwargs))
+    raise AssertionError("reference resolution attempted source acquisition")
+
+with patch.object(Genome, "_set_local_paths", forbid_source_acquisition):
+    import tests.conftest
+    from varcode.reference import infer_genome
+
+    references = ["GRCh38", "hg38", "B38", "GRCh38.p13",
+                  "/references/GRCh38.d1.vd1.fa"]
+    selected = [infer_genome(name)[0] for name in references]
+    assert not attempts, "reference selection tried to acquire missing sources"
+    assert all(genome.release == 81 for genome in selected)
+    # Explicit choices bypass the test default, including object identity.
+    assert infer_genome("GRCh38:95")[0].release == 95
+    assert infer_genome(95)[0].release == 95
+    supplied = cached_release(93)
+    assert infer_genome(supplied)[0] is supplied
+
+assert snapshot() == before
+'''
+    result = subprocess.run(
+        [sys.executable, "-c", script],
+        cwd=Path(__file__).resolve().parents[1],
+        env=dict(os.environ, PYENSEMBL_CACHE_DIR=str(tmp_path),
+                 VARCODE_TEST_CACHE_STATE=cache_state),
+        capture_output=True, text=True, timeout=30)
+    assert result.returncode == 0, result.stdout + result.stderr

varcode/version.py

@@ -1 +1 @@
-__version__ = "10.11.1"
+__version__ = "10.11.2"