Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/api/qc.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,17 +138,17 @@ return similarity >= qc.SIMILARITY_THRESHOLD # 0.5

### SapBERT Model

**Model:** `cambridgeltl/SapBERT-from-PubMedBERT-fulltext`
**Model:** `RenaissanceComputing/SapBERT-PubMedBERT-BabelSynonyms-v2025-01-23`

**Backend:** [sentence-transformers](https://www.sbert.net/) (PyTorch). Embeddings are compared with scikit-learn's `cosine_similarity`. SapBERT's self-alignment pretraining pulls UMLS synonym pairs together in embedding space, which fits this stage's task, deciding whether two names denote the same entity, better than the NLI/STS-trained BioBERT it replaced. The 0.5 threshold was carried over from that BioBERT gate and has not been re-tuned for SapBERT's score distribution.
**Backend:** [sentence-transformers](https://www.sbert.net/) (PyTorch). Embeddings are compared with scikit-learn's `cosine_similarity`. SapBERT's self-alignment pretraining pulls synonym pairs together in embedding space; this checkpoint is RENCI's fine-tune of `cambridgeltl/SapBERT-from-PubMedBERT-fulltext` on synonym pairs sampled from NCATS Translator Babel identifier cliques, so its training signal matches this stage's task, deciding whether two names denote the same entity. The 0.5 threshold was carried over from the earlier BioBERT gate and has not been re-tuned for SapBERT's score distribution.

**Lazy-loaded** on the first `fullmap_audit()` call that reaches the embedding stage via `get_sapbert()`, then cached globally for the lifetime of the process.

### Model Caching

`get_sapbert()` loads the model from the local cache when present; otherwise it downloads `cambridgeltl/SapBERT-from-PubMedBERT-fulltext` and saves it for future runs.
`get_sapbert()` loads the model from the local cache when present; otherwise it downloads `RenaissanceComputing/SapBERT-PubMedBERT-BabelSynonyms-v2025-01-23` and saves it for future runs.

**Cache location:** `.tablassert/sapbert/` on disk (`qc.MODEL`); the loaded model object is also cached in memory for the lifetime of the process.
**Cache location:** `.tablassert/sapbert-babel/` on disk (`qc.MODEL`); the loaded model object is also cached in memory for the lifetime of the process. Existing `.tablassert/sapbert/` directories hold the previous base checkpoint and are stale; they can be deleted.

### Example Usage

Expand Down
14 changes: 8 additions & 6 deletions src/tablassert/qc.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,13 @@

logger = cat("QC")

MODEL: Path = BASE / "sapbert"
# Cache dir is keyed to the model: swapping HF repos must move the dir, or
# existing installs would silently keep serving the previous checkpoint.
MODEL: Path = BASE / "sapbert-babel"

# Pass threshold for the SapBERT cosine-similarity stage. Carried over from the
# BioBERT gate; SapBERT's UMLS synonym-alignment training shifts true synonym
# pairs upward, so a tuning pass against real reject logs is the natural follow-up.
# BioBERT gate; synonym-alignment fine-tuning shifts true synonym pairs upward,
# so a tuning pass against real reject logs is the natural follow-up.
SIMILARITY_THRESHOLD: float = 0.5
FUZZ_RATIO_THRESHOLD: float = 70.0
FUZZ_PARTIAL_THRESHOLD: float = 80.0
Expand Down Expand Up @@ -103,8 +105,8 @@ def get_sapbert() -> object:
"""Lazy-load and memoize the SapBERT sentence-transformer (``functools.cache``).

Loads from the local cache at ``MODEL`` when present; otherwise downloads
``cambridgeltl/SapBERT-from-PubMedBERT-fulltext`` and saves it for future
runs.
``RenaissanceComputing/SapBERT-PubMedBERT-BabelSynonyms-v2025-01-23`` and
saves it for future runs.

Returns:
The memoized ``SentenceTransformer`` instance.
Expand All @@ -117,7 +119,7 @@ def get_sapbert() -> object:
model: object = sentence_transformers.SentenceTransformer(str(MODEL)) # pyright: ignore
else:
model = sentence_transformers.SentenceTransformer( # pyright: ignore
"cambridgeltl/SapBERT-from-PubMedBERT-fulltext"
"RenaissanceComputing/SapBERT-PubMedBERT-BabelSynonyms-v2025-01-23"
)
MODEL.mkdir(parents=True, exist_ok=True)
model.save(MODEL) # pyright: ignore
Expand Down
2 changes: 1 addition & 1 deletion tests/test_cover_qc.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
import tablassert.qc as qc
from tablassert.errors import QcRuntimeMissingError

HF_REPO: str = "cambridgeltl/SapBERT-from-PubMedBERT-fulltext"
HF_REPO: str = "RenaissanceComputing/SapBERT-PubMedBERT-BabelSynonyms-v2025-01-23"


class FakeModel:
Expand Down
Loading