diff --git a/docs/configuration/table.md b/docs/configuration/table.md index 3dfcec8..2ffd621 100644 --- a/docs/configuration/table.md +++ b/docs/configuration/table.md @@ -187,7 +187,7 @@ Defines subject-predicate-object relationships. | `predicate` | String | No | Biolink predicate. Defaults to `"related_to"`. Three DAKP prevention predicates are deliberate local extensions ahead of Biolink support (see `biolink.PREDICATE_OVERRIDES`): `prevents`, `applied_to_prevent`, `contraindicated_in_the_prevention_of`. | | `object` | NodeEncoding | Yes | Object entity configuration | | `qualifiers` | List[Qualifier] | No | Edge qualifiers (context) | -| `category_override` | Map[Categories, EdgeCategories] | No | Pin the association class per resolved object category (bare names, no `biolink:` prefix), replacing the derived (subject, object) pair lookup for those rows. Rows whose object category is absent from the map derive as before. Pinned classes are still reconciled against the section predicate (a class whose `predicate` slot rejects it is walked up the association hierarchy, with a warning at config time). Use it when one section mixes object categories the pair lookup merges: e.g. `Disease` and `PhenotypicFeature` both derive to `ChemicalEntityToDiseaseOrPhenotypicFeatureAssociation`, but only `EntityToDiseaseAssociation` / `EntityToPhenotypicFeatureAssociation` receive the class-scoped `regulatory_approvals` grant and declare slots like `number_of_cases`. Rows pinned to either of those classes may additionally carry `disease_context_qualifier` plus DAKP's sparse qualifier stack (`anatomical_context_qualifier`, `sex_qualifier`, `population_context_qualifier`, `frequency_qualifier`, `temporal_context_qualifier`) under deliberate Tablassert policy grants. Deliberately ungranted: `species_context_qualifier` (disabled -- never emittable) plus `temporal_interval_qualifier` and `severity_qualifier` (unsatisfiable -- attached to no class, so a grant could never validate). | +| `category_override` | Map[Categories, EdgeCategories] | No | Pin the association class per resolved object category (bare names, no `biolink:` prefix), replacing the derived (subject, object) pair lookup for those rows. Rows whose object category is absent from the map derive as before. Pinned classes are still reconciled against the section predicate (a class whose `predicate` slot rejects it is walked up the association hierarchy, with a warning at config time). Use it when one section mixes object categories the pair lookup merges: e.g. `Disease` and `PhenotypicFeature` both derive to `ChemicalEntityToDiseaseOrPhenotypicFeatureAssociation`, but only `EntityToDiseaseAssociation` / `EntityToPhenotypicFeatureAssociation` declare slots like `regulatory_approvals` and `number_of_cases`. Rows pinned to either of those classes may additionally carry DAKP's sparse qualifier stack ahead of the model under deliberate Tablassert policy grants: `population_context_qualifier` and `temporal_context_qualifier` on both classes, plus `sex_qualifier` on the disease side. Deliberately ungranted: `species_context_qualifier` (disabled -- never emittable) plus `temporal_interval_qualifier` and `severity_qualifier` (unsatisfiable -- attached to no class, so a grant could never validate). | **Example:** ```yaml @@ -274,7 +274,7 @@ annotations: The separator is a property of the data, not of the slot. Inspect the table's cells and set `split_by` to the separator the cells actually use: `","` for comma-joined ids like `"EFO:0001,EFO:0002"` above, `";"` for `"EFO:0001;EFO:0002"`, `"|"` only if the cells happen to be pipe-joined. -For the canonical `regulatory_approvals` annotation, a pipe-delimited source column uses `split_by: "|"` and emits a real per-row array such as `["011111", "022222"]`; combine it with a class-scoped `category_override` when only the intended association classes receive that grant. +For the canonical `regulatory_approvals` annotation, a pipe-delimited source column uses `split_by: "|"` and emits a real per-row array such as `["011111", "022222"]`; combine it with a class-scoped `category_override` when only the intended association classes declare it (unpinned rows on classes without the slot are pruned). `split_by` is the one multivalued encoding: every row's cell becomes its own JSON array, so an array that differs per row, the shape a literal can never express, is declared directly. Values are trimmed and blanks dropped; a null cell stays null. @@ -590,7 +590,7 @@ annotations: Annotation names fall into three groups at build time: -- **Allowed edge fields:** names on the edge allow-list: [Biolink Association](https://biolink.github.io/biolink-model/) slots, qualifier slots, and curated KGX/Tablassert edge fields (e.g. `p_value`, `adjusted_p_value`, `knowledge_level`, `supporting_text`, `publications`, `effect_size`, `effect_type`, `statistical_significance_qualifier`, qualifier slots like `severity_qualifier` / `disease_context_qualifier`) are written to edges verbatim. `effect_size` and `effect_type` became real Association slots in biolink-model 4.4.4 (PR #1774) and emit as real JSON numbers and enum tokens. Annotation names are matched case-insensitively against the allow-list and emitted under the canonical slot spelling. The class-scoped `regulatory_approvals` grant is available on `EntityToDiseaseAssociation` and `EntityToPhenotypicFeatureAssociation`; it is a multivalued slot, so declare [`split_by`](#split_by) for pipe-joined cells like `011111|022222`. The same two classes also carry class-scoped grants for DAKP's sparse qualifier stack (`anatomical_context_qualifier`, `sex_qualifier`, `population_context_qualifier`, `frequency_qualifier`, `temporal_context_qualifier`) plus `disease_context_qualifier`; without a grant `prune_to_class` nulls those qualifiers off the pinned rows. `species_context_qualifier` stays disabled, and `temporal_interval_qualifier` / `severity_qualifier` stay unsatisfiable, so none of the three is granted. +- **Allowed edge fields:** names on the edge allow-list: [Biolink Association](https://biolink.github.io/biolink-model/) slots, qualifier slots, and curated KGX/Tablassert edge fields (e.g. `p_value`, `adjusted_p_value`, `knowledge_level`, `supporting_text`, `publications`, `effect_size`, `effect_type`, `statistical_significance_qualifier`, qualifier slots like `severity_qualifier` / `disease_context_qualifier`) are written to edges verbatim. `effect_size` and `effect_type` became real Association slots in biolink-model 4.4.4 (PR #1774) and emit as real JSON numbers and enum tokens. Annotation names are matched case-insensitively against the allow-list and emitted under the canonical slot spelling. `regulatory_approvals` (multivalued, so declare [`split_by`](#split_by) for pipe-joined cells like `011111|022222`), `disease_context_qualifier`, `anatomical_context_qualifier`, and `frequency_qualifier` are declared natively on `EntityToDiseaseAssociation` / `EntityToPhenotypicFeatureAssociation` by biolink-model 4.4.5, and `sex_qualifier` on the phenotypic-feature class, so those rows keep them without any override. The remaining class-scoped grants cover DAKP's sparse qualifier stack ahead of the model: `population_context_qualifier` and `temporal_context_qualifier` on both classes, plus `sex_qualifier` on `EntityToDiseaseAssociation`; without a grant `prune_to_class` nulls those qualifiers off the pinned rows. `species_context_qualifier` stays disabled, and `temporal_interval_qualifier` / `severity_qualifier` stay unsatisfiable, so none of the three is granted. - **Study metadata:** `study_size`, `study_cohort`, `study_context`, `study_date_range`, `study_method_description`, and `study_method_types` are current Biolink `Study` node properties ([biolink-model PR #1770](https://github.com/biolink/biolink-model/pull/1770)). They describe the study itself, not an association, so their values are carried on the edge's **inlined supporting Study** (`has_supporting_studies` to `Study`, the COHD/ICEES pattern) rather than emitted as edge fields. Deprecated `supporting_study_*` spellings and `sample_size` are accepted as aliases and renamed onto canonical `study_*` names by coercion. They never appear in final JSON. Declaring any of them is legal and emits a `BiolinkRelocationWarning` naming where the value went. - **Unsatisfiable slots:** names the Biolink LinkML schema declares but attaches to **no** Pydantic class, derived from the installed `biolink-model`. A record carrying one could never validate, so its value is preserved in the inlined supporting study's `StudyResult.description`. A slot leaves the set automatically once a release attaches it. diff --git a/pyproject.toml b/pyproject.toml index a676fed..efcb617 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -47,7 +47,7 @@ classifiers = [ ] requires-python = ">=3.11" dependencies = [ - "biolink-model>=4.4.4", + "biolink-model>=4.4.5", "polars>=2.0.0rc2", "pyarrow>=21.0.0", "rapidfuzz>=3.14.3", diff --git a/rust/src/fullmap.rs b/rust/src/fullmap.rs index 8c4ecfc..9ba5710 100644 --- a/rust/src/fullmap.rs +++ b/rust/src/fullmap.rs @@ -2950,11 +2950,11 @@ fn extract_validate_rename( archive.display() )) }; - let database = open_read_only(&primary).map_err(&validation_ctx)?; - validate_schema(&database).map_err(&validation_ctx)?; - let build_id = read_build_id(&database).map_err(&validation_ctx)?; - let shard_count = shard_count_of(&database).map_err(&validation_ctx)?; - let recorded_allowlist = read_taxon_allowlist_identity(&database).map_err(&validation_ctx)?; + let database = open_read_only(&primary).map_err(validation_ctx)?; + validate_schema(&database).map_err(validation_ctx)?; + let build_id = read_build_id(&database).map_err(validation_ctx)?; + let shard_count = shard_count_of(&database).map_err(validation_ctx)?; + let recorded_allowlist = read_taxon_allowlist_identity(&database).map_err(validation_ctx)?; drop(database); // A published archive built WITHOUT the required filter is not the database diff --git a/src/tablassert/biolink.py b/src/tablassert/biolink.py index a050b92..c1fff6f 100644 --- a/src/tablassert/biolink.py +++ b/src/tablassert/biolink.py @@ -591,28 +591,8 @@ class EffectTypes(str, Enum): CLASS_FIELD_OVERRIDES: dict[str, frozenset[str]] = { - "EntityToDiseaseAssociation": frozenset( - { - "anatomical_context_qualifier", - "disease_context_qualifier", - "frequency_qualifier", - "population_context_qualifier", - "regulatory_approvals", - "sex_qualifier", - "temporal_context_qualifier", - } - ), - "EntityToPhenotypicFeatureAssociation": frozenset( - { - "anatomical_context_qualifier", - "disease_context_qualifier", - "frequency_qualifier", - "population_context_qualifier", - "regulatory_approvals", - "sex_qualifier", - "temporal_context_qualifier", - } - ), + "EntityToDiseaseAssociation": frozenset({"population_context_qualifier", "sex_qualifier", "temporal_context_qualifier"}), + "EntityToPhenotypicFeatureAssociation": frozenset({"population_context_qualifier", "temporal_context_qualifier"}), } """Per-class grants of edge fields the resolved association class does not declare. @@ -620,22 +600,21 @@ class EffectTypes(str, Enum): the slots ``lib.prune_to_class`` keeps on rows resolved to that class even though the installed model attaches them elsewhere. -The motivating case is a DAKP contraindication edge: ``regulatory_approvals`` is a -canonical slot not yet attached by the installed model, while -``disease_context_qualifier`` is declared only on the -``ChemicalEntityToDiseaseOrPhenotypicFeatureAssociation`` lineage -- so one edge can -natively carry both only through these explicit class-scoped grants. The same gap -covers DAKP's sparse qualifier stack (``anatomical_context_qualifier``, -``sex_qualifier``, ``population_context_qualifier``, ``frequency_qualifier``, -``temporal_context_qualifier``): each is a satisfiable Biolink qualifier slot that no -pinned class declares, so without a grant ``prune_to_class`` would null it off the -edge. Deliberately excluded: ``species_context_qualifier`` (in -``DISABLED_EDGE_FIELDS`` -- never emittable), ``temporal_interval_qualifier`` and -``severity_qualifier`` (in ``UNSATISFIABLE_EDGE_FIELDS`` -- attached to no class, so -a grant could never validate). Tablassert -deliberately emits the granted fields on the pinned classes ahead of the pinned model +The motivating case is a DAKP contraindication edge carrying the sparse qualifier +stack ahead of the pinned model: ``population_context_qualifier`` and +``temporal_context_qualifier`` are satisfiable qualifier slots the pinned classes do +not declare, and ``sex_qualifier`` is declared only on the phenotypic-feature side of +the family -- so without a grant ``prune_to_class`` would null them off the edge. +Upstream Biolink has absorbed most of the original stack (the biolink-model 4.4.5 +mixin consolidation attached ``disease_context_qualifier``, +``anatomical_context_qualifier``, and ``frequency_qualifier`` to the whole +disease/phenotype family, renamed ``FDA_regulatory_approvals`` to the canonical +``regulatory_approvals`` on the pinned classes, and added ``sex_qualifier`` to +``EntityToPhenotypicFeatureAssociation``); the tripwire removed those grants the day +the release landed, and the remaining ones stay deliberately ahead of the model (pending an upstream Biolink widening). ``_validation_record`` strips granted fields -before record validation so the deliberate gap is not reported as ``extra_forbidden``. +before record validation so the deliberate gap is not reported as +``extra_forbidden``. A tripwire test asserts every granted field is still absent from its class: the moment a biolink-model release attaches the slot, the suite fails and the stale grant is diff --git a/tests/test_biolink.py b/tests/test_biolink.py index 4aa74eb..9057f3d 100644 --- a/tests/test_biolink.py +++ b/tests/test_biolink.py @@ -412,39 +412,20 @@ def test_class_field_overrides_track_the_installed_model() -> None: Tripwire: the moment a biolink-model release attaches a granted slot to the class, this fails and the stale grant is removed from ``CLASS_FIELD_OVERRIDES`` (same philosophy as the ``UNSATISFIABLE_EDGE_FIELDS`` derivation guard). A field the - family allow-list would strip anyway must never be granted. The canonical - ``regulatory_approvals`` grant is intentionally present on exactly the two - association classes that need it while the installed model catches up, alongside - DAKP's sparse qualifier stack (``anatomical_context_qualifier``, - ``sex_qualifier``, ``population_context_qualifier``, ``frequency_qualifier``, - ``temporal_context_qualifier`` -- each satisfiable, none declared by the pinned - classes). Deliberately excluded from the grants: ``species_context_qualifier`` - (disabled) plus ``temporal_interval_qualifier`` and ``severity_qualifier`` - (unsatisfiable -- a grant could never validate). + family allow-list would strip anyway must never be granted. The biolink-model + 4.4.5 mixin consolidation (plus the ``regulatory_approvals`` rename and the + phenotypic-feature ``sex_qualifier`` attachment) fired exactly this tripwire for + the original seven-field grant stack; the remaining grants are DAKP's sparse + qualifier slots still absent from the pinned classes + (``population_context_qualifier`` / ``temporal_context_qualifier`` on both, and + ``sex_qualifier`` on the disease side only). Deliberately excluded from the + grants: ``species_context_qualifier`` (disabled) plus + ``temporal_interval_qualifier`` and ``severity_qualifier`` (unsatisfiable -- a + grant could never validate). """ assert { - "EntityToDiseaseAssociation": frozenset( - { - "anatomical_context_qualifier", - "disease_context_qualifier", - "frequency_qualifier", - "population_context_qualifier", - "regulatory_approvals", - "sex_qualifier", - "temporal_context_qualifier", - } - ), - "EntityToPhenotypicFeatureAssociation": frozenset( - { - "anatomical_context_qualifier", - "disease_context_qualifier", - "frequency_qualifier", - "population_context_qualifier", - "regulatory_approvals", - "sex_qualifier", - "temporal_context_qualifier", - } - ), + "EntityToDiseaseAssociation": frozenset({"population_context_qualifier", "sex_qualifier", "temporal_context_qualifier"}), + "EntityToPhenotypicFeatureAssociation": frozenset({"population_context_qualifier", "temporal_context_qualifier"}), } == CLASS_FIELD_OVERRIDES for class_name, fields in CLASS_FIELD_OVERRIDES.items(): cls: type[Any] = association_class(f"biolink:{class_name}") @@ -460,6 +441,9 @@ def test_validate_record_tolerates_class_field_override_grants() -> None: The grant is a deliberate, class-scoped step ahead of the pinned model, so its ``extra_forbidden`` must not surface on either intended target -- while the same field on an ungranted class stays a real defect through the installed-model boundary. + ``population_context_qualifier`` is the probe because it is still granted on both + targets: ``regulatory_approvals`` was absorbed by the model itself in 4.4.5, so a + probe on it would pass natively and exercise nothing. """ base: dict[str, Any] = { "id": "e1", @@ -468,13 +452,13 @@ def test_validate_record_tolerates_class_field_override_grants() -> None: "object": "MONDO:0005148", "knowledge_level": "statistical_association", "agent_type": "data_analysis_pipeline", - "regulatory_approvals": ["FDA:1"], + "population_context_qualifier": "NCIT:C25667", } for category in ("EntityToDiseaseAssociation", "EntityToPhenotypicFeatureAssociation"): record: dict[str, Any] = {**base, "category": [f"biolink:{category}"]} assert validate_record(record, edge=True) == [] control: dict[str, Any] = {**base, "category": ["biolink:GeneToDiseaseAssociation"]} - assert "regulatory_approvals: extra_forbidden" in validate_record(control, edge=True) + assert "population_context_qualifier: extra_forbidden" in validate_record(control, edge=True) unknown: dict[str, Any] = {**base, "category": ["biolink:EntityToDiseaseAssociation"], "not_a_biolink_field": "x"} assert "not_a_biolink_field: extra_forbidden" in validate_record(unknown, edge=True) diff --git a/tests/test_lib.py b/tests/test_lib.py index cd8f3c2..f59b5d6 100644 --- a/tests/test_lib.py +++ b/tests/test_lib.py @@ -1993,15 +1993,17 @@ def test_prune_to_class_keeps_override_only_slots() -> None: def test_prune_to_class_keeps_class_field_override_grants() -> None: """Slots granted to a class by CLASS_FIELD_OVERRIDES survive prune_to_class. - ``disease_context_qualifier`` is declared only on the - ``ChemicalEntityToDiseaseOrPhenotypicFeatureAssociation`` lineage, but the policy - grant keeps it on ``EntityToDiseaseAssociation`` / - ``EntityToPhenotypicFeatureAssociation`` rows so a pinned edge can carry it - alongside ``regulatory_approvals`` -- and the same holds for DAKP's sparse - qualifier stack (``anatomical_context_qualifier``, ``sex_qualifier``, - ``population_context_qualifier``, ``frequency_qualifier``, - ``temporal_context_qualifier``), none of which the pinned classes declare. - Classes without the grant still prune them. + DAKP's sparse qualifier stack rides the pinned ``EntityToDiseaseAssociation`` / + ``EntityToPhenotypicFeatureAssociation`` rows through ``prune_to_class`` via the + grants: ``population_context_qualifier`` / ``temporal_context_qualifier`` (both + classes) and ``sex_qualifier`` (disease side only) are kept even though the + installed model does not declare them there. ``regulatory_approvals`` needs no + grant anymore -- biolink-model 4.4.5 attached it natively to the two pinned + classes -- but the ungranted control class still prunes it, alongside the + still-granted qualifiers. The qualifiers the 4.4.5 mixin consolidation attached + to the whole disease/phenotype family (``disease_context_qualifier``, + ``anatomical_context_qualifier``, ``frequency_qualifier``) survive on every row + including the control. """ from tablassert.lib import PRUNED_COLUMN, prune_to_class @@ -2022,20 +2024,15 @@ def test_prune_to_class_keeps_class_field_override_grants() -> None: } ) out: pl.DataFrame = prune_to_class(lf).collect() - assert out["disease_context_qualifier"].to_list() == ["MONDO:0005148", None, "MONDO:0005015"] - assert out["regulatory_approvals"].to_list() == ["FDA:1", None, "FDA:3"] - for col in ("anatomical_context_qualifier", "sex_qualifier", "population_context_qualifier", "frequency_qualifier", "temporal_context_qualifier"): + assert out["disease_context_qualifier"].to_list() == ["MONDO:0005148", "MONDO:0005148", "MONDO:0005015"] + # Native multivalued slots wrap kept values; granted fields are passed through verbatim. + assert out["anatomical_context_qualifier"].to_list() == [["UBERON:0001557"]] * 3 + assert out["frequency_qualifier"].to_list() == ["HP:0012823"] * 3 + assert out["regulatory_approvals"].to_list() == [["FDA:1"], None, ["FDA:3"]] + for col in ("sex_qualifier", "population_context_qualifier", "temporal_context_qualifier"): assert out[col].to_list() == [out[col].to_list()[0], None, out[col].to_list()[2]], col pruned_second: list[str] = out[PRUNED_COLUMN].to_list()[1] - for col in ( - "disease_context_qualifier", - "regulatory_approvals", - "anatomical_context_qualifier", - "sex_qualifier", - "population_context_qualifier", - "frequency_qualifier", - "temporal_context_qualifier", - ): + for col in ("regulatory_approvals", "sex_qualifier", "population_context_qualifier", "temporal_context_qualifier"): assert any(entry.startswith(f"{col}=") for entry in pruned_second), col assert out[PRUNED_COLUMN].to_list()[0] == [] assert out[PRUNED_COLUMN].to_list()[2] == [] diff --git a/uv.lock b/uv.lock index b31a715..7c570b5 100644 --- a/uv.lock +++ b/uv.lock @@ -319,15 +319,15 @@ wheels = [ [[package]] name = "biolink-model" -version = "4.4.4" +version = "4.4.5" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "linkml-runtime" }, { name = "pydantic" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/85/f6/0691b06fd1f258d1c1e5e3cdd8293c2c9beed12a2d956ecfaa7280e58694/biolink_model-4.4.4.tar.gz", hash = "sha256:100e84f6ea05c22d8333a8d478f0a4e6b8036ff2afbc5e116a6dc286a06deeb4", size = 4042819, upload-time = "2026-08-10T23:34:47.678Z" } +sdist = { url = "https://files.pythonhosted.org/packages/cc/95/b63c5b6bb70863585314a6ff14df32b91eaba5f38450a1e94228eef971ed/biolink_model-4.4.5.tar.gz", hash = "sha256:68534bbfa19c6ba979088bcb30e32fac081db36418ea574e7fbd4b03ab4febe4", size = 4115710, upload-time = "2026-09-18T15:06:06.971Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/62/de/a81e071f0aa904cb2e787b26f35e3d3e4c76db23eb22f4a663867bccd387/biolink_model-4.4.4-py3-none-any.whl", hash = "sha256:320ca2d83672e7cc24184cf2f2a415d755273c56f236128712ec457b2cf83c8b", size = 360399, upload-time = "2026-08-10T23:34:46.031Z" }, + { url = "https://files.pythonhosted.org/packages/19/aa/1a95a68693332df121e143ad84ff1f640cbe5717e5c7ec5dfe521d97d001/biolink_model-4.4.5-py3-none-any.whl", hash = "sha256:62fbd39eb81a41fbf7fa150b550115fe5d1b5b076dc908c32486e37bd7ca7251", size = 370799, upload-time = "2026-09-18T15:06:05.22Z" }, ] [[package]] @@ -3549,7 +3549,7 @@ typecheck = [ [package.metadata] requires-dist = [ { name = "aria2", marker = "extra == 'aria2'", specifier = "==0.0.1b0" }, - { name = "biolink-model", specifier = ">=4.4.4" }, + { name = "biolink-model", specifier = ">=4.4.5" }, { name = "cyclopts", marker = "extra == 'cli'", specifier = ">=1.0.0" }, { name = "datasets", marker = "extra == 'distill'", specifier = ">=3.0.0" }, { name = "dspy", marker = "extra == 'optimize'", specifier = ">=3.2.1" },