Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/configuration/table.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ Defines subject-predicate-object relationships.
| `predicate` | String | No | Biolink predicate. Defaults to `"related_to"`. Three DAKP prevention predicates are deliberate local extensions ahead of Biolink support (see `biolink.PREDICATE_OVERRIDES`): `prevents`, `applied_to_prevent`, `contraindicated_in_the_prevention_of`. |
| `object` | NodeEncoding | Yes | Object entity configuration |
| `qualifiers` | List[Qualifier] | No | Edge qualifiers (context) |
| `category_override` | Map[Categories, EdgeCategories] | No | Pin the association class per resolved object category (bare names, no `biolink:` prefix), replacing the derived (subject, object) pair lookup for those rows. Rows whose object category is absent from the map derive as before. Pinned classes are still reconciled against the section predicate (a class whose `predicate` slot rejects it is walked up the association hierarchy, with a warning at config time). Use it when one section mixes object categories the pair lookup merges: e.g. `Disease` and `PhenotypicFeature` both derive to `ChemicalEntityToDiseaseOrPhenotypicFeatureAssociation`, but only `EntityToDiseaseAssociation` / `EntityToPhenotypicFeatureAssociation` receive the class-scoped `regulatory_approvals` grant and declare slots like `number_of_cases`. Rows pinned to either of those classes may additionally carry `disease_context_qualifier` plus DAKP's sparse qualifier stack (`anatomical_context_qualifier`, `sex_qualifier`, `population_context_qualifier`, `frequency_qualifier`, `temporal_context_qualifier`) under deliberate Tablassert policy grants. Deliberately ungranted: `species_context_qualifier` (disabled -- never emittable) plus `temporal_interval_qualifier` and `severity_qualifier` (unsatisfiable -- attached to no class, so a grant could never validate). |
| `category_override` | Map[Categories, EdgeCategories] | No | Pin the association class per resolved object category (bare names, no `biolink:` prefix), replacing the derived (subject, object) pair lookup for those rows. Rows whose object category is absent from the map derive as before. Pinned classes are still reconciled against the section predicate (a class whose `predicate` slot rejects it is walked up the association hierarchy, with a warning at config time). Use it when one section mixes object categories the pair lookup merges: e.g. `Disease` and `PhenotypicFeature` both derive to `ChemicalEntityToDiseaseOrPhenotypicFeatureAssociation`, but only `EntityToDiseaseAssociation` / `EntityToPhenotypicFeatureAssociation` declare slots like `regulatory_approvals` and `number_of_cases`. Rows pinned to either of those classes may additionally carry DAKP's sparse qualifier stack ahead of the model under deliberate Tablassert policy grants: `population_context_qualifier` and `temporal_context_qualifier` on both classes, plus `sex_qualifier` on the disease side. Deliberately ungranted: `species_context_qualifier` (disabled -- never emittable) plus `temporal_interval_qualifier` and `severity_qualifier` (unsatisfiable -- attached to no class, so a grant could never validate). |

**Example:**
```yaml
Expand Down Expand Up @@ -274,7 +274,7 @@ annotations:

The separator is a property of the data, not of the slot. Inspect the table's cells and set `split_by` to the separator the cells actually use: `","` for comma-joined ids like `"EFO:0001,EFO:0002"` above, `";"` for `"EFO:0001;EFO:0002"`, `"|"` only if the cells happen to be pipe-joined.

For the canonical `regulatory_approvals` annotation, a pipe-delimited source column uses `split_by: "|"` and emits a real per-row array such as `["011111", "022222"]`; combine it with a class-scoped `category_override` when only the intended association classes receive that grant.
For the canonical `regulatory_approvals` annotation, a pipe-delimited source column uses `split_by: "|"` and emits a real per-row array such as `["011111", "022222"]`; combine it with a class-scoped `category_override` when only the intended association classes declare it (unpinned rows on classes without the slot are pruned).

`split_by` is the one multivalued encoding: every row's cell becomes its own JSON array, so an array that differs per row, the shape a literal can never express, is declared directly. Values are trimmed and blanks dropped; a null cell stays null.

Expand Down Expand Up @@ -590,7 +590,7 @@ annotations:

Annotation names fall into three groups at build time:

- **Allowed edge fields:** names on the edge allow-list: [Biolink Association](https://biolink.github.io/biolink-model/) slots, qualifier slots, and curated KGX/Tablassert edge fields (e.g. `p_value`, `adjusted_p_value`, `knowledge_level`, `supporting_text`, `publications`, `effect_size`, `effect_type`, `statistical_significance_qualifier`, qualifier slots like `severity_qualifier` / `disease_context_qualifier`) are written to edges verbatim. `effect_size` and `effect_type` became real Association slots in biolink-model 4.4.4 (PR #1774) and emit as real JSON numbers and enum tokens. Annotation names are matched case-insensitively against the allow-list and emitted under the canonical slot spelling. The class-scoped `regulatory_approvals` grant is available on `EntityToDiseaseAssociation` and `EntityToPhenotypicFeatureAssociation`; it is a multivalued slot, so declare [`split_by`](#split_by) for pipe-joined cells like `011111|022222`. The same two classes also carry class-scoped grants for DAKP's sparse qualifier stack (`anatomical_context_qualifier`, `sex_qualifier`, `population_context_qualifier`, `frequency_qualifier`, `temporal_context_qualifier`) plus `disease_context_qualifier`; without a grant `prune_to_class` nulls those qualifiers off the pinned rows. `species_context_qualifier` stays disabled, and `temporal_interval_qualifier` / `severity_qualifier` stay unsatisfiable, so none of the three is granted.
- **Allowed edge fields:** names on the edge allow-list: [Biolink Association](https://biolink.github.io/biolink-model/) slots, qualifier slots, and curated KGX/Tablassert edge fields (e.g. `p_value`, `adjusted_p_value`, `knowledge_level`, `supporting_text`, `publications`, `effect_size`, `effect_type`, `statistical_significance_qualifier`, qualifier slots like `severity_qualifier` / `disease_context_qualifier`) are written to edges verbatim. `effect_size` and `effect_type` became real Association slots in biolink-model 4.4.4 (PR #1774) and emit as real JSON numbers and enum tokens. Annotation names are matched case-insensitively against the allow-list and emitted under the canonical slot spelling. `regulatory_approvals` (multivalued, so declare [`split_by`](#split_by) for pipe-joined cells like `011111|022222`), `disease_context_qualifier`, `anatomical_context_qualifier`, and `frequency_qualifier` are declared natively on `EntityToDiseaseAssociation` / `EntityToPhenotypicFeatureAssociation` by biolink-model 4.4.5, and `sex_qualifier` on the phenotypic-feature class, so those rows keep them without any override. The remaining class-scoped grants cover DAKP's sparse qualifier stack ahead of the model: `population_context_qualifier` and `temporal_context_qualifier` on both classes, plus `sex_qualifier` on `EntityToDiseaseAssociation`; without a grant `prune_to_class` nulls those qualifiers off the pinned rows. `species_context_qualifier` stays disabled, and `temporal_interval_qualifier` / `severity_qualifier` stay unsatisfiable, so none of the three is granted.
- **Study metadata:** `study_size`, `study_cohort`, `study_context`, `study_date_range`, `study_method_description`, and `study_method_types` are current Biolink `Study` node properties ([biolink-model PR #1770](https://github.com/biolink/biolink-model/pull/1770)). They describe the study itself, not an association, so their values are carried on the edge's **inlined supporting Study** (`has_supporting_studies` to `Study`, the COHD/ICEES pattern) rather than emitted as edge fields. Deprecated `supporting_study_*` spellings and `sample_size` are accepted as aliases and renamed onto canonical `study_*` names by coercion. They never appear in final JSON. Declaring any of them is legal and emits a `BiolinkRelocationWarning` naming where the value went.
- **Unsatisfiable slots:** names the Biolink LinkML schema declares but attaches to **no** Pydantic class, derived from the installed `biolink-model`. A record carrying one could never validate, so its value is preserved in the inlined supporting study's `StudyResult.description`. A slot leaves the set automatically once a release attaches it.

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ classifiers = [
]
requires-python = ">=3.11"
dependencies = [
"biolink-model>=4.4.4",
"biolink-model>=4.4.5",
"polars>=2.0.0rc2",
"pyarrow>=21.0.0",
"rapidfuzz>=3.14.3",
Expand Down
10 changes: 5 additions & 5 deletions rust/src/fullmap.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2950,11 +2950,11 @@ fn extract_validate_rename(
archive.display()
))
};
let database = open_read_only(&primary).map_err(&validation_ctx)?;
validate_schema(&database).map_err(&validation_ctx)?;
let build_id = read_build_id(&database).map_err(&validation_ctx)?;
let shard_count = shard_count_of(&database).map_err(&validation_ctx)?;
let recorded_allowlist = read_taxon_allowlist_identity(&database).map_err(&validation_ctx)?;
let database = open_read_only(&primary).map_err(validation_ctx)?;
validate_schema(&database).map_err(validation_ctx)?;
let build_id = read_build_id(&database).map_err(validation_ctx)?;
let shard_count = shard_count_of(&database).map_err(validation_ctx)?;
let recorded_allowlist = read_taxon_allowlist_identity(&database).map_err(validation_ctx)?;
drop(database);

// A published archive built WITHOUT the required filter is not the database
Expand Down
53 changes: 16 additions & 37 deletions src/tablassert/biolink.py
Original file line number Diff line number Diff line change
Expand Up @@ -591,51 +591,30 @@ class EffectTypes(str, Enum):


CLASS_FIELD_OVERRIDES: dict[str, frozenset[str]] = {
"EntityToDiseaseAssociation": frozenset(
{
"anatomical_context_qualifier",
"disease_context_qualifier",
"frequency_qualifier",
"population_context_qualifier",
"regulatory_approvals",
"sex_qualifier",
"temporal_context_qualifier",
}
),
"EntityToPhenotypicFeatureAssociation": frozenset(
{
"anatomical_context_qualifier",
"disease_context_qualifier",
"frequency_qualifier",
"population_context_qualifier",
"regulatory_approvals",
"sex_qualifier",
"temporal_context_qualifier",
}
),
"EntityToDiseaseAssociation": frozenset({"population_context_qualifier", "sex_qualifier", "temporal_context_qualifier"}),
"EntityToPhenotypicFeatureAssociation": frozenset({"population_context_qualifier", "temporal_context_qualifier"}),
}
"""Per-class grants of edge fields the resolved association class does not declare.

Keys are bare association class names (``association_class(cat).__name__``), values
the slots ``lib.prune_to_class`` keeps on rows resolved to that class even though the
installed model attaches them elsewhere.

The motivating case is a DAKP contraindication edge: ``regulatory_approvals`` is a
canonical slot not yet attached by the installed model, while
``disease_context_qualifier`` is declared only on the
``ChemicalEntityToDiseaseOrPhenotypicFeatureAssociation`` lineage -- so one edge can
natively carry both only through these explicit class-scoped grants. The same gap
covers DAKP's sparse qualifier stack (``anatomical_context_qualifier``,
``sex_qualifier``, ``population_context_qualifier``, ``frequency_qualifier``,
``temporal_context_qualifier``): each is a satisfiable Biolink qualifier slot that no
pinned class declares, so without a grant ``prune_to_class`` would null it off the
edge. Deliberately excluded: ``species_context_qualifier`` (in
``DISABLED_EDGE_FIELDS`` -- never emittable), ``temporal_interval_qualifier`` and
``severity_qualifier`` (in ``UNSATISFIABLE_EDGE_FIELDS`` -- attached to no class, so
a grant could never validate). Tablassert
deliberately emits the granted fields on the pinned classes ahead of the pinned model
The motivating case is a DAKP contraindication edge carrying the sparse qualifier
stack ahead of the pinned model: ``population_context_qualifier`` and
``temporal_context_qualifier`` are satisfiable qualifier slots the pinned classes do
not declare, and ``sex_qualifier`` is declared only on the phenotypic-feature side of
the family -- so without a grant ``prune_to_class`` would null them off the edge.
Upstream Biolink has absorbed most of the original stack (the biolink-model 4.4.5
mixin consolidation attached ``disease_context_qualifier``,
``anatomical_context_qualifier``, and ``frequency_qualifier`` to the whole
disease/phenotype family, renamed ``FDA_regulatory_approvals`` to the canonical
``regulatory_approvals`` on the pinned classes, and added ``sex_qualifier`` to
``EntityToPhenotypicFeatureAssociation``); the tripwire removed those grants the day
the release landed, and the remaining ones stay deliberately ahead of the model
(pending an upstream Biolink widening). ``_validation_record`` strips granted fields
before record validation so the deliberate gap is not reported as ``extra_forbidden``.
before record validation so the deliberate gap is not reported as
``extra_forbidden``.

A tripwire test asserts every granted field is still absent from its class: the moment
a biolink-model release attaches the slot, the suite fails and the stale grant is
Expand Down
50 changes: 17 additions & 33 deletions tests/test_biolink.py
Original file line number Diff line number Diff line change
Expand Up @@ -412,39 +412,20 @@ def test_class_field_overrides_track_the_installed_model() -> None:
Tripwire: the moment a biolink-model release attaches a granted slot to the class,
this fails and the stale grant is removed from ``CLASS_FIELD_OVERRIDES`` (same
philosophy as the ``UNSATISFIABLE_EDGE_FIELDS`` derivation guard). A field the
family allow-list would strip anyway must never be granted. The canonical
``regulatory_approvals`` grant is intentionally present on exactly the two
association classes that need it while the installed model catches up, alongside
DAKP's sparse qualifier stack (``anatomical_context_qualifier``,
``sex_qualifier``, ``population_context_qualifier``, ``frequency_qualifier``,
``temporal_context_qualifier`` -- each satisfiable, none declared by the pinned
classes). Deliberately excluded from the grants: ``species_context_qualifier``
(disabled) plus ``temporal_interval_qualifier`` and ``severity_qualifier``
(unsatisfiable -- a grant could never validate).
family allow-list would strip anyway must never be granted. The biolink-model
4.4.5 mixin consolidation (plus the ``regulatory_approvals`` rename and the
phenotypic-feature ``sex_qualifier`` attachment) fired exactly this tripwire for
the original seven-field grant stack; the remaining grants are DAKP's sparse
qualifier slots still absent from the pinned classes
(``population_context_qualifier`` / ``temporal_context_qualifier`` on both, and
``sex_qualifier`` on the disease side only). Deliberately excluded from the
grants: ``species_context_qualifier`` (disabled) plus
``temporal_interval_qualifier`` and ``severity_qualifier`` (unsatisfiable -- a
grant could never validate).
"""
assert {
"EntityToDiseaseAssociation": frozenset(
{
"anatomical_context_qualifier",
"disease_context_qualifier",
"frequency_qualifier",
"population_context_qualifier",
"regulatory_approvals",
"sex_qualifier",
"temporal_context_qualifier",
}
),
"EntityToPhenotypicFeatureAssociation": frozenset(
{
"anatomical_context_qualifier",
"disease_context_qualifier",
"frequency_qualifier",
"population_context_qualifier",
"regulatory_approvals",
"sex_qualifier",
"temporal_context_qualifier",
}
),
"EntityToDiseaseAssociation": frozenset({"population_context_qualifier", "sex_qualifier", "temporal_context_qualifier"}),
"EntityToPhenotypicFeatureAssociation": frozenset({"population_context_qualifier", "temporal_context_qualifier"}),
} == CLASS_FIELD_OVERRIDES
for class_name, fields in CLASS_FIELD_OVERRIDES.items():
cls: type[Any] = association_class(f"biolink:{class_name}")
Expand All @@ -460,6 +441,9 @@ def test_validate_record_tolerates_class_field_override_grants() -> None:
The grant is a deliberate, class-scoped step ahead of the pinned model, so its
``extra_forbidden`` must not surface on either intended target -- while the same
field on an ungranted class stays a real defect through the installed-model boundary.
``population_context_qualifier`` is the probe because it is still granted on both
targets: ``regulatory_approvals`` was absorbed by the model itself in 4.4.5, so a
probe on it would pass natively and exercise nothing.
"""
base: dict[str, Any] = {
"id": "e1",
Expand All @@ -468,13 +452,13 @@ def test_validate_record_tolerates_class_field_override_grants() -> None:
"object": "MONDO:0005148",
"knowledge_level": "statistical_association",
"agent_type": "data_analysis_pipeline",
"regulatory_approvals": ["FDA:1"],
"population_context_qualifier": "NCIT:C25667",
}
for category in ("EntityToDiseaseAssociation", "EntityToPhenotypicFeatureAssociation"):
record: dict[str, Any] = {**base, "category": [f"biolink:{category}"]}
assert validate_record(record, edge=True) == []
control: dict[str, Any] = {**base, "category": ["biolink:GeneToDiseaseAssociation"]}
assert "regulatory_approvals: extra_forbidden" in validate_record(control, edge=True)
assert "population_context_qualifier: extra_forbidden" in validate_record(control, edge=True)
unknown: dict[str, Any] = {**base, "category": ["biolink:EntityToDiseaseAssociation"], "not_a_biolink_field": "x"}
assert "not_a_biolink_field: extra_forbidden" in validate_record(unknown, edge=True)

Expand Down
Loading
Loading