Skip to content

fix(sqlite): negate token :not at the resource level - #480

Open
angela-helios wants to merge 1 commit into
mainfrom
fix/473-token-not-semantics
Open

fix(sqlite): negate token :not at the resource level#480
angela-helios wants to merge 1 commit into
mainfrom
fix/473-token-not-semantics

Conversation

@angela-helios

Copy link
Copy Markdown
Contributor

Closes #473

The bug

On sqlite, the token :not modifier negated the row predicate inside the per-parameter subquery:

resource_id IN (SELECT resource_id FROM search_index
                WHERE param_name = 'x' AND NOT (value_token_code = ?))

That asks "does any indexed row differ from the value", not FHIR's "does no value of the parameter match". Two failure modes, both seen in the #448 sweep:

  • Multi-valued elements leak back in. Patient?language:not=urn:ietf:bcp:47|en-US returned all 27 patients — every patient with a second communication row (or a second coding, or a second identifier) satisfied NOT(row = x) via its other rows. Same story for Observation?combo-code:not (1400/1400) and ExplanationOfBenefit?status:not=active (matched a phantom contained-resource row).
  • Resources missing the element are dropped. A resource with no rows for the parameter never joins the index at all, so Encounter?reason-code:not=x returned 245 instead of ≥370 — but the spec is explicit that absent-element resources match :not.

Only single-row scalar params (Patient?gender:not=male) happened to behave.

The fix

The token handler now always builds the positive predicate, and the query builder negates at the resource level by flipping the subquery membership:

resource_id NOT IN (SELECT resource_id FROM search_index
                    WHERE param_name = 'x' AND (value_token_code = ?))

One change handles both failure modes: a resource is excluded iff any of its rows matches, and resources with no rows for the parameter fall out of the subquery and are kept. Multi-value :not=a,b composes correctly too (the OR'd positives sit inside the NOT IN).

Postgres already implements these semantics (postgres_integration_search_not_modifier asserts the missing-element case); this aligns sqlite with it.

Tests

  • Unit: the :not fragment stays positive (negation is the builder's job).
  • Integration (search_suite, sqlite): missing-element inclusion (gender:not=male returns the no-gender patient) and multi-valued exclusion (an Observation coded both LOINC and SNOMED must not leak through its SNOMED coding).
  • Full sqlite runs green: persistence lib 758 passed, search_suite 83 passed, clippy clean.

The :not modifier wrapped the row predicate in NOT(...) inside the
per-parameter subquery, which asks "does any indexed row differ from the
value" instead of FHIR's "does no value match". Multi-valued elements
(identifier arrays, CodeableConcept codings, communication.language)
leaked back in through their non-matching rows, and resources without
the element at all -- matches, per the spec -- never joined the index and
were silently dropped.

The token handler now always builds the positive predicate and the query
builder flips the subquery membership to resource_id NOT IN (...) when
the modifier is :not, which handles both failure modes in one place.
Postgres already implemented these semantics; this aligns sqlite with it.

Closes #473
@claude

claude Bot commented Jul 31, 2026

Copy link
Copy Markdown

Code review

Found 2 issues — both regressions in contained-resource handling introduced by this PR's :not fix. Details in inline comments.

@codecov

codecov Bot commented Jul 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

search: token :not is per-row, not per-resource — wrong on every nested/array-valued parameter

2 participants