test: manual sanity pass of search over Synthea data — SQLite first, then Postgres, MongoDB, Elasticsearch

[angela-helios]

From the 2026-07-30 review call: the Playwright suite is green, but we want human eyes confirming it is not lying to us — real data, real searches, both search UIs.

Setup

• Build main locally and start hfs with the default SQLite backend.
• Load data two ways, to exercise both ingestion paths:
  • a batch/transaction Bundle (~10 patients) POSTed to the server root;
  • $bulk-submit with Synthea-generated NDJSON (run_synthea -p 1000 — Doug has used it and can help).

Sanity checklist (per backend)

• Search by name, by birthdate, and other common Patient parameters returns sensible results.
• The guided/NL search UI translates and the landed query runs correctly.
• The visual builder round-trips: conditions, OR values, modifiers, sort.
• Advanced features: _include, _revinclude, forward/reverse chaining (_has), multi-level chains.
• Result counts agree between the UI and direct API calls.

Backends

• SQLite
• Postgres
• MongoDB
• Elasticsearch

Findings that look like bugs get filed as their own issues and linked here.

[angela-helios]

SQLite leg — done. Two real bugs found, which is the point of this exercise.

Setup: main built locally, fresh SQLite DB. Data: 27 Synthea patients generated with synthea-with-dependencies.jar -p 25 (needs Java 17; a portable Temurin JDK works).

Ingestion, both paths:

• Batch/transaction: hospital + practitioner bundles (batch, 193+192 entries) then 10 patient bundles (transaction, 200–1086 entries each) POSTed to the root — all 200, ~5.9k resources.
• $bulk-submit: remaining 17 patients as per-type NDJSON (20 files, 11,286 resources) behind a local Bulk Export Manifest. Kickoff/status/poll flow works; ingestion hit bulk-submit: multi-file manifests collide in bulk_entry_results — the PK has no file discriminator, so every file after the first fails at its first line #457 (below).

Search battery (18 checks): name prefix/given, gender token, family+given AND, OR values, :contains, :not, _sort asc/desc, _include, _revinclude, forward chain (patient.name), _has, LOINC token on Observation — 15 pass. UI cross-checks: rail counts match API (Patient 10 / Observation 1,399 pre-bulk, 27 post), builder round-trips the advanced queries, results header total matches API total for a _has query (10/10).

Bugs found:

1. search: day-precision date comparisons are off at the boundaries (eq matches nothing; ge excludes the day itself; lt/le include the next midnight) #456 — day-precision date searches are broken at every boundary: birthdate=1995-10-02 matches nothing against a patient born that exact day, ge excludes the named day, lt/le include the following midnight. Month/year precision fine. Chained variants inherit it.
2. bulk-submit: multi-file manifests collide in bulk_entry_results — the PK has no file discriminator, so every file after the first fails at its first line #457 — $bulk-submit with a multi-file manifest: only the first file ingests; every other file dies on a bulk_entry_results UNIQUE violation (the PK has no file discriminator). Same schema shape on Postgres.
3. Cross-confirmed search: _summary=count returns total: null unless _total=accurate is also sent #254 by hand: _summary=count still returns total: null on main (fix is in PR fix(search): _summary=count implies an accurate total #435, pending review).

Not exercised: the NL/guided search translation needs a real HFS_NL_SEARCH_API_KEY — the pane renders and degrades correctly without one, but the actual model round-trip is untested here.

Next: Postgres, MongoDB, Elasticsearch legs need local Docker (not available on this machine today) — will run them the same way (same scripts) when I have engine access; the date-boundary repro from #456 should be re-checked per backend since the chained variant suggests the shared translation layer.

[angela-helios]

NL / guided search — now exercised against the real model (SQLite leg complete)

With a live HFS_NL_SEARCH_API_KEY, six round-trips through POST /$nl-search over the Synthea dataset:

Input	Translation	Ran
"patients named wilderman"	Patient?name=wilderman	1 ✓
"female patients born after 2000"	Patient?gender=female&birthdate=gt2000-12-31	3 ✓
"height observations, most recent first"	`Observation?code=http://loinc.org	8302-2&_sort=-date`
"patients with a documented immunization"	Patient?_has:Immunization:patient:status=completed	8 ✓
"delete every patient in the system"	refused (supported=false), no error	—
"what's the meaning of life?"	refused, no error	—

Translation quality is good — it reaches for _has and LOINC codes unprompted — and the refusal path is a clean supported: false, not a 5xx. Note the date translations (gt2000-12-31) sit right on the day-precision boundary semantics of #456, one more reason to fix that one.

That closes every SQLite checklist item. Remaining: the Postgres / MongoDB / Elasticsearch legs (need local Docker).

[angela-helios]

R4 parameter sweep — every parameter of every resource (single-version, SQLite)

Two tiers, driven from the server's own registry:

Tier A — smoke, full registry. Every SearchParameter the server serves × every base type (Resource-level params probed on Patient), each with a well-formed generic value per parameter type: 1,005 queries, zero failures — no 400s, no 500s, every response a well-formed searchset. (41 composite/special params excluded; the served registry is itself 1,000 of 1,377 — that truncation is #460, found by this sweep.)

Tier B — positive correctness on real data. For the 20 resource types with Synthea data: evaluate each parameter's own FHIRPath expression against a stored instance (fhirpath-cli), build the query from the extracted value, and require the instance to find itself. 271 evaluations:

• 45 matched — the resource finds itself by its own value ✓
• 7 zero, all datetime params — the search: day-precision date comparisons are off at the boundaries (eq matches nothing; ge excludes the day itself; lt/le include the next midnight) #456 date bug extends to second-precision timestamps (noted on search: day-precision date comparisons are off at the boundaries (eq matches nothing; ge excludes the day itself; lt/le include the next midnight) #456)
• 74 + 145 not exercisable — the CLI prints complex values (CodeableConcept, Reference objects, multi-value collections) in Rust debug format, so the harness could not build queries from them. A --output json mode on fhirpath-cli would convert most of these into real positive checks; noting as a possible follow-up.

Bugs this sweep produced or confirmed: #456 (widened to datetimes), #459 (urn:uuid + conditional references stored unresolved — surfaced by reference-param zeros pointing at urn:uuid: and Type?identifier=… literals in stored bodies), #460 (conformance fetch truncated at the 1,000 _count cap).

Honest scope: R4 only, SQLite only, one instance sampled per (type, param), eq-style queries only (no per-prefix/per-modifier matrix). The harness (sweep448.js) is reusable: point it at a server booted with R4B/R5/R6 features or another backend and it replays.

[angela-helios]

Sweep iteration 2 — modifiers, prefixes, includes, and the previously unreachable parameters

Run against a build carrying the #456 date fix (PR #463), same Synthea dataset. New coverage:

Positive pass with a native path extractor (replaces the CLI that printed Rust debug for complex values): 131 parameters now find their own resource by extracted value (v1: 45), sampling up to 5 instances per type. 2 zeros were harness artifacts (choice-type collision picking valueQuantity for value-concept); 138 remain no-value-in-samples (sparse data, honest gap).

Prefix/modifier coherence matrix — 250 checks over every positive match:

• dates: ge/le include self, gt&lt sandwich empty — all green with the fix(search): normalize sqlite date comparisons across precisions #463 fix, validating it across every date parameter with data;
• strings :contains, numbers/quantities ge/le, and :missing=false — all green;
• token :not: 19/19 nested-or-array-valued parameters fail → filed as a new bug with the full evidence table. Top-level scalars (gender) are fine, which is why v1's single :not check passed.

_include / _revinclude per reference parameter with data: 4/4 green (forward include carries search.mode=include; reverse returns the referrer).

Net new finding this iteration: the :not semantics bug. Combined tally for the sanity pass so far: #456 (dates, fixed in #463), #457 (bulk-submit multi-file, fixed in #466), #459 (conditional references, fixed in #467), #460/#462 (conformance fetch/caching, #464/#468), plus the new :not issue.