Skip to content

Add a toggle to match any or all of the selected labels in the search filter #324

Open

Description

@samuelvkwong

Motivation

Selecting several labels in the search Filters panel always means "reports carrying at least one of them" (union). The more common clinical question is the intersection -- "reports with pneumonia AND pleural effusion" -- and it cannot be expressed today. The only workarounds are searching one label at a time and intersecting by hand, or running an extraction. From the September 2026 ADIT/RADIS brainstorming: "Be able to toggle between UNION and INTERSECTION for labels".

OR was a deliberate choice in the auto-labeling design (docs/superpowers/specs/2026-05-21-auto-labeling-design.md:33-34; docs/superpowers/plans/2026-06-04-auto-labeling.md:125-130, which also rejected a typed label: syntax). This issue proposes revisiting it with a small user-facing switch, not moving label logic into the query language.

Current behaviour

  • SearchForm builds a labels MultipleChoiceField from active labels (values are label names) and renders it only when at least one active label exists (radis/search/forms.py:44-48, :76-84).
  • SearchView threads cleaned_data["labels"] straight into SearchFilters; there is no mode field (radis/search/views.py:40, :68-79).
  • SearchFilters.labels is documented as "ANY of the given label names" (radis/search/site.py:49-50, :69).
  • _build_filter_query implements this as one subquery: reports having a LabelResult whose label name is IN the list and whose value is in LabelResult.SURFACING_VALUES, ANDed in as report__in (radis/pgsearch/providers.py:225-233). One matching row qualifies the report, so it is OR across labels. Introduced by Feature/auto labeling #231 (commit 1bcd6b9) and unchanged since.
  • The same _build_filter_query serves search(), count() and retrieve() through _fuse_hybrid (:354, called at :444, :535, :539) and the subscription filter() provider (:547-552), so a change there reaches search, extraction retrieval and subscriptions at once.
  • test_label_filter_matches_any_label pins OR (radis/labels/tests/test_search_filter.py:56-93); the user guide states it at docs/user-docs/user-guide.md:59 and :79.
  • Filters only run when the query text parses to a node (radis/search/views.py:62; QueryParser.parse returns None for empty text at radis/search/utils/query_parser.py:310-311). A label-only intersection with no free text is therefore blocked by the sibling bullet "Searching without query text doesn't work (for all filters)".

Proposal

  • A second control directly under the labels listbox, inside the same "only when labels exist" gate: labels_mode as a ChoiceField(required=False, choices=[("", "Match any of the selected"), ("all", "Match all of the selected")]), cleaned to "any" / "all".
  • SearchFilters.labels_mode: Literal["any", "all"] = "any" (Literal is already used for patient_sex, site.py:63); reword the labels docstring. A string enum leaves room for a third value (e.g. exclude) without changing the form contract.
  • In _build_filter_query, mode any keeps today's code path unchanged; mode all requires a surfacing result for every distinct selected label.
  • Recommended default: any, so existing URLs, bookmarks and every other SearchFilters caller keep their behaviour. all is arguably the better clinical default, but it would silently change those; see open question 1.
  • Update user-guide.md:59 and :79. docs/user-docs/admin-guide.md:179-180 and :188 describe the filter without stating OR/AND and only need a re-read; the spec/plan under docs/superpowers are historical and need no edit.

Open questions

  1. Default mode: any (recommended, backwards compatible) or all?
  2. Wording: "Match any / Match all" (proposed) vs the brainstorming's "Union / Intersection".
  3. Always show the mode control while the labels field is shown, or hide it until two labels are selected? Simpler: always shown.
  4. Should the same toggle be added to the extraction and subscription forms here? Those forms carry no labels filter at all today (radis/extractions/forms.py:139-149, radis/extractions/views.py:412-422, radis/extractions/tasks.py:87-97, radis/subscriptions/tasks.py:87-96). Recommendation: separate issues for the sibling bullets; they inherit labels_mode through SearchFilters for free.
  5. Search results do not render label badges (only radis/reports/templates/reports/report_detail.html:71-73 does), so a user cannot see why a result matched an intersection. Fine for a first cut; possibly a follow-up.

Implementation notes

Why a <select> with a "" default and not a checkbox or radio. "Reset filters" sets .value = "" on every #filters input, #filters select except language/age (radis/search/static/search/search.js:16-37); that does not uncheck a checkbox or radio, so those would survive a reset. x-ignore-empty-inputs disables ""-valued inputs on submit so they are dropped from the GET URL (radis/core/static/core/core.js:29-46, radis/search/templates/search/search.html:12-15), which keeps today's URLs unchanged when the default is chosen.

SQL for all, cheapest first. Build the subquery on LabelResult with .values("report_id") rather than via Report, to avoid the extra join.

  • A (recommended): one subquery per distinct name, ANDed together -- for name in set(labels): fq &= Q(report__in=LabelResult.objects.filter(label__name=name, value__in=LabelResult.SURFACING_VALUES).values("report_id")). Same shape as today, uses the (label, value) index (radis/labels/models.py:70), N is a handful.
  • B: single grouped subquery, LabelResult.objects.filter(label__name__in=names, value__in=LabelResult.SURFACING_VALUES).values("report_id").annotate(n=Count("label_id", distinct=True)).filter(n=len(names)).values("report_id"). .values() must precede .annotate() so Django groups by report_id only. Not verified against a database.
  • Not an array/containment column on ReportSearchIndex: the Make the FTS candidate query single-table with a search projection #292 design doc keeps labels as a subquery because LabelResult churns under relabeling (branch docs/superpowers/specs/2026-08-29-fts-query-shape-performance-design.md:150-151).

Name-based matching is sound: Label.name is globally unique (labels/models.py:39) and the form validates against active labels, so duplicates only come from hand-edited URLs and set() suffices.

Scope and privacy. Labels are system-wide (Label.group is a LabelGroup, labels/models.py:29, not an auth group); report visibility is enforced by fq = Q(report__groups=filters.group) at providers.py:200, under which the label predicate is ANDed. No new leak surface, no permission changes.

Known limitation. LabelResult.is_stale (labels/models.py:77-82) is ignored by the filter today (admin-guide.md:188), so an intersection over a partially relabeled corpus can include stale rows, exactly as any does.

Tests. all with both/edema-only/pneumonia-only/neither (a report with PRESENT for A and ABSENT for B must not match); all with one label equals any; all with zero labels adds no predicate; duplicate names; form default/cleaning and omission when no labels exist; view threading of labels_mode=all.

Sequencing with #292. The branch (45f8435) touches the same lines only to rename report__in to report_id__in (branch providers.py:237-245). Land in either order; the other rebases in one identifier. If #292 lands first, add an id="labels-all" param and a mode branch to both the new implementation and _build_filter_query_legacy in radis/pgsearch/tests/test_filter_query_equivalence.py (branch :61-68, :217-219). #230 (report API) does not build SearchFilters, so it needs nothing.

Related

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions