Skip to content

Offer the label filter in the extraction wizard's search step #325

Open

Description

@samuelvkwong

Motivation

Labels can be used as a filter on the search page only. A user who narrows a search by label cannot use the same criterion to scope the corpus of an extraction job, although the query engine already supports it: the wizard's live preview count, the "Preview Search Results" link and the job's retrieval all go through _build_filter_query, which applies SearchFilters.labels. The gap is entirely in the extractions app. (From the September 2026 ADIT/RADIS brainstorming: "Labels should also be in extractions query step".)

Current behaviour

  • The search page builds a labels multi-select of active label names, hides it when there are no active labels (radis/search/forms.py:44-48, :76-80) and passes the names into SearchFilters.labels (radis/search/views.py:40, :78).
  • _build_filter_query applies the label filter as an ANY-of subquery over surfacing LabelResult rows (radis/pgsearch/providers.py:225-233); it is shared by search(), count() and retrieve().
  • ExtractionJob stores every other filter as a column or M2M but has no labels relation (radis/extractions/models.py:36-53).
  • SearchForm.Meta.fields has no labels (radis/extractions/forms.py:26-40), and SearchFilters is built without labels in three places: the form's clean() (radis/extractions/forms.py:139-149), the live preview endpoint (radis/extractions/views.py:412-422) and the job task (radis/extractions/tasks.py:87-97).
  • The preview's hx-include is a hardcoded list of 1--prefixed field names (radis/extractions/templates/extractions/_search_preview_form_section.html:6), the preview view special-cases only modalities for setlist (radis/extractions/views.py:350-356), and the "Preview Search Results" URL is assembled from a hand-written parameter list (radis/extractions/views.py:448-474). A new field that is not added to all three is silently ignored by the preview.
  • The summary step and the detail page list the search parameters without labels (radis/extractions/templates/extractions/extraction_job_wizard_summary.html:31-70, radis/extractions/templates/extractions/extraction_job_detail.html:61-98).
  • Prerequisite defect: the wizard's done() saves the job with save(commit=False) (radis/extractions/views.py:297) + job.save() (:314) and never calls save_m2m(), so ExtractionJob.modalities is not persisted today. A labels M2M would be lost the same way. Filed as a separate bug (see Related).

Proposal

  • ExtractionJob.labels = ManyToManyField(Label, blank=True) plus migration 0008 (latest is 0007_alter_extractionjob_language.py).
  • A shared create_label_field() in radis/core/form_fields.py, mirroring create_modality_field (:82-97): use_pk=True returns a ModelMultipleChoiceField over Label.objects.filter(active=True).order_by("name"), use_pk=False returns the name-valued field the search form builds inline today, so SearchForm in radis/search can switch to the factory. Callers hide the field when it has no choices, as search does. form_fields.py imports only radis.core.constants and radis.reports.models (:12-13), so importing radis.labels.models there creates no cycle.
  • Add labels to SearchForm.Meta.fields and to the filters column after modalities; pass labels=list(labels.values_list("name", flat=True)) at the three SearchFilters call sites; add [name='1-labels'] to hx-include, setlist handling for labels next to modalities, and search_params["labels"] in the preview URL; show labels on the summary and detail pages. Label.__str__ returns the name (radis/labels/models.py:42-43), so search.labels|join:", " in the summary template should work like search.modalities|join does today (not verified in a running app).
  • Semantics: ANY-of, as SearchFilters documents (radis/search/site.py:49-50) and the search page implements. The union/intersection toggle is a separate brainstorming bullet; whatever the search page ends up with should be reused here, so the help text should not hardcode more than the current "any of".
  • Extractions are one-shot, so the only timing effect is that reports not yet labeled by the nightly scan (LABELING_SCAN_CRON, radis/settings/base.py:749) are excluded from the corpus. A sentence in the step-2 help text and in the user guide (docs/user-docs/user-guide.md:144) is probably enough; see open question 3.

Open questions

  1. ANY-of or ALL-of? Depends on the union/intersection bullet; this issue should just follow the search page.
  2. Inactive or deleted labels. The form offers active labels only, but the subquery does not check Label.active (radis/pgsearch/providers.py:229-233), so a label deactivated after the job was created still matches through its stale results, exactly as on the search page. Drop inactive labels from the run-time filter, or leave it consistent with search? The M2M cascade on Label delete silently removes the label from the job.
  3. Should the preview show how many matching reports still await labeling (complement of _needs_work_queryset, radis/labels/scope.py:8-11), or is a help-text sentence enough for a one-shot job? The sentence is the cheaper option.

Implementation notes

  • Fix the missing save_m2m() first; a regression test can follow the pattern in radis/extractions/tests/test_forms.py:250-284 (save(commit=False) + job.save() + refresh_from_db()), with a non-empty modalities list.
  • Group scoping is unchanged: labels are system-wide admin objects (radis/labels/models.py:27-39, docs/user-docs/admin-guide.md:174) and the report set stays bounded by Q(report__groups=filters.group) (radis/pgsearch/providers.py:200). The only new exposure is that label names appear on one more form, as on the search page.
  • Tests: factory (active labels only, alphabetical, hidden when empty), form clean() passes labels, preview passes labels and includes them in the search URL, task passes labels from job.labels, wizard persists the M2M.
  • Docs: docs/user-docs/user-guide.md:144 (Step 2 filter list), CLAUDE.md:62 (labels section mentions the search filter only).

Related

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions