Repository navigation
Offer the label filter in the extraction wizard's search step #325
Copy link
Copy link
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Motivation
Labels can be used as a filter on the search page only. A user who narrows a search by label cannot use the same criterion to scope the corpus of an extraction job, although the query engine already supports it: the wizard's live preview count, the "Preview Search Results" link and the job's retrieval all go through _build_filter_query, which applies SearchFilters.labels. The gap is entirely in the extractions app. (From the September 2026 ADIT/RADIS brainstorming: "Labels should also be in extractions query step".)
Current behaviour
- The search page builds a
labelsmulti-select of active label names, hides it when there are no active labels (radis/search/forms.py:44-48,:76-80) and passes the names intoSearchFilters.labels(radis/search/views.py:40,:78). _build_filter_queryapplies the label filter as an ANY-of subquery over surfacingLabelResultrows (radis/pgsearch/providers.py:225-233); it is shared bysearch(),count()andretrieve().ExtractionJobstores every other filter as a column or M2M but has no labels relation (radis/extractions/models.py:36-53).SearchForm.Meta.fieldshas nolabels(radis/extractions/forms.py:26-40), andSearchFiltersis built without labels in three places: the form'sclean()(radis/extractions/forms.py:139-149), the live preview endpoint (radis/extractions/views.py:412-422) and the job task (radis/extractions/tasks.py:87-97).- The preview's
hx-includeis a hardcoded list of1--prefixed field names (radis/extractions/templates/extractions/_search_preview_form_section.html:6), the preview view special-cases onlymodalitiesforsetlist(radis/extractions/views.py:350-356), and the "Preview Search Results" URL is assembled from a hand-written parameter list (radis/extractions/views.py:448-474). A new field that is not added to all three is silently ignored by the preview. - The summary step and the detail page list the search parameters without labels (
radis/extractions/templates/extractions/extraction_job_wizard_summary.html:31-70,radis/extractions/templates/extractions/extraction_job_detail.html:61-98). - Prerequisite defect: the wizard's
done()saves the job withsave(commit=False)(radis/extractions/views.py:297) +job.save()(:314) and never callssave_m2m(), soExtractionJob.modalitiesis not persisted today. A labels M2M would be lost the same way. Filed as a separate bug (see Related).
Proposal
ExtractionJob.labels = ManyToManyField(Label, blank=True)plus migration0008(latest is0007_alter_extractionjob_language.py).- A shared
create_label_field()inradis/core/form_fields.py, mirroringcreate_modality_field(:82-97):use_pk=Truereturns aModelMultipleChoiceFieldoverLabel.objects.filter(active=True).order_by("name"),use_pk=Falsereturns the name-valued field the search form builds inline today, soSearchForminradis/searchcan switch to the factory. Callers hide the field when it has no choices, as search does.form_fields.pyimports onlyradis.core.constantsandradis.reports.models(:12-13), so importingradis.labels.modelsthere creates no cycle. - Add
labelstoSearchForm.Meta.fieldsand to the filters column aftermodalities; passlabels=list(labels.values_list("name", flat=True))at the threeSearchFilterscall sites; add[name='1-labels']tohx-include,setlisthandling forlabelsnext tomodalities, andsearch_params["labels"]in the preview URL; show labels on the summary and detail pages.Label.__str__returns the name (radis/labels/models.py:42-43), sosearch.labels|join:", "in the summary template should work likesearch.modalities|joindoes today (not verified in a running app). - Semantics: ANY-of, as
SearchFiltersdocuments (radis/search/site.py:49-50) and the search page implements. The union/intersection toggle is a separate brainstorming bullet; whatever the search page ends up with should be reused here, so the help text should not hardcode more than the current "any of". - Extractions are one-shot, so the only timing effect is that reports not yet labeled by the nightly scan (
LABELING_SCAN_CRON,radis/settings/base.py:749) are excluded from the corpus. A sentence in the step-2 help text and in the user guide (docs/user-docs/user-guide.md:144) is probably enough; see open question 3.
Open questions
- ANY-of or ALL-of? Depends on the union/intersection bullet; this issue should just follow the search page.
- Inactive or deleted labels. The form offers active labels only, but the subquery does not check
Label.active(radis/pgsearch/providers.py:229-233), so a label deactivated after the job was created still matches through its stale results, exactly as on the search page. Drop inactive labels from the run-time filter, or leave it consistent with search? The M2M cascade onLabeldelete silently removes the label from the job. - Should the preview show how many matching reports still await labeling (complement of
_needs_work_queryset,radis/labels/scope.py:8-11), or is a help-text sentence enough for a one-shot job? The sentence is the cheaper option.
Implementation notes
- Fix the missing
save_m2m()first; a regression test can follow the pattern inradis/extractions/tests/test_forms.py:250-284(save(commit=False)+job.save()+refresh_from_db()), with a non-emptymodalitieslist. - Group scoping is unchanged: labels are system-wide admin objects (
radis/labels/models.py:27-39,docs/user-docs/admin-guide.md:174) and the report set stays bounded byQ(report__groups=filters.group)(radis/pgsearch/providers.py:200). The only new exposure is that label names appear on one more form, as on the search page. - Tests: factory (active labels only, alphabetical, hidden when empty), form
clean()passes labels, preview passes labels and includes them in the search URL, task passes labels fromjob.labels, wizard persists the M2M. - Docs:
docs/user-docs/user-guide.md:144(Step 2 filter list),CLAUDE.md:62(labels section mentions the search filter only).
Related
- Companion issue: Offer the label filter in the subscription form without losing unlabeled reports #326 (same factory and M2M pattern, plus a timing design for refreshes).
- Bug: Extraction wizard never calls save_m2m(), so the modalities filter is dropped from the saved job #327 (prerequisite; also covers the
extraction_job_detail.html:85Modalities row). - Automatically generate the pre-filter query from extraction fields #138 -- auto-generated pre-filter query for extractions; a label filter is a deterministic complement to it.
- Feature/auto labeling #231 (merged) introduced the labels app and
SearchFilters.labels; its design spec lists neither extractions nor subscriptions in scope or out of scope. - Make the FTS candidate query single-table with a search projection #292 -- rewrites
_build_filter_query; no change toradis/pgsearch/providers.pyis needed for this half, so no overlap. - Add a toggle to match any or all of the selected labels in the search filter #324 -- decides the semantics this form should follow.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Activity