Create a batch transfer directly from batch query results without re-uploading the exported file #419

Motivation

From the September 2026 ADIT/RADIS brainstorming: the batch query -> batch transfer workflow is a file round trip. The user downloads the result xlsx, adds or fills a Pseudonym column if none was given at query time, deletes rows for studies or patients that should not be transferred, splits the file if it exceeds the 500-row limit, and uploads it again while re-selecting source and destination. Nothing needed to create the transfer is missing in ADIT: every BatchQueryResult already holds patient_id, study_uid, series_uid and the pseudonym piped from the query task, and the source node is on the task. Marked LOW PRIO in the document.

This is the lighter alternative to #152 (merge the apps), which medihack left open as unsure because batch query is useful on its own. It keeps batch query standalone and adds a bridge.

Current behaviour

• The export always contains every result of every task; there is no selection (adit/batch_query/utils/exporters.py:11, :23-25). The results page offers only Download (adit/batch_query/templates/batch_query/batch_query_result_list.html:20-32).
• The pseudonym from the query file is copied onto every result of the task explicitly to pipe it through to a batch transfer (adit/batch_query/models.py:104, :190; adit/batch_query/processors.py:194, :233; commit bda0c99, 2021-01-27). Nothing consumes it yet except the export.
• Each BatchQueryTask carries its own source (adit/batch_query/forms.py:112-117, adit/batch_query/processors.py:22-24).
• On the transfer side, BatchTransferFileParser.parse groups rows by StudyInstanceUID into one task, and a row without SeriesInstanceUID makes the task transfer the whole study (adit/batch_transfer/parsers.py:47-72). BatchTransferTaskSerializer enforces pseudonym-required and the cross-row checks (adit/batch_transfer/serializers.py:12-34, :70-73). The form stamps source and destination on every task (adit/batch_transfer/forms.py:152-160); TODOs at forms.py:155-156 and serializers.py:43-44 say per-row source/destination from the file is planned.
• Failed query tasks produce no rows (adit/batch_query/processors.py:32-43). A PatientName+BirthDate query matching several patients exports all their studies with only an "Indistinct patients" warning (:81-88, :171-178, :208-215), so the user removes the wrong patient's rows by hand.
• MAX_BATCH_QUERY_SIZE = 1000 limits query tasks, MAX_BATCH_TRANSFER_SIZE = 500 limits rows before grouping (adit/settings/base.py:510, :514; adit/core/parsers.py:62-63), so an export can easily exceed the transfer limit.
• Selective transfer already creates tasks from checkbox-selected rows without a file (adit/selective_transfer/templates/selective_transfer/_query_results_table.html:22-30, adit/selective_transfer/consumers.py:375-399), but it applies one job-level pseudonym to all studies and caps non-staff at 10 (:380-381). Neither fits multi-patient batch results.
• DICOM node access is scoped to the user's active group (adit/core/models.py:64); the batch transfer form re-checks both nodes at submit time (adit/batch_transfer/forms.py:112-122).

Proposal

Options, cheapest first:

A -- "Transfer all results of this job" button. No selection UI. Removes the round trip for the clean case; multi-study and indistinct-patient cases still need the file path.

B -- "Create batch transfer" from selected results (recommended). A checkbox column on the batch query results table plus a file-less transfer form. Details below.

C -- Full merge (#152, TODO.md:82-89). One app with QUERYING/READY states and a cache of found studies. Largest change; contradicts the comment on #152.

Shape of B:

• Put the creation logic in batch_transfer, not in a batch_query view: either a view in batch_transfer that takes a batch query job id and selected result pks, or a "from batch query job" option on BatchTransferJobCreateView with batch_file required only when no job is chosen. Both reuse permission_required = "batch_transfer.add_batchtransferjob" and BatchTransferLockedMixin (adit/batch_transfer/views.py:53-57) and avoid batch_query importing batch_transfer (there are no imports in either direction today). Factor the core ("build a BatchTransferJob from a list of {patient_id, study_uid, series_uids, pseudonym, lines} dicts") into a service function; a RADIS -> ADIT bridge (openradx/radis TODO.md:47) would need exactly that.
• Form: BatchTransferJobForm minus batch_file and source. project_name / project_description can be prefilled from the BatchQueryJob (adit/batch_query/models.py:28-29). ethics_application_id cannot: it exists only on BatchTransferJob (adit/batch_transfer/models.py:20) and is required by default (adit/settings/base.py:525, forms.py:87-88).
• Source comes from the selected results' query tasks and is shown read-only; refuse if they span more than one source. Re-check source.is_accessible_by_user(user, "source") and the chosen destination (adit/core/models.py:107-111) at creation time: the query may have run under another active group, access may have been revoked, or staff may be acting on another user's job.
• Build one dict per selected result with series_uids = [series_uid] or [] and lines = result.query.lines (BatchTransferTask.lines has no default, adit/batch_transfer/models.py:34), validate with BatchTransferTaskSerializer(many=True, can_transfer_unpseudonymized=...), group by study with the step extracted from parsers.py:47-72 so parser and service share it, bulk_create with source/destination, then finish like DicomJobCreateView.form_valid (adit/core/views.py:109-121, START_BATCH_TRANSFER_UNVERIFIED at adit/settings/base.py:412) and redirect to the new job.
• Missing pseudonyms: show the serializer error with the offending rows and tell the user to re-run the query with a Pseudonym column. Do not copy the single job-level pseudonym from selective transfer.
• Only offer the action once the query job is in a terminal status (the set is_restartable checks, adit/core/models.py:354-360); results are bulk-created per task (processors.py:43), so a job in progress has partial results.
• Results table: BatchQueryResultTable shows only task_id, patient_id, study_date_time, study_description, modalities, image_count (adit/batch_query/tables.py:31-38). Add pseudonym and series columns so the user sees what will be transferred. Selection across a paginated, filtered table needs an explicit "all results of task N" (filter task_id exists, adit/batch_query/filters.py:22) and "all results of the job" mode; batch_query.js holds only preference hooks, so select-all is new code.
• Keep the owner-or-staff gating of the results views (adit/batch_query/views.py:170-174); the batch_transfer permission alone must not widen who sees patient names and birth dates.

Open questions

1. Bridge (this issue) or merge (Merge Batch Query and Batch Transfer #152)? Worth settling before B is built; the sample-file/help-text fix in the companion issue is independent of it.
2. Missing pseudonyms: reject with a hint (smallest), a per-row pseudonym input on the results page (close to the wontfix in Add web xlsx editor to batch transfer and batch query view #140), or server-side generation reusing compute_pseudonym (adit/core/utils/pseudonymizer.py:13-25, as mass transfer does with a salt or a random per-study seed, adit/mass_transfer/processors.py:419-431)? The last changes who controls the pseudonym mapping; a policy decision.
3. Should MAX_BATCH_TRANSFER_SIZE count selected results (as the file path does) or resulting tasks after grouping? Series-level results inflate row counts; counting after grouping seems more useful.
4. Selection granularity: per result row only, or also per task and whole job?
5. Let the user override the source, or always use the query task's source?
6. Record the origin BatchQueryJob on the BatchTransferJob? An FK means a migration and a cross-app model dependency; a text field or nothing avoids both.

Implementation notes

• No migration is needed for B without an origin FK. A per-row exclude flag on BatchQueryResult would need one.
• Tests: unit test for the extracted grouping function; a view test posting selected pks that asserts created tasks, the pseudonym-required error, mixed-source refusal and foreign-source refusal. Factories exist (BatchQueryResultFactory, adit/batch_query/factories.py:41; BatchTransferTaskFactory, adit/batch_transfer/factories.py:22).
• Not executed end to end; the round trip was established by reading exporter, parser and serializer.
• TODO.md:96-97 says multiple patients per PatientName+BirthDate are not allowed; processors.py:81-88 and :171-178 do allow it with a warning, so that note is stale.

Related

• Merge Batch Query and Batch Transfer #152 -- merge batch query and batch transfer; this is the alternative that keeps batch query standalone.
• Add web xlsx editor to batch transfer and batch query view #140 -- web xlsx editor, wontfix; option 2 above deliberately stops short of it.
• Export also dicom task results that did not succeed #20 and TODO.md:11-19 -- export changes; handled in the companion sample-file/help-text issue.
• Add series level search and transfer to SelectiveTransfer #141 -- series-level transfer in selective transfer; B gives a series-level transfer path via batch query results.
• Preliminarily exclude modalities before selective transfer/download #251 -- modalities excluded when a pseudonym is given; a results-to-transfer table has the same display question.
• openradx/radis TODO.md:47 (export collections to ADIT) depends on the four-column batch file contract staying stable; B does not change it.