Dealing with large inputs in job inputs provenance endpoint

[fmigneault]

Note

Originally mentioned in #580

With #584, it is proposed to transfer the /jobs/{jobId}/inputs endpoint from Part 4 to Part 1 because it is useful to know what a process was executed with (as a reminder, that endpoint return not only inputs, but also headers and other relevant details to allow re-executing the process exactly as originally submitted).

One possible ambiguity arises with large inputs. If a multi-GB/TB data input was provided inline (using value) for process execution, it is most realistically not desirable (or even feasible with server constraints) to respond with the same large embedded data in /jobs/{jobId}/inputs.

Therefore, it might be necessary to introduce a permission for the server to request (or auto-select) a Prefer: return=minimal definition, such that the large inline data could be substituted to an equivalent already-staged URI (using href). That would be essentially equivalent to the current Job outputs (or inline sync response) behaviour where if the "server deems that the size of an output value is large", it can decide to return it differently.

There might be other Content-Transfer-Encoding and base64 considerations to include as well in that response.

Note

Edit
Following #593, the endpoint is now /jobs/{jobId}/definition (see Part 4 - Job Management Definition) as it contains everything that defines the job, including inputs, but also other parameterization details (e.g. headers for Prefe execution modes).

[bpross-52n]

SWG Meeting from 2026-06-08: Add clarifying examples to Part 1, based on the discussion in this issue.