Part 3: Add multipart ad-hoc workflow execution with CWL

[fmigneault]

With the addition of Part 2 multipart deployment:

• CWL guidance improvements + Multipart Deploy/Replace #588
• https://github.com/opengeospatial/ogcapi-processes/blob/master/extensions/deploy_replace_undeploy/standard/recommendations/cwl/PER_body-cwl-multipart.adoc

A server may deploy CWL parts (namely, a Workflow and its CommandLineTool/ExpressionTool dependencies) as separate part contents just as they would be defined under separate .cwl files on disk. This avoids users having to manually combine these documents by nesting them or aggregating them with $graph pattern (i.e.: https://www.commonwl.org/v1.2/Workflow.html#Packed_documents).

Building upon a similar pattern, Part 3 Worklows and Chaining may be able to address the "Ad-hoc CWL workflows" problem:

• Ad-hoc CWL workflows not applicable #500

Note

For context, ad-hoc CWL do not work natively because CWL is just a workflow "definition". Contrary to Part 3 Nested Processes, it cannot have the runtime "input parametrization" values directly embedded in the workflow, unless the CWL definition is itself modified (e.g.: by injecting valueFrom fields, https://www.commonwl.org/v1.2/Workflow.html#WorkflowStepInput). Having to modify the CWL for each input value defeats its reusable purpose.

Therefore, the proposal is to define a multipart request as follows, which can include simultaneously the CWL "definition" and the OGC execution "input parametrization" values. An advantage of this multipart approach (on top of separating concerns and preserving the CWL structure as it would be deployed), is that each part may supply its own headers. For example, the request could be adjusted with another content schema/profile on the execution side without impacting CWL part, and vice-versa. Obviously, other contents (e.g.: openEO or native OAP process description instead of CWL) could work the same way.

POST /jobs
Content-type: multipart/mixed; boundary="cwl_ad_hoc_workflow_boundary"
Prefer: respond-async

--cwl_ad_hoc_workflow_boundary
Content-Type: application/cwl+json

{
  "doc": "The Workflow *Definition*",
  "class": "Workflow", 
  "inputs": { "<input-id>": {"type": "<cwl-type>"} },
  "outputs": { "<output-id>": {"type": "<cwl-type>"} },
  "steps": { "step-1": "{...}", "step-2": "{...}" }
}

--cwl_ad_hoc_workflow_boundary
Content-Type: application/json
Content-Schema: <https://raw.githubusercontent.com/opengeospatial/ogcapi-processes/refs/heads/master/openapi/schemas/processes-workflows/execute-workflows.yaml>

{ 
  "description": "The Workflow *Values* / Job Parameters",
  "inputs": { "<input-id>": "the-value" },
  "outputs": { "<output-id>": { "format": { "mediaType": "iana:application/x-what-i-want" } } }
}

--cwl_ad_hoc_workflow_boundary--