question: add arguments to workflow step inputs

[matthdsm]

Hi,
I'm trying to add an stepInput to a workflow step whose value is based on another input
e.g.

        self.input(
            "readGroupHeaderLine",
            String(),
            value=StringFormatter(
                "@RG\\tID:{name}\\tSM:{name}\\tLB:{name}\\tPL:{pl}",
                name=InputSelector("sampleName"),
                pl=InputSelector("platform"),
            ),
        )
        generateAlignedSam = self.step(
            "generateAlignedSam",
            biotools.bwa.BwaMem_0_7_15(
                reads=[self.unalignedReadsR1, self.unalignedReadsR2],
                reference=self.referenceFasta,
                readGroupHeaderLine=self.readGroupHeaderLine,
                threads=self.threads,
            ),
        )

This seems to work, but when translating to CWL, the readGroupHeaderLine input is still required. Is there a way to add an input to a step, so it doesn't show up in the workflow inputs?

Thanks
M

[illusional]

Hey @matthdsm, for workflow step inputs, this is planned to be resolved by #8. I can't give you an exact ETA, but I'm intending to start work on this soon.

[matthdsm]

Great to hear! I'll be monitoring the repo for changes I can test.

Thanks again for this amazing initiative
M

[illusional]

Hey @matthdsm, thanks for standing by while I've been working on this feature. It's still WIP, but just wanted to let you know that in the Expressions PR (#8), this functionality is working in WDL. I've added this as a test case which executes as expected in MiniWDL.

Note the difference is you assign the expression to the default which would then get baked into the workflow.

https://github.com/PMCC-BioinformaticsCore/janis-core/blob/df661812b23518e4eb1a548d192fce7848b62237/janis_core/tests/test_translation_wdl.py#L949-L984

This translates to:

workflow wf {
  input {
    String sampleName
    String platform
    String? readGroupHeaderLine = "@RG\\tID:~{sampleName}\\tSM:~{sampleName}\\tLB:~{sampleName}\\tPL:~{platform}"
  }
  call echo as print {
    input:
      inp=select_first([readGroupHeaderLine, "@RG\\tID:~{sampleName}\\tSM:~{sampleName}\\tLB:~{sampleName}\\tPL:~{platform}"])
  }
  output {
    File out = print.out
  }
}

Nb: expressions haven't been formally released yet.

[matthdsm]

Awesome! Looking good!
As it stands now, we're converging on CWL for our workflows, so I you don't mind, I'll hold out on testing until that part's finished too.

Thanks for all the great work already, I've been pushing this library in my research group.
Cheers
M

[illusional]

Hey @matthdsm, I'm edging closer towards the generalised expressions. Specifically the CWL implementation for this feature has been completed, and the example workflow from before converts to the following CWL:

#!/usr/bin/env cwl-runner
class: Workflow
cwlVersion: v1.0
requirements:
  InlineJavascriptRequirement: {}
  StepInputExpressionRequirement: {}
inputs:
  platform:
    id: platform
    type: string
  readGroupHeaderLine:
    id: readGroupHeaderLine
    type:
    - string
    - 'null'
  sampleName:
    id: sampleName
    type: string
outputs:
  out:
    id: out
    type: File
    outputSource: print/out
steps:
  print:
    in:
      _print_inp_platform:
        id: _print_inp_platform
        source: platform
      _print_inp_readGroupHeaderLine:
        id: _print_inp_readGroupHeaderLine
        source: readGroupHeaderLine
      _print_inp_sampleName:
        id: _print_inp_sampleName
        source: sampleName
      inp:
        id: inp
        valueFrom: |-
          $((inputs._print_inp_readGroupHeaderLine != null) ? inputs._print_inp_readGroupHeaderLine : "@RG\\tID:{name}\\tSM:{name}\\tLB:{name}\\tPL:{pl}".replace(/\{name\}/g, inputs._print_inp_sampleName).replace(/\{pl\}/g, inputs._print_inp_platform))
    run: tools/EchoTestTool.cwl
    out:
    - out
id: wf

You could also apply this directly on the step input:

wf = WorkflowBuilder("wf")
wf.input("sampleName", str)
wf.input("platform", str)

wf.step(
    "print",
    EchoTestTool(
        inp=StringFormatter(
            "@RG\\tID:{name}\\tSM:{name}\\tLB:{name}\\tPL:{pl}",
            name=wf.sampleName,
            pl=wf.platform,
        )
    ),
)
wf.output("out", source=wf.print)

You should be able to test the CWL translations with no worries. WDL requires the following updated (and unreleased) module:

pip3 install --no-dependencies --upgrade git+https://github.com/PMCC-BioinformaticsCore/python-wdlgen.git@cleanup-commandbuilder

[matthdsm]

Thanks! I'll be sure to give it a go ASAP!

Cheers
M