A GitHub Action to launch a workflow using Seqera Platform - https://cloud.seqera.io.
This action uses the Tower CLI.
Contributed with ❤️ from the @nf-core community.
Note
Yes we know, Nextflow Tower was renamed to Seqera Platform a while back, but renaming the GitHub Action would mean breaking everyone's workflows.
This runs the current GitHub repository pipeline on Seqera Platform at the current commit hash when pushed to the dev branch. The workflow runs on the user's personal workspace.
on:
push:
branches: [dev]
jobs:
run-tower:
runs-on: ubuntu-latest
steps:
- uses: seqeralabs/action-tower-launch@v1
# Use repository secrets for sensitive fields
with:
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}This example never runs automatically, but creates a button under the GitHub repository Actions tab that can be used to manually trigger the workflow.
It runs on a specified installation of Seqera Platform, with a specific organisation workspace. It also calls an external pipeline to be run at a pinned version tag.
The --outdir parameter is used to save results to a separate directory in the AWS bucket and the pipeline uses two config profiles.
name: Launch on Seqera Platform
# Manually trigger the action with a button in GitHub
# Alternatively, trigger on release / push etc.
on:
workflow_dispatch:
jobs:
run-tower:
name: Launch on Seqera Platform
# Don't try to run on forked repos
if: github.repository == 'YOUR_USERNAME/REPO'
runs-on: ubuntu-latest
steps:
- uses: seqeralabs/action-tower-launch@v1
# Use repository secrets for sensitive fields
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
api_endpoint: ${{ secrets.TOWER_API_ENDPOINT }}
compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
pipeline: YOUR_USERNAME/REPO
revision: v1.2.1
run_name: ${{ github.job }}_${{ github.run_attempt }}
workdir: ${{ secrets.AWS_S3_BUCKET }}/work/${{ github.sha }}
# Set any custom pipeline params here - JSON object as a string
parameters: |
{
"outdir": "${{ secrets.AWS_S3_BUCKET }}/results/${{ github.sha }}"
}
# List of pipeline config profiles to use - comma separated list as a string
profiles: test,aws_tower
- uses: actions/upload-artifact@v3
with:
name: ${{ needs.getdate.outputs.date }}_run_logs
path: |
tower_action_*.log
tower_action_*.jsonPlease note that a number of these inputs are sensitive and should be kept secure. We recommend saving them as appropriate using GitHub repository encrypted secrets. They can then be accessed with ${{ secrets.SECRET_NAME }} in your GitHub actions workflow.
Note that if you are using secrets, these will be screened in the GitHub Actions log and appear as ***.
Warning
Secret masking only covers the live GitHub Actions log view. It does not extend to the tower_action_*.log/.json files this action writes. If you interpolate a secret into nextflow_config, parameters or pre_run_script and enable the verbose action input, it will appear in plain text in the log. Prefer resolving such secrets via Seqera Platform pipeline secrets (secrets.MY_SECRET in Nextflow config) instead of GitHub Actions secrets wherever possible, since Platform secrets are never sent through this action or its logs at all.
[Required] Seqera Platform personal access token.
Visit https://cloud.seqera.io/tokens to generate a new access token.
See the Seqera Platform documentation for more details:
[Optional] Seqera Platform workspace ID.
Seqera Platform organisations can have multiple Workspaces. Use this field to choose a specific workspace.
Default: Your personal user's workspace.
Your Workspace ID can be found in the organisation's Workspaces tab:
Default: Your primary workspace.
[Optional] Seqera Platform compute environment name (not ID).
Default: Your primary compute environment.
[Optional] Seqera Platform API URL endpoint.
Default: api.cloud.seqera.io
[Optional] Workspace pipeline name or full pipeline URL.
For example, https://github.com/nf-core/sarek.
Can also be the name of a preconfigured pipeline in Seqera Platform.
Default: The current GitHub repository (https://github.com/${{github.repository}}).
[Optional] Pipeline revision.
A pipeline release tag, branch or commit hash.
Default: The revision specified in Seqera Platform or the default branch of the repo.
[Optional] Nextflow work directory.
The location that temporary working files should be stored. Must be accessible in the Seqera Platform compute environment used.
[Optional] Pipeline parameters.
Additional pipeline parameters.
These should be supplied as a valid JSON object, quoted as a string in your GitHub Actions workflow. See example usage above for an example.
JSON is required (not YAML) because we do some parsing and dumping to ensure that the action handles multi-line string formatting correctly.
[Optional] Nextflow config profiles.
Pipeline config profiles to use. Should be comma separated without spaces.
[Optional] Seqera Platform run name
Provide a name for the run in Seqera Platform.
[Optional] Nextflow config options.
Useful to pass custom Nextflow config options to the tw launch command e.g.
jobs:
run-tower:
steps:
- uses: seqeralabs/action-tower-launch@v1
with:
nextflow_config: |
process.errorStrategy = 'retry'
process.maxRetries = 3
# Truncated..[Optional] Pre-run script before launch.
Pre-run script executed before pipeline launch. This would be particularly useful if you wanted to use a different version of Nextflow than the default available in Seqera Platform. You can set this in the pipeline Github Actions:
jobs:
run-tower:
steps:
- uses: seqeralabs/action-tower-launch@v1
with:
pre_run_script: "export NXF_VER=21.10.3"
# Truncated..[Optional] Set GitHub action to wait for pipeline completion
The default setting is for GitHub actions to wait until a pipeline runs to completion. If you want GitHub actions to launch the workflow and then finish you can set the wait to false:
jobs:
run-tower:
steps:
- uses: nf-core/tower-action@v2
with:
wait: false
# Truncated..[Optional] Enable verbose logging in the Tower CLI, for debugging launch failures.
Default: false.
Warning
Verbose mode makes the CLI log full HTTP request/response bodies to tower_action_*.log, including the literal contents of nextflow_config, parameters and pre_run_script. GitHub's secret masking does not cover this file.
jobs:
run-tower:
steps:
- uses: seqeralabs/action-tower-launch@v2
with:
verbose: true
# Truncated..The action creates the output variable json which is a JSON string of metadata created by the Seqera Platform API. It looks like this and can be parsed using the built in fromJSON() method.
{
"workflowId" : "7f061c344df044",
"workflowUrl" : "https://cloud.seqera.io/orgs/myorg/workspaces/myworkspace/watch/7f061c344df044",
"workspaceId" : 123456789,
"workspaceRef" : "[myorg / myworkspace]"
}
In addition, each variable is available as a separate output available under the following IDs:
workflowIdworkflowUrlworkspaceIdworkspaceRef
From the example above, we can now extend it to use the output variables in downstream steps and jobs. In this example we use the output variables twice:
- We capture the output variables of the step and use it as an output of the job. We then use it in a second job.
- We use the outputs in a following step where we echo them to the console.
on:
pull_request:
push:
branches: [dev]
jobs:
run-tower:
runs-on: ubuntu-latest
# Capture action outputs as outputs for the job
outputs:
workflow_id: ${{ steps.run.outputs.workflowId }}
workspace_id: ${{ steps.run.outputs.workspaceId }}
steps:
- uses: seqeralabs/action-tower-launch@v1
with:
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
- name: Comment PR
uses: thollander/actions-comment-pull-request@v2
with:
message: |
Pipeline launched, monitor progress [here](${{ steps.runs.outputs.workflowUrl }})
Details:
- Workflow ID: ${{ steps.runs.outputs.workflowId }}
- Workspace: ${{ steps.runs.outputs.WorkspaceRef }}
- Workspace ID: ${{ steps.runs.outputs.workspaceId }}
- Workflow URL: ${{ steps.runs.outputs.workflowUrl }}
comment_tag: towerrun
# Capture JSON + logs and save as artifacts
- uses: actions/upload-artifact@v3
if: success() || failure()
with:
name: ${{ needs.getdate.outputs.date }}_run_logs
path: |
tower_action_*.log
tower_action_*.json
# We install the Tower CLI and use the variables to get the details of the pipeline run.
get_details:
runs-on: ubuntu-latest
needs: [run-tower]
steps:
- name: Get run details
run: |
# Install TW CLI
wget -L https://github.com/seqeralabs/tower-cli/releases/download/v0.7.3/tw-0.7.3-linux-x86_64
sudo mv tw-* /usr/local/bin/tw
chmod +x /usr/local/bin/tw
# Use variables with ${{ needs.id.outputs.variable }} syntax
tw -o json runs view \
-w ${{ needs.run-tower.outputs.workspace_id }} \
-i ${{ needs.run-tower.outputs.workflow_id }}The action prints normal stdout info-level log messages to the actions console. However, it also saves a verbose log file and an output JSON of job details as a file. We recommend using actions/upload-artifact in your GitHub Actions workflow as shown in the examples above, this will then expose this file to be used in subsequent jobs and as a download through the workflow summary page.
The output log file is saved as tower_action_$(timestamp).log and can be captured using `actions/upload-artifact using the following settings:
- uses: actions/upload-artifact@v4
with:
name: Seqera Platform debug log file
path: tower_action_*.logThe action writes a JSON file which has the same format as the outputs.json used above. This is wrtten to a file called tower_action_$(uuidgen).json. It can be captured in a similar manner:
- uses: actions/upload-artifact@v4
with:
name: Seqera Platform output JSON file
path: tower_action_*.jsonThis GitHub Action was written by Phil Ewels (@ewels), with help from and based on earlier work by Gisela Gabernet (@ggabernet).