SLURM doesn't explicity create DAGs. But the dagger syntax could be used to generate SBATCH files, and define job dependencies like afterok and afterany etc.
Is it worthwhile using the slurmpy/pyslurm Python bindings? Or using Python to just write the batch headers at the top of the script?
Some thought needs to be given to how to account for the SBATCH vs executed script flexibility. They can all be in the same script, or the sbatch could call into another script - we want to support both if we implement this.
SLURM doesn't explicity create DAGs. But the dagger syntax could be used to generate
SBATCHfiles, and define job dependencies likeafterokandafteranyetc.Is it worthwhile using the
slurmpy/pyslurmPython bindings? Or using Python to just write the batch headers at the top of the script?Some thought needs to be given to how to account for the SBATCH vs executed script flexibility. They can all be in the same script, or the sbatch could call into another script - we want to support both if we implement this.