Asked copilot based on the snakemake repo how it handles provenance records: provenance.md. Note that snakemake deposits "provenance" info about output files to help it decide when a file needs to be regenerated. Thus, much of the provenance info is not very useful to users of DLN. It also may lack information we would be interested in.
Snakemake keeps records in .snakemake/metadata and .snakemake/incomplete of files depending on their status. Looks like three statuses: incomplete with no job, incomplete with job, and complete.
SQLite seems best since it can utilize a shared filesystem like Turbo instead of requiring network connectivity between the HPC and the DLN server
Asked copilot based on the snakemake repo how it handles provenance records: provenance.md. Note that snakemake deposits "provenance" info about output files to help it decide when a file needs to be regenerated. Thus, much of the provenance info is not very useful to users of DLN. It also may lack information we would be interested in.
Snakemake keeps records in
.snakemake/metadataand.snakemake/incompleteof files depending on their status. Looks like three statuses: incomplete with no job, incomplete with job, and complete.SQLite seems best since it can utilize a shared filesystem like Turbo instead of requiring network connectivity between the HPC and the DLN server