Calloc error with cactus_consolidated

[mufernando]

Hi,

First, thanks for developing this great software!

Over the past week I have been running into this error in the cactus-align step:

[2026-04-12T16:22:18-0700] [MainThread] [I] [toil.leader] Issued job 'cactus_cons' kind-cactus_cons/instance-dlauexhg v1 with job batch system ID: 8 and disk: 31.8 Gi, memory: 364.9 Gi, cores: 44, accelerators: [], preemptible: False
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] 2026-04-12 16:22:26.088363: Running the command: "cactus_consolidated --seqFile /tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.seqfile --logLevel INFO --alignments /tmp/toilwf-5fd68012247f5d71b9
7bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1_primary.paf --params /tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.config.xml --outputFile /tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.c2h --outputHalFastaFile /tmp/toilwf-5fd6801
2247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.c2h.fa --outputReferenceFile /tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.ref --outgroupEvents Strix_varia Strix_aluco Strix_nebulosa --referenceEvent Anc1 --threads 44"
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Params file: /tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.config.xml
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Output file string : /tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.c2h
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Output hal fasta file string : /tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.c2h.fa
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Output reference fasta file string : /tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.ref
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Sequence files and events: (null)
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Alignments file: /tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1_primary.paf
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Secondary alignments file: (null)
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Constraint alignments file: (null)
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Species tree: (null)
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Outgroup events: Strix_varia Strix_aluco Strix_nebulosa
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Reference event: Anc1
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Loaded the parameters files, 0 seconds have elapsed
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Set up the cactus disk, 0 seconds have elapsed
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Constructed the first flower
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Going to build the event tree with newick string: ((Lophostrix_cristata:0.00958,Strix_seloputo:0.0089)Anc1:0.012074999999999999,((Strix_nebulosa:0.00612,Strix_aluco:0.0057)Anc3:0.00105,Strix_
varia:0.006059999999999999)Anc2:0.002495)Anc0;
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Parsed the tree
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Constructed the basic event tree
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Assigning sequence Strix_seloputo.fa to Strix_seloputo
[2026-04-12T16:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Processing file: Strix_seloputo.fa
[2026-04-12T16:22:34-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Assigning sequence Lophostrix_cristata.fa to Lophostrix_cristata
[2026-04-12T16:22:34-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Processing file: Lophostrix_cristata.fa
[2026-04-12T16:22:41-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Assigning sequence Strix_nebulosa.fa to Strix_nebulosa
[2026-04-12T16:22:41-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Processing file: Strix_nebulosa.fa
[2026-04-12T16:22:48-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Assigning sequence Strix_aluco.fa to Strix_aluco
[2026-04-12T16:22:48-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Processing file: Strix_aluco.fa
[2026-04-12T16:22:56-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Assigning sequence Strix_varia.fa to Strix_varia
[2026-04-12T16:22:56-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Processing file: Strix_varia.fa
[2026-04-12T16:23:04-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Constructed the initial flower with 129274 sequences and 10 events with string: (((Lophostrix_cristata:0.00958,Strix_seloputo:0.0089)Anc1:0.012075,((Strix_nebulosa:0.00612,Strix_aluco:0.0057)
Anc3:0.00105,Strix_varia:0.00606)Anc2:0.002495)Anc0:9.22337e+18)ROOT:9.22337e+18;
[2026-04-12T16:23:04-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Established the first Flower in the hierarchy, 38 seconds have elapsed
[2026-04-12T16:23:06-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Converted alignment coordinates, 40 seconds have elapsed
[2026-04-12T16:23:06-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Stripped any unique IDs, 40 seconds have elapsed
[2026-04-12T16:23:06-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): We found a max path distance between ingroups in the tree ((((Lophostrix_cristata:0.00958,Strix_seloputo:0.0089)Anc1:0.012075,((Strix_nebulosa:0.00612,Strix_aluco:0.0057)Anc3:0.00105,Strix_va
ria:0.00606)Anc2:0.002495)Anc0:9.22337e+18)ROOT:9.22337e+18;) of 0.018480, giving us and min final chain length of: 2048
[2026-04-12T16:23:06-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Starting annealing round with a minimum chain length of 2048 and an alignment trim of 3
[2026-04-12T16:23:37-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Sequence graph statistics after annealing:
[2026-04-12T16:23:52-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): There were 7572752 blocks in the sequence graph, representing 3647262476 total aligned bases
[2026-04-12T16:23:53-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Block degree stats: min 1, avg 4.422241, median 3, max 177250
[2026-04-12T16:23:53-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Block support stats: min 0.000000, avg 0.778192, median 0.750000, max 2.000000
[2026-04-12T16:23:55-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Starting melting round with a minimum chain length of 2
[2026-04-12T16:24:55-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): A melting round is destroying 313862 blocks with an average degree of 21.548031 from chains with length less than 2. Total aligned bases lost: 6763108
[2026-04-12T16:25:05-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Starting melting round with a minimum chain length of 32
[2026-04-12T16:26:04-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): A melting round is destroying 3294472 blocks with an average degree of 4.003317 from chains with length less than 32. Total aligned bases lost: 74347112
[2026-04-12T16:26:13-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Starting melting round with a minimum chain length of 256
[2026-04-12T16:26:52-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): A melting round is destroying 185355 blocks with an average degree of 8.650018 from chains with length less than 256. Total aligned bases lost: 41275626
[2026-04-12T16:27:44-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): A melting round is destroying 294512 blocks with an average degree of 5.310582 from chains with length less than 2048. Total aligned bases lost: 111320641
[2026-04-12T16:27:52-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): ERROR: Calloc failed with request for 1 lots of 56 bytes: Cannot allocate memory
[2026-04-12T16:27:52-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc1): Command exited with non-zero status 1
[2026-04-12T16:27:52-0700] [Thread-1 (daddy)] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker cactus_cons file:/tmp/Anc1-align kind-cactus_cons/instance-dlauexhg.
[2026-04-12T16:27:52-0700] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'cactus_cons' kind-cactus_cons/instance-dlauexhg v1
Exit reason: None
[2026-04-12T16:27:52-0700] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'cactus_cons' kind-cactus_cons/instance-dlauexhg v2
[2026-04-12T16:27:52-0700] [MainThread] [W] [toil.leader] Log from job "kind-cactus_cons/instance-dlauexhg" follows:
=========>
        [2026-04-12T16:22:18-0700] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG---
        [2026-04-12T16:22:18-0700] [MainThread] [I] [toil] Running Toil version 9.1.2-355b72ba0e425619c0367562caf2a337078dba65 on host cnode-12-14.local.
        [2026-04-12T16:22:18-0700] [MainThread] [I] [toil.worker] Working on job 'cactus_cons' kind-cactus_cons/instance-dlauexhg v1
        [2026-04-12T16:22:25-0700] [MainThread] [I] [toil.worker] Loaded body Job('cactus_cons' kind-cactus_cons/instance-dlauexhg v1) from description 'cactus_cons' kind-cactus_cons/instance-dlauexhg v1
        [2026-04-12T16:22:25-0700] [MainThread] [W] [toil.statsAndLogging] Deprecated toil method that will be moved/replaced in a future release."
        [2026-04-12T16:22:26-0700] [MainThread] [W] [root] Deprecated toil method.  Please call "logging.getLevelName" directly.
        [2026-04-12T16:22:26-0700] [MainThread] [I] [cactus.shared.common] Running the command ['cactus_consolidated', '--seqFile', '/tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.seqfile', '--logLevel', 'INFO', '--alignments', '/tmp/toilwf-5fd6801
2247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1_primary.paf', '--params', '/tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.config.xml', '--outputFile', '/tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.c2h', '--outputHalFas
taFile', '/tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.c2h.fa', '--outputReferenceFile', '/tmp/toilwf-5fd68012247f5d71b97bfcffd3e9797f/8487/job/tmpzu1d2bwj/Anc1.ref', '--outgroupEvents', 'Strix_varia Strix_aluco Strix_nebulosa', '--referenceEvent
', 'Anc1', '--threads', '44']

I have tried increasing RAM or decreasing number of cores, but it did not help. I am using cactus v3.1.4 (non-gpu version).

Thanks again!

Murillo

[glennhickey]

The key part of the message is

ERROR: Calloc failed with request for 1 lots of 56 bytes: Cannot allocate memory

which is almost certainly due to it running out of memory. It's at a part of the code (CAF) where the number of cores doesn't take effect. How much RAM do you have?

[mufernando]

Thanks for the prompt response! I am giving it 450G of RAM. I tried increasing it up to 550G and it didn't help.

I am thinking that one of my assemblies might be trash? I soft masked repeats using repeatmasker, but I didn't have a group-specific repeat library. The genomes are not that big, around 1.5Gbp

FYI This is the command that I ran:

026-04-12 16:21:23 - RULE align-Anc1 - INFO - runCommand0 - Checking for tmp dir: /home/exacloud/gscratch/WallLab/rodrigmu/tmp/Anc1-align - False
2026-04-12 16:21:23 - RULE align-Anc1 - INFO - runCommand1 - Running command: singularity exec --cleanenv --bind /home/exacloud/gscratch/WallLab/rodrigmu/tmp:/tmp /home/exacloud/gscratch/ConradLab/rodrigmu/cactus-snakemake/cactus_v3.1.4.sif cactus-align /tmp/Anc1-align /h
ome/exacloud/gscratch/ConradLab/rodrigmu/owl_wga/cactus_output/cactus_input_minimal.txt /home/exacloud/gscratch/ConradLab/rodrigmu/owl_wga/cactus_output/Anc1.paf /home/exacloud/gscratch/ConradLab/rodrigmu/owl_wga/cactus_output/Anc1.hal --root Anc1 --logInfo --retryCount 0
 --maxCores 44 --maxMemory 450G

[mufernando]

I ran another trial overnight giving the job 1Tb of RAM and it is still erroring out. I am now testing an older version of cactus (2.9.9) to see if I get the same error.

[mufernando]

Hi, @glennhickey,

Over the past weeks I tried:

• Increasing the RAM to a total of 1Tb
• Using an older version of cactus (2.9.9)
• Trimming 1Kb contigs or shorter
• Subset genomes with high N50 (>50Mb)
• Now I am running a new try with the subset of high N50 genomes + removing contigs smaller than 1Mb

Any insight you can give I'll take...

Thanks,

[glennhickey]

Is the log identical when you have 1TB of ram? Or is it getting any further? Are you able to see with top or something like that how much memory it's using when it dies? How big is the input PAF file? Can yo u share the inputs of your failing command?

[mufernando]

Hi @glennhickey,

Thanks for the quick response.

It seems to be getting further than before:

2026-04-20 06:36:59 - RULE align-Anc4 - INFO - runCommand0 - Checking for tmp dir: /home/exacloud/gscratch/ConradLab/rodrigmu/tmp/Anc4-align - False
2026-04-20 06:36:59 - RULE align-Anc4 - INFO - runCommand1 - Running command: singularity exec --cleanenv --bind /home/exacloud/gscratch/ConradLab/rodrigmu/tmp:/tmp /home/exacloud/gscratch/ConradLab/rodrigmu/cactus-snakemake/cactus_v3.1.4.sif cactus-align /tmp/Anc4-align /home/exacloud/gscratch/ConradLab/rodrigmu/owl_wga/cactus_output/cactus_input_HQ_min.txt /home/exacloud/gscratch/ConradLab/rodrigmu/owl_wga/cactus_output/Anc4.paf /home/exacloud/gscratch/ConradLab/rodrigmu/owl_wga/cactus_output/Anc4.hal --root Anc4 --logInfo --retryCount 0 --maxCores 24 --maxMemory 1000G
-------------------- COMMAND LOG BEGIN --------------------
...
...
...
[2026-04-20T06:42:44-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Attaching the sequence to the cactus root 1914, header JASWHU010000061.1 with length 1201027 and 2 total bases aligned and 0 bases aligned to other chromosome threads
[2026-04-20T06:42:44-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Attaching the sequence to the cactus root 336, header CBDBYL010000535.1 with length 2457639 and 2 total bases aligned and 0 bases aligned to other chromosome threads
[2026-04-20T06:42:44-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Attaching the sequence to the cactus root 1932, header JASWHU010000064.1 with length 1128768 and 2 total bases aligned and 0 bases aligned to other chromosome threads
[2026-04-20T06:42:44-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Attaching the sequence to the cactus root 444, header CBDBYL010000836.1 with length 1372480 and 2 total bases aligned and 0 bases aligned to other chromosome threads
[2026-04-20T06:42:44-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Attaching the sequence to the cactus root 294, header CBDBYL010000417.1 with length 5265043 and 2 total bases aligned and 0 bases aligned to other chromosome threads
[2026-04-20T06:42:44-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Attaching the sequence to the cactus root 174, header OZ243526.1 with length 8536262 and 2 total bases aligned and 0 bases aligned to other chromosome threads
[2026-04-20T06:44:01-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Ran cactus caf, 360 seconds have elapsed
[2026-04-20T06:44:24-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Ran extended flowers ready for bar on 1896695 flowers, 383 seconds have elapsed
[2026-04-20T07:22:22-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Ran cactus bar (use poa:1), 2661 seconds have elapsed
[2026-04-20T07:22:25-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): There are 7 layers in the flowers hierarchy
[2026-04-20T07:22:25-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): In the 0 layer there are 1 flowers in the flowers hierarchy
[2026-04-20T07:22:26-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Building a matching for 78 stub nodes in the top level problem from 656 total stubs of which 276 attached , 3709514 total ends, 3179 chains, 1854429 blocks 1851846 groups and 328 sequences
[2026-04-20T07:22:51-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): In the 1 layer there are 1851846 flowers in the flowers hierarchy
[2026-04-20T07:23:00-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): In the 2 layer there are 1304366 flowers in the flowers hierarchy
[2026-04-20T07:23:17-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): In the 3 layer there are 40702 flowers in the flowers hierarchy
[2026-04-20T07:23:18-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): In the 4 layer there are 2132 flowers in the flowers hierarchy
[2026-04-20T07:23:18-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): In the 5 layer there are 138 flowers in the flowers hierarchy
[2026-04-20T07:23:18-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): In the 6 layer there are 26 flowers in the flowers hierarchy
[2026-04-20T07:23:18-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Ran cactus make reference, 2717 seconds have elapsed
[2026-04-20T07:27:16-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Ran cactus make reference bottom up coordinates, 2955 seconds have elapsed
[2026-04-20T07:27:25-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Ran cactus make reference top down coordinates, 2964 seconds have elapsed
[2026-04-20T07:27:37-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): cactus_consolidated(+0x63c12)[0x55cd74230c12]
[2026-04-20T07:27:37-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): cactus_consolidated(+0x63d01)[0x55cd74230d01]
[2026-04-20T07:27:37-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): cactus_consolidated(+0x6ab1f)[0x55cd74237b1f]
[2026-04-20T07:27:37-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): cactus_consolidated(+0x1122d)[0x55cd741de22d]
[2026-04-20T07:27:37-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): cactus_consolidated(+0x80dc)[0x55cd741d50dc]
[2026-04-20T07:27:37-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): cactus_consolidated(+0x5e9c)[0x55cd741d2e9c]
[2026-04-20T07:27:37-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f5e8cc84d90]
[2026-04-20T07:27:37-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f5e8cc84e40]
[2026-04-20T07:27:37-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): cactus_consolidated(+0x6355)[0x55cd741d3355]
[2026-04-20T07:27:37-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): ERROR: Malloc failed with a request for: 1138269 bytes: Cannot allocate memory
[2026-04-20T07:27:38-0700] [MainThread] [I] [toil-rt] cactus_consolidated(Anc4): Command exited with non-zero status 1

The input PAF file is 200M. I was able to see how much ram the SLURM process used with seff and it says it only used ~30Gb out of 1Tb, though I would take this RAM usage estimate with a grain of salt.

What inputs do you need exactly? I can share the PAF and fastas no problem.

Thanks,

Murillo

[glennhickey]

Too bad, it almost finished on that last run! It's clearly running out of memory. Given the size of the input and the fast runtime, I'd be extremely shocked if it's actually getting up to 1 TB (the 30Gb slurm is reporting seems more likely). If you're sure it is using that much memory and want to share every input file to cactus-align with me (seqfile, paf, fastas) I can try to reproduce here.