feat(bam): --outBAMsortingBinsN, spilling the coordinate sort to disk bins

[BenjaminDEMAILLE]

Six commits. The first five are the existing bd/bam-sort-bins work, which was finished and pushed but had no PR open. The sixth fixes a gap I found while validating it.

The feature

--limitBAMsortRAM was a threshold the sort died on rather than a bound it respected: every record stayed resident until finish(), and exceeding the limit aborted the run telling the user to raise it or give up on sorting. This partitions records by reference sequence into --outBAMsortingBinsN temporary BAMs, then reads them back one bin at a time. The bins are coordinate-disjoint and already ordered relative to each other, so no k-way merge is needed.

The gap, and the fix

Spilling only happened from finish(). write_batch accumulated every record with no bound check, so by the time the first bin was written the whole run was already resident and the peak had been reached. The bound was respected during the sort and nowhere else, which is not where the memory goes.

Measured, 560k records, yeast (17 references), --limitBAMsortRAM 100000000:

	peak RSS
in-memory sort	1980 MB
spill at finish	1967 MB

The feature was buying nothing. The last commit opens the bins on the first crossing of the bound and pushes the buffer out as batches arrive:

records	spill at finish	incremental
560k	1967 MB	1744 MB
1.3M	2925 MB	2254 MB

The gap widens with record count, which is the property that was missing: the buffer no longer scales with the run. Peak is not flat, and the commit says so: the index, the alignment pipeline's own batches, and the allocator's retained pages sit underneath it and are untouched by this.

Caveat worth stating

Binning is by reference sequence, so the benefit scales with reference count. On a single-contig genome every mapped record lands in one bin and this buys nothing (that is how I first mis-measured it, on a 1-chromosome test index). Human, with its 24+ references, is the case it is for.

Correctness

• Default path (no bins) byte-identical to current main.
• Binned path: 561,312 decoded records identical to the in-memory sort.
• Old binary aborts on --limitBAMsortRAM 100000000 for this input; this completes.
• Tests pass, 0 clippy warnings, cargo fmt --check clean.

Independent of #256 / #257 / #261; touches only src/io/bam.rs and src/params/mod.rs.

🤖 Generated with Claude Code

[BenjaminDEMAILLE]

Closing this as a duplicate of #214, which I should have found before opening it.

#214 has been open since 2026-08-06 and solves the same problem better:

	#214 (external merge)	this PR (per-reference bins)
peak memory	flat in output size	still grows: 1744 MB at 560k records, 2254 MB at 1.3M
single-contig genome	works	no benefit — every record lands in one bin
bin/run count	balanced merge passes, guards against EMFILE	bounded by reference count
tie order	stable merge on run index	stable within bin

The measurement I took here is the one useful thing to carry over: the existing bd/bam-sort-bins work enforced the bound only in finish(), so write_batch accumulated the whole run first and the peak had already been reached. In-memory sort 1980 MB vs spill-at-finish 1967 MB, i.e. it bought nothing. Whatever lands, the bound has to be enforced while records arrive, not after. #214 does that by construction.

Also worth noting for #214: --limitBAMsortRAM 0 changing from unlimited to 512 MiB is a behaviour change against STAR, where 0 means unlimited. Worth being deliberate about, since ruSTAR is a faithful port.

Sorry for the noise.