From b47d52573ca837abae14fb9b4b3906196743a74d Mon Sep 17 00:00:00 2001 From: Benjamin Demaille Date: Thu, 27 Aug 2026 21:59:42 +0200 Subject: [PATCH] docs(roadmap): record where the port stands, measured The roadmap tracked phases; it did not say how close the output actually is, and the figures quoted elsewhere came from runs nobody could reproduce. Two commands now produce these numbers, and both are in the tree: test/yeast_tier.sh and test/nfcore_diff.py. Yeast tier, 50 000 pairs: 98.658% of mates at the same chromosome, position and CIGAR, 99.964% with the same NH. Of the 1 127 differences, 1 113 are multi-mappers whose primary differs, and 1 110 of those have an identical set of loci on both sides with an identical primary score, which makes them the documented tie-break divergence rather than a gap. That leaves about 15 mates in 84 006, all of them window-formation or stitch-preference differences. Each is named with its CIGAR on both sides, and for the largest one the cause is identified: rerunning it with --alignIntronMax 1000000 reproduces STAR's alignment exactly, so the binning that follows from alignIntronMax=0 is what keeps the two pieces apart. --- ROADMAP.md | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/ROADMAP.md b/ROADMAP.md index d419f612..b7c47c1e 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -348,3 +348,63 @@ Single-cell quantification layered around the existing aligner: the cDNA read al **Phase 14.5–14.11 + performance** (2026-07): completed the feature-parity set — `Summary.csv` (STARsolo-faithful, CellRanger funnel split to its own file), `--soloCellFilter` CellRanger2.2/TopCells/**EmptyDrops_CR** (Monte-Carlo ambient rescue in the `filtered/` writer), `--soloFeatures` **GeneFull/SJ/Velocyto** (spliced/unspliced/ambiguous per Sullivan 2025), `--soloMultiMappers` Uniform/PropUnique/EM/Rescue, chemistries **CB_UMI_Complex** (multi-segment) and **SmartSeq** (plate-based, SE + PE fragment counts), and a rustar-vs-STARsolo SJ + multi-mapper diff harness. Performance: pipelined solo FASTQ decode, parallelized matrix build + EmptyDrops MC, libdeflate/zlib-rs for matrix gzip + BGZF, and an **O(log n + k) segment-tree gene-overlap query** (replacing STAR's linear scan — the #1 solo hotspot, ~14% wall reduction). Sparse suffix array (`--genomeSAsparseD`, byte-identical to STAR's D=2) for a 31% smaller index. 516 tests, 0 clippy warnings. **Native three-way benchmark** (2026-07, `test/aws/`): fresh single-instance EC2 comparison on a real 10x dataset (`5k_Mouse_PBMCs_5p_gem-x_GEX`, 5′ GEM-X, GRCm39-2024-A), all native x86_64, 10 threads, NVMe, page cache dropped, no BAM. Wall / peak RSS / cells: **STARsolo 2.7.11b** 87 s / 28.3 GB / 4,061; **rustar-aligner** 121 s / 25.7 GB / 3,689 (→ ~105 s with the segment-tree query merged after this run); **rustar `--genomeSAsparseD 2`** 119 s / **17.7 GB** / 3,692; **CellRanger 10.0.0** 347 s / 13.1 GB / 3,858. `Gene/raw` matrix byte-identical to STARsolo's. Supersedes the earlier Docker-emulation numbers above (those were penalized by Rosetta/virtiofs). Remaining gap to STARsolo is small and output-identical; rustar owns the memory frontier via sparse SA. + +--- + +## Where the port actually stands (2026-08-27, measured) + +Numbers below come from two runs anyone can reproduce, not from recollection: +`test/yeast_tier.sh` (ERR12389696 against the R64-1-1 genome, no annotation) +and `test/nfcore_diff.py` (the nf-core/rnaseq test data). Both compare against +STAR 2.7.11b on the same inputs. + +### Yeast tier, 50 000 pairs, 84 006 mates compared + +| | count | share | +|---|---|---| +| same chromosome, position and CIGAR | 82 879 | 98.658% | +| same NH | 83 976 | 99.964% | +| mates only in STAR's output | 8 | | +| mates only in rustar-aligner's output | 2 | | + +Of the 1 127 mates placed differently, **1 113 are multi-mappers whose primary +differs**, and those are ties rather than disagreements: 1 110 of them have an +identical set of loci on both sides, and 1 108 have an identical primary +alignment score. That is the documented tie-break divergence +([DIVERGENCE.md](DIVERGENCE.md) §1.1), not a faithfulness gap. + +What remains after excluding ties is **about 15 mates in 84 006**, in three +shapes: + +1. **STAR splices where we soft-clip** (the largest group). `ERR12389696.13842` + mate2: STAR `143M831334N7M` at AS 291, rustar `144M6S` at AS 288. Running + the same read with `--alignIntronMax 1000000` produces STAR's alignment + exactly, so the cause is window binning, not scoring: with + `alignIntronMax=0` the bin width stays at `2^winBinNbits`, and the two + pieces land in windows too far apart to stitch. STAR reaches it anyway. +2. **The reverse**: `ERR12389696.15864` is a pair only rustar reports, both + mates spliced across ~420 kb. +3. **A different stitch of the same locus**: `ERR12389696.27612` mate2, STAR + `32M288N97M21S` at AS 252 against rustar `129M21S` at AS 250 — the spliced + alternative exists on mate1 and is chosen there, but not on mate2. + +### nf-core/rnaseq test data, 50 000 pairs + +| | STAR | rustar-aligner | +|---|---|---| +| uniquely mapped | 41 691 | 41 684 | +| multi-mapped | 934 | 942 | +| deepest multimapper (NH) | 14 | 14 | +| unmapped: too short | 3 766 | 3 778 | +| unmapped: other | 3 609 | 3 596 | + +Both remaining gaps on this dataset were bugs found from user reports and are +fixed: the unmapped-reason split (#48) and the multimapper depth (#31). + +### What "finished" needs, from here + +- The ten themes of #143, most of which have a rebased pull request waiting. +- The 19 STAR parameters still in `NOT_YET_ACCEPTED`, every one of which is + implemented by one of those pull requests. +- The ~15 non-tie mates above, which are all window-formation or + stitch-preference differences, i.e. the P1 theme rather than new features.