Skip to content

fix(filter): take outFilterMismatchNoverLmax over the mapped length - #252

Open
BenjaminDEMAILLE wants to merge 1 commit into
mainfrom
fix/mismatch-nover-lmax
Open

fix(filter): take outFilterMismatchNoverLmax over the mapped length#252
BenjaminDEMAILLE wants to merge 1 commit into
mainfrom
fix/mismatch-nover-lmax

Conversation

@BenjaminDEMAILLE

Copy link
Copy Markdown
Contributor

Closes #238, which I filed while adding --outFilterMismatchNoverReadLmax and deliberately did not bundle into that PR.

The rule

ReadAlign_mappedFilter.cpp:

} else if ( (trBest->nMM > outFilterMismatchNmaxTotal)
         || (double(trBest->nMM)/double(trBest->rLength) > P.outFilterMismatchNoverLmax) ) {

rLength is the transcript's mapped length. This code divided by the read length, which is the denominator of a different parameter, --outFilterMismatchNoverReadLmax.

Why it matters

The two denominators agree on a fully aligned read and diverge on a soft-clipped one, in the permissive direction: a 50-base alignment of a 100-base read carrying 16 mismatches read as 0.16 rather than 0.32, so it passed the 0.3 default that STAR uses to reject it. The filter was loosest exactly where the alignment was weakest.

What it changes in practice

Measured on the nf-core/rnaseq test data (50 000 pairs, the fixture from #31 and #48): output-neutral. Uniquely mapped 41 665 and multi-mapped 961 with and without the change, because at default settings the absolute --outFilterMismatchNmax cap of 10 rejects those alignments first.

So this is a correctness fix that shows up when that cap is raised, not a change to today's numbers. I would rather say that plainly than imply a measurement it does not have; the yeast faithfulness benchmark is where a difference would surface, and that dataset is not one I can fetch here.

Tests

The ratio is now a named function instead of an expression inside the retain closure, so it is testable: the 50-of-100-bases case above, a two-exon transcript where the mapped length is the sum of the exons rather than the genomic span, and a transcript with no exons that must not divide by zero.

cargo test, cargo clippy --all-targets -- -D warnings on a cold cache, and cargo fmt --check are green.

🤖 Generated with Claude Code

STAR divides the mismatch count by the transcript's mapped length
(trBest->rLength in ReadAlign_mappedFilter.cpp). This divided by the read
length, which is the denominator of a different parameter,
outFilterMismatchNoverReadLmax.

The two agree on a fully aligned read and diverge on a soft-clipped one, in
the permissive direction: a 50-base alignment of a 100-base read carrying 16
mismatches read as 0.16 rather than as 0.32, so it passed the 0.3 default that
STAR uses to reject it. The filter was loosest exactly where the alignment was
weakest.

Measured on the nf-core/rnaseq test data (50 000 pairs), the change is
output-neutral: uniquely mapped 41665 and multi-mapped 961 either way, because
the absolute --outFilterMismatchNmax cap of 10 dominates at default settings.
It bites when that cap is raised, which is what the unit tests cover.

The ratio is now a named function rather than an expression inside the retain
closure, so it can be tested directly.

Closes #238.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

outFilterMismatchNoverLmax is applied to the read length, not the mapped length

1 participant