Mean + median read length

[ewels]

FastQC reports a range for read length in Basic Statistics, but it's a common ask to know the mean + median read lengths too. MultiQC tries its best to estimate this, but it'd be better if FastQC could calculate the accurate numbers.

History / context

Back in 2022, @jchorl added some code to the FastQC module in MultiQC that attempts to calculate the median read length.

This works relatively well in most cases, but includes a bit of a fudge:

# if the distribution-entry is a range, we use the average of the range.
# this isn't technically correct, because we can't know what the distribution
# is within that range. Probably good enough though.

I suggested that we try to raise this upstream and get FastQC to generate the metrics as it has the required numbers, but I don't think it ever made it.

Anyway, 4 years later I'm getting reports from folks ('cc @jfy133) that maybe the approach isn't really good enough and it's "generating crazy numbers that do not correspond to anything". I figure better late than never - maybe we can look into getting FastQC to report this.

[jfy133]

Correct: for context, the reason why we (@sofstam and me) found out the 'fudge' code in MultiQC doesn't appear to work when handling very long read/nanopore data it seems:

Screenshots from MultiQC - the General Stats column 'median length' from the FastQC module (not FastQC itself!) is reporting all numbers ending in 999, with one absolutely massive number

But the NANOQ MultiQC (which I guess is similar to FastQC in some ways) reports 'normal' looking number for the Median length, without the massively skewed thing from the one very long read

[s-andrews]

Fastqc calculates a read length distribution as one of its specific analysis modules so there's already a detailed breakdown of the lengths there. The reason for putting the limit values in the overall summary was because the distributions of lengths are often skewed, certainly for short read data, so the mean and median aren't really a useful number.

Mean length is trivial - we report total bases and total sequences already.

Median could be calculated from the length distribution module pretty easily but we'd need a fallback if that was disabled in the config file (it's probably never going to be, but you never know).

[s-andrews]

I've added a quick implementation for this. I don't think it matters if the module is ignored in the report as the metrics look to be calculated anyway.

Not tested beyond a super quick sanity check as yet.

[ewels]

@s-andrews did you mean to remove Sequences flagged as poor quality ?

[s-andrews]

Yes, slightly rash decision but it's something which was specific to very old illumina runs when we used to routinely call all clusters, bypassing the default filters. You can do that still but it is very little used and it doesn't work for other sources of fastq file. I'd taken the option out of the launch script and it probably doesn't need to be in the analysis either.

[ewels]

Ok great 👍🏻 In that case my tidy-up PR is very small: #205