Updates to ONT project summary reports by almahans · Pull Request #175 · NationalGenomicsInfrastructure/ngi_reports

almahans · 2025-11-11T14:49:22Z

Added read counts and avg. read length per sample
Added sequencing method to FC table
Added separate file showing samples sequenced per FC

Added sample read counts and samples sequenced for FCs

Added the average read length per sample and fixed the units for total read counts

Set option --skip_fastq to True for ONT runs

Calculate avg. read length by fetching FC sequenced for each sample. Also add a separate table for FC-sample info, and add Lib prep to the library info table

Change prep.label to prep ID to display it in all cases, and remove the space from "Flow cell" in the headers to make the collapse/expand function work.

Change where the stats are taken from when splitting by barcode

Added seq. method to flowcell info table, and remove the FC-sample table from report to keep only in a separate file

Add a warning for cases where there is no way to assign reads to samples due to barcode naming.

Changed the searching for barcodes to look for only argument --split_files_by_barcode

Update version number and versionlog

aanil · 2025-11-12T13:09:13Z

ngi_reports/utils/entities.py

+                f"Flowcell {self.run_name} has no LIMS information, please check and amend report manually"
+            )
+            self.exclude = True
+        else:


If you have a return instead of else, you could avoid one level of indentation below

aanil · 2025-11-12T13:14:08Z

ngi_reports/utils/entities.py

+                .get("sample_data", [])
+            )
+            self.fc_sample_barcodes = {}
+            for lims_sample in lims_samples:


We loop through lims_samples 3 times here and below. Its better to minimise the number of times we loop through the same variable

We only ever seem to use the sample name, we could just have a list of sample names instead.

aanil · 2025-11-12T13:17:25Z

ngi_reports/utils/entities.py

+                "user_specified_flow_cell_id"
+            )
+            run_arguments = fc_runparameters.get("args")
+            for arg in run_arguments:


If run_arguments is a list, we don't need to loop through it, we can just check "min_qscore" in run_arguments

aanil · 2025-11-12T13:22:18Z

ngi_reports/utils/entities.py

+            for arg in run_arguments:
+                if "--split_files_by_barcode=on" in arg:


Same here, you don't need to go through every argument if run_arguments is a list

aanil · 2025-11-12T13:25:42Z

ngi_reports/utils/entities.py

+                        self.sample_reads[sample_id] = float(
+                            final_acquisition.get("acquisition_run_info")
+                            .get("yield_summary")
+                            .get("basecalled_pass_read_count")


This calculation does not need to be in the loop

aanil · 2025-11-12T13:26:30Z

ngi_reports/utils/entities.py

+                                    self.sample_reads[sample_id] = float(
+                                        barcode.get("snapshots")[-1]
+                                        .get("yield_summary")
+                                        .get("basecalled_pass_read_count")
+                                    )


This calculation does not need to be in the loop

Oh wait, strike that, you use the barcode here

aanil · 2025-11-12T14:19:03Z

ngi_reports/utils/entities.py

+                fcObj.populate_ont_flowcell(log)
+                if fcObj.exclude:
+                    continue
+                else:


You don't really need the else here

Small changes in indentations and looping

ngi_reports/utils/entities.py

Co-authored-by: Anandashankar Anil <aanil@users.noreply.github.com>

aanil · 2025-11-26T12:49:14Z

ngi_reports/utils/entities.py

                self.qual_threshold = float(arg.split("=")[-1])
+            if "split_files_by_barcode" in arg:
+                split_files_by_barcode = arg.split("=")[-1]
+                self.qual_threshold = float(arg.split("=")[-1])


Suggested change

self.qual_threshold = float(arg.split("=")[-1])

ngi_reports/utils/entities.py

aanil · 2025-11-26T13:00:53Z

ngi_reports/utils/entities.py

+            sample_id = lims_sample.get("sample_name", "")
+            self.fc_sample_barcodes[sample_id] = lims_sample.get(
+                "ont_barcode", "NoIndex"
+            )
+        self.samples_run = []
+        for sample in self.fc_sample_barcodes.keys():
+            self.samples_run.append(f"{sample}")
+        self.samples_run = ", ".join(self.samples_run)
+
+        self.sample_reads = {}
+        self.average_read_length_passed = {}
+        fc_barcode_info = final_acquisition.get("acquisition_output")[1]
+
+        if "--split_files_by_barcode=on" in run_arguments:
+            for barcode in fc_barcode_info.get("plot")[0].get("snapshots"):
+                barcode_name = barcode.get("filtering")[0].get("barcode_name")
+                barcode_alias = barcode.get("filtering")[0].get("barcode_alias")
+                if barcode_name != barcode_alias:
+                    for lims_sample in lims_samples:
+                        sample_id = lims_sample.get("sample_name", "")
+                        if sample_id == barcode_alias:
+                            self.sample_reads[sample_id] = float(
+                                barcode.get("snapshots")[-1]
+                                .get("yield_summary")
+                                .get("basecalled_pass_read_count")
+                            )
+                            self.average_read_length_passed[sample_id] = (
+                                float(
+                                    barcode.get("snapshots")[-1]
+                                    .get("yield_summary")
+                                    .get("basecalled_pass_bases")
+                                )
+                                / self.sample_reads[sample_id]
+                            )
+            if self.sample_reads == {}:
+                log.warning(
+                    f"Flowcell {self.run_name} has no barcode aliases corresponding to sample IDs."
+                )
+
+        elif "--split_files_by_barcode=off" in run_arguments:
+            for lims_sample in lims_samples:
+                sample_id = lims_sample.get("sample_name", "")
+                self.sample_reads[sample_id] = float(
+                    final_acquisition.get("acquisition_run_info")
+                    .get("yield_summary")
+                    .get("basecalled_pass_read_count")
+                )
+                self.average_read_length_passed[sample_id] = round(
+                    float(
+                        final_acquisition.get("acquisition_run_info")
+                        .get("yield_summary")
+                        .get("basecalled_pass_bases")
+                    )
+                    / self.sample_reads[sample_id]
+                )


Suggested change

sample_id = lims_sample.get("sample_name", "")

self.fc_sample_barcodes[sample_id] = lims_sample.get(

"ont_barcode", "NoIndex"

)

self.samples_run = []

for sample in self.fc_sample_barcodes.keys():

self.samples_run.append(f"{sample}")

self.samples_run = ", ".join(self.samples_run)

self.sample_reads = {}

self.average_read_length_passed = {}

fc_barcode_info = final_acquisition.get("acquisition_output")[1]

if "--split_files_by_barcode=on" in run_arguments:

for barcode in fc_barcode_info.get("plot")[0].get("snapshots"):

barcode_name = barcode.get("filtering")[0].get("barcode_name")

barcode_alias = barcode.get("filtering")[0].get("barcode_alias")

if barcode_name != barcode_alias:

for lims_sample in lims_samples:

sample_id = lims_sample.get("sample_name", "")

if sample_id == barcode_alias:

self.sample_reads[sample_id] = float(

barcode.get("snapshots")[-1]

.get("yield_summary")

.get("basecalled_pass_read_count")

)

self.average_read_length_passed[sample_id] = (

float(

barcode.get("snapshots")[-1]

.get("yield_summary")

.get("basecalled_pass_bases")

)

/ self.sample_reads[sample_id]

)

if self.sample_reads == {}:

log.warning(

f"Flowcell {self.run_name} has no barcode aliases corresponding to sample IDs."

)

elif "--split_files_by_barcode=off" in run_arguments:

for lims_sample in lims_samples:

sample_id = lims_sample.get("sample_name", "")

self.sample_reads[sample_id] = float(

final_acquisition.get("acquisition_run_info")

.get("yield_summary")

.get("basecalled_pass_read_count")

)

self.average_read_length_passed[sample_id] = round(

float(

final_acquisition.get("acquisition_run_info")

.get("yield_summary")

.get("basecalled_pass_bases")

)

/ self.sample_reads[sample_id]

)

aanil · 2025-11-26T13:02:53Z

ngi_reports/utils/entities.py

-
        self.lanes = OrderedDict()
        self.fc_sample_qvalues = defaultdict(dict)
+        self.exclude = False


Suggested change

self.exclude = False

aanil · 2025-11-26T13:03:03Z

ngi_reports/utils/entities.py

+            log.warning(
+                f"Flowcell {self.run_name} has no LIMS information, please check and amend report manually"
+            )
+            self.exclude = True


Suggested change

self.exclude = True

aanil · 2025-11-26T13:03:37Z

ngi_reports/utils/entities.py

                fcObj = Flowcell(fc, self.ngi_name, ontcon)
-                fcObj.populate_ont_flowcell()
+                fcObj.populate_ont_flowcell(log)
+                if fcObj.exclude:


Suggested change

if fcObj.exclude:

if not fcObj:

change from FC.exclude to return values for FC without LIMS information

ngi_reports/utils/entities.py

ngi_reports/reports/ont_project_summary.py

aanil

👍

Co-authored-by: Anandashankar Anil <aanil@users.noreply.github.com>

almahans added 12 commits October 15, 2025 09:11

sample read counts & FC samples

7924b03

Added sample read counts and samples sequenced for FCs

Add read length and read count units

9b87b79

Added the average read length per sample and fixed the units for total read counts

set skip fastq true for ONT

bd7d7d1

Set option --skip_fastq to True for ONT runs

Add read length and FC-sample info table

820123e

Calculate avg. read length by fetching FC sequenced for each sample. Also add a separate table for FC-sample info, and add Lib prep to the library info table

Fix collapsible FC tables

0619743

Change prep.label to prep ID to display it in all cases, and remove the space from "Flow cell" in the headers to make the collapse/expand function work.

Change view for barcode stats

b6e7c7f

Change where the stats are taken from when splitting by barcode

Add seq method

10b9262

Added seq. method to flowcell info table, and remove the FC-sample table from report to keep only in a separate file

Merge branch 'NationalGenomicsInfrastructure:master' into master

ba2daa5

add warning for barcodes

ce92a99

Add a warning for cases where there is no way to assign reads to samples due to barcode naming.

Merge branch 'master' of https://github.com/almahans/ngi_reports

29f84bc

updated barcode search

f7f2d7c

Changed the searching for barcodes to look for only argument --split_files_by_barcode

Update version

f14d09b

Update version number and versionlog

almahans requested review from FranBonath, aanil and ssjunnebo November 11, 2025 14:49

almahans self-assigned this Nov 11, 2025

almahans added the validation label Nov 11, 2025

aanil reviewed Nov 12, 2025

View reviewed changes

minor changes

84426c4

Small changes in indentations and looping

aanil reviewed Nov 26, 2025

View reviewed changes

ngi_reports/utils/entities.py Show resolved Hide resolved

aanil reviewed Nov 26, 2025

View reviewed changes

ngi_reports/utils/entities.py Show resolved Hide resolved

aanil reviewed Nov 26, 2025

View reviewed changes

ngi_reports/utils/entities.py Show resolved Hide resolved

almahans and others added 2 commits November 26, 2025 13:29

Update ngi_reports/utils/entities.py

b11e42a

Co-authored-by: Anandashankar Anil <aanil@users.noreply.github.com>

Update ngi_reports/utils/entities.py

0d6da58

Co-authored-by: Anandashankar Anil <aanil@users.noreply.github.com>

Update ngi_reports/utils/entities.py

5be0eb1

Co-authored-by: Anandashankar Anil <aanil@users.noreply.github.com>

almahans requested a review from aanil November 26, 2025 12:30

aanil reviewed Nov 26, 2025

View reviewed changes

ngi_reports/utils/entities.py Show resolved Hide resolved

aanil reviewed Nov 26, 2025

View reviewed changes

almahans added 2 commits November 26, 2025 15:09

Small fixes after suggestions

f04e499

Change return values

3ec3728

change from FC.exclude to return values for FC without LIMS information

almahans requested a review from aanil November 26, 2025 14:24

aanil reviewed Nov 26, 2025

View reviewed changes

ngi_reports/utils/entities.py Outdated Show resolved Hide resolved

aanil reviewed Nov 26, 2025

View reviewed changes

ngi_reports/utils/entities.py Show resolved Hide resolved

small changes

cb3955e

almahans requested a review from aanil November 26, 2025 16:28

aanil reviewed Nov 26, 2025

View reviewed changes

ngi_reports/utils/entities.py Show resolved Hide resolved

aanil reviewed Nov 27, 2025

View reviewed changes

ngi_reports/reports/ont_project_summary.py Outdated Show resolved Hide resolved

aanil approved these changes Nov 27, 2025

View reviewed changes

Update ngi_reports/reports/ont_project_summary.py

fb55e17

Co-authored-by: Anandashankar Anil <aanil@users.noreply.github.com>

aanil merged commit d3d20d8 into NationalGenomicsInfrastructure:master Jan 8, 2026
1 check passed

		for arg in run_arguments:
		if "--split_files_by_barcode=on" in arg:

Conversation

almahans commented Nov 11, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aanil left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants