Add DIAMOND blastp subworkflow for protein homology search against NCBI RefSeq by tracelail · Pull Request #50 · nf-core/proteinannotator

tracelail · 2025-06-02T18:46:25Z

Protein functional annotation at scale demands efficient sequence alignment against large reference databases. BLAST, the traditional tool for this task, is computationally expensive for large datasets. [DIAMOND](https://github.com/bbuchfink/diamond) (Buchfink et al., [Nature Methods 2021](https://doi.org/10.1038/s41592-021-01101-x)) provides BLAST-compatible sensitivity with significantly higher throughput, up to 10,000× faster than BLAST on large protein databases by using a double-indexed alignment algorithm optimized for modern hardware. Adding DIAMOND blastp to proteinannotator enables users to run protein homology searches against the full NCBI RefSeq protein database as part of their functional annotation workflow, which would be impractical with BLAST at scale.

This PR adds [DIAMOND blastp](https://github.com/bbuchfink/diamond) ([Buchfink et al., Nature Methods 2021](https://doi.org/10.1038/s41592-021-01101-x)) to the pipeline, using the existing nf-core diamond/blastp and [diamond/makedb](https://nf-co.re/modules/diamond_makedb/) modules, plus two new local modules:

New local modules:

ncbirefseqdownload : Downloads and concatenates NCBI RefSeq protein FASTAs for a specified release category (e.g. complete, other) into a single compressed reference FASTA for use by diamond/makedb
diamondpreparetaxa : Downloads and extracts NCBI taxonomy files (nodes.dmp, names.dmp) required for taxonomic classification in diamond/makedb

New subworkflows:

subworkflows/local/diamond : Orchestrates the full DIAMOND pipeline: RefSeq download → taxonomy preparation → DIAMOND_MAKEDB → DIAMOND_BLASTP. Supports all seven DIAMOND output formats (blast, xml, txt, daa, sam, tsv, paf) via params.diamond_outfmt and params.diamond_blast_columns
subworkflows/local/functional_annotation — Integrates the DIAMOND subworkflow alongside the existing InterProScan and KOfamScan logic. All three tools are independently gated by their own skip flag (params.skip_diamond, params.skip_interproscan, params.skip_kofamscan); FUNCTIONAL_ANNOTATION passes through all seven of DIAMOND's real output channels (diamond_blast, diamond_xml, diamond_txt, diamond_daa, diamond_sam, diamond_tsv, diamond_paf) rather than hardcoding a single outfmt-specific emit

New parameters (added to nextflow_schema.json and nextflow.config):

-skip_diamond — Skip the DIAMOND BLASTP taxonomic classification step entirely (default: false)
-refseq_release — NCBI RefSeq release category (default: complete)
-taxondmp_zip — URL to NCBI taxonomy dump archive
-taxonmap — URL to compressed protein accession-to-taxid map
-diamond_outfmt — Output format code (default: 6, tabular)
-diamond_blast_columns — Optional column list for tabular output

Testing:

All new local modules have nf-test test suites with both live and stub tests:

ncbirefseqdownload — tested against the NCBI other RefSeq release via rclone (NCBI [permanently discontinued rsync support](https://ncbiinsights.ncbi.nlm.nih.gov/2026/03/25/retire-rsync-support-ftp-downloads/) as of June 1, 2026); stub uses echo "" | gzip to produce a valid (non-empty) gzip stub
diamondpreparetaxa — tested against a minimal mini_taxdump.tar.gz fixture rather than NCBI's full real taxonomy dump, to keep CI fast and network-independent; stub creates placeholder nodes.dmp and names.dmp

The diamond subworkflow is tested end-to-end using a miniature mini_prot.accession2taxid.gz taxon map and a small test_refseq.fasta. Tests cover five scenarios:

Live run with tabular output (outfmt 6, no columns)
Stub: outfmt 6, no columns
Stub: outfmt 6, with custom columns
Stub: outfmt 0 (pairwise BLAST format)
Stub: two-sample input, regression-testing a fixed channel cardinality bug (DIAMOND_MAKEDB's output now correctly broadcasts to every sample via .first(), rather than pairing positionally and silently dropping any sample beyond the first)

The functional_annotation subworkflow is tested with skip_interproscan = true and skip_diamond = false for DIAMOND-focused tests (live and stub), confirming the DIAMOND path runs correctly independently. Pre-existing InterProScan- and KOfamScan-focused tests now set skip_diamond = true, since they were previously running the full DIAMOND pipeline unconditionally despite being unrelated to what they test.

All stub tests are tagged CI for fast pipeline CI runs. Live module tests require -profile docker and are tagged accordingly.

subworkflows/local/functional_annotation — Integrates the DIAMOND subworkflow alongside the existing InterProScan and KOfamScan logic. All three tools are independently gated by their own skip flag. FUNCTIONAL_ANNOTATION passes through all seven of DIAMOND's real output channels; consuming these outputs downstream in proteinannotator.nf (e.g. a comparison/reporting step) is left to a follow-up PR.

PR checklist

…nto functional_annotation subworkflow.

nf-core-bot · 2025-06-02T18:46:58Z

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.3.1.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

…nd makedb.

…nto unfinished-diamond-blastp

…o unfinished-diamond-blastp

…aretaxa local module template.

…t and wrote an initial main.nf.test for the process

…t process is successful.

…utilized tests/nextflow.config to fix memory error.

…to diamondpreparetaxa module

…est1.fasta and test2.fasta to the diamond subworkflow directory for diamond/blastp nf-test input.

…iamond subworkflow Diamond_blastp output. Included Diamond subworkflow execution in the functional annotation subworkflow.

…ules.

olgabot

Turns out I had started this review a while ago... But here are a bunch of suggestions that will hopefully deal with the container issues.

Co-authored-by: Olga Botvinnik <olga.botvinnik@gmail.com>

…w config processes for DIAMOND_MAKEDB and DIAMOND_BLASTP.

…o unfinished-diamond-blastp

…blastp' into unfinished-diamond-blastp

… modules and subworkflows. Added some tests and confirmed updated snapshots.

… merged back in if needed or fixed if requested. updated docs.md with some breaks in markdown and output examples of PAF. Fixed some dev merging issues in functional_annotation/meta.yml.

…and annotation information.

…ond test had a typo of tsv output when it should have been txt. Updated snapshots are also included.

…hannels from diamond subworkflow main and nf-test assertions.

erikrikarddaniel

Quite some work left.

erikrikarddaniel · 2026-08-13T14:34:17Z

+    DIAMOND(
+        ch_fasta
+    )
+    ch_diamond_tsv = DIAMOND.out.tsv


diamond_tsv is wired to DIAMOND.out.tsv, which only gets populated for outfmt==102 -- not the pipeline's default outfmt=6 (txt) -- and this emit has no downstream consumer in this PR at all

Failure scenario: Run with the default --diamond_outfmt 6: DIAMOND_BLASTP writes a .txt file, but FUNCTIONAL_ANNOTATION.out.diamond_tsv stays empty -- proven directly by the PR's own committed snapshot (subworkflows/local/functional_annotation/tests/main.nf.test.snap, test 'Diamond execution subworkflow success', diamond_outfmt=6): "diamond_tsv": [] while workflow.success is true.

Checked workflows/proteinannotator.nf: FUNCTIONAL_ANNOTATION.out.diamond_tsv isn't referenced anywhere downstream either, matching this PR's own description ("not consumed downstream... deferred to follow-up PR"). Given that, I'd recommend removing ch_diamond_tsv/the diamond_tsv emit entirely for now rather than fixing its wiring -- it's currently dead code on both ends (never correctly populated under default settings, and never read), so there's nothing to keep. When the follow-up PR actually wires in a comparison module, it can pick the emit that matches whatever --diamond_outfmt it's designed around, deliberately, instead of inheriting this mismatch.

Posted by an automated code review (recall-biased, high effort). Please verify before acting.

erikrikarddaniel · 2026-08-13T14:34:17Z

                { assert snapshot(
                    path(workflow.out.interproscan_tsv[0][1]).readLines()[0]
-                        .contains("GI|225038609|EFDID|719595|FULL	079fff43a0270e432d339ea71b6f0acf	350	SFLD	SFLDS00057	Glutaminase/Asparaginase	17	347	0.0	T"),
+                        .contains("GI|225038609|EFDID|719595|FULL       079fff43a0270e432d339ea71b6f0acf        350     SFLD    SFLDS00057      Glutaminase/Asparaginase        17      347     0.0     T"),


The two pre-existing InterProScan tests lost their snapshot baselines, and the one retained assertion string was silently corrupted from tab-delimited to space-delimited

Confirmed via cat -A against origin/dev: the retained assertion expects space-separated output where the pre-existing baseline (and the tool's actual output format) uses tabs. As written, this assertion no longer checks what it's meant to check.

Posted by an automated code review (recall-biased, high effort). Please verify before acting.

erikrikarddaniel · 2026-08-13T14:34:17Z

+        ch_fasta
+    )
+    ch_diamond_tsv = DIAMOND.out.tsv
+    ch_versions = ch_versions.mix(DIAMOND.out.versions.first())


.first() on DIAMOND.out.versions (a channel that legitimately carries two version entries -- rsync + wget) silently discards one, non-deterministically

Failure scenario: The standalone DIAMOND subworkflow's own test snapshot shows two versions.yml entries for this exact scenario, but the same run through FUNCTIONAL_ANNOTATION collapses to one -- confirmed by diffing the two committed snapshots. The aggregated versions.yml/MultiQC table will be missing either the rsync or wget version line whenever DIAMOND runs through FUNCTIONAL_ANNOTATION.

Worth being precise about blast radius: this channel only ever feeds the software-versions report, never any data/process-invocation logic, so this doesn't affect actual DIAMOND results -- it's a reproducibility/traceability bug, not a correctness one.

Both NCBIREFSEQDOWNLOAD and DIAMONDPREPARETAXA already have the topic-channel version of their own versions emit commented out in place (// updated versioning method to be implemented, followed by the real tuple(...), emit: versions, topic: versions line). Switching those two on would sidestep this whole .mix()/.first() bookkeeping for these two new modules specifically, without needing the rest of the pipeline to migrate first -- topic channels coexist fine alongside the existing versions.yml-file pattern used elsewhere. Given the PR description notes topic-channel migration is deferred pending an nf-core-wide mandate timeline, this bug is a concrete example of the exact risk that pattern avoids -- worth reconsidering for at least these two new modules, even if the full pipeline migration stays deferred.

Posted by an automated code review (recall-biased, high effort). Please verify before acting.

erikrikarddaniel · 2026-08-13T14:34:17Z

-                    "versions.yml:md5,f1d8a406d3dcb97a7c15e9c810926de1"
+                    "versions.yml:md5,a74a0c8fcb741e59bc14424f612b8d09"
                ],
                "funfam_domains": [


Unrelated subworkflow (domain_annotation, not touched by this PR's own stated scope) has a regressed snapshot -- empty arrays plus a duplicate JSON key

Corroborated by a stray committed .nf-test.log in this PR's diff -- looks like an accidental local --update-snapshot run swept this in alongside the intended changes.

Posted by an automated code review (recall-biased, high effort). Please verify before acting.

erikrikarddaniel · 2026-08-13T14:34:17Z

+    """
+    mkdir -p taxa/
+    wget -q ${taxondmp_zip}
+    tar -xzf taxdump.tar.gz -C taxa/


tar -xzf taxdump.tar.gz extracts a hardcoded filename, but the actual downloaded file's name depends on whatever --taxondmp_zip URL was given

Failure scenario: Any schema-valid --taxondmp_zip URL with a different basename than taxdump.tar.gz -- the file wget saves will have that different name, and the hardcoded tar -xzf taxdump.tar.gz will fail to find it.

Posted by an automated code review (recall-biased, high effort). Please verify before acting.

erikrikarddaniel · 2026-08-13T14:34:17Z

@@ -0,0 +1,51 @@
+process DIAMONDPREPARETAXA {


DIAMONDPREPARETAXA hand-rolls wget+tar as a new bespoke local module, duplicating a fetch-and-unpack pattern this same PR's sibling functional_annotation/main.nf:37-41 already solves generically via the vendored ARIA2+UNTAR modules

Failure scenario: Two different, inconsistent 'download a reference URL and unpack it' implementations now coexist one function-call apart in the same subworkflow tree: nf-core-standard ARIA2 (retry-capable) + UNTAR on one side, a bespoke wget -q (no retry) + tar on the other. A future robustness fix (retries, checksum validation) applied to ARIA2 won't benefit this path, and a new contributor has two inconsistent patterns to learn instead of one.

Worth distinguishing from the sibling NCBIREFSEQDOWNLOAD module: that one uses rsync, and there's currently no rsync module in nf-core/modules to install instead, so a local module is more defensible there. This one has no such excuse -- ARIA2+UNTAR are already proven working in this exact PR.

Posted by an automated code review (recall-biased, high effort). Please verify before acting.

erikrikarddaniel · 2026-08-13T14:34:17Z

@@ -0,0 +1,122 @@
+process DIAMOND_BLASTP {


DIAMOND_BLASTP has no publishDir configured anywhere -- not in the module, not in conf/modules.config, no pipeline-wide fallback either

Checked conf/modules.config, conf/base.config, nextflow.config, and this vendored module itself: none of them publish any of DIAMOND_BLASTP's outputs. Combined with the diamond_tsv issue above, this means running the pipeline today produces no visible DIAMOND result file in the results directory at all, under any --diamond_outfmt setting -- the actual homology-search hits only ever exist transiently in work/. Given this PR's stated purpose is enabling DIAMOND-based homology search "as part of their functional annotation workflow," I'd expect at least the raw hits file to be published even before the comparison-module integration lands in the follow-up PR.

Posted by an automated code review (recall-biased, high effort). Please verify before acting.

erikrikarddaniel · 2026-08-13T14:34:17Z

+            }
+            workflow {
+                """
+                input[0] = [ [id:'test'], file("${moduleTestDir}/test_refseq.fasta", checkIfExists: true)]


No test anywhere in this PR passes more than one sample through ch_fasta

Checked all six test cases touching the new Diamond code (this file's 4 tests, plus subworkflows/local/functional_annotation/tests/main.nf.test's 2 Diamond-focused tests): every single one uses input[0] = [ [id:'test'], file(...)] -- a single tuple, never a multi-item channel. This isn't "coverage happens to miss the multi-sample case" -- structurally, none of these tests could have caught the ch_diamond_db cardinality issue flagged in subworkflows/local/diamond/main.nf:39, since that failure mode only manifests when ch_fasta has more than one item. Worth adding at least one test case with channel.of([...], [...]) carrying two samples before merge -- that would immediately confirm or rule out the cardinality concern empirically, and closes a real coverage gap either way.

Posted by an automated code review (recall-biased, high effort). Please verify before acting.

erikrikarddaniel · 2026-08-13T14:34:17Z

+        when {
+            process {
+                """
+                input[0] = 'ftp://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz'


Every "live" (non-stub) test that touches DIAMONDPREPARETAXA downloads the full, real NCBI taxonomy dump -- repeated identically across three separate test files

This exact taxondmp_zip = 'ftp://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz' (full-size, unminified) appears in this file, subworkflows/local/diamond/tests/main.nf.test, and subworkflows/local/functional_annotation/tests/main.nf.test. Contrast with taxonmap and the FASTA input in the same tests, which were both thoughtfully shrunk to small fixtures (mini_prot.accession2taxid.gz, test_refseq.fasta) -- the same care wasn't applied here. Three separate live CI runs each pulling the full NCBI taxdump is real CI cost and a source of network-flakiness risk that didn't need to exist; a small hand-built nodes.dmp/names.dmp fixture (matching what mini_prot.accession2taxid.gz already required for the taxonmap side) would let these tests run fast and deterministically.

Posted by an automated code review (recall-biased, high effort). Please verify before acting.

erikrikarddaniel · 2026-08-13T14:44:41Z

+- [[PR #13](https://github.com/nf-core/proteinannotator/pull/13)] Add nf-core seqkit/stats module
+- [[PR #9](https://github.com/nf-core/proteinannotator/pull/9)] Add [InterProScan](https://interproscan-docs.readthedocs.io/) module
+- [#90](https://github.com/nf-core/proteinannotator/pull/90) - Added the option to download and use the latest `metagRoot` HMM library (or use path to an existing one) for domain annotation. (by @angelphanth)
 - [#90](https://github.com/nf-core/proteinannotator/pull/90) - Added the option to download and use the latest `metagRoot` HMM library (or use path to an existing one) for domain annotation. (by @angelphanth)


Inconsistent format (sometimes PR sometimes not) and not ordered.

duplication

heuermh · 2026-08-13T20:36:34Z

Would you need help resolving conflicts here? It is somewhat difficult to review if conflicts remain.

Then

--skip_diamond parameter deferred to follow-up PR

Does this mean the DIAMOND blastp subworkflow will run by default now?

If so, that could be surprising to existing users. I might think it better to include --skip_diamond in this pull request and set it to true by default.

tracelail · 2026-08-14T13:45:08Z

@heuermh

Would you need help resolving conflicts here? It is somewhat difficult to review if conflicts remain.

Then

--skip_diamond parameter deferred to follow-up PR

Does this mean the DIAMOND blastp subworkflow will run by default now?

If so, that could be surprising to existing users. I might think it better to include --skip_diamond in this pull request and set it to true by default.

I can address the conflicts. I should have checked again for conflicts before requesting review. On the the --skip_diamond you're absolutely right and an oversight on my part. I got lost in executing diamond and forgot that it should be implemented in this PR.

vagkaratzas

quite a few non nf-core stuff around that need to be fixed..the latest dev should be merged in this branch and conflicts should be resolved.

I was wondring if this should stay under the functional_annotation subworkflow, or become its own derive_taxonomy subworkflow (or something like that) that is currently missing from the pipeline and this seems to fill that gap.

when finalized and before merging, the metro map should be updated accordingly (I can do that)

vagkaratzas · 2026-08-15T13:36:16Z

@@ -1,3 +1,6 @@
 {


please gitignore non-standard nf-core template files such as this one

vagkaratzas · 2026-08-15T13:39:28Z


 </details>

+#### Diamond


I'd argue this is too long for usage.md. I would trim down extensively, or have DIAMOND.md file, and/or link to the external software? What do you think @erikrikarddaniel ? Since you also recently linked some non nf-core md files for extra info

vagkaratzas · 2026-08-15T13:42:31Z

again, non-standard file...this information should be reflected in the updated metro-map instead

vagkaratzas · 2026-08-15T13:45:16Z

@@ -10,4 +10,34 @@ params {
    pipelines_testdata_base_path = 'https://raw.githubusercontent.com/nf-core/test-datasets/refs/heads/proteinannotator'


remove the extra bloat from this file --only keep the few lines that are in the template.

vagkaratzas · 2026-08-15T13:45:42Z

.log files should be gitignored

vagkaratzas · 2026-08-15T13:46:06Z

+- [[PR #13](https://github.com/nf-core/proteinannotator/pull/13)] Add nf-core seqkit/stats module
+- [[PR #9](https://github.com/nf-core/proteinannotator/pull/9)] Add [InterProScan](https://interproscan-docs.readthedocs.io/) module
+- [#90](https://github.com/nf-core/proteinannotator/pull/90) - Added the option to download and use the latest `metagRoot` HMM library (or use path to an existing one) for domain annotation. (by @angelphanth)
 - [#90](https://github.com/nf-core/proteinannotator/pull/90) - Added the option to download and use the latest `metagRoot` HMM library (or use path to an existing one) for domain annotation. (by @angelphanth)


duplication

…for funtional annotation testing.

… for testing local referencing. Tested and udpated snapshot.

…ded a multi item input channel stub step with updated snapshot.

…. Updated meta and environment to match. snap needs pruning once ncbi real test can run after ban.

Added publish dir entries for diamond makedb and diamond blastp.

…ync 2026-06-01) Ref: https://ncbiinsights.ncbi.nlm.nih.gov/2026/03/25/retire-rsync-support-ftp-downloads/ Also add --user-agent "Mozilla/5.0": NCBI blocks rclone's default UA, independent of IP or request rate.

eweizy and others added 5 commits March 26, 2025 19:08

Install diamond/blastp

53aa47c

Unfinished diamond/blastp integraton

55b7632

Add diamond

3c5b661

reinstalled diamond/blastp module. Installed blast/makeblastdb

b1fdc74

wrote draft integration of BLAST_MAKEBLASTDB and NCBIREFSEQDOWNLOAD i…

9f1ea67

…nto functional_annotation subworkflow.

tracelail added 7 commits June 3, 2025 14:47

installed diamond makedb

52506f7

cleared nf-test logs and added tuple output for main.nf.test of diamo…

49ee17c

…nd makedb.

Merge branch 'dev' of https://github.com/tracelail/proteinannotator i…

6ab82f3

…nto unfinished-diamond-blastp

Merge branch 'dev' of https://github.com/nf-core/proteinannotator int…

89fb03e

…o unfinished-diamond-blastp

removed blast/makeblastdb nf-core module and create local diamondprep…

f12c619

…aretaxa local module template.

finished up writing the ncbirefseqdownload process for the first draf…

4f4db82

…t and wrote an initial main.nf.test for the process

edited ncbirefseqdownload script to a working stated where the nf-tes…

3946ba5

…t process is successful.

olgabot mentioned this pull request Jun 24, 2025

Unfinished diamond/blastp integration #18

Closed

11 tasks

tracelail and others added 10 commits June 26, 2025 08:59

created a working ncbirefseqdownload module with basic nf-test. Also …

94ebe1b

…utilized tests/nextflow.config to fix memory error.

added more working tests to ncbirefseqdownload and organized.

06bf5e8

Added diamondpreparetaxa main.nf script and a process.success nf-test.

7681f82

Merge branch 'dev' into unfinished-diamond-blastp

0cf27f7

added working snapshot assertions for process.out match and versions …

92c847c

…to diamondpreparetaxa module

Added output documentation for all seven Diamond subworkflow outputs.

043451f

wrote a potential subworkflow for diamond as well as a test. Copied t…

f59c0b7

…est1.fasta and test2.fasta to the diamond subworkflow directory for diamond/blastp nf-test input.

added stub portion to ncbirefseqdownload. Added all output emits to d…

cf259d3

…iamond subworkflow Diamond_blastp output. Included Diamond subworkflow execution in the functional annotation subworkflow.

Added stub section to diamondpreparetaxa module.

9fd5d3f

created a simple flow diagram of the diamond subworkflow and it's mod…

fa390ce

…ules.

olgabot requested changes Jul 22, 2025

View reviewed changes

tracelail and others added 5 commits July 28, 2025 12:21

Apply suggestions from code review

4eb5ad1

Co-authored-by: Olga Botvinnik <olga.botvinnik@gmail.com>

Created workflow success tests for diamond subworkflow. Added nextflo…

71ff9ef

…w config processes for DIAMOND_MAKEDB and DIAMOND_BLASTP.

Merge branch 'dev' of https://github.com/nf-core/proteinannotator int…

a4f00be

…o unfinished-diamond-blastp

Merge remote-tracking branch 'refs/remotes/origin/unfinished-diamond-…

16e10b2

…blastp' into unfinished-diamond-blastp

corrected typo in diamondpreparetaxa container

c970041

tracelail added 7 commits April 3, 2026 18:47

minor updates for cleaning and future version implementation in local…

6a39985

… modules and subworkflows. Added some tests and confirmed updated snapshots.

Updated modules.json to remove mmseqs/search that would break. Can be…

8dfc3ea

… merged back in if needed or fixed if requested. updated docs.md with some breaks in markdown and output examples of PAF. Fixed some dev merging issues in functional_annotation/meta.yml.

Addressed some liniting issues in modules and schema json

0c9ba7d

Updated schema and config to fix lint error. Updated readme with diam…

60e36fa

…and annotation information.

diamond subworkflow needed a default with elvis operator and the diam…

a2e2c02

…ond test had a typo of tsv output when it should have been txt. Updated snapshots are also included.

updated diamond nf-core modules

ac1a9c8

Diamond modules update had new versioning. Removed old version emit c…

9a7dace

…hannels from diamond subworkflow main and nf-test assertions.

tracelail marked this pull request as ready for review April 9, 2026 14:55

tracelail added 2 commits April 9, 2026 11:12

updated resolution conflicts

67617ad

Merge branch 'dev' into unfinished-diamond-blastp

63facff

tracelail changed the title ~~wrote draft integration of BLAST_MAKEBLASTDB and NCBIREFSEQDOWNLOAD into functional_annotation subworkflow.~~ Integration of BLAST_MAKEBLASTDB and NCBIREFSEQDOWNLOAD into functional_annotation subworkflow. Apr 9, 2026

tracelail changed the title ~~Integration of BLAST_MAKEBLASTDB and NCBIREFSEQDOWNLOAD into functional_annotation subworkflow.~~ Add DIAMOND blastp subworkflow for protein homology search against NCBI RefSeq Apr 9, 2026

tracelail requested a review from olgabot April 9, 2026 15:17

tracelail mentioned this pull request Jul 6, 2026

Added blog post, From Confused to Contributing nf-core/website#4303

Closed

erikrikarddaniel requested changes Aug 13, 2026

View reviewed changes

vagkaratzas requested changes Aug 15, 2026

View reviewed changes

tracelail added 12 commits August 18, 2026 18:48

Merge dev into unfinished-diamond-blastp, resolve conflicts

83a764a

Updated versioning to new topic standard and added additional inputs …

02f1e38

…for funtional annotation testing.

Untrack .nf-test.log (already covered by .gitignore)

afb321a

created test taxdump to reduce CI bloat and changed wget tool to curl…

04dc134

… for testing local referencing. Tested and udpated snapshot.

updated container and dependencies

20784fc

Addressed diamond database channel as a single item queue channel. Ad…

b4d352d

…ded a multi item input channel stub step with updated snapshot.

updated ncbi download to use rclone since rsync is depricated by ncbi…

5ba47b3

…. Updated meta and environment to match. snap needs pruning once ncbi real test can run after ban.

restored domain_annotation via git checkout dev.

b0da026

Added publish dir entries for diamond makedb and diamond blastp.

updated modules.config

5dee8c0

Added skip_diamond functionality to pipeline and ran stub tests.

e204f10

Replace rsync with rclone in NCBIREFSEQDOWNLOAD (NCBI discontinued rs…

43bae90

…ync 2026-06-01) Ref: https://ncbiinsights.ncbi.nlm.nih.gov/2026/03/25/retire-rsync-support-ftp-downloads/ Also add --user-agent "Mozilla/5.0": NCBI blocks rclone's default UA, independent of IP or request rate.

updated diamond output markdown with publishdir subdirectories.

b96323c

		@@ -10,4 +10,34 @@ params {
		pipelines_testdata_base_path = 'https://raw.githubusercontent.com/nf-core/test-datasets/refs/heads/proteinannotator'

Uh oh!

Conversation

tracelail commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR checklist

Uh oh!

nf-core-bot commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

olgabot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

erikrikarddaniel left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

heuermh commented Aug 13, 2026

Uh oh!

tracelail commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vagkaratzas left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

tracelail commented Jun 2, 2025 •

edited

Loading

nf-core-bot commented Jun 2, 2025 •

edited

Loading

tracelail commented Aug 14, 2026 •

edited

Loading