MetaT Eukaryotic Metatranscriptomics Pipeline

SLURM-based metatranscriptomic pipeline for marine eukaryotic gene expression (AtlantECO / Mission Microbiome Atlantic).

Recommended production path (TD2, no clustering)

Use TD2 for ORF calling and skip amino-acid clustering.
This maximises recovery of coding sequences without collapsing near-identical peptides, and uses a modern ORF scorer (PSAURON + revised retention) instead of six-frame transeq.

Raw reads
  → QC / trim (Trimmomatic + FastQC)
  → rRNA removal (SortMeRNA) + MEGAHIT assembly
      (optional: + rnaSPAdes → merge with CD-HIT-EST)
  → TD2 LongOrfs + Predict          ← instead of 6-frame transeq
  → [skip clustering]
  → Taxonomy (Diamond + MarFERReT) on TD2 peptides
  → Function (KofamScan + EggNOG) on TD2 peptides
  → Extract contig NT for TD2 ORFs → Kallisto TPM
  → Merge results table

One-command submit (login node)

# After configuring 00_config.sh and placing raw reads:
chmod +x run_pipeline_td2_nocluster.sh
./run_pipeline_td2_nocluster.sh

# Preview only:
./run_pipeline_td2_nocluster.sh --dry-run

# Resume from TD2 onward (assembly already done):
./run_pipeline_td2_nocluster.sh --from 3 --to 10

This runner sets ORF_SOURCE=td2 so taxonomy, Kofam, NT extract, Kallisto, and merge use TD2 peptides and do not require step 5 clustering.

Manual step list (same path)

Order	Script	Notes
1	01_qc_trim.slurm	FastQC + Trimmomatic
2	02_megahit_assembly.slurm	SortMeRNA + MEGAHIT
2b/2c	optional	02b_rnaspades_assembly.slurm + 02c_merge_assemblies.slurm
3	03f_td2.slurm	TD2 ORFs → ${TD2_DIR}/${SAMPLE_ID}/${SAMPLE_ID}.megahit.td2.pep.gz
—	skip	03_translate_6frame, 04_select_best_frame, 05_cluster_*
6	06_taxonomic_annotation.slurm	ORF_SOURCE=td2
7	07_functional_annotation.slurm	KofamScan; ORF_SOURCE=td2
7b	07b_eggnog_td2.slurm	EggNOG on TD2 peptides
8	08_extract_nt_sequences.slurm	Contig NT for TD2 ORF IDs; ORF_SOURCE=td2
9	09_kallisto_quantify.slurm	TPM on those contigs
10	10_merge_results.slurm	Taxonomy + Kofam + EggNOG + abundance

Submit downstream jobs with:

sbatch --export=ALL,ORF_SOURCE=td2 06_taxonomic_annotation.slurm

Why this path?

Choice	Rationale
TD2 instead of transeq	Explicit coding models with modern filters; strong MarFERReT agreement with six-frame (~83% same best subject when both hit on ST106) while avoiding frame-suffix bookkeeping for production
No clustering	Keeps full peptide diversity for metaT (rare transcripts, near-identical isoforms); “max sensitivity”
EggNOG + Kofam + MarFERReT	Function + taxonomy on the same TD2 peptide set
Kallisto on contigs carrying TD2 ORFs	Expression linked to the same coding calls used for annotation

Optional parallel tracks (not required for production)

• Six-frame (03 + 04): wide ORF universe / novelty audits; compare with compare_orf_prediction_td2.slurm
• TransDecoder (03b / 03d): complete vs partial CDS slices
• MetaEuk (03e): orthology-aware predictions vs a reference DB
• AGNOSTOS (11_agnostos_profile_search.slurm): unknown / dark CDS space
• Multi-sample TD2 rollup: scripts/collect_td2_sixframe_metrics_all_samples.py + plot_td2_sixframe_multisample.R
• Top-term comparison plots: copy scripts/methods_transeq_vs_td2.example.json, edit paths, then scripts/top_annotations_among_hits.py + plot_top_annotations.R

---

Legacy / comparison path (six-frame ± clustering)

The original runner ./run_pipeline.sh still submits six-frame translation → best frame → optional clustering → annotate → quantify. Use that for method comparisons or if you intentionally want transeq ORFs. Prefer run_pipeline_td2_nocluster.sh for new samples.

ORF_SOURCE (scripts/select_orf_input.sh):

Value	Protein input
td2	TD2 pep (recommended)
sixframe	*.megahit.6tr.bf${MIN_ORF_LENGTH}.fasta.gz
auto	clustered → six-frame → TD2 (first found)

---

Quick start

1. Clone

git clone https://github.com/erocke/Eukarion.git
cd Eukarion   # or metaT_euk_pipeline if nested

2. Configure

cp 00_config.template.sh 00_config.sh
# Edit SAMPLE_ID, BASE_DIR, database paths, modules, conda envs

3. Environments (minimum for recommended path)

conda create -n megahit -c bioconda megahit -y
conda create -n sortmerena -c bioconda sortmerna -y
conda create -n td2 -c bioconda td2 -y          # TD2.LongOrfs / TD2.Predict
conda create -n seqkit-env -c bioconda seqkit -y
conda create -n kofamscan -c bioconda kofamscan -y
conda create -n eggnog_clean -c bioconda eggnog-mapper -y   # name must match EGGNOG_CONDA_ENV

4. Databases

• MarFERReT Diamond DB (MARFERRET_DMND)
• KofamScan profiles (KOFAM_DIR)
• EggNOG-mapper data (EGGNOG_DB)
• SortMeRNA / SILVA (as used in assembly job)
• NCBI taxonomy dump for LCA naming (as in 06_taxonomic_annotation.slurm)

5. Input reads

source 00_config.sh && create_dirs
cp sample_R1.fastq.gz "${RAW_DIR}/${SAMPLE_ID}_R1_001.fastq.gz"
cp sample_R2.fastq.gz "${RAW_DIR}/${SAMPLE_ID}_R2_001.fastq.gz"

6. Run

./run_pipeline_td2_nocluster.sh
squeue -u $USER

---

Pipeline scripts (all steps)

Step	Script	Tool	Role
1	01_qc_trim.slurm	FastQC + Trimmomatic	QC / adapters
2	02_megahit_assembly.slurm	MEGAHIT + SortMeRNA	Assembly
2b	02b_rnaspades_assembly.slurm	rnaSPAdes	Optional
2c	02c_merge_assemblies.slurm	CD-HIT-EST	Optional merge
3f	03f_td2.slurm	TD2	Recommended ORFs
3	03_translate_6frame.slurm	transeq	Optional / comparison
4	04_select_best_frame.slurm	custom	Six-frame only
5	05_cluster_*.slurm	CD-HIT / MMseqs2	Skip for recommended path
6	06_taxonomic_annotation.slurm	Diamond + MarFERReT	Taxonomy
7	07_functional_annotation.slurm	KofamScan	KO
7b	07b_eggnog_td2.slurm	EggNOG-mapper	GO / Pfam / KO (TD2)
8	08_extract_nt_sequences.slurm	seqkit	Contigs for Kallisto
9	09_kallisto_quantify.slurm	Kallisto	TPM
10	10_merge_results.slurm	Python	Final table
11	11_agnostos_profile_search.slurm	MMseqs2	Optional unknowns

---

Key config (00_config.sh)

export SAMPLE_ID="MMA_ST106_L004"
export BASE_DIR="/scratch/${USER}/metaT_euk"
export MIN_ORF_LENGTH=100
export ORF_SOURCE=td2          # recommended; used by runner and steps 6–9
export TD2_CONDA_ENV=td2
export TD2_DIR="${ASSEMBLY_DIR}/td2"

Do not commit 00_config.sh (local paths). Commit 00_config.template.sh only.

---

Outputs (recommended path)

Path	Content
assemblies/td2/${SAMPLE_ID}/${SAMPLE_ID}.megahit.td2.pep.gz	TD2 peptides
assemblies/annotations/${SAMPLE_ID}.bf100.td2_mft_lca_named.tab.gz	Taxonomy
assemblies/annotations/${SAMPLE_ID}.bf100.td2.aa.best.kofam.tsv.gz	Kofam
assemblies/annotations/${SAMPLE_ID}.td2.eggnog.simple.tsv.gz	EggNOG simple
assemblies/kallisto/${SAMPLE_ID}/	Abundance
results/${SAMPLE_ID}.merged_results.tsv*	Merged table

---

Citation notes

Cite MEGAHIT, SortMeRNA, Diamond, MarFERReT, KofamScan, EggNOG-mapper, Kallisto, and TD2 (Sommer et al., bioRxiv) as appropriate for your manuscript.

License

MIT — see LICENSE.

Authors

Emma Rocke (@erocke) — University of Cape Town / AtlantECO