Benchmarking-component-choices-for-unpaired-single-cell-RNA-and-epigenomic-integration

---

Overview

Benchmarking is done in 3 steps

• Gene activity score methods
• Dimension reduction methods
• Clustering and labeling methods

At each stage, conduct all possible combinations from different steps and evaluate the results using evaluation matrices(Methods).

---

1. Gene activity score

Methods for Gene Activity Score (GAS)

We have benchmarked 4 gene activity methods, each representing different approaches to predicting gene activity from scATAC-seq data. We have followed the online tutorial for each of the methods :

1. Signac(1.10.0), Tutorial
2. LIGER (v2.1.0), Tutorial
3. Cicero(1.3.9) , Tutorial
4. MAESTRO(1.5.1), directly followed the description of the function

Evaluation metrics for gene activity score

1. Pearson correlation coefficient (PCC) of each gene's scRNA-seq data and GAS data.
2. Local neighborhood consistency (LNC) of each cell's scRNA-seq profile and GAS profile.
3. Cell-type Average Silhouette width (ASW) of each cell type on PCA reduced embedding of GAS profile.

---

2.Dimension reduction

Methods for Dimension Reduction

We benchmarked multiple dimension-reduction methods that embed scRNA-seq and scATAC-seq cells into a shared latent space. For each method, we followed the official tutorial:

1. Seurat (v4.3.0) — Tutorial
2. bindSC (v1.0.0) — Tutorial
3. CoupledNMF (MATLAB implementation) — Code & docs
4. LIGER (v2.1.0) — Tutorial
5. scJoint — Tutorial
6. scDART — Repo
7. uniPort (v1.2.2) — Tutorial
8. GLUE (v0.3.2) — Tutorial
9. MultiMAP — Tutorial
10. SIMBA (v1.2) — Tutorial

Evaluation metrics for common embeddings

1. Percentage of same cell's different profiles are mutually in the top 1% nearest neighbors for each other(MNN%).
2. Contrastive similarity of a cell to its paired cell against the mean similarity to all unpaired cells.
3. Cell-type Average Silhouette width (ASW) of each cell type in RNA and ATAC embedding.

---

3.Clustering and labeling

Methods for Clustering and Labeling

We benchmarked multiple strategies for clustering or transferring labels:

1. Seurat anchor-based label transfer — using FindTransferAnchors and TransferData with default parameters
2. FigR anchor-based label transfer — Tutorial
3. Leiden clustering — performed in Scanpy on the joint embedding (cosine distance; neighbors = 20, except BMMC = 200; resolution = 1.0)
4. KNN label transfer — KNeighborsClassifier (Euclidean distance; k = 5)
5. Optimal transport (OT) label transfer — Moscot (v0.4.0) — Tutorial

Evaluation metrics for clustering

1. Normalized mutual information(NMI): between RNA label and ATAC label
2. Average label consistency percentage of label match, per cluster ( average of two percentages: for each cluster RNA cluster, percentage of matched ATAC cells with the same label. Calculate the same for ATAC. Then, we get the mean of all RNA clusters and all ATAC clusters. )

---

Script examples

This section provides example scripts to demonstrate:

• Running each method (GAS → dimension reduction → clustering/labeling)
• Evaluating outputs: GAS| Joint embeddings | RNA, and ATAC clustering labels

---

Running the methods

1. Gene activity score (GAS) calculation

   • Example script

2. Dimension reduction (joint embedding)

   • Example script
   • coverage: Seurat / LIGER / bindSC / scJoint / scDART / uniPort / GLUE / MultiMAP / SIMBA / CoupledNMF

3. Clustering & labeling / label transfer

   • Example script
   • coverage: OT (Moscot)/FigR /Leiden / KNN /

---

Evaluation

1. GAS evaluation

   • Script for GAS evaluation

2. Integration evaluation

   • Example script

3. Clustering/labeling evaluation

   • Example script

4. Evaluation functions (metrics & utilities)

   • Evaluation functions

---