CYP2D6 Variant–Drug Metabolic Activity Prediction via Fine-tuned Protein Language Modelling

Code accompanying the paper "Predicting CYP2D6 variant–drug metabolic activity through experimental profiling and contrastive protein language modelling"

This repository provides the fine-tuning pipeline for the CYP2D6-adapted ESM3 encoder (FT-ESM) and the downstream variant–drug regression pipeline (XGBoost) used in the paper. The CYP2 subfamily fine-tuning corpus, the in-house CYP2D6 variant–drug measurement dataset, the pre-trained FT-ESM checkpoint, and the pre-extracted FT-ESM and MoleBERT embedding pickles are distributed separately through the accompanying Google Drive (see data/README.md).

---

Repository layout

CYP2D6-FTESM/
├── ftesm/                       # Fine-tuning of the ESM3 protein language model
│   ├── model_ftesm.py           # LoRA + FiLM + CWCL model definition
│   └── train_3task_cwcl.py      # Main training entry point (3-target regression + CWCL)
│
├── downstream/                  # Variant–drug regression pipeline (XGBoost)
│   ├── train_nestedCV.py        # Nested-CV training driver (10-fold stratified nested CV)
│   └── bayesopt_xgb_v4.py       # XGBoost hyperparameter Bayesian optimization
│
├── configs/
│   └── run_reg_long.sh          # Reference training launcher (multi-GPU torchrun)
│
├── data/                        # (Not tracked in git) — see data/README.md
│
├── requirements.txt
├── LICENSE
└── README.md

---

External resources (Google Drive)

The following are hosted on Google Drive because of size:

1. CYP2 subfamily fine-tuning corpus — 3,708 missense variants across CYP2B6, CYP2C8, CYP2E1, CYP2S1, and CYP2D6, with AlphaMissense, ESM-1v log-likelihood, and FoldX ΔΔG labels.
2. In-house CYP2D6 variant–drug dataset — nine haplotypes × 35 drugs (291 pairs) with intrinsic clearance labels.
3. Pre-trained FT-ESM checkpoint — SWA-averaged encoder used throughout the paper.
4. FT-ESM variant embedding pickle — pre-extracted per-allele embeddings.
5. MoleBERT drug embedding pickle — pre-extracted per-substrate embeddings.

Download link and folder layout are described in data/README.md.

---

Installation

Tested with Python 3.11 on Linux (CUDA 11.8, 4 × NVIDIA A100-40GB).

git clone https://github.com/sunset222/cypvar-esm.git
cd cypvar-esm
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

---

Reproducing the paper

1. Fine-tune ESM3 on the CYP2 subfamily variants

torchrun --nproc_per_node=4 ftesm/train_3task_cwcl.py \
    --epochs 200 --lora_lr 2e-5 --other_lr 2e-5 \
    --temperature 0.05 --batch_size 8 \
    --cwcl_v2l_weight 0.7 --lambda_cls 1.0 --lambda_cwcl 1.0 \
    --regression --cwcl_use_diff \
    --data_file data/cyp2_family_45pct_cutoff.pkl \
    --output_dir runs/ftesm_reg_cwcl

The exact hyperparameters used in the paper are wrapped in configs/run_reg_long.sh.

2. Train the downstream XGBoost regressor

# 10-fold stratified nested CV using pre-extracted FT-ESM + MoleBERT features
python downstream/train_nestedCV.py \
    --features data/CYP2D6_variant_FAM_reg_ep200_SWA5_KIT.pkl \
    --drug_features data/MoleBERT_Substrate.pkl \
    --regressor xgb \
    --output runs/downstream_ft_xgb

XGBoost hyperparameters are optimized by Bayesian optimization inside the inner CV loop (see downstream/bayesopt_xgb_v4.py).

---

Citation

Please cite the accompanying paper if you use this code:

---

License

Released under the MIT License. See LICENSE.