mdforge framework: general, reproducible GROMACS MD analysis

.github/workflows/ci.yml

@@ -0,0 +1,28 @@
+name: CI
+
+on:
+  push:
+    branches: [main, mdforge-framework]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install package (+ dev extras)
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -e ".[dev]"
+      - name: Lint (ruff, non-blocking)
+        run: ruff check mdforge || true
+      - name: Run tests
+        run: pytest -q

Dockerfile

@@ -0,0 +1,16 @@
+# mdforge -- reproducible container image.
+# Build:  docker build -t mdforge .
+# Run:    docker run --rm -v /data/sim:/sim mdforge analyze --input /sim --output /sim/results
+FROM mambaorg/micromamba:1.5-jammy
+
+WORKDIR /app
+COPY --chown=$MAMBA_USER:$MAMBA_USER environment.yml /app/environment.yml
+RUN micromamba install -y -n base -f /app/environment.yml && \
+    micromamba clean --all --yes
+
+COPY --chown=$MAMBA_USER:$MAMBA_USER . /app
+ARG MAMBA_DOCKERFILE_ACTIVATE=1
+RUN pip install --no-deps -e .
+
+ENTRYPOINT ["mdforge"]
+CMD ["--help"]

README.md

@@ -0,0 +1,111 @@
+# mdforge — reusable, reproducible GROMACS MD analysis
+
+[![CI](https://github.com/SamDozer/molecular-dynamics-forge/actions/workflows/ci.yml/badge.svg)](https://github.com/SamDozer/molecular-dynamics-forge/actions)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Python](https://img.shields.io/badge/python-3.10%2B-blue.svg)](pyproject.toml)
+
+**mdforge** turns a GROMACS simulation directory into a complete, publication-quality,
+fully reproducible analysis — with minimal input. Point it at a folder; it discovers
+the files, **detects the system** (protein / ligand / DNA / RNA / membrane / ions /
+multi-chain / …), **auto-selects the right analyses**, runs them with streaming-friendly
+performance, and produces figures, tables, a provenance manifest, and a report.
+
+> This repository began as a project-specific pipeline for an α-zein simulation
+> (now preserved in [`legacy/`](legacy) and [`examples/`](examples)) and was
+> refactored into this general toolkit.
+
+---
+
+## Highlights
+
+- **Zero-config detection** — recursively finds `*.xtc/.trr/.tpr/.gro/.edr/.ndx/.top`,
+  validates them (clear messages for anything missing), and classifies the system.
+- **Automatic module selection** — each analysis declares the system types and files
+  it supports; the pipeline runs exactly what applies (`--plan` shows *why*).
+- **Extensible via plugins** — drop a `BaseAnalysis` subclass into
+  `mdforge/analysis/plugins/` (or a `--plugin-dir`) and it is auto-discovered.
+- **Config-driven** — describe a whole run in `config.yaml` and re-run with one command
+  (ideal for HPC/batch).
+- **Reproducible by construction** — every run writes `manifest.json/yaml` with library
+  versions, git commit, input-file fingerprints, seeds, parameters and runtimes.
+- **Publication-quality output** — 300-dpi PNG **and** vector PDF, consistent
+  Nature-like style, plus Markdown + self-contained HTML (+ optional PDF) reports.
+- **Scales** — streams frame-by-frame and caches a solute-only trajectory, so large
+  (100 GB+) explicit-solvent runs stay tractable.
+
+## Supported systems
+
+protein-only · protein–protein · protein–peptide · protein–ligand · protein–DNA ·
+protein–RNA · protein–membrane · multi-chain · ions · cofactors · mixed biomolecular
+systems (detected automatically; override with `--system-type`).
+
+## Install
+
+```bash
+git clone https://github.com/SamDozer/molecular-dynamics-forge
+cd molecular-dynamics-forge
+python -m pip install -e ".[all]"      # or ".[dev]" for tests
+```
+
+## Usage
+
+```bash
+mdforge detect  --input /path/to/sim_dir            # what's in my system?
+mdforge analyze --input /path/to/sim_dir --plan     # what would run, and why?
+mdforge analyze --input /path/to/sim_dir -o results # run everything applicable
+mdforge analyze --config examples/alpha_zein_A8HNE1/config.yaml   # reproducible
+mdforge list-analyses                                # registered analyses (incl. plugins)
+```
+
+See [`docs/QUICKSTART.md`](docs/QUICKSTART.md) for all options and the plugin template.
+
+## Architecture
+
+```
+mdforge/
+  core/       system.py (detection) · base.py (BaseAnalysis) · registry.py (+plugins)
+              context.py · config.py (YAML+CLI) · provenance.py · pipeline.py
+  io/         discovery.py · validation.py
+  statistics/ descriptive · timeseries · correlation · bootstrap
+  plotting/   style · figures (PNG+PDF, 300 dpi)
+  analysis/   rmsd · rmsf · rog · sasa · … + plugins/ (auto-discovered)
+  report/     generator (Markdown + HTML + PDF)
+  cli/        main (analyze/detect/list) · interactive
+tests/  ·  examples/  ·  docs/  ·  legacy/  ·  Dockerfile  ·  .github/workflows/
+```
+
+Every analysis subclasses `BaseAnalysis`, declaring `required_files`,
+`supported_systems`, `outputs` and `default_params`; subclasses auto-register, so
+**detection → selection → run → report** is entirely data-driven.
+
+## Analyses (24 built-in)
+
+| System scope | Analyses |
+|---|---|
+| **Any system** | RMSD, radius of gyration, SASA, COM, H-bonds, energies (`.edr`), ProLIF, MM/PBSA workflow, statistics |
+| **Protein** | RMSF, structural descriptors (Dmax/κ²/volume), native contacts (Q), contact map, DSSP secondary structure, RIN, PCA + free-energy landscape, DCCM, clustering, salt bridges, convergence (RMSIP/block-avg), end-to-end *(plugin)* |
+| **Complex** (protein–protein/–nucleic) | interface (BSA, contacts, iRMSD) |
+| **Protein–ligand** | ligand RMSD, ligand contacts, binding pocket |
+| **Protein–DNA/RNA** | protein–nucleic contacts, nucleic RMSD |
+
+Each is a drop-in `BaseAnalysis`; the pipeline runs only those applicable to the
+detected system (`mdforge list-analyses` shows all; `--plan` shows what runs and why).
+The complex/ligand/nucleic modules are implemented and gate correctly but await
+validation on a matching test trajectory.
+
+## Reproducibility
+
+```bash
+cat results/manifest.json     # versions, git commit, seeds, params, input hashes, runtimes
+```
+
+## Container
+
+```bash
+docker build -t mdforge .
+docker run --rm -v /data/sim:/sim mdforge analyze --input /sim --output /sim/results
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

docs/QUICKSTART.md

@@ -0,0 +1,80 @@
+# mdforge — Quick Start
+
+## Install
+
+```bash
+git clone https://github.com/SamDozer/molecular-dynamics-forge
+cd molecular-dynamics-forge
+python -m pip install -e ".[all]"     # core + energy + fingerprints + pdf + ui
+```
+or with conda:
+```bash
+conda env create -f environment.yml && conda activate zein-md
+python -m pip install -e .
+```
+
+## 1. Detect your system
+
+```bash
+mdforge detect --input /path/to/simulation_dir
+```
+Prints the discovered files, validation status, and the detected composition
+(protein chains, ligands, nucleic acids, lipids, ions, water) and system type.
+
+## 2. See what would run (dry-run)
+
+```bash
+mdforge analyze --input /path/to/simulation_dir --plan
+```
+Shows which analyses are selected for the detected system — and *why* each other
+one was skipped.
+
+## 3. Run the full analysis
+
+```bash
+mdforge analyze --input /path/to/simulation_dir --output results/
+```
+Discovers files → detects the system → auto-selects analyses → runs them →
+writes CSVs, 300-dpi PNG+PDF figures, a `manifest.json/yaml` provenance record,
+and a Markdown+HTML report.
+
+### Handy options
+```bash
+--stride 10                 # analyse every 10th frame (quick look)
+--analyses rmsd,rog,sasa    # run a specific subset
+--exclude sasa              # skip an analysis
+--all                       # run every applicable analysis
+--interactive               # ask when the choice is ambiguous
+--report md,html,pdf        # report formats
+--plugin-dir ./my_plugins   # load extra drop-in analyses
+--config config.yaml        # drive everything from a YAML file
+```
+
+## 4. Reproducible, config-driven workflow
+
+```bash
+mdforge analyze --config examples/alpha_zein_A8HNE1/config.yaml
+```
+Every run writes a `manifest.json` capturing library versions, git commit, input
+fingerprints, seeds, parameters and per-analysis runtimes — enough for another
+researcher to reproduce the analysis exactly.
+
+## 5. Add your own analysis (plugin)
+
+Drop a file into `mdforge/analysis/plugins/` (or any `--plugin-dir`):
+
+```python
+from mdforge.core.base import BaseAnalysis
+
+class MyAnalysis(BaseAnalysis):
+    name = "my_analysis"
+    label = "My analysis"
+    supported_systems = {"*"}
+    def run(self, ctx):
+        u = ctx.core_universe()
+        ...                       # compute
+        ctx.write_csv(df, "my_analysis.csv")
+        return {"figure": "my_analysis"}
+```
+It is auto-discovered and appears in `mdforge list-analyses` immediately.
+See `mdforge/analysis/plugins/example_end_to_end.py` for a complete template.

environment.yml

@@ -0,0 +1,27 @@
+# Conda environment for the alpha-zein (A8HNE1) MD analysis pipeline.
+#   conda env create -f environment.yml
+#   conda activate zein-md
+#
+# conda-forge provides pre-built wheels for rdkit / mdtraj / MDAnalysis, which is
+# the most reliable way to install the interaction-fingerprint stack.
+name: zein-md
+channels:
+  - conda-forge
+dependencies:
+  - python=3.11
+  - numpy>=1.24
+  - scipy>=1.10
+  - pandas>=2.0
+  - matplotlib>=3.7
+  - seaborn>=0.12
+  - tqdm>=4.65
+  - pyyaml>=6.0
+  - scikit-learn>=1.3
+  - networkx>=3.0
+  - mdanalysis>=2.7
+  - mdtraj>=1.10
+  - rdkit>=2023.3
+  - pip
+  - pip:
+      - panedr>=0.8
+      - prolif>=2.0

examples/alpha_zein_A8HNE1/README.md

@@ -0,0 +1,26 @@
+# Example: α-zein (A8HNE1)
+
+This directory holds the **reference example** that seeded the toolkit: a 100 ns
+GROMACS simulation of an AlphaFold 3 α-zein model (187 residues, explicit
+solvent, CHARMM36).
+
+- `config.yaml` — a complete, reproducible mdforge workflow for this system.
+- `figures/`, `results/`, `tables/`, `report/` — outputs produced by the original
+  project-specific pipeline (now preserved in [`../../legacy/`](../../legacy)),
+  kept here as a worked example of what a full analysis looks like.
+
+## Reproduce with mdforge
+
+```bash
+# edit input_dir in config.yaml to point at your copy of the raw files, then:
+mdforge analyze --config examples/alpha_zein_A8HNE1/config.yaml
+```
+
+The raw simulation files (`step5_production.xtc/.tpr/.edr/...`) are **not**
+included (multi-GB); point `input_dir` at your own copy.
+
+## Key finding (reference)
+
+The extended AlphaFold model **compacts** (Rg 4.2 → 3.5 nm) while retaining its
+helical content (~55 %) and most native contacts (Q ≈ 0.89) — characteristic of
+an elongated, flexible seed-storage protein. See `report/analysis_report.md`.

examples/alpha_zein_A8HNE1/config.yaml

@@ -0,0 +1,37 @@
+# ---------------------------------------------------------------------------
+# Reproducible mdforge workflow for the alpha-zein (A8HNE1) system.
+#
+# Run it with a single command:
+#     mdforge analyze --config examples/alpha_zein_A8HNE1/config.yaml
+#
+# CLI flags override anything set here, so the same file works on HPC/batch.
+# ---------------------------------------------------------------------------
+
+# Point this at your GROMACS simulation directory (searched recursively).
+input_dir: /path/to/A8HNE1_MDS
+output_dir: examples/alpha_zein_A8HNE1/mdforge_out
+
+# Trajectory slicing / performance
+start: 0
+end: null            # null => last frame
+stride: 1            # analyse every frame (increase for a quick look)
+core_extract: true   # cache a solute-only trajectory for fast structural analyses
+
+# Analysis selection
+analyses: null       # null => auto-select for the detected system
+exclude: []          # e.g. [sasa] to skip
+plugin_dirs: []      # extra folders of drop-in analysis plugins
+
+# Reproducibility / reporting
+seed: 0
+report_formats: [md, html]
+
+# Per-analysis parameter overrides (optional)
+params:
+  rmsd:
+    ref_frame: 0
+    window: 20
+  rmsf:
+    ref_frame: 0
+  rog:
+    window: 20

mdforge/__init__.py

@@ -0,0 +1,38 @@
+"""
+mdforge -- a reusable, extensible framework for analysing GROMACS molecular
+dynamics simulations of arbitrary biomolecular systems.
+
+Point it at a simulation directory and it will discover the files, detect the
+system composition (protein / ligand / nucleic acid / membrane / ions / ...),
+select the appropriate analyses, run them with streaming-friendly performance,
+and produce publication-quality figures plus a fully reproducible report.
+
+Public API
+----------
+    from mdforge import analyze, detect_system, __version__
+"""
+
+from __future__ import annotations
+
+__version__ = "0.2.0"
+__author__ = "Hossam Mahmoud"
+
+# Re-export the most commonly used entry points (kept import-light).
+from mdforge.core.system import detect_system, SystemInfo, SystemType, ComponentType  # noqa: E402
+from mdforge.core.registry import registry  # noqa: E402
+
+__all__ = [
+    "__version__",
+    "detect_system",
+    "SystemInfo",
+    "SystemType",
+    "ComponentType",
+    "registry",
+    "analyze",
+]
+
+
+def analyze(*args, **kwargs):
+    """Convenience wrapper around :func:`mdforge.core.pipeline.run_pipeline`."""
+    from mdforge.core.pipeline import run_pipeline
+    return run_pipeline(*args, **kwargs)

mdforge/analysis/__init__.py

@@ -0,0 +1,7 @@
+"""
+Built-in analysis modules.
+
+Every module here defines one or more :class:`~mdforge.core.base.BaseAnalysis`
+subclasses, which auto-register on import.  The registry imports all modules in
+this package (except ``plugins``, handled separately) at selection time.
+"""