Focus, Align, and Diffuse:
Time-Series-Aware Keyframe Diffusion for Cardiac Dynamic Synthesis from Sparse Observations

Junkai Liu¹, Haofan Wu¹, Nay Aung², Joao A. C. Lima³, Steffen E. Petersen², Le Zhang¹

¹University of Birmingham   ²Queen Mary University of London   ³Johns Hopkins University

CardioFAD is a time-series-aware keyframe diffusion framework for cardiac dynamic synthesis from sparse observations.

📑 Table of Contents

• 🔎 Overview
• 💡 Key Ideas
• 🎬 Video Comparisons
• 📊 Experimental Scope
• 💿 Data
• 📁 Repository Layout
• 🛠️ Usage
• 📜 License
• 🙏 Acknowledgements
• 📮 Contact
• 🔗 Citation

🔎 Overview

CardioFAD synthesizes complete cardiac dynamics from a single CMR volume, CMR slice, or ECHO frame. It learns motion-salient keyframe anchors for trajectory recovery and ECG–visual alignment, then uses two-stage diffusion to generate the full sequence. This design improves visual quality, temporal coherence, and cardiac functional consistency under extreme one-shot sparsity.

💡 Key Ideas

The Focus–Align–Diffuse framework has three components:

Component	Description
🎯 Focus	Learns trajectory-sufficient keyframes without predefined ED/ES landmarks, guided by the Trajectory Sufficiency Principle.
🔗 Align	Uses keyframe-weighted ECG–visual alignment to form reliable cross-modal anchors, guided by the Anchor Alignment Principle.
🌊 Diffuse	Uses two-stage diffusion to generate keyframes first, then interpolate the remaining frames.

🎬 Video Comparisons

Left to right: Ground truth and CardioFAD. These examples use the videos from our project page, where more examples and interactive controls are available.

CMR · Volume2Seq

One-volume-conditioned cardiac dynamics, shown at the middle slice (7 / 13).

cmr-comparison.mp4

Echo · Frame2Video

One-frame-conditioned echocardiography video from EchoNet-Dynamic.

echo-comparison.mp4

📊 Experimental Scope

CardioFAD is evaluated on both 3D+t CMR and 2D+t ECHO synthesis tasks.

3D+t CMR synthesis

• Volume2Seq
• Slice2Seq

2D+t ECHO synthesis

• Frame2Video on EchoNet-Dynamic
• Frame2Video on EchoNet-Pediatric A4C
• Frame2Video on EchoNet-Pediatric PSAX

Evaluation covers visual quality and temporal coherence (PSNR, SSIM, LPIPS, FID, and FVD), alongside cardiac functional consistency assessed through ejection fraction (EF) estimation.

💿 Data

The two echocardiography datasets used in our experiments are available from their official download pages:

Dataset	Views used in CardioFAD	Download
EchoNet-Dynamic	Apical four-chamber (A4C)	Download
EchoNet-Pediatric	A4C and parasternal short-axis (PSAX)	Download

📁 Repository Layout

CardioFAD/
├── tokenizer/
│   ├── CMR/          # CMR tokenizer training
│   └── Echo/         # Echo preprocessing and tokenizer training
├── generation/
│   ├── CMR/          # Latent extraction, Slice2Temp, and Volume2Temp
│   └── Echo/         # Latent extraction and Frame2Video
├── assets/           # README figures and video comparisons
├── requirements.txt
├── LICENSE           # Apache-2.0
└── README.md

🛠️ Usage

Use each script's --help to view its required arguments.

CMR

Step	Script
Train the tokenizer	train_tokenizer_CMR.py
Extract latents	get_latent.py
Train Slice2Temp	train_slice2temp.py
Sample Slice2Temp	sample_slice2temp.py
Train Volume2Temp	train_volume2temp.py
Sample Volume2Temp	sample_volume2temp.py

Echo

Step	Script
Generate pseudo ECG	generate_pseudoECG.py
Preprocess videos	preprocess_echo.py
Train the tokenizer	train_tokenizer_ECHO.py
Extract latents	get_latent.py
Train Frame2Video	train_frame2video.py
Sample Frame2Video	sample_frame2video.py

📜 License

This project is released under the Apache-2.0 license.

Third-party components and datasets remain subject to their respective licenses and terms of use.

🙏 Acknowledgements

This repository builds on MONAI, LeanVAE, DiT, SiT, and Latte. We thank the authors and contributors for sharing their code and research.

📮 Contact

For questions, please contact Junkai Liu at jxl1920@student.bham.ac.uk or liujk1997@gmail.com.

🔗 Citation

If our work helps your research, please consider giving us a star ⭐ or citing CardioFAD.

@article{liu2026focus,
  title={Focus, Align, and Diffuse: Time-Series-Aware Keyframe Diffusion for Cardiac Dynamic Synthesis from Sparse Observations},
  author={Liu, Junkai and Wu, Haofan and Aung, Nay and Lima, Joao AC and Petersen, Steffen E and Zhang, Le},
  journal={arXiv preprint arXiv:2610.19050},
  year={2026}
}