Expressive Portrait Image Animation for Live Streaming

Zhiyuan Li^1,2,3 · Chi-Man Pun¹ 📪 · Chen Fang² · Jue Wang² · Xiaodong Cun³ 📪

¹ University of Macau    ² Dzine.ai    ³ GVC Lab, Great Bay University

  

📣 Updates

• [2025.12.12] 🔥 Release inference code, config and pretrained weights!

⚙️ Framework

We present PersonaLive, a real-time and streamable diffusion framework capable of generating infinite-length portrait animations on a single 12GB GPU.

🚀 Getting Started

🛠 Installation

# clone this repo
git clone https://github.com/GVCLab/PersonaLive
cd PersonaLive

# Create conda environment
conda create -n personalive python=3.10
conda activate personalive

# Install packages with pip
pip install -r requirements.txt

⏬ Download weights

1. Download pre-trained weight of based models and other components (SD V1.5, sd-vae-ft-mse and image_encoder)

2. Download pre-trained weights into the ./pretrained_weights folder.

   

Finally, these weights should be orgnized as follows:

pretrained_weights
├── onnx
│   ├── unet_opt
│   │   ├── unet_opt.onnx
│   │   └── unet_opt.onnx.data
│   └── unet
├── personalive
│   ├── denoising_unet.pth
│   ├── motion_encoder.pth
│   ├── motion_extractor.pth
│   ├── pose_guider.pth
│   ├── reference_unet.pth
│   └── temporal_module.pth
├── sd-vae-ft-mse
│   ├── diffusion_pytorch_model.bin
│   └── config.json
└── sd-image-variations-diffusers
│   ├── image_encoder
│   │   ├── pytorch_model.bin
│   │   └── config.json
│   ├── unet
│   │   ├── diffusion_pytorch_model.bin
│   │   └── config.json
│   └── model_index.json
└── tensorrt
    └── unet_work.engine

🎞️ Offline Inference

python inference_offline.py

📸 Online Inference

📦 Setup Web UI

# install Node.js 18+
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
nvm install 18

cd webcam
source start.sh

🏎️ Acceleration (Optional)

Converting the model to TensorRT can significantly speed up inference (~ 2x ⚡️). Building the engine may take about 20 minutes depending on your device. Note that TensorRT optimizations may lead to slight variations or a small drop in output quality.

python torch2trt.py

▶️ Start Streaming

python inference_online.py

then open http://0.0.0.0:7860 in your browser. (*If http://0.0.0.0:7860 does not work well, try http://localhost:7860)

❤️ Acknowledgement

This code is mainly built upon Moore-AnimateAnyone, X-NeMo, StreamDiffusion, RAIN and LivePortrait, thanks to their invaluable contributions.