Pipeline-Parallel Lecture: Simplest Dualpipe Implementation.
Inspired by DeepSeek-V3 and DualPipe, We designed and implemented a simplest dualpipe-like schedule.
Blog: 【手撕DualPipe】让我们一步步把 MoE EP 通信 "消除"（附代码）
run dualpipe_step.py and dualpipe.py
this repo is part of lecture: "LLM from scratch". This repo Not to be used for any commercial purposes without permission.
License: CC-BY-NC-ND-4.0