Skip to content

update parallel state following Megatron-LM parallel_state.py 2025.1 - #32

Open
1195343015 wants to merge 4 commits into
aliyun:masterfrom
1195343015:update-parallel-state
Open

update parallel state following Megatron-LM parallel_state.py 2025.1#32
1195343015 wants to merge 4 commits into
aliyun:masterfrom
1195343015:update-parallel-state

Conversation

@1195343015

Copy link
Copy Markdown
Contributor
  1. update parallel state following Megatron-LM parallel_state.py 2025.1
  2. add cp argument and cp_group for developing in future
  3. fix some small issues (like when world_size or comm_size is 1, there should be no communications)

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@1195343015

Copy link
Copy Markdown
Contributor Author

CC @zhouheyang-alibaba

@1195343015
1195343015 force-pushed the update-parallel-state branch from 1cdfeaf to eac4527 Compare May 30, 2025 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants