Repository navigation
feat(worker): allow fixed TP groups in Elastic EP - #743
Merged
Merged
Conversation
Default omitted TP to one and support two whole GPUs per DP member. Preserve width semantics and master identity during scale-up. Task 1 of 3: Elastic EP tensor parallel groups
Count each full GPU allocation as one DP member. Require unit-GPU placement bundles on the native actor node for TP=2 and hold uncertain observations. Task 2 of 3: Elastic EP tensor parallel groups
Document immutable TP size, DP member width, GPU demand, and pending post-merge physical acceptance. Register the owning spec in the module map. Task 3 of 3: Elastic EP tensor parallel groups
|
✅ OpenCodeReview: Review complete: 0 finding(s) across 2 selected item(s). Reviewed |
Collaborator
Author
|
Review round 1 is addressed at 5105958. Read all feedback buckets: one top-level review, three inline findings, and two additional findings in the original summary. The incremental review now reports zero new findings.
Validation after the fixes: clean-source full make test with race/shuffle, make lint, make lint docs, and make generate all passed. Physical TP=2 testing remains pending the merged CI image. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind enhancement
/area worker
What this PR does / why we need it:
The finalized main implementation allows fixed TP groups in Elastic EP. TP is read from the existing engine
extraArgsparser and remains immutable after creation.elasticEp.widthchanges the DP member count.Which issue(s) this PR links to:
NONE
Special notes for your reviewer:
e9d9c47c3b8baa1e6289d484daf0f092a0f93b4cthrough an explicitly authorized history rewrite. This description records the finalized main contract; the original PR head and merge record remain historical.NotEnoughResources; TP=2 inference and DP2-to-DP4 resize did not execute. The test resources were fully cleaned up.Does this PR introduce a user-facing change?