Skip to content

feat(worker): allow fixed TP groups in Elastic EP - #743

Merged
thxCode merged 5 commits into
mainfrom
spec/elastic-ep-tensor-parallel
Oct 7, 2026
Merged

thxCode merged 5 commits into
mainfrom
spec/elastic-ep-tensor-parallel

Conversation

@thxCode

@thxCode thxCode commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

What type of PR is this?

/kind enhancement
/area worker

What this PR does / why we need it:

The finalized main implementation allows fixed TP groups in Elastic EP. TP is read from the existing engine extraArgs parser and remains immutable after creation. elasticEp.width changes the DP member count.

  • Remove the earlier TP=1 profile restriction without adding a TP API field or a TP=2 upper limit.
  • Default and validate each member's whole-GPU quantity against its configured TP size.
  • Count only complete TP allocations and Ray placement groups. Reject incomplete, duplicate, ambiguous, or cross-node groups.
  • Preserve the master during DP scale-up and reject online changes to the effective TP size. Equivalent argument spellings remain valid.
  • Update the existing Elastic EP guide and owning spec. No separate TP spec is retained.

Which issue(s) this PR links to:

NONE

Special notes for your reviewer:

  • The original squash merge was superseded on main by e9d9c47c3b8baa1e6289d484daf0f092a0f93b4c through an explicitly authorized history rewrite. This description records the finalized main contract; the original PR head and merge record remain historical.
  • Scale-up and unchanged width are supported. Scale-down remains outside this change. Online TP adjustment is not supported.
  • CPU regression cases cover TP=1, 2, 4, and 8, argument parsing and immutable updates, complete-card allocation, and Ray placement counterexamples. These cases do not establish physical support for every model or hardware shape.
  • The full race/shuffle/coverage suite passed in a clean source export. Core tests, source/docs lint, generation, and independent read-only review also passed.
  • The published amd64 image and a live CPU worker binary matched the finalized source commit. The preemptible eight-GPU test environment reported NotEnoughResources; TP=2 inference and DP2-to-DP4 resize did not execute. The test resources were fully cleaned up.
  • The related infrastructure recovery documentation is in PR 744.

Does this PR introduce a user-facing change?

Elastic EP members can use fixed TP settings from engine extraArgs. Width changes the DP member count, and each member holds a complete TP GPU group. TP remains fixed after creation.

Default omitted TP to one and support two whole GPUs per DP member. Preserve width semantics and master identity during scale-up.

Task 1 of 3: Elastic EP tensor parallel groups
Count each full GPU allocation as one DP member. Require unit-GPU placement bundles on the native actor node for TP=2 and hold uncertain observations.

Task 2 of 3: Elastic EP tensor parallel groups
Document immutable TP size, DP member width, GPU demand, and pending post-merge physical acceptance. Register the owning spec in the module map.

Task 3 of 3: Elastic EP tensor parallel groups
@thxCode thxCode added kind/enhancement New feature or request kind/api-change Adds, removes, or changes an API area/worker The worker control plane and its controllers labels Oct 7, 2026
@gpustack-code-review

gpustack-code-review Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

✅ OpenCodeReview: Review complete: 0 finding(s) across 2 selected item(s).

Reviewed 257dea5..5105958 only; earlier commits in this PR were reviewed in a previous run.

Comment thread api/worker/v1alpha1/model_deployment.go
Comment thread pkg/worker/controllers/worker/model_deployment_elastic_observer.go
Comment thread pkg/worker/webhooks/worker/model_deployment.go
@thxCode

thxCode commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator Author

Review round 1 is addressed at 5105958.

Read all feedback buckets: one top-level review, three inline findings, and two additional findings in the original summary. The incremental review now reports zero new findings.

Finding Outcome
API generation reminder Generated API artifacts are included. Regeneration leaves the current tree unchanged; verify-generated CI passed.
Empty placement-bundle resources Kept the fail-closed contract. CPU-only bundles with stated CPU resources pass. Missing resources remain Unknown because they cannot establish GPU bundle cardinality. Missing actor/node identity is refused by the join before this helper.
Webhook generation and tests reminder Generated webhook output remains unchanged. Admission/default/immutability regression tests are included.
Empty sibling allocation records Fixed in 50fd8e0. Empty sibling records are accepted. Sibling containers that hold accelerators are still refused with a specific reason. A regression test failed before the fix and passed after it.
Guide validation rationale Updated in 5105958. The guide keeps the model compatibility requirement and links to the owning specification for validation scope.

Validation after the fixes: clean-source full make test with race/shuffle, make lint, make lint docs, and make generate all passed. Physical TP=2 testing remains pending the merged CI image.

@thxCode
thxCode merged commit dc3efd7 into main Oct 7, 2026
8 checks passed
@thxCode thxCode changed the title feat(worker): support Elastic EP tensor parallel groups feat(worker): allow fixed TP groups in Elastic EP Oct 7, 2026
@thxCode
thxCode deleted the spec/elastic-ep-tensor-parallel branch October 8, 2026 03:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/worker The worker control plane and its controllers kind/api-change Adds, removes, or changes an API kind/enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant