Skip to content

feat(model-deployment): allow ordinary role configuration updates - #757

Merged
thxCode merged 19 commits into
mainfrom
pteropod
Oct 9, 2026
Merged

thxCode merged 19 commits into
mainfrom
pteropod

Conversation

@thxCode

@thxCode thxCode commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

What type of PR is this?

/kind enhancement
/area worker

What this PR does / why we need it:

  • Allows ordinary ModelDeployment.spec.roles[] updates to size, resources, instanceType, and command through whole-replica replacement with fresh Kueue admission.
  • Preserves role identity, untouched Prefill/Decode siblings, deployed routing, leader selection, and peer DNS while old members remain during replacement.
  • Retains peer Services for desired and still-standing surplus ordinals, then prunes them after the last member leaves.
  • Adds case 115 scheduling coverage for queued edits, size changes, whole-card/slice transitions, command updates, P/D sibling identity, and ownership-safe cleanup after an ambiguous create response.
  • Keeps Elastic EP and hardware-specific restrictions explicit in the documentation.

Validation:

  • make test passed for the full repository with race detection, shuffled tests, and coverage.
  • make lint, make lint docs, make lint agents-shell, make build, make generate (two passes), and git diff --check passed.
  • Independent Spec and Standards reviews found no confirmed Spec or Service-rendering defect. The final local review fixed the case 115 ambiguous-create cleanup gap before this PR.
  • Runtime validation passed the applicable role-update and inference legs. Ascend, native RDMA, Elastic EP replacement, and unavailable hardware geometry remain documented limits.

Which issue(s) this PR links to:

NONE

Special notes for your reviewer:

  • member-index=0 is shape-dependent: ordinary multi-member roles select the leader, External DP selects qualified ranks, and mixed old/new configurations use the deployed member shape until replacement completes.
  • Runtime evidence includes real Kubernetes/Kueue object transitions and serving checks. Placeholder legs do not claim engine inference or serving continuity.
  • The Spec records only role-mutability evidence and its limits; unrelated campaign failures remain outside the Spec.

Does this PR introduce a user-facing change?

ModelDeployment roles can update size, resources, instanceType, and command through safe whole-replica replacement while preserving role identity and peer routing during rollout.

Signed-off-by: thxCode <thxcode0824@gmail.com>
Read running group size and execution from member Pods. Keep health, cache, status, joint admission, and retirement independent of the replacement configuration.

Task 1 of model-deployment-role-mutability.

Signed-off-by: thxCode <thxcode0824@gmail.com>
- Keep role names and kinds fixed while allowing shape and command edits.
- Revalidate changed role dependencies and preserve external drift access.
- Retain Elastic EP constraints and document whole-group replacement.

Task 2 of model-deployment-role-mutability.

Signed-off-by: thxCode <thxcode0824@gmail.com>
Describe recovery in the active replica slot and retain deployed routing during updates. State that fresh admission can advance replacement before the engine is ready.

Task 2 of model-deployment-role-mutability.

Signed-off-by: thxCode <thxcode0824@gmail.com>
Persist one replacement per role through cleanup, creation, and fresh admission. Preserve deployed routing and peer services during size and command changes. Recover interrupted writes and member loss without replacing healthy siblings.

Task 3 of model-deployment-role-mutability.

Signed-off-by: thxCode <thxcode0824@gmail.com>
Generate from the verified source commit and compare two complete generation passes. Keep ordinary role editability and Elastic EP restrictions aligned in published schemas and clients.

Task 4 of model-deployment-role-mutability.

Signed-off-by: thxCode <thxcode0824@gmail.com>
Read current members before synchronizing Services. Preserve their
answering shape before the first replacement appears. Avoid empty
replacement candidates when a legacy Pod has no ordinal.

Assert complete selectors and retain qualification revocation.
Document parallelism as the desired role configuration.

Signed-off-by: thxCode <thxcode0824@gmail.com>
Mark all implementation tasks complete. Record local race tests, lint,
generation parity, compilation, and review corrections.

Keep runtime acceptance pending for user-arranged testing before PR creation.

Signed-off-by: thxCode <thxcode0824@gmail.com>
@gpustack-code-review

gpustack-code-review Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

✅ OpenCodeReview: Review complete: 0 finding(s) across 1 selected item(s).

Reviewed 21b4c35..e88f1e2 only; earlier commits in this PR were reviewed in a previous run.

Comment thread pkg/worker/controllers/worker/model_deployment_pod_group.go Outdated
Comment thread pkg/worker/controllers/worker/model_deployment_replacement_slot.go
Comment thread pkg/worker/controllers/worker/model_deployment_replacement_slot.go
Comment thread pkg/worker/controllers/worker/model_deployment_service.go Outdated
Comment thread pkg/worker/controllers/worker/model_deployment.go
Signed-off-by: thxCode <thxcode0824@gmail.com>
Ignore placement pins added only to Workload templates when proving fresh replacement admission. Preserve member affinity and all existing ownership, resource and queue checks.

Add regression controls for sliced, shared and model-manager placement pins. Record the runtime validation still in progress.

Task 5 of model-deployment-role-mutability.

Signed-off-by: thxCode <thxcode0824@gmail.com>
Preserve per-replica peer DNS while deployed members remain during role configuration replacement. Cover standing and terminating surplus members, harden case 115 ownership-safe cleanup, and record the scoped runtime validation.

Signed-off-by: thxCode <thxcode0824@gmail.com>
…ement

Scope the front Service gate to the front role's standing members,
treat an unreadable External-DP reading as unknown instead of
leader-served, and return replacement server read failures as errors
so the slot requeues instead of waiting for an event that never comes.

Signed-off-by: thxCode <thxcode0824@gmail.com>
Signed-off-by: thxCode <thxcode0824@gmail.com>
@thxCode

thxCode commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed in ca03323 and 2e85bc2.

Fixed:

  • model_deployment_pod_group.go: unreadable External-DP reading now holds membership (tri-state).
  • model_deployment_replacement_slot.go: server read failures are returned as errors, so the slot requeues.
  • model_deployment_service.go: the front Service gate reads only the front role's standing members.
  • admission.md: the stop gate is stated for the whole template, with spec.volume as the only unconditional exception.
  • elastic-ep.md: the Limitations entry now points to the section above instead of repeating the frozen fields.
  • status.md: the rollout rule is left to deployment.md#rollout-behavior.
  • prefill-decode.md: the shared-document mechanism now lives in deployment.md only.

Not changed here: the two performance comments (per-pass uncached Workload lists and per-role Service reads). The uncached reads guard against informer lag in the replacement window, so replacing them needs its own change. Follow-up.

Also added the missing Signed-off-by to one earlier commit; the branch history was rewritten with no content change. Tests: go test ./pkg/worker/..., make lint and make lint docs pass.

Comment thread pkg/worker/controllers/worker/model_deployment_initial_queue.go
Comment thread pkg/worker/controllers/worker/model_deployment_replacement_slot.go Outdated
Comment thread pkg/worker/controllers/worker/model_deployment_replacement_slot.go
Comment thread pkg/worker/controllers/worker/model_deployment.go
Comment thread pkg/worker/controllers/worker/model_deployment.go
Comment thread pkg/worker/controllers/worker/model_deployment.go
Comment thread pkg/worker/webhooks/worker/model_deployment.go
Comment thread pkg/worker/webhooks/worker/model_deployment.go
…ents

Signed-off-by: thxCode <thxcode0824@gmail.com>
Signed-off-by: thxCode <thxcode0824@gmail.com>
Signed-off-by: thxCode <thxcode0824@gmail.com>
@thxCode

thxCode commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed the latest review round (8 inline threads and 6 summary-routed findings, 14 in total, matching the bot's count). The fixes are committed as fixups of the commits that own the code and will be pushed as a fast-forward.

Fixed

  • Initial-queue check (model_deployment_initial_queue.go): cached status that already shows a node assignment or container status now refuses the replica before any API-server Get. Test added.
  • Replacement slot cleanup (model_deployment_replacement_slot.go): replacementWorkloadDeletes deletes only the Workload the slot captured; the unreachable uncaptured-delete branch and its unused parameters are removed. Test added.
  • Retirement hold reason (model_deployment_retirement.go): a replica with no members now reports the replica has no members to read instead of a missing engine container. Test added.
  • Rollout log (model_deployment.go): the broken-first path now logs short of its members as its reason.
  • kubectl-shim: trap collapsed to a single expansion, same behavior.
  • docs/modules/kv-cache/pool.md: the deletion sentence moved after the reason list so the paragraph reads as one unit.

Not changed, with reasons (replied in each thread)

  • Index-out-of-range panic at model_deployment.go:589: desired[role] is a map keyed by ordinal, so a missing ordinal returns nil and does not panic; verified.
  • Namespace-wide Workload lists (two threads): Kueue copies only configured labelKeysToCopy onto the Workload and this repo configures none, so the suggested label filter would match nothing; the vacancy read also has to come from the API server.
  • Uncached pod list before the Service sync: deliberate, documented in the code, to avoid retiring peer records of members still running.
  • poolRef re-judge in the webhook: spec.kvCache is rejected on update by the identity check, so it cannot change.

Already satisfied (no change): the three reminders to include regression tests and regenerated output. Tests for those contracts are in the PR, and this round changes no API types or markers.

Checks run locally: make lint (0 issues) and go test ./pkg/worker/... ./api/worker/... pass. make generate cannot run in this worktree because the generators require a checkout path ending in gpustack.ai/gpustack; CI's verify-generated covers it.

Resolved the two threads whose fixes landed; the others stay open for the reviewer.

Comment thread pkg/worker/controllers/worker/model_deployment_replacement_slot.go
…ents

Signed-off-by: thxCode <thxcode0824@gmail.com>
…ents

Signed-off-by: thxCode <thxcode0824@gmail.com>
@thxCode

thxCode commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed the follow-up review on the previous fixups (1 inline thread, no findings routed to the summary).

Fixed

  • Replacement slot cleanup (model_deployment_replacement_slot.go): a slot that captured no Workload UID, because the Workload had not reached the informer yet, can now still release it when the Workload names a captured member as an owner. Name matches and ownerless Workloads remain untouched. Added an end-to-end test of cleanReplacementSlot for the empty-WorkloadUID case, and confirmed it fails without the fix.

Verification

  • make lint reports 0 issues, go test ./pkg/worker/... passes, and make generate leaves the tree clean on the fix commit (the commit after it only adds a test).

@thxCode
thxCode merged commit 10b5a0b into main Oct 9, 2026
12 checks passed
@thxCode
thxCode deleted the pteropod branch October 10, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant