Skip to content

feat(modeldeployment): configure per-role shared memory - #748

Merged
thxCode merged 2 commits into
mainfrom
spec/model-role-shm-size
Oct 8, 2026
Merged

thxCode merged 2 commits into
mainfrom
spec/model-role-shm-size

Conversation

@thxCode

@thxCode thxCode commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

What type of PR is this?

/kind enhancement
/kind api-change
/area worker

What this PR does / why we need it:

ModelDeployment roles cannot size their shared memory. Ordinary Pods use the container runtime default;
Elastic GPU members use a fixed 512Mi volume, and their CPU head has no default mount.

Add optional roles[].shmSize as a positive Kubernetes quantity. Render an isolated memory-backed
/dev/shm volume with a default limit of 16Gi for ordinary roles, take-over roles, and Elastic head/master/workers.
Preserve explicit /dev/shm mounts in additionalVolumes. Only the main container mounts the generated volume.
Regenerate the API schema, protobuf, deepcopy, OpenAPI and apply configuration.

Which issue(s) this PR links to:

NONE

Special notes for your reviewer:

  • Ordinary capacity edits follow recreate rollout. Existing Elastic members keep their mounts; new or replacement members use the current value.
  • The limit does not reserve RAM or raise memory requests. Kubelet can lower the mount size to the Pod or node memory budget.
  • 16Gi is a project default, not a guarantee that every engine workload fits. Dynamic TP>1 inference remains unverified.
  • Regression cases cover both engines, ordinary and Elastic Pods, custom capacities, explicit mount precedence, invalid quantities, spec hash changes, and existing Elastic member retention.
  • All nine serialized render fixtures were compared with the prior renderer. Only the shared-memory volume, main mount and spec hash changed.
  • make test passed for 99 packages with race, shuffle and coverage. Focused controller/webhook tests, make lint, make lint docs and post-lint generation passed.
  • No GPU workload or paid cluster was started for this change. In-cluster verification remains part of the planned combined test window.

Does this PR introduce a user-facing change?

Configure ModelDeployment shared memory through roles[].shmSize, defaulting to a 16Gi memory-backed /dev/shm limit per Pod. Explicit additionalVolumes mounts take precedence. Action required: upgrading adds the default mount to ordinary roles and triggers their recreate rollout. Existing Elastic members retain their mounts until replacement.

Add a positive shmSize quantity with a 16Gi rendered default. Preserve explicit mounts and existing Elastic members. Update generated APIs, render checks and the owning guides.

Signed-off-by: thxCode <thxcode0824@gmail.com>
@gpustack-code-review

gpustack-code-review Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

✅ OpenCodeReview: Review complete: 0 finding(s) across 1 selected item(s).

Reviewed 24ac71f..1e368ee only; earlier commits in this PR were reviewed in a previous run.

Signed-off-by: thxCode <thxcode0824@gmail.com>
@thxCode

thxCode commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator Author

Review round 1: checked all feedback buckets (0 reviews, 0 inline comments, 1 summary finding).

The documentation finding is valid. The Elastic EP page now describes only its head/member mounts and replacement behavior. It links to the deployment configuration page for defaults, explicit mount precedence and memory budgeting.

make lint docs passed after this edit. make generate completed with no generated drift. This is a documentation-only fixup of the feature commit; the existing full CPU test evidence still applies. No GPU verification is claimed.

@thxCode
thxCode merged commit 0c3d9e7 into main Oct 8, 2026
10 checks passed
@thxCode
thxCode deleted the spec/model-role-shm-size branch October 8, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant