Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 0 additions & 40 deletions api/worker/v1alpha1/generated.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 5 additions & 14 deletions api/worker/v1alpha1/generated.proto

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 5 additions & 14 deletions api/worker/v1alpha1/model_deployment.go
Original file line number Diff line number Diff line change
Expand Up @@ -557,15 +557,15 @@ type ModelDeploymentRole struct {
// one dedicated logical Ray cluster for the deployment, with a CPU-only control-plane head
// and one whole-GPU TP group per DP engine rank. PRESENCE IS THE PROFILE DISCRIMINANT: a role
// without it renders exactly as it did before this field existed, and a profile that is
// turned on and off again describes a different deployment, so the field's presence and
// HeadInstanceType are frozen at creation. Width is editable for scale-up; scale-down is
// turned on and off again describes a different deployment. The field's presence is frozen
// at creation. Width is editable for scale-up; scale-down is
// refused by admission in this release.
//
// The profile admits exactly one such role per deployment, running the vLLM engine as a
// single Server role of one instance of one Pod. Every member -- the reserved API/DP-master
// and every Ray-only worker -- takes a complete TP group declared through ExtraArgs.
// All GPUs are whole, unsliced and unpartitioned. Admission binds the resource count to TP.
// The head uses HeadInstanceType's own queue and never counts toward Width.
// The auxiliary head requests no container resources and never counts toward Width.
//
// +optional
ElasticEP *ModelDeploymentRoleElasticEP `json:"elasticEp,omitempty" protobuf:"bytes,18,opt,name=elasticEp"`
Expand All @@ -576,7 +576,8 @@ type ModelDeploymentRole struct {
// Width is the only mutable field: it answers how large the collective currently should be,
// which is a running-state question. This release accepts increases and unchanged values;
// admission refuses a decrease until Elastic EP scale-down support is complete.
// Profile presence, HeadInstanceType and the TP degree in ExtraArgs are fixed at creation.
// The role cannot enable or disable this profile after creation.
// Its tensor parallel size comes from ExtraArgs and cannot change after creation.
type ModelDeploymentRoleElasticEP struct {
// Width is the total number of GPU engines in the elastic collective, INCLUDING the
// reserved API/DP-master member; width-1 of them are Ray-only workers. It is the total
Expand All @@ -587,16 +588,6 @@ type ModelDeploymentRoleElasticEP struct {
// +k8s:validation:minimum=2
// +k8s:validation:maximum=64
Width int32 `json:"width" protobuf:"varint,1,name=width"`

// HeadInstanceType is the name of the CPU-only InstanceType the Ray control-plane head
// runs against. The head is admitted as its own CPU-accounted workload on that type's
// queue: it takes no accelerator, joins no GPU queue at zero charge, and is not a width
// member.
//
// +required
// +k8s:validation:minLength=1
// +k8s:validation:maxLength=253
HeadInstanceType string `json:"headInstanceType" protobuf:"bytes,2,name=headInstanceType"`
}

// ModelDeploymentRoleTopology is the topology request for one independent replica group.
Expand Down
9 changes: 1 addition & 8 deletions api/worker/v1alpha1/zz_generated.crds.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 3 additions & 13 deletions api/worker/zz_generated.openapi.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 10 additions & 1 deletion docs/modules/model-deployment/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,11 +298,20 @@ webhook-level one, so nothing here should be read as a claim about how that issu

### Shared memory

`roles[].shmSize` sets the capacity limit of `/dev/shm` in each role Pod. It accepts a positive Kubernetes
`spec.roles[].shmSize` sets the capacity limit of `/dev/shm` in each role Pod. It accepts a positive Kubernetes
quantity, such as `32Gi`. Omission renders a `16Gi` memory-backed `EmptyDir`.
This applies to both engines, take-over roles, and the Elastic Ray head and GPU members.
Each Pod has its own volume. Only the main container mounts it.

Set it in the role configuration:

```yaml
spec:
roles:
- name: server
shmSize: 32Gi
```

An explicit `/dev/shm` mount in `additionalVolumes` takes precedence, including when `shmSize` is set.
The operator preserves that mount's capacity and backing.
The default is a project capacity choice, not a universal engine requirement.
Expand Down
Loading
Loading