[janosmurai]

Type of change

• New feature
• Bug fix
• Breaking change
• Refactoring or maintenance
• Documentation update

Related issue

Closes:

• SHD-1091 [gateway-provider] llm-d v1.0 CRDs and Istio
• SHD-1092 [app-serving] llm-d v1.0 model serving
• SHD-1093 [Installer] llm-d v1.0 model serving
• SHD-1095 inference simulator as a selectable model (app-serving and installer)

Part of SHD-907 (llm-d v1.0 update). Does not close it: its server part, SHD-1094, is axem-solutions/shaide_server#38.

Description

Moves model serving to llm-d v1.0 and keeps Gateway Mode (one Istio Gateway per model).

• gateway-provider: Gateway API and Inference Extension CRDs v1.6.2, the llm-d router CRDs and Istio 1.29.2.

• app-serving: each model is described by one model.yaml (shared schema in pkg/iac/serving/modeldef) instead of per-chart values files. The retired llm-d-infra, inferencepool and llm-d-modelservice charts are replaced by:

  • a platform-managed Istio Gateway, with the discovery Service shaide-server already uses;
  • the vendored llm-d-router-gateway v1.0.0 chart (InferencePool and EPP, FailOpen);
  • a platform-managed vLLM Deployment and HTTPRoute.

  Before deploying, a migration step deletes the old modelservice Deployments so their GPUs are freed. Router and proxy requests are sized to fit 4 vCPU nodes.

• installer: reads model.yaml in the catalog, mirrors vllm/vllm-openai, the llm-d router EPP and Istio 1.29.2, checks driver compatibility against the vLLM image and bundles the new chart.

• simulator (SHD-1095): a single generative/Simulator model runs llm-d-inference-sim (default image provided by the stack, simulatorImage). The installer offers it like any generative model. It needs no weights and no GPU, and its image is mirrored only when it is served. Simulated embedders stay for manual stacks only.

Validation

• Changed Go files were formatted with gofmt.
• go test ./... passed in every affected Go module.

Additional validation details:

• pkg, installer, app_serving, infra/gateway-provider: gofmt -l, go vet ./... and go test ./... clean.
• Installer run on a dev cluster over an existing install. It updated the CRDs and Istio and uninstalled GPT-OSS-20B (both the migrated v1.0 and the legacy Deployments removed).
• Through the discovery Service: /v1/models, a chat completion and a streamed chat completion succeeded (Gateway, EPP, simulator). HTTPRoute Accepted/ResolvedRefs.

Deployment and operational impact

• Existing installs migrate in place. Old modelservice pods are deleted before the new server starts, so each model has a short downtime during the update.
• Breaking for manual stacks: model folders now need a model.yaml. The old gaie-*/ms-* values files are removed.

Documentation

Updated the architecture docs (model serving, deployment flow, gateway, traffic flow, storage, observability, installer), the installer guide and troubleshooting, the operations guide, the local cluster guides and the Pulumi templates.

Checklist

• I added or updated tests for changed behavior, or explained why tests are not needed.
• I updated documentation for deployment, configuration, topology, or operational changes.
• My commits follow the Conventional Commits specification.
• Every commit includes a Signed-off-by trailer (git commit -s).
• I did not commit Pulumi state, kubeconfigs, chart archives, model artifacts, credentials, tokens, or plaintext secrets.

Additional information

• installer/build/manifests/images.yaml also sets the shaide-webapp tag to dev-shd-993.

AI assistance:

• Tool: Claude Code
• Model: Claude Opus 5.5
• Used for: implementation, tests, chart vendoring, docs and this description
• Human verification: done