feat(deploy): add container image, Helm chart and Envoy AI Gateway, litellm examples

[ansjindal]

What

Adds the packaging needed to run switchyard-server on Kubernetes:

• Dockerfile — multi-stage release build of the switchyard-server
  binary on debian:bookworm-slim. 40 MB, runs as UID 65532, works under a
  read-only root filesystem. Distinct from benchmark/*.Dockerfile, which
  build the Python launcher for benchmark harnesses.
• .dockerignore — adds target. Currently absent, so any build from a
  developer checkout ships a multi-gigabyte Rust build directory as context.
• deploy/helm/switchyard/ — chart for the standalone proxy: deployment
  TOML rendered into a ConfigMap, provider keys from a Secret, health probes,
  graceful-shutdown-aware termination, and optional ServiceMonitor, PDB, HPA,
  TLS and routing-log volumes.
• examples/kubernetes/ — Envoy AI Gateway integration in both
  directions, along with LiteLLM.

No Rust or Python source is touched; this is additive packaging only.

Its solving: #318

Why

switchyard-server already behaves like a Kubernetes workload — one port
serving the LLM API, /health and /metrics; credentials read from
environment variables named by api_key_env rather than baked into config;
SIGTERM draining in-flight requests for --shutdown-timeout. What is missing
is the packaging that turns those properties into a deployment, so every
adopter currently rebuilds it.

The Envoy AI Gateway examples are included because the two projects solve
adjacent problems and compose in both directions, with materially different
security properties:

	Envoy AI Gateway in front	Switchyard in front
Chain	client → Envoy → Switchyard → provider	client → Switchyard → Envoy → provider
Provider credentials	Switchyard pod env	BackendSecurityPolicy, never in the pod
Client identity	SecurityPolicy on the Gateway	nothing by default

How tested

• uv run ruff check . clean — not run; no Python changed. CI covers it.
• uv run mypy switchyard clean — not run; no Python changed.
• uv run pytest tests/ green — not run; no Python changed.
• Manual smoke — deployed and exercised end to end on a live cluster.

Cluster: single-node k3s v1.34.10, Envoy Gateway v1.8.3, Envoy AI Gateway
v1.0.0, against nemotron-3-nano-30b-a3b, -super-v3 and -ultra.

Image:

• builds in ~3 min; switchyard-server --version reports 0.2.0
• --config routes.toml --dry-run exits non-zero on a missing api_key_env
  and zero on a valid deployment

Chart:

• helm lint --strict clean; renders against both example values files
• /health 200, /v1/models lists the configured route ids, /metrics
  exports switchyard_build_info{version="0.2.0"}
• ConfigMap edits roll the Deployment via the checksum/config annotation

Both topologies:

• both Gateways reach Programmed=True; every AIGatewayRoute,
  AIServiceBackend, Backend, BackendSecurityPolicy reports Accepted
• real completions returned through each chain
• Envoy in front: response carries
  x-model-router-selected-model showing the tier Switchyard picked
• Switchyard in front: Envoy access log confirms transit — route match,
  schema.prefix path rewrite, upstream TLS to the provider
• client auth: no key → 401, wrong key → 401, valid key → 200
• NetworkPolicy: unlabelled pod refused

Checklist

• One class per file; filename = snake_case of the primary class. —
  N/A, no source files.
• New public symbols exported from switchyard/__init__.py.__all__. —
  N/A, no public API change.
• Unit tests added for new components / bug fixes. — N/A, no behaviour
  change. Validation is the manual smoke above; happy to add a kind-based
  e2e job if you want one.
• README / --help updated if customer-facing surface changed. — no CLI
  change; new docs are deploy/helm/switchyard/README.md and
  examples/kubernetes/README.md.
• Commits signed off per the DCO.

Notes for reviewers

Please confirm the paths before deep review. deploy/helm/switchyard/ and
examples/kubernetes/ are guesses at your conventions; charts/ is equally
easy. Likewise the chart defaults image.repository to
ghcr.io/nvidia-nemo/switchyard/switchyard-server, which does not exist yet —
the follow-up branch adds the workflow that would publish it.

Summary by CodeRabbit

• New Features

  • Added a production-ready Docker image for Switchyard Server.
  • Added a Helm chart for Kubernetes deployments, including configurable services, secrets, TLS, health checks, autoscaling, disruption budgets, and Prometheus monitoring.
  • Added support for custom Kubernetes resources and secure deployment defaults.
  • Added Kubernetes examples for deployments with Envoy AI Gateway, provider authentication, routing, rate limits, and network access controls.

• Documentation

  • Added installation, configuration, topology, and operational guidance for Docker, Helm, and Kubernetes deployments.

[coderabbitai]

Walkthrough

The change adds a production Docker image, a configurable Helm chart, Kubernetes deployment resources, and two Envoy AI Gateway integration examples.

Changes

Switchyard deployment packaging

Layer / File(s)	Summary
Production container image .dockerignore, Dockerfile	Adds a cached multi-stage Rust build and a Debian slim runtime image. The runtime uses a non-root user, CA certificates, port 4000, and a default routes configuration path.
Helm chart contracts and defaults deploy/helm/switchyard/Chart.yaml, deploy/helm/switchyard/values.yaml, deploy/helm/switchyard/templates/_helpers.tpl, deploy/helm/switchyard/templates/configmap.yaml, deploy/helm/switchyard/README.md	Adds chart metadata, naming helpers, ConfigMap and Secret resolution, deployment TOML defaults, security settings, service settings, optional operational features, and chart documentation.
Helm workload and Kubernetes resources deploy/helm/switchyard/templates/deployment.yaml, deploy/helm/switchyard/templates/service.yaml, deploy/helm/switchyard/templates/secret.yaml, deploy/helm/switchyard/templates/serviceaccount.yaml, deploy/helm/switchyard/templates/hpa.yaml, deploy/helm/switchyard/templates/pdb.yaml, deploy/helm/switchyard/templates/servicemonitor.yaml, deploy/helm/switchyard/templates/extra-objects.yaml, deploy/helm/switchyard/templates/NOTES.txt	Adds Deployment, Service, Secret, ServiceAccount, HPA, PDB, ServiceMonitor, extra-object, and installation-note templates.
Integration topology documentation examples/kubernetes/README.md	Documents both Envoy AI Gateway topologies, installation steps, authentication ownership, request behavior, and operational settings.
Envoy in front of Switchyard examples/kubernetes/envoy-ai-gateway-in-front/*	Adds Gateway infrastructure, Switchyard backend routing, forwarded-host removal, API-key authentication, token-based limits, and routing values.
Switchyard in front of Envoy examples/kubernetes/switchyard-in-front-of-envoy-ai-gateway/*	Adds an internal provider gateway, provider TLS and API-key configuration, forwarded-host removal, network restrictions, and Switchyard routing values.

Estimated code review effort: 5 (Critical) | ~120 minutes

Poem

I’m a rabbit with a chart in my paws,
Helm pods rise by Kubernetes laws.
A Docker stage builds swift and bright,
Envoy guides requests left and right.
Secrets, routes, and probes align—
Hop, hop, the deployment is fine!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the container image, Helm chart, and Envoy AI Gateway examples added by the pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@deploy/helm/switchyard/templates/deployment.yaml`:
- Around line 80-85: Update the deployment health probe definitions to render
scheme: HTTPS when .Values.tls.enabled is true and no custom probe scheme is
configured. Preserve any explicitly configured scheme and the existing HTTP
behavior when TLS is disabled.

In `@deploy/helm/switchyard/templates/pdb.yaml`:
- Around line 13-16: Update the podDisruptionBudget conditional so it checks
whether maxUnavailable differs from its unset default "" rather than relying on
truthiness. This must preserve and render an explicit numeric maxUnavailable
value of 0; only an unset maxUnavailable should select minAvailable.

In `@examples/kubernetes/README.md`:
- Around line 102-105: Update the primary Kubernetes installation sequence in
the README to create the switchyard-client-keys secret and apply
04-client-auth.yaml before exposing the topology. Update the corresponding
request example to include the required x-api-key header, preserving the
existing installation and request flow.

In `@examples/kubernetes/switchyard-in-front-of-envoy-ai-gateway/01-gateway.yaml`:
- Around line 43-49: Add a NetworkPolicy in the envoy-gateway-system resources
that targets the generated Envoy workload and permits ingress only from
Switchyard client pods selected by both the switchyard namespace and the
authorized pod labels. Verify the generated Envoy labels used by the policy
match the deployed workload, while preserving the existing ClusterIP service
configuration.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7497c1e2-f427-4afd-a068-b3a5c7e55dd9

📥 Commits

Reviewing files that changed from the base of the PR and between 70aeb1f and 2151a4e.

📒 Files selected for processing (28)

• .dockerignore
• Dockerfile
• deploy/helm/switchyard/.helmignore
• deploy/helm/switchyard/Chart.yaml
• deploy/helm/switchyard/README.md
• deploy/helm/switchyard/templates/NOTES.txt
• deploy/helm/switchyard/templates/_helpers.tpl
• deploy/helm/switchyard/templates/configmap.yaml
• deploy/helm/switchyard/templates/deployment.yaml
• deploy/helm/switchyard/templates/extra-objects.yaml
• deploy/helm/switchyard/templates/hpa.yaml
• deploy/helm/switchyard/templates/pdb.yaml
• deploy/helm/switchyard/templates/secret.yaml
• deploy/helm/switchyard/templates/service.yaml
• deploy/helm/switchyard/templates/serviceaccount.yaml
• deploy/helm/switchyard/templates/servicemonitor.yaml
• deploy/helm/switchyard/values.yaml
• examples/kubernetes/README.md
• examples/kubernetes/envoy-ai-gateway-in-front/01-gateway.yaml
• examples/kubernetes/envoy-ai-gateway-in-front/02-switchyard-backend.yaml
• examples/kubernetes/envoy-ai-gateway-in-front/03-forwarded-host.yaml
• examples/kubernetes/envoy-ai-gateway-in-front/04-client-auth.yaml
• examples/kubernetes/envoy-ai-gateway-in-front/values.switchyard.yaml
• examples/kubernetes/switchyard-in-front-of-envoy-ai-gateway/01-gateway.yaml
• examples/kubernetes/switchyard-in-front-of-envoy-ai-gateway/02-provider-backend.yaml
• examples/kubernetes/switchyard-in-front-of-envoy-ai-gateway/03-forwarded-host.yaml
• examples/kubernetes/switchyard-in-front-of-envoy-ai-gateway/04-restrict-access.yaml
• examples/kubernetes/switchyard-in-front-of-envoy-ai-gateway/values.switchyard.yaml