diff --git a/deploy/helm/llm-request-router/README.md b/deploy/helm/llm-request-router/README.md index 7a31534f7..d1683c608 100644 --- a/deploy/helm/llm-request-router/README.md +++ b/deploy/helm/llm-request-router/README.md @@ -75,7 +75,8 @@ Important settings to review before deployment: - `llmRequestRouter.metrics.enabled` to expose the metrics port on the Service (default: `false`) - `llmRequestRouter.metrics.serviceMonitor.enabled` to create a Prometheus `ServiceMonitor` (requires `metrics.enabled`) - `llmRequestRouter.certificate.*` to let cert-manager issue the Stargate QUIC server certificate -- `llmRequestRouter.tls.*` to mount the issued TLS Secret and pass cert/key paths to Stargate +- `llmRequestRouter.tls.*` to mount the TLS Secret and pass cert/key paths to Stargate +- `llmRequestRouter.tls.mode` to choose the source of the QUIC server identity. `certManager` (default) mounts the Secret cert-manager writes for `certificate.*`. `existingSecret` mounts a pre-created Secret instead: the chart renders no `Certificate` and adds no issuer dependency, `certificate.enabled` must stay `false`, `tls.secretName`, `tls.certPath`, and `tls.keyPath` are required, and the operator owns issuance, renewal, rotation, and recovery. The Secret must provide the `tls.crt` and `tls.key` entries. The chart cannot read a pre-created Secret, so it does not validate its SANs or expiry. - `llmRequestRouter.pki.*` to provision the OpenBao service-issuing PKI hierarchy that cert-manager mints the Certificate from. Opt-in via `pki.enabled=true`. Mirrors the SIS chart's `hook-lls-migrations.yaml` pattern: a Helm pre-install/pre-upgrade Job runs the `nvcf-openbao-migrations` image with `CORE_MIGRATIONS_ENABLED=false` + `ADDONS_LLM_ENABLED=true` so only the LLM addon executes. `pki.allowedDomains` (comma-separated DNS suffixes) is required when enabled and is the OpenBao PKI role's `allowed_domains` security constraint. Typically this is `,cluster.local`. Job-level fail-hard is handled by `restartPolicy: OnFailure` + `pki.backoffLimit` combined with the migrations image's `FAILED_MIGRATIONS` accumulator (image `>= 0.12.1`). - `llmRequestRouter.vault.audience` for the projected ServiceAccount token audience used to authenticate to OpenBao - `llmRequestRouter.vault.noVaultAnnotations` to disable Vault Agent injection (useful for local testing without OpenBao) diff --git a/deploy/helm/llm-request-router/llm-request-router/templates/_helpers.tpl b/deploy/helm/llm-request-router/llm-request-router/templates/_helpers.tpl index 829d42a43..512a69cea 100644 --- a/deploy/helm/llm-request-router/llm-request-router/templates/_helpers.tpl +++ b/deploy/helm/llm-request-router/llm-request-router/templates/_helpers.tpl @@ -180,6 +180,43 @@ comparing suffixes. {{- end -}} {{- end }} +{{/* +Validate the QUIC server identity source. certManager keeps cert-manager as the +owner of issuance and renewal. existingSecret mounts a pre-created TLS Secret, +renders no Certificate, and makes the operator the owner. The two are mutually +exclusive, and existingSecret needs the Secret name plus both file paths because +the mount and the Stargate arguments are all conditional on them. +*/}} +{{- define "llm-request-router.validateTlsIdentity" -}} +{{- $tls := .Values.llmRequestRouter.tls | default dict -}} +{{- $certificate := .Values.llmRequestRouter.certificate | default dict -}} +{{- $mode := $tls.mode | default "certManager" -}} +{{- if not (has $mode (list "certManager" "existingSecret")) -}} +{{- fail (printf "llmRequestRouter.tls.mode must be certManager or existingSecret, got %q" (toString $mode)) -}} +{{- end -}} +{{- if eq $mode "existingSecret" -}} +{{- if $certificate.enabled -}} +{{- fail "llmRequestRouter.certificate.enabled must be false when llmRequestRouter.tls.mode is existingSecret; cert-manager and the operator cannot both own the request-router certificate" -}} +{{- end -}} +{{- if not $tls.secretName -}} +{{- fail "llmRequestRouter.tls.secretName is required when llmRequestRouter.tls.mode is existingSecret" -}} +{{- end -}} +{{- if not $tls.certPath -}} +{{- fail "llmRequestRouter.tls.certPath is required when llmRequestRouter.tls.mode is existingSecret" -}} +{{- end -}} +{{- if not $tls.keyPath -}} +{{- fail "llmRequestRouter.tls.keyPath is required when llmRequestRouter.tls.mode is existingSecret" -}} +{{- end -}} +{{- $tlsMountPath := include "llm-request-router.tlsMountPath" . | trim -}} +{{- if not (hasPrefix "/" $tlsMountPath) -}} +{{- fail "llmRequestRouter.tls.mountPath must be an absolute path when llmRequestRouter.tls.mode is existingSecret" -}} +{{- end -}} +{{- if or (ne $tlsMountPath (dir $tls.certPath)) (ne $tlsMountPath (dir $tls.keyPath)) -}} +{{- fail "llmRequestRouter.tls.mountPath must match the directory containing llmRequestRouter.tls.certPath and llmRequestRouter.tls.keyPath when llmRequestRouter.tls.mode is existingSecret" -}} +{{- end -}} +{{- end -}} +{{- end }} + {{- define "llm-request-router.tlsSecretName" -}} {{- $tls := .Values.llmRequestRouter.tls | default dict -}} {{- $certificate := .Values.llmRequestRouter.certificate | default dict -}} diff --git a/deploy/helm/llm-request-router/llm-request-router/templates/deployment.yaml b/deploy/helm/llm-request-router/llm-request-router/templates/deployment.yaml index 6f80715ca..6d5059d5f 100644 --- a/deploy/helm/llm-request-router/llm-request-router/templates/deployment.yaml +++ b/deploy/helm/llm-request-router/llm-request-router/templates/deployment.yaml @@ -31,6 +31,11 @@ tunnel target. {{- if and $disableDnsDiscovery (gt $replicaCount 1) }} {{- fail "llmRequestRouter.discovery.disableDnsDiscovery cannot be true when llmRequestRouter.replicaCount is greater than 1; multi-replica routers require DNS discovery" }} {{- end }} +{{- /* +The identity guard lives here, not in certificate.yaml: existingSecret mode +renders no Certificate, so a guard in that template would never run. +*/}} +{{- include "llm-request-router.validateTlsIdentity" . }} {{- $advertisedHostnameTemplate := include "llm-request-router.advertisedHostnameTemplate" . }} spec: serviceName: {{ .Values.llmRequestRouter.service.headlessName }} diff --git a/deploy/helm/llm-request-router/llm-request-router/values.yaml b/deploy/helm/llm-request-router/llm-request-router/values.yaml index da50fa8a8..a0a3429dc 100644 --- a/deploy/helm/llm-request-router/llm-request-router/values.yaml +++ b/deploy/helm/llm-request-router/llm-request-router/values.yaml @@ -175,6 +175,17 @@ llmRequestRouter: audience: "" tls: + # Source of the QUIC server identity. + # certManager - certificate.enabled controls cert-manager issuance and + # the chart mounts the Secret cert-manager writes. + # existingSecret - mount a pre-created TLS Secret. The chart renders no + # Certificate and adds no issuer dependency. secretName, + # certPath, and keyPath are required, certificate.enabled + # must stay false, and the operator owns issuance, + # renewal, rotation, and recovery. + mode: certManager + # Secret holding the server certificate. It must provide the tls.crt and + # tls.key entries, as a kubernetes.io/tls Secret does. # The mount directory is polled every 30 seconds for certificate and key # updates, which apply to new connections without a pod restart. Keep both # files in one Secret so Kubernetes projects one atomic generation. diff --git a/deploy/helm/llm-request-router/scripts/check-pki-render.sh b/deploy/helm/llm-request-router/scripts/check-pki-render.sh index 5004aade5..8dbfcc263 100644 --- a/deploy/helm/llm-request-router/scripts/check-pki-render.sh +++ b/deploy/helm/llm-request-router/scripts/check-pki-render.sh @@ -290,4 +290,144 @@ if render_certificate_case \ fail "advertised hostname containing non-DNS braces unexpectedly rendered" fi +# Pass 3: existing-Secret identity mode. The operator owns issuance, so the +# chart must mount the pre-created Secret without rendering a Certificate or +# the OpenBao provisioning hook. +existing_secret_manifest="${tmp_dir}/existing-secret.yaml" +render_existing_secret_case() { + output="$1" + shift + + helm template llm-request-router ./llm-request-router \ + --namespace nvcf \ + --values ./llm-request-router/values.yaml \ + --set llmRequestRouter.image.repository=stargate \ + --set-string llmRequestRouter.tls.mode=existingSecret \ + --set llmRequestRouter.tls.quicInsecure=false \ + "$@" \ + > "${output}" +} + +render_existing_secret_case \ + "${existing_secret_manifest}" \ + --set-string llmRequestRouter.tls.secretName=operator-quic-tls \ + --set-string llmRequestRouter.tls.certPath=/etc/stargate/tls/tls.crt \ + --set-string llmRequestRouter.tls.keyPath=/etc/stargate/tls/tls.key + +existing_secret_cert="$(yq -rN 'select(.kind == "Certificate") | .metadata.name' "${existing_secret_manifest}" | head -n1)" +[ -z "${existing_secret_cert}" ] || fail "existing-Secret mode rendered a Certificate: ${existing_secret_cert}" + +existing_secret_job="$(yq -rN 'select(.kind == "Job" and .metadata.name == "addons-llm-migrations") | .metadata.name' "${existing_secret_manifest}" | head -n1)" +[ -z "${existing_secret_job}" ] || fail "existing-Secret mode rendered the OpenBao provisioning hook" + +existing_secret_volume="$(yq -rN 'select(.kind == "StatefulSet" and .metadata.name == "llm-request-router") | .spec.template.spec.volumes[] | select(.name == "stargate-tls" and .secret.secretName == "operator-quic-tls") | .name' "${existing_secret_manifest}")" +[ "${existing_secret_volume}" = "stargate-tls" ] || fail "existing-Secret mode did not mount the pre-created Secret" + +existing_secret_mount="$(yq -rN 'select(.kind == "StatefulSet" and .metadata.name == "llm-request-router") | .spec.template.spec.containers[0].volumeMounts[] | select(.name == "stargate-tls" and .mountPath == "/etc/stargate/tls" and .readOnly == true) | .name' "${existing_secret_manifest}")" +[ "${existing_secret_mount}" = "stargate-tls" ] || fail "existing-Secret mode did not mount stargate-tls read-only" + +existing_secret_args="$(yq -rN 'select(.kind == "StatefulSet" and .metadata.name == "llm-request-router") | .spec.template.spec.containers[0].args[]' "${existing_secret_manifest}")" +printf '%s\n' "${existing_secret_args}" | grep -qx -- "--tls-cert-path=/etc/stargate/tls/tls.crt" || fail "existing-Secret mode did not pass the certificate path" +printf '%s\n' "${existing_secret_args}" | grep -qx -- "--tls-key-path=/etc/stargate/tls/tls.key" || fail "existing-Secret mode did not pass the private key path" +if printf '%s\n' "${existing_secret_args}" | grep -qx -- "--quic-insecure"; then + fail "existing-Secret mode enabled insecure request-router transport" +fi + +# Mixed ownership: cert-manager and the operator cannot both own the identity. +mixed_ownership_error="${tmp_dir}/mixed-ownership.err" +if render_existing_secret_case \ + /dev/null \ + --set llmRequestRouter.certificate.enabled=true \ + --set-string llmRequestRouter.certificate.issuerRef.name=nvcf-openbao-pki \ + --set-string 'llmRequestRouter.certificate.dnsNames[0]=*.llm-request-router-headless.nvcf.svc.cluster.local' \ + --set-string llmRequestRouter.tls.secretName=operator-quic-tls \ + --set-string llmRequestRouter.tls.certPath=/etc/stargate/tls/tls.crt \ + --set-string llmRequestRouter.tls.keyPath=/etc/stargate/tls/tls.key \ + 2> "${mixed_ownership_error}"; then + fail "mixed certificate ownership unexpectedly rendered" +fi +grep -Fq \ + "llmRequestRouter.certificate.enabled must be false when llmRequestRouter.tls.mode is existingSecret" \ + "${mixed_ownership_error}" || fail "mixed ownership render did not return the expected guard message" + +# Incomplete configuration: every required value reports itself by name. The +# mount and the Stargate arguments are conditional, so a missing value would +# otherwise leave the router on plaintext QUIC without any diagnostic. +check_required_existing_secret_value() { + case_name="$1" + expected_message="$2" + shift 2 + + error_file="${tmp_dir}/${case_name}.err" + if render_existing_secret_case /dev/null "$@" 2> "${error_file}"; then + fail "existing-Secret render without ${case_name} unexpectedly succeeded" + fi + grep -Fq "${expected_message}" "${error_file}" || + fail "${case_name} render did not return the expected guard message" +} + +check_required_existing_secret_value \ + secret-name \ + "llmRequestRouter.tls.secretName is required when llmRequestRouter.tls.mode is existingSecret" \ + --set-string llmRequestRouter.tls.certPath=/etc/stargate/tls/tls.crt \ + --set-string llmRequestRouter.tls.keyPath=/etc/stargate/tls/tls.key + +check_required_existing_secret_value \ + cert-path \ + "llmRequestRouter.tls.certPath is required when llmRequestRouter.tls.mode is existingSecret" \ + --set-string llmRequestRouter.tls.secretName=operator-quic-tls \ + --set-string llmRequestRouter.tls.keyPath=/etc/stargate/tls/tls.key + +check_required_existing_secret_value \ + key-path \ + "llmRequestRouter.tls.keyPath is required when llmRequestRouter.tls.mode is existingSecret" \ + --set-string llmRequestRouter.tls.secretName=operator-quic-tls \ + --set-string llmRequestRouter.tls.certPath=/etc/stargate/tls/tls.crt + +# The Secret is mounted as a directory, so changing its mount location without +# changing the certificate paths would leave the router unable to read them. +mismatched_mount_path_error="${tmp_dir}/mismatched-mount-path.err" +if render_existing_secret_case \ + /dev/null \ + --set-string llmRequestRouter.tls.secretName=operator-quic-tls \ + --set-string llmRequestRouter.tls.mountPath=/var/run/router-tls \ + --set-string llmRequestRouter.tls.certPath=/etc/stargate/tls/tls.crt \ + --set-string llmRequestRouter.tls.keyPath=/etc/stargate/tls/tls.key \ + 2> "${mismatched_mount_path_error}"; then + fail "existing-Secret mode accepted a mount path that does not contain the TLS files" +fi +grep -Fq \ + "llmRequestRouter.tls.mountPath must match the directory containing llmRequestRouter.tls.certPath and llmRequestRouter.tls.keyPath when llmRequestRouter.tls.mode is existingSecret" \ + "${mismatched_mount_path_error}" || fail "mismatched mount path did not return the expected guard message" + +# Kubernetes volume mounts must be absolute paths. Matching relative paths +# would otherwise pass the directory consistency check but fail at deployment. +relative_mount_path_error="${tmp_dir}/relative-mount-path.err" +if render_existing_secret_case \ + /dev/null \ + --set-string llmRequestRouter.tls.secretName=operator-quic-tls \ + --set-string llmRequestRouter.tls.mountPath=tls \ + --set-string llmRequestRouter.tls.certPath=tls/tls.crt \ + --set-string llmRequestRouter.tls.keyPath=tls/tls.key \ + 2> "${relative_mount_path_error}"; then + fail "existing-Secret mode accepted a relative TLS mount path" +fi +grep -Fq \ + "llmRequestRouter.tls.mountPath must be an absolute path when llmRequestRouter.tls.mode is existingSecret" \ + "${relative_mount_path_error}" || fail "relative mount path did not return the expected guard message" + +# An unknown mode must fail rather than silently fall back to cert-manager. +invalid_mode_error="${tmp_dir}/invalid-mode.err" +if helm template llm-request-router ./llm-request-router \ + --namespace nvcf \ + --values ./llm-request-router/values.yaml \ + --set llmRequestRouter.image.repository=stargate \ + --set-string llmRequestRouter.tls.mode=externalSecret \ + > /dev/null 2> "${invalid_mode_error}"; then + fail "unknown llmRequestRouter.tls.mode unexpectedly rendered" +fi +grep -Fq \ + 'llmRequestRouter.tls.mode must be certManager or existingSecret, got "externalSecret"' \ + "${invalid_mode_error}" || fail "unknown mode render did not return the expected guard message" + echo "PKI render checks passed" diff --git a/deploy/stacks/self-managed/environments/base.yaml b/deploy/stacks/self-managed/environments/base.yaml index 4b97eb8f9..8d451b369 100644 --- a/deploy/stacks/self-managed/environments/base.yaml +++ b/deploy/stacks/self-managed/environments/base.yaml @@ -257,21 +257,35 @@ addons: # LLM addon: gateway + request router (stargate) for LLM function invocation llm: enabled: false - # QUIC TLS certificate for the request router (Stargate). When enabled, - # the chart requests a Certificate from the configured issuer and mounts - # the resulting Secret. Managed mode also provisions the OpenBao - # service-issuing hierarchy and defaults to - # ClusterIssuer/nvcf-openbao-pki. Disabled by default; opt in per env. + # QUIC TLS certificate for the request router (Stargate). Disabled by + # default; opt in per env. + # + # mode selects who owns the server identity: + # certManager - the chart requests a Certificate from the configured + # issuer and mounts the resulting Secret. Managed mode + # also provisions the OpenBao service-issuing hierarchy + # and defaults to ClusterIssuer/nvcf-openbao-pki. + # existingSecret - the chart mounts a Secret you created and manages no + # issuance. Requires secretName. clusterIssuer.enabled, + # dnsNames, and allowedDomains must be unset because + # they only steer stack-managed issuance. You own + # issuance, renewal, rotation, and recovery, and the + # stack validates neither the SANs nor the expiry. pki: enabled: false + mode: certManager # REQUIRED for a managed issuer. Comma-separated DNS suffixes the # OpenBao PKI role accepts. Typically the customer domain plus # cluster.local for in-cluster service identity. allowedDomains: "" - # REQUIRED when enabled. SANs requested on the issued certificate. + # REQUIRED for mode certManager. SANs requested on the issued + # certificate. Must be empty for mode existingSecret. dnsNames: [] - # Optional overrides; defaults are usually correct. + # REQUIRED for mode existingSecret; the kubernetes.io/tls Secret in the + # nvcf namespace holding the tls.crt and tls.key entries. Optional for + # mode certManager, where it names the Secret cert-manager writes. # secretName: stargate-quic-tls + # Optional overrides; defaults are usually correct. # issuerKind: ClusterIssuer # issuerName: nvcf-openbao-pki # Stack management defaults to true only for the default diff --git a/deploy/stacks/self-managed/global.yaml.gotmpl b/deploy/stacks/self-managed/global.yaml.gotmpl index ec7491f72..5f4238f2f 100644 --- a/deploy/stacks/self-managed/global.yaml.gotmpl +++ b/deploy/stacks/self-managed/global.yaml.gotmpl @@ -869,6 +869,57 @@ llmRequestRouter: so an operator staging pki.enabled=true ahead of llm.enabled=true should not be punished with confusing required errors during render. */ -}} {{- if and (dig "addons" "llm" "enabled" false .Values) (dig "addons" "llm" "pki" "enabled" false .Values) }} + {{- /* Identity ownership. certManager keeps cert-manager as the owner of + issuance and renewal. existingSecret points the router at a Secret the + operator already created, so the stack renders no Certificate and pulls + in no issuer dependency. Validate the same way as issuerKind: dig only + falls back for a missing path, so explicit null and wrongly typed values + must be rejected rather than coerced into the default. */ -}} + {{- $pkiMode := dig "addons" "llm" "pki" "mode" "certManager" .Values }} + {{- if not (kindIs "string" $pkiMode) }} + {{- fail "addons.llm.pki.mode must be the string \"certManager\" or \"existingSecret\"" }} + {{- end }} + {{- if not (has $pkiMode (list "certManager" "existingSecret")) }} + {{- fail (printf "addons.llm.pki.mode must be exactly \"certManager\" or \"existingSecret\", got %q" $pkiMode) }} + {{- end }} + {{- if eq $pkiMode "existingSecret" }} + {{- /* The operator owns issuance in this mode, so every value that only + steers stack-managed issuance is a mixed-ownership conflict. Fail loudly + instead of silently ignoring it: a leftover dnsNames list reads as "the + stack still issues my certificate" and would go unnoticed until the + certificate expired. */ -}} + {{- $existingSecretPkiValues := dig "addons" "llm" "pki" dict .Values }} + {{- if kindIs "map" $existingSecretPkiValues }} + {{- $existingSecretClusterIssuer := dig "clusterIssuer" dict $existingSecretPkiValues }} + {{- if and (kindIs "map" $existingSecretClusterIssuer) (hasKey $existingSecretClusterIssuer "enabled") }} + {{- if index $existingSecretClusterIssuer "enabled" }} + {{- fail "addons.llm.pki.clusterIssuer.enabled must be false or unset when addons.llm.pki.mode is existingSecret; cert-manager and the operator cannot both own the request-router certificate" }} + {{- end }} + {{- end }} + {{- end }} + {{- $existingSecretDnsNames := dig "addons" "llm" "pki" "dnsNames" (list) .Values }} + {{- if not (kindIs "slice" $existingSecretDnsNames) }} + {{- fail "addons.llm.pki.dnsNames must be a list when addons.llm.pki.mode is existingSecret" }} + {{- end }} + {{- if gt (len $existingSecretDnsNames) 0 }} + {{- fail "addons.llm.pki.dnsNames applies only to a stack-issued Certificate and must be empty when addons.llm.pki.mode is existingSecret" }} + {{- end }} + {{- if dig "addons" "llm" "pki" "allowedDomains" "" .Values }} + {{- fail "addons.llm.pki.allowedDomains constrains the managed OpenBao signing role only and must be empty when addons.llm.pki.mode is existingSecret" }} + {{- end }} + {{- /* No default: the Secret is the operator's, so guessing a name would + mount the wrong identity or fail at pod start instead of at render. */ -}} + {{- $existingSecretName := required "addons.llm.pki.secretName is required when addons.llm.pki.mode is existingSecret" (dig "addons" "llm" "pki" "secretName" "" .Values) }} + certificate: + enabled: false + tls: + mode: existingSecret + secretName: {{ $existingSecretName | quote }} + mountPath: {{ dig "addons" "llm" "pki" "mountPath" "/etc/stargate/tls" .Values | quote }} + certPath: {{ dig "addons" "llm" "pki" "certPath" "/etc/stargate/tls/tls.crt" .Values | quote }} + keyPath: {{ dig "addons" "llm" "pki" "keyPath" "/etc/stargate/tls/tls.key" .Values | quote }} + quicInsecure: false + {{- else }} {{- $secretName := dig "addons" "llm" "pki" "secretName" "stargate-quic-tls" .Values }} {{- /* dig only falls back for a missing path, so explicit null, empty, and wrongly typed values reach these checks. Reject them instead of coercing @@ -952,6 +1003,7 @@ llmRequestRouter: {{- end }} {{- end }} {{- end }} + {{- end }} {{- with dig "llmRequestRouter" "podDisruptionBudget" dict .Values }} podDisruptionBudget: {{- toYaml . | nindent 4 }} diff --git a/deploy/stacks/self-managed/helmfile.d/01-dependencies.yaml.gotmpl b/deploy/stacks/self-managed/helmfile.d/01-dependencies.yaml.gotmpl index 8a65fc3e2..8afe676c8 100644 --- a/deploy/stacks/self-managed/helmfile.d/01-dependencies.yaml.gotmpl +++ b/deploy/stacks/self-managed/helmfile.d/01-dependencies.yaml.gotmpl @@ -12,7 +12,19 @@ environments: {{- $issuerKind := dig "addons" "llm" "pki" "issuerKind" "ClusterIssuer" .Values }} {{- $issuerName := dig "addons" "llm" "pki" "issuerName" "nvcf-openbao-pki" .Values }} {{- $manageIssuer := false }} +{{- /* existingSecret hands issuance to the operator, so the stack must not + create an issuer or require OpenBao for one. Validate the mode here as + well as in global.yaml.gotmpl so both states reject the same inputs. */ -}} +{{- $pkiMode := dig "addons" "llm" "pki" "mode" "certManager" .Values }} {{- if $llmPkiActive }} +{{- if not (kindIs "string" $pkiMode) }} +{{- fail "addons.llm.pki.mode must be the string \"certManager\" or \"existingSecret\"" }} +{{- end }} +{{- if not (has $pkiMode (list "certManager" "existingSecret")) }} +{{- fail (printf "addons.llm.pki.mode must be exactly \"certManager\" or \"existingSecret\", got %q" $pkiMode) }} +{{- end }} +{{- end }} +{{- if and $llmPkiActive (eq ($pkiMode | toString) "certManager") }} {{- /* dig only falls back for a missing path, so explicit null, empty, and wrongly typed values reach these checks. Reject them instead of coercing them: a malformed management flag that silently resolves to false skips diff --git a/deploy/stacks/self-managed/tests/check-llm-pki-issuer.sh b/deploy/stacks/self-managed/tests/check-llm-pki-issuer.sh index 9310a0e27..ffe0f9918 100755 --- a/deploy/stacks/self-managed/tests/check-llm-pki-issuer.sh +++ b/deploy/stacks/self-managed/tests/check-llm-pki-issuer.sh @@ -240,6 +240,81 @@ expect_external_router() { fi } +# existingSecret mode cannot reuse render_router: that helper always passes +# dnsNames, which this mode rejects as a mixed-ownership conflict. +render_existing_secret_router() { + local case_name="$1" + shift + local values_file="$work_dir/$case_name.router-values.yaml" + local router_chart="$stack_dir/../../helm/llm-request-router/llm-request-router" + + HELMFILE_ENV=base HELMFILE_CACHE_HOME="$work_dir/helmfile-cache" helmfile \ + --file "$stack_dir/helmfile.d/02-core.yaml.gotmpl" \ + --environment default \ + --selector name=llm-request-router \ + --chart "$router_chart" \ + --skip-deps \ + "${core_state_values[@]}" \ + --state-values-set addons.llm.enabled=true \ + --state-values-set addons.llm.pki.enabled=true \ + --state-values-set-string addons.llm.pki.mode=existingSecret \ + "$@" \ + write-values \ + --output-file-template "$values_file" \ + >"$work_dir/$case_name.router.log" 2>&1 +} + +# The conflict guards live in global.yaml.gotmpl, which only 02-core loads, so +# these cases cannot go through expect_failure. +expect_router_failure() { + local case_name="$1" + local expected_error="$2" + shift 2 + + if render_existing_secret_router "$case_name" "$@"; then + fail "$case_name rendered successfully" + fi + grep -Fq "$expected_error" "$work_dir/$case_name.router.log" || + fail "$case_name did not return the expected error: $expected_error" +} + +# The operator owns issuance, so the router mounts their Secret and the stack +# renders neither a Certificate nor the OpenBao provisioning hook. +expect_existing_secret_router() { + local case_name="$1" + local secret_name="$2" + local values_file="$work_dir/$case_name.router-values.yaml" + local manifests_file="$work_dir/$case_name.router-manifests.yaml" + local router_chart="$stack_dir/../../helm/llm-request-router/llm-request-router" + + helm template llm-request-router "$router_chart" \ + --namespace nvcf \ + --values "$values_file" \ + >"$manifests_file" + + local rendered_cert + rendered_cert="$(yq -rN 'select(.kind == "Certificate") | .metadata.name' "$manifests_file" | head -1)" + test -z "$rendered_cert" || + fail "$case_name rendered a Certificate in existingSecret mode: $rendered_cert" + + if grep -Fq 'name: addons-llm-migrations' "$manifests_file"; then + fail "$case_name rendered the managed OpenBao provisioning hook" + fi + + local mounted_secret + mounted_secret="$(yq -rN 'select(.kind == "StatefulSet") | .spec.template.spec.volumes[] | select(.name == "stargate-tls") | .secret.secretName' "$manifests_file" | head -1)" + test "$mounted_secret" = "$secret_name" || + fail "$case_name mounted secret $mounted_secret, expected $secret_name" + + grep -Fq -- '--tls-cert-path=/etc/stargate/tls/tls.crt' "$manifests_file" || + fail "$case_name did not pass the request-router certificate path" + grep -Fq -- '--tls-key-path=/etc/stargate/tls/tls.key' "$manifests_file" || + fail "$case_name did not pass the request-router private key path" + if grep -Fq -- '--quic-insecure' "$manifests_file"; then + fail "$case_name enabled insecure request-router transport" + fi +} + expect_managed_router() { local case_name="${1:-managed-defaults}" local issuer_name="${2:-nvcf-openbao-pki}" @@ -599,4 +674,64 @@ render_list_all external-issuer-all \ "${router_dns_names[@]}" expect_declared_all external-issuer-all 0 +# Cases 27 to 32: existingSecret mode. The operator owns issuance, renewal, +# rotation, and recovery, so the stack must add no issuer or cert-manager +# ownership and must not require OpenBao. +existing_secret_overrides=( + --state-values-set openbao.enabled=false + --state-values-set-string addons.llm.pki.mode=existingSecret + --state-values-set-string addons.llm.pki.secretName=operator-quic-tls +) + +render_list existing-secret \ + --state-values-set addons.llm.enabled=true \ + --state-values-set addons.llm.pki.enabled=true \ + "${existing_secret_overrides[@]}" +expect_enabled existing-secret false + +render_existing_secret_router existing-secret \ + --state-values-set openbao.enabled=false \ + --state-values-set-string addons.llm.pki.secretName=operator-quic-tls || + fail "existing-secret router render failed" +expect_existing_secret_router existing-secret operator-quic-tls +render_list_all existing-secret-all \ + --state-values-set addons.llm.enabled=true \ + --state-values-set addons.llm.pki.enabled=true \ + "${existing_secret_overrides[@]}" +expect_declared_all existing-secret-all 0 + +# Values that only steer stack-managed issuance are conflicts, not no-ops. +expect_router_failure existing-secret-managed-issuer \ + 'addons.llm.pki.clusterIssuer.enabled must be false or unset when addons.llm.pki.mode is existingSecret' \ + --state-values-set-string addons.llm.pki.secretName=operator-quic-tls \ + --state-values-set addons.llm.pki.clusterIssuer.enabled=true + +expect_router_failure existing-secret-dns-names \ + 'addons.llm.pki.dnsNames applies only to a stack-issued Certificate and must be empty when addons.llm.pki.mode is existingSecret' \ + --state-values-set-string addons.llm.pki.secretName=operator-quic-tls \ + --state-values-set-string 'addons.llm.pki.dnsNames[0]=llm-request-router.nvcf.svc.cluster.local' + +expect_router_failure existing-secret-scalar-dns-names \ + 'addons.llm.pki.dnsNames must be a list when addons.llm.pki.mode is existingSecret' \ + --state-values-set-string addons.llm.pki.secretName=operator-quic-tls \ + --state-values-set-string addons.llm.pki.dnsNames=llm-request-router.nvcf.svc.cluster.local + +expect_router_failure existing-secret-allowed-domains \ + 'addons.llm.pki.allowedDomains constrains the managed OpenBao signing role only and must be empty when addons.llm.pki.mode is existingSecret' \ + --state-values-set-string addons.llm.pki.secretName=operator-quic-tls \ + --state-values-set-string addons.llm.pki.allowedDomains=nvcf.svc.cluster.local + +# Without a Secret name the chart would silently leave the router on plaintext +# QUIC, so the stack must fail at render instead. +expect_router_failure existing-secret-missing-name \ + 'addons.llm.pki.secretName is required when addons.llm.pki.mode is existingSecret' + +# An unknown mode must fail in the dependency state too, so a typo cannot skip +# the issuer release while the core state still issues a Certificate. +expect_failure existing-secret-unknown-mode \ + 'addons.llm.pki.mode must be exactly "certManager" or "existingSecret", got "existingsecret"' \ + --state-values-set addons.llm.enabled=true \ + --state-values-set addons.llm.pki.enabled=true \ + --state-values-set-string addons.llm.pki.mode=existingsecret + echo "check-llm-pki-issuer: all checks passed" diff --git a/docs/user/llm-function-enablement.md b/docs/user/llm-function-enablement.md index 4265665e9..28995c9ce 100644 --- a/docs/user/llm-function-enablement.md +++ b/docs/user/llm-function-enablement.md @@ -35,8 +35,9 @@ When enabled, the stack creates: Production deployments must secure the QUIC transport between each LLM worker and the request router. The request router presents a certificate issued by -cert-manager. Each compute plane receives the public root CA certificate and -uses the combined system and private trust bundle in the `llm-worker` sidecar. +cert-manager, or one you issue yourself and supply in a pre-created Secret. +Each compute plane receives the public root CA certificate and uses the +combined system and private trust bundle in the `llm-worker` sidecar. The request-router address configured for the compute plane must use a DNS name listed in the certificate SANs. For a single-cluster deployment, use @@ -156,6 +157,71 @@ set, and accepts any supported `routingMethod` from a function. When a load-balancer configuration is set, a function can only select an algorithm that the configuration enables. +### Pre-created request-router Secret + +Set `mode: existingSecret` when you already issue the request-router server +certificate yourself and want the stack to mount it without managing issuance: + +```yaml +addons: + llm: + enabled: true + pki: + enabled: true + mode: existingSecret + secretName: stargate-quic-tls +``` + +The stack renders no `Certificate`, installs no `ClusterIssuer`, and adds no +cert-manager or OpenBao dependency for the request router. You can set +`certManager.enabled: false` and `openbao.enabled: false` when no other stack +component needs them. + +Create the Secret in the `nvcf` namespace before installing the stack. It must +carry the `tls.crt` and `tls.key` entries, as a `kubernetes.io/tls` Secret +does: + +```bash +kubectl create secret tls stargate-quic-tls \ + --namespace nvcf \ + --cert path/to/tls.crt \ + --key path/to/tls.key +``` + +`clusterIssuer.enabled`, `dnsNames`, and `allowedDomains` only steer +stack-managed issuance. Rendering fails if any of them is set in this mode, so +a configuration that expects the stack to issue a certificate cannot be +mistaken for one that expects you to. + +The certificate must carry a SAN covering the router's advertised hostname and +the address workers connect to. At the default single-replica configuration +that is `llm-request-router.nvcf.svc.cluster.local`. At higher replica counts +the router advertises per-pod headless names, so use a leftmost wildcard such +as `*.llm-request-router-headless.nvcf.svc.cluster.local`. Include any external +name set in `global.workerEndpoints.llmRequestRouterAddress`. The stack cannot +read your Secret at render time, so it validates neither the SANs nor the +expiry. A certificate that does not cover the advertised hostname fails at +worker connection time, not at install time. + +You own issuance, renewal, rotation, and recovery in this mode: + +- Renewal and rotation: update the Secret, then restart the router with + `kubectl rollout restart statefulset/llm-request-router --namespace nvcf`. + The router reads the certificate at startup. +- Expiry: track it yourself. Nothing in the stack renews the certificate or + alerts on an approaching expiry. +- Recovery: if the Secret is deleted or malformed, the router pods fail to + start. Restore the Secret and roll the StatefulSet. + +Compute-plane trust works exactly as described in +[Compute-plane trust](#compute-plane-trust). Author the `transportTls` block +in the control-plane profile by hand with the public root CA that signed your +certificate, as you would for an external issuer. The profile exporter sources +a bundle only from the managed OpenBao hierarchy, so it leaves the block empty +here. The bundle must contain `CERTIFICATE` blocks only. Profile validation +rejects a private key, so the request-router private key never reaches the +compute plane. + ### External cert-manager Set `certManager.enabled: false` when cert-manager is installed and managed @@ -457,9 +523,10 @@ kubectl get httproute -A | grep llm ## Certificate Renewal cert-manager renews the request-router certificate and updates -`Secret/stargate-quic-tls`. The request router loads its certificate when the -pod starts. Restart the StatefulSet after renewal so every replica uses the -updated certificate: +`Secret/stargate-quic-tls`. With `mode: existingSecret` there is no renewal +loop and you update the Secret yourself. Either way, the request router loads +its certificate when the pod starts. Restart the StatefulSet after renewal so +every replica uses the updated certificate: ```bash kubectl -n nvcf rollout restart statefulset/llm-request-router