feat: allow an external autoscaler to own proxy Deployment replicas - #9628
feat: allow an external autoscaler to own proxy Deployment replicas#9628guanchzhou wants to merge 2 commits into
Conversation
Add EnvoyProxy.spec.provider.kubernetes.envoyDeployment.replicasManagedByExternalAutoscaler. When true, Envoy Gateway omits the replicas field from the generated proxy Deployment so an external autoscaler (an external HorizontalPodAutoscaler, a KEDA ScaledObject, etc.) can own the replica count without Envoy Gateway reverting it on the next reconcile. This reuses the same server-side-apply ownership behavior already used for the built-in envoyHpa, which only supports what a native HPA metrics spec can express. Fixes envoyproxy#9627 Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com>
✅ Deploy Preview for cerulean-figolla-1f9435 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9d0fadb8a4
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9628 +/- ##
=======================================
Coverage 76.04% 76.04%
=======================================
Files 259 259
Lines 43275 43277 +2
=======================================
+ Hits 32907 32912 +5
+ Misses 8178 8176 -2
+ Partials 2190 2189 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…nt, regen helm goldens - Honor replicasManagedByExternalAutoscaler in the ratelimit Deployment renderer, since KubernetesDeploymentSpec is shared with rateLimitDeployment. Previously the flag was silently ignored there and EG kept owning spec.replicas. - Add a with-external-autoscaler golden test for the ratelimit deployment. - Regenerate gateway-crds-helm test goldens to include the new CRD field (gen-check). Signed-off-by: Andrey Maltsev <maltsev.andrey@gmail.com>
|
does #9594 fix this issue ? |
|
@arkodg Not quite — #9594 and this PR are complementary:
if provider.GetEnvoyProxyKubeProvider().EnvoyHpa != nil ||
ptr.Deref(deploymentConfig.ReplicasManagedByExternalAutoscaler, false) {
replicas = nil
}The motivating case is a KEDA Happy to rebase or adjust naming (boolean vs an enum like |
|
|
What this PR does
Adds
replicasManagedByExternalAutoscalerto the Envoy proxyKubernetesDeploymentSpec. When set totrue, Envoy Gateway omits thereplicasfield from the generated proxy Deployment, so an external autoscaler — a user-managedHorizontalPodAutoscaler, a KEDAScaledObject, etc. — can own the replica count without Envoy Gateway reverting it on the next reconcile.Why
Envoy Gateway applies the proxy Deployment with server-side apply +
ForceOwnership, so it ownsspec.replicasand reverts any externally computed value unless the built-inenvoyHpais configured. ButenvoyHpaonly supports what a native HPA metrics spec can express — it can't drive Prometheus-query / event-driven scaling that KEDA provides.Today the only workaround is to template-unset
envoyDeployment.replicasso Envoy Gateway doesn't own the field — brittle, and it has to be repeated per gateway. This is the same class of problem as the stale #5751; the recent fix that omitsspec.replicasunder an HPA only covers the built-inenvoyHpa, not external autoscalers.Fixes #9627.
Implementation
api/v1alpha1/shared_types.go): new optionalReplicasManagedByExternalAutoscaler *boolonKubernetesDeploymentSpec.internal/infrastructure/kubernetes/proxy/resource_provider.go):replicasis leftnilwhen the flag is set orEnvoyHpa != nil. Semantics mirror the existingenvoyHpapath — when set,Replicasis ignored (precedence, no hard rejection, which is friendlier for GitOps migrations). Deployment only (DaemonSet has no replicas).envoyproxiesCRDs, and the API reference.customize-envoyproxy.md, including how to target the Deployment via a pinnedenvoyDeployment.nameor the stablegateway.envoyproxy.io/owning-gateway-*labels, with a KEDAScaledObjectexample.Testing
New golden test case
with-external-autoscalerassertsspec.replicasis absent when the flag is set and noenvoyHpais configured (the exact fingerprint of the bug), and unchanged otherwise.Open question for maintainers
Boolean (
replicasManagedByExternalAutoscaler, as here) vs. an enum (replicaManagement: EnvoyGateway | External). Happy to switch to whichever you prefer. A follow-up PR can mirror the field on the rate limit deployment for symmetry.