Skip to content

feat: add EnvoyProxy.spec.runtime served over RTDS #9671

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
kanurag94 wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

feat: add EnvoyProxy.spec.runtime served over RTDS #9671

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions api/v1alpha1/envoyproxy_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
package v1alpha1

import (
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
gwapiv1 "sigs.k8s.io/gateway-api/apis/v1"
)
Expand Down Expand Up @@ -67,6 +68,32 @@ type EnvoyProxySpec struct {
// +optional
Bootstrap *ProxyBootstrap `json:"bootstrap,omitempty"`

// Runtime defines Envoy runtime values for managed proxies, as a map of runtime key
// to value. Values are typed: numbers, booleans and strings are all preserved as
// written, which matters because Envoy resolves boolean runtime guards only from real
// booleans and numeric limits only from numbers or numeric strings.
//
// Runtime values are delivered over RTDS, so changes apply to running proxies without
// a restart and without triggering a CDS update.
//
// One use is tuning circuit breakers on clusters that Envoy Gateway does not build from
// its own configuration, such as clusters added by an extension server, which otherwise
// keep Envoy's built-in defaults:
//
// runtime:
// circuit_breakers.my-extension-cluster.default.max_requests: 16384
//
// Runtime keys are an Envoy-level interface and are not validated by Envoy Gateway.
// Visit https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime
// to learn more about the available keys.
//
// Note: this has no effect when spec.bootstrap uses the Replace type with a bootstrap
// that omits the RTDS layer, since the proxy then has no layer for these values to
// land in.
//
// +optional
Runtime map[string]apiextensionsv1.JSON `json:"runtime,omitempty"`

// Concurrency defines the number of worker threads to run. If unset, it defaults to
// the number of cpuset threads on the platform.
//
Expand Down
7 changes: 7 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 21 additions & 0 deletions charts/gateway-crds-helm/templates/generated/gateway.envoyproxy.io_envoyproxies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11299,6 +11299,27 @@ spec:
RoutingType can be set to "Service" to use the Service Cluster IP for routing to the backend,
or it can be set to "Endpoint" to use Endpoint routing. The default is "Endpoint".
type: string
runtime:
additionalProperties:
x-kubernetes-preserve-unknown-fields: true
description: "Runtime defines Envoy runtime values for managed proxies,
as a map of runtime key\nto value. Values are typed: numbers, booleans
and strings are all preserved as\nwritten, which matters because
Envoy resolves boolean runtime guards only from real\nbooleans and
numeric limits only from numbers or numeric strings.\n\nRuntime
values are delivered over RTDS, so changes apply to running proxies
without\na restart and without triggering a CDS update.\n\nOne use
is tuning circuit breakers on clusters that Envoy Gateway does not
build from\nits own configuration, such as clusters added by an
extension server, which otherwise\nkeep Envoy's built-in defaults:\n\n\truntime:\n\t
\ circuit_breakers.my-extension-cluster.default.max_requests: 16384\n\nRuntime
keys are an Envoy-level interface and are not validated by Envoy
Gateway.\nVisit https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime\nto
learn more about the available keys.\n\nNote: this has no effect
when spec.bootstrap uses the Replace type with a bootstrap\nthat
omits the RTDS layer, since the proxy then has no layer for these
values to\nland in."
type: object
shutdown:
description: Shutdown defines configuration for graceful envoy shutdown
process.
Expand Down
21 changes: 21 additions & 0 deletions charts/gateway-helm/charts/crds/crds/generated/gateway.envoyproxy.io_envoyproxies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11298,6 +11298,27 @@ spec:
RoutingType can be set to "Service" to use the Service Cluster IP for routing to the backend,
or it can be set to "Endpoint" to use Endpoint routing. The default is "Endpoint".
type: string
runtime:
additionalProperties:
x-kubernetes-preserve-unknown-fields: true
description: "Runtime defines Envoy runtime values for managed proxies,
as a map of runtime key\nto value. Values are typed: numbers, booleans
and strings are all preserved as\nwritten, which matters because
Envoy resolves boolean runtime guards only from real\nbooleans and
numeric limits only from numbers or numeric strings.\n\nRuntime
values are delivered over RTDS, so changes apply to running proxies
without\na restart and without triggering a CDS update.\n\nOne use
is tuning circuit breakers on clusters that Envoy Gateway does not
build from\nits own configuration, such as clusters added by an
extension server, which otherwise\nkeep Envoy's built-in defaults:\n\n\truntime:\n\t
\ circuit_breakers.my-extension-cluster.default.max_requests: 16384\n\nRuntime
keys are an Envoy-level interface and are not validated by Envoy
Gateway.\nVisit https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime\nto
learn more about the available keys.\n\nNote: this has no effect
when spec.bootstrap uses the Replace type with a bootstrap\nthat
omits the RTDS layer, since the proxy then has no layer for these
values to\nland in."
type: object
shutdown:
description: Shutdown defines configuration for graceful envoy shutdown
process.
Expand Down
12 changes: 12 additions & 0 deletions internal/cmd/egctl/testdata/translate/out/default-resources.all.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,12 @@ envoyProxyForGatewayClass:
static_layer:
re2.max_program_size.error_level: 4294967295
re2.max_program_size.warn_level: 1000
- name: envoy-gateway-runtime
rtds_layer:
name: envoy-gateway-runtime
rtds_config:
ads: {}
resource_api_version: V3
dynamic_resources:
ads_config:
api_type: DELTA_GRPC
Expand Down Expand Up @@ -604,6 +610,12 @@ xds:
staticLayer:
re2.max_program_size.error_level: 4294967295
re2.max_program_size.warn_level: 1000
- name: envoy-gateway-runtime
rtdsLayer:
name: envoy-gateway-runtime
rtdsConfig:
ads: {}
resourceApiVersion: V3
node:
locality:
zone: $(ENVOY_SERVICE_ZONE)
Expand Down
10 changes: 10 additions & 0 deletions internal/cmd/egctl/testdata/translate/out/from-gateway-api-to-xds.all.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,16 @@
"re2.max_program_size.error_level": 4294967295,
"re2.max_program_size.warn_level": 1000
}
},
{
"name": "envoy-gateway-runtime",
"rtdsLayer": {
"name": "envoy-gateway-runtime",
"rtdsConfig": {
"ads": {},
"resourceApiVersion": "V3"
}
}
}
]
},
Expand Down
6 changes: 6 additions & 0 deletions internal/cmd/egctl/testdata/translate/out/from-gateway-api-to-xds.all.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ xds:
staticLayer:
re2.max_program_size.error_level: 4294967295
re2.max_program_size.warn_level: 1000
- name: envoy-gateway-runtime
rtdsLayer:
name: envoy-gateway-runtime
rtdsConfig:
ads: {}
resourceApiVersion: V3
node:
locality:
zone: $(ENVOY_SERVICE_ZONE)
Expand Down
6 changes: 6 additions & 0 deletions internal/cmd/egctl/testdata/translate/out/from-gateway-api-to-xds.bootstrap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,12 @@ xds:
staticLayer:
re2.max_program_size.error_level: 4294967295
re2.max_program_size.warn_level: 1000
- name: envoy-gateway-runtime
rtdsLayer:
name: envoy-gateway-runtime
rtdsConfig:
ads: {}
resourceApiVersion: V3
node:
locality:
zone: $(ENVOY_SERVICE_ZONE)
Expand Down
10 changes: 10 additions & 0 deletions internal/cmd/egctl/testdata/translate/out/jwt-single-route-single-match-to-xds.all.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,16 @@
"re2.max_program_size.error_level": 4294967295,
"re2.max_program_size.warn_level": 1000
}
},
{
"name": "envoy-gateway-runtime",
"rtdsLayer": {
"name": "envoy-gateway-runtime",
"rtdsConfig": {
"ads": {},
"resourceApiVersion": "V3"
}
}
}
]
},
Expand Down
6 changes: 6 additions & 0 deletions internal/cmd/egctl/testdata/translate/out/jwt-single-route-single-match-to-xds.all.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ xds:
staticLayer:
re2.max_program_size.error_level: 4294967295
re2.max_program_size.warn_level: 1000
- name: envoy-gateway-runtime
rtdsLayer:
name: envoy-gateway-runtime
rtdsConfig:
ads: {}
resourceApiVersion: V3
node:
locality:
zone: $(ENVOY_SERVICE_ZONE)
Expand Down
6 changes: 6 additions & 0 deletions ...rnal/cmd/egctl/testdata/translate/out/jwt-single-route-single-match-to-xds.bootstrap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,12 @@ xds:
staticLayer:
re2.max_program_size.error_level: 4294967295
re2.max_program_size.warn_level: 1000
- name: envoy-gateway-runtime
rtdsLayer:
name: envoy-gateway-runtime
rtdsConfig:
ads: {}
resourceApiVersion: V3
node:
locality:
zone: $(ENVOY_SERVICE_ZONE)
Expand Down
6 changes: 6 additions & 0 deletions internal/cmd/egctl/testdata/translate/out/no-service-cluster-ip.all.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ xds:
staticLayer:
re2.max_program_size.error_level: 4294967295
re2.max_program_size.warn_level: 1000
- name: envoy-gateway-runtime
rtdsLayer:
name: envoy-gateway-runtime
rtdsConfig:
ads: {}
resourceApiVersion: V3
node:
locality:
zone: $(ENVOY_SERVICE_ZONE)
Expand Down
75 changes: 75 additions & 0 deletions internal/gatewayapi/testdata/envoyproxy-runtime.in.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
envoyProxyForGatewayClass:
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: custom-proxy-config
namespace: envoy-gateway-system
spec:
runtime:
circuit_breakers.ai-gateway-extproc-uds.default.max_requests: 16384
envoy.reloadable_features.some_feature: false
some.string.key: "a string"
gateways:
- apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
namespace: envoy-gateway
name: gateway-1
spec:
gatewayClassName: envoy-gateway-class
listeners:
- name: http
protocol: HTTP
port: 80
allowedRoutes:
namespaces:
from: All
httpRoutes:
- apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
namespace: envoy-gateway
name: httproute-1
spec:
parentRefs:
- namespace: envoy-gateway
name: gateway-1
sectionName: http
rules:
- matches:
- path:
value: "/"
backendRefs:
- name: service-1
port: 8080
services:
- apiVersion: v1
kind: Service
metadata:
namespace: envoy-gateway
name: service-1
spec:
selector:
app: service-1
ports:
- name: http
protocol: TCP
port: 8080
endpointslices:
- apiVersion: discovery.k8s.io/v1
kind: EndpointSlice
metadata:
namespace: envoy-gateway
name: service-1
labels:
kubernetes.io/service-name: service-1
addressType: IPv4
ports:
- name: http
protocol: TCP
port: 8080
endpoints:
- addresses:
- 7.7.7.7
conditions:
ready: true
Loading
Loading