Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ metadata:
namespace: galactic-system
spec:
vpcRef: "60"
vpcAttachmentRef: "60"
vipAddresses:
- 2001:db8:6060:1::1
protocol: tcp
Expand All @@ -24,7 +23,6 @@ metadata:
namespace: galactic-system
spec:
vpcRef: "60"
vpcAttachmentRef: "60"
vipAddresses:
- 2001:db8:6060:1::1
protocol: udp
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ metadata:
namespace: galactic-system
spec:
vpcRef: "60"
vpcAttachmentRef: "60"
vipAddresses:
- 2001:db8:6060:3::1
protocol: tcp
Expand All @@ -24,7 +23,6 @@ metadata:
namespace: galactic-system
spec:
vpcRef: "60"
vpcAttachmentRef: "60"
vipAddresses:
- 2001:db8:6060:3::1
protocol: udp
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ metadata:
namespace: galactic-system
spec:
vpcRef: "60"
vpcAttachmentRef: "60"
vipAddresses:
- 2001:db8:6060:2::1
protocol: tcp
Expand All @@ -24,7 +23,6 @@ metadata:
namespace: galactic-system
spec:
vpcRef: "60"
vpcAttachmentRef: "60"
vipAddresses:
- 2001:db8:6060:2::1
protocol: udp
Expand Down
13 changes: 6 additions & 7 deletions docs/agents/ARCHITECTURE-GATEWAY.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,10 +103,10 @@ per-VPC gateway design to colocate with the workload's own VRF.

### The two CRDs

| CRD | Scope | Written by | Purpose |
| ---------------- | -------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `NetworkGateway` | Namespaced, one per gateway node (`spec.targetRef.name` = node name) | Operator, once per gateway node (see [worked example](#worked-containerlab-example) below) | Node-scoped root object, mirroring `BGPRouter`'s pattern. Identifies which nodes participate in the anycast mesh and surfaces each node's engine health via `status.conditions`. Carries **no** self-address field — DSR rewrites nothing, so there is no translation source to publish. |
| `NetworkRule` | Namespaced, tenant-writable | Tenant, via an admission webhook that verifies VPC/VPCAttachment ownership (**webhook not yet deployed in this repo** — see Known Constraints) | Ingress load-balancing spec: `vpcRef`/`vpcAttachmentRef` (opaque tenant identifiers), `vipAddresses` (1–8, IPv4/IPv6), `protocol` (`tcp`/`udp`), `port`, `backends` (1–64 `address:port` pairs). Served by every `NetworkGateway` in the namespace identically. |
| CRD | Scope | Written by | Purpose |
| ---------------- | -------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `NetworkGateway` | Namespaced, one per gateway node (`spec.targetRef.name` = node name) | Operator, once per gateway node (see [worked example](#worked-containerlab-example) below) | Node-scoped root object, mirroring `BGPRouter`'s pattern. Identifies which nodes participate in the anycast mesh and surfaces each node's engine health via `status.conditions`. Carries **no** self-address field — DSR rewrites nothing, so there is no translation source to publish. |
| `NetworkRule` | Namespaced, tenant-writable | Tenant, via an admission webhook that verifies VPC ownership (**webhook not yet deployed in this repo** — see Known Constraints) | Ingress load-balancing spec: `vpcRef` (opaque tenant identifier), `vipAddresses` (1–8, IPv4/IPv6), `protocol` (`tcp`/`udp`), `port`, `backendSelector` (the VPCAttachments in `vpcRef` that serve it), `backendPort`. Served by every `NetworkGateway` in the namespace identically. |

Both are defined in `go.datum.net/network`'s `api/v1alpha1` package
(`gateway_types.go`, `rule_types.go`) — the same external CRD module the
Expand Down Expand Up @@ -724,9 +724,8 @@ pod on that node is not a supported configuration.
- **No tenant dimension in the datapath.** `vip_table` is keyed by `(proto,
VIP port, VIP address)` only — a VIP is globally unique by construction,
so no tenant/VPC field is needed to disambiguate ingress traffic.
`DesiredRule.VPCRef`/`VPCAttachmentRef` are carried through purely for
telemetry labeling and admission-webhook auditing, never consulted by the
datapath itself.
`DesiredRule.VPCRef` is carried through only for the per-tenant rule
quota, never consulted by the datapath itself.
- **uSID resolution for backends.** There is no exported "IP → uSID" query
anywhere else in this codebase (`internal/runtime/gobgp/monitor.go`
decodes EVPN Prefix-SID attributes purely internally, for local kernel
Expand Down
20 changes: 9 additions & 11 deletions docs/gateway/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -447,16 +447,15 @@ gateway node, named after that node (see the worked example above).

### `NetworkRule` — tenant-writable ingress load-balancing spec

| Field | Required | Type | Description |
| ----------------------- | -------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `spec.vpcRef` | Yes | `string` | Opaque VPC identifier (owned by the companion operator, not validated here beyond non-emptiness). Only attachments whose `status.vpc` equals it can be backends. |
| `spec.vpcAttachmentRef` | Yes | `string` | Opaque VPC attachment identifier, paired with `vpcRef`. |
| `spec.vipAddresses` | Yes | `[]string` (1–8) | Ingress VIP addresses this rule provisions. The backend nodes translate IPv6 VIPs only (#705). |
| `spec.protocol` | Yes | `tcp` \| `udp` | Transport protocol matched by `vipAddresses`/`port`. |
| `spec.port` | Yes | `int32` (1–65535) | Ingress port on `vipAddresses` this rule load-balances. |
| `spec.backendSelector` | Yes | `metav1.LabelSelector` | Selects the `VPCAttachment`s (`cloud.datumapis.com`, any namespace) that serve the rule. Each contributes its IPv6 interface addresses as backends. Must not be empty. |
| `spec.backendPort` | Yes | `int32` (1–65535) | Destination port on every selected backend. |
| `status.conditions` | — | — | `Accepted` (set `True` once gateway nodes exist; see the admission-webhook caveat below), `<gateway-node>/Programmed` per gateway node, and `<backend-node>/BackendsBound` per node hosting a backend. |
| Field | Required | Type | Description |
| ---------------------- | -------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `spec.vpcRef` | Yes | `string` | Opaque VPC identifier (owned by the companion operator, not validated here beyond non-emptiness). Only attachments whose `status.vpc` equals it can be backends. |
| `spec.vipAddresses` | Yes | `[]string` (1–8) | Ingress VIP addresses this rule provisions. The backend nodes translate IPv6 VIPs only (#705). |
| `spec.protocol` | Yes | `tcp` \| `udp` | Transport protocol matched by `vipAddresses`/`port`. |
| `spec.port` | Yes | `int32` (1–65535) | Ingress port on `vipAddresses` this rule load-balances. |
| `spec.backendSelector` | Yes | `metav1.LabelSelector` | Selects the `VPCAttachment`s (`cloud.datumapis.com`, any namespace) that serve the rule. Each contributes its IPv6 interface addresses as backends. Must not be empty. |
| `spec.backendPort` | Yes | `int32` (1–65535) | Destination port on every selected backend. |
| `status.conditions` | — | — | `Accepted` (set `True` once gateway nodes exist; see the admission-webhook caveat below), `<gateway-node>/Programmed` per gateway node, and `<backend-node>/BackendsBound` per node hosting a backend. |

Example, the TCP rule from `deploy/containerlab/resources/galactic-gateway/iad/networkrules.yaml`:

Expand All @@ -468,7 +467,6 @@ metadata:
namespace: galactic-system
spec:
vpcRef: "60"
vpcAttachmentRef: "60"
vipAddresses:
- 2001:db8:6060:3::1
protocol: tcp
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ require (
github.com/spf13/viper v1.21.0
github.com/vishvananda/netlink v1.3.2-0.20261002201547-1d1c62507e3c
go.datum.net/cloud v0.1.1-0.20260918213150-260f797bbc5c
go.datum.net/network v0.3.1-0.20261008204016-fc3154bde4d8
go.datum.net/network v0.3.1-0.20261009013235-5e00bde12789
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.69.0
go.opentelemetry.io/otel v1.44.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.44.0
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -204,8 +204,8 @@ github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
go.datum.net/cloud v0.1.1-0.20260918213150-260f797bbc5c h1:k0P5980+kS90zCa0M4MohyS0woJL3bh/uEarRrW9PmY=
go.datum.net/cloud v0.1.1-0.20260918213150-260f797bbc5c/go.mod h1:HTeaepXL0EqiqNsQ13K4/kGEwv+VLJ//BGrUfVIFfEA=
go.datum.net/network v0.3.1-0.20261008204016-fc3154bde4d8 h1:SN5aP+EC7S1JRrW57BQ8d9VzQvE9duXZAyMDgGn1pRg=
go.datum.net/network v0.3.1-0.20261008204016-fc3154bde4d8/go.mod h1:LawI34k+ddtv3kRbmjiss0Lb6QKpSr60YP3fX1IG5RU=
go.datum.net/network v0.3.1-0.20261009013235-5e00bde12789 h1:dPm2P7Ao4yRhuZe67WWXHohEo6IJZgT8lavJRn6hE4Y=
go.datum.net/network v0.3.1-0.20261009013235-5e00bde12789/go.mod h1:LawI34k+ddtv3kRbmjiss0Lb6QKpSr60YP3fX1IG5RU=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.69.0 h1:2yEATaop1/a1I4psnSLgWVPLWwCzkqWakgJy7xTDVy0=
Expand Down
9 changes: 4 additions & 5 deletions internal/controller/networkgateway_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -782,11 +782,10 @@ func buildDesiredRule(
}

return gateway.DesiredRule{
Key: rule.Namespace + "/" + rule.Name,
VPCRef: rule.Spec.VPCRef,
VPCAttachmentRef: rule.Spec.VPCAttachmentRef,
VIPAddresses: vips,
Protocol: string(rule.Spec.Protocol),
Key: rule.Namespace + "/" + rule.Name,
VPCRef: rule.Spec.VPCRef,
VIPAddresses: vips,
Protocol: string(rule.Spec.Protocol),
//nolint:gosec // rule.Spec.Port is CRD-validated to [1,65535] (Minimum/Maximum markers on NetworkRuleSpec.Port)
Port: uint16(rule.Spec.Port),
Backends: backends,
Expand Down
9 changes: 4 additions & 5 deletions internal/controller/networkrule_controller_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -88,11 +88,10 @@ func newTestRule(name, vpcRef string, vips ...string) *bgpv1alpha1.NetworkRule {
return &bgpv1alpha1.NetworkRule{
ObjectMeta: metav1.ObjectMeta{Namespace: testNamespace, Name: name},
Spec: bgpv1alpha1.NetworkRuleSpec{
VPCRef: vpcRef,
VPCAttachmentRef: "attach-1",
VIPAddresses: vips,
Protocol: bgpv1alpha1.NetworkRuleProtocolTCP,
Port: 443,
VPCRef: vpcRef,
VIPAddresses: vips,
Protocol: bgpv1alpha1.NetworkRuleProtocolTCP,
Port: 443,
BackendSelector: metav1.LabelSelector{
MatchLabels: map[string]string{testBackendLabel: testBackendLabelValue},
},
Expand Down
1 change: 0 additions & 1 deletion internal/gateway/diff.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ func diffRuleKeys(active map[string]DesiredRule, desired map[string]DesiredRule)
func rulesEqual(a, b DesiredRule) bool {
return a.Key == b.Key &&
a.VPCRef == b.VPCRef &&
a.VPCAttachmentRef == b.VPCAttachmentRef &&
a.Protocol == b.Protocol &&
a.Port == b.Port &&
slices.Equal(a.VIPAddresses, b.VIPAddresses) &&
Expand Down
2 changes: 1 addition & 1 deletion internal/gateway/diff_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ func TestRulesEqual(t *testing.T) {
return DesiredBackend{Address: netip.MustParseAddr("2001:db8:1::10"), Port: 8080, USID: usid}
}
base := DesiredRule{
Key: testKeyA, VPCRef: "vpc", VPCAttachmentRef: "att",
Key: testKeyA, VPCRef: "vpc",
VIPAddresses: []netip.Addr{vip}, Protocol: "udp", Port: 80,
Backends: []DesiredBackend{backend(usidA)},
}
Expand Down
10 changes: 4 additions & 6 deletions internal/gateway/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,10 @@ type DesiredRule struct {
// NetworkRule), used as the map key in Engine's convergence pass.
Key string

// VPCRef and VPCAttachmentRef are opaque tenant identifiers, carried for
// telemetry labels and admission auditing. The datapath never needs them: a
// VIP is globally unique, so no tenant dimension disambiguates ingress
// traffic.
VPCRef string
VPCAttachmentRef string
// VPCRef is the tenant's opaque VPC identifier, used to count rules against
// the per-tenant quota. The datapath never needs it: a VIP is globally
// unique, so no tenant dimension disambiguates ingress traffic.
VPCRef string

// VIPAddresses are the ingress VIP addresses this rule provisions.
VIPAddresses []netip.Addr
Expand Down
14 changes: 7 additions & 7 deletions internal/webhook/networkrule_webhook.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ import (

// NetworkRuleValidator is the admission validator for NetworkRule, verifying
// through the pluggable Authorizer that the requesting identity may act on the
// VPC and attachment the rule names before a create or update is admitted.
// VPC the rule names before a create or update is admitted.
type NetworkRuleValidator struct {
// Authorizer performs the actual check. Production callers must not leave
// this as AllowAllAuthorizer.
Expand All @@ -26,16 +26,16 @@ type NetworkRuleValidator struct {
var _ admission.Validator[*networkv1alpha1.NetworkRule] = &NetworkRuleValidator{}

// ValidateCreate verifies the requester is authorized for the NetworkRule's
// vpc/vpcattachment before create is admitted.
// VPC before create is admitted.
func (v *NetworkRuleValidator) ValidateCreate(
ctx context.Context, rule *networkv1alpha1.NetworkRule,
) (admission.Warnings, error) {
return nil, v.authorize(ctx, rule)
}

// ValidateUpdate re-verifies authorization on update: a rule's VPC and
// attachment references could otherwise be changed after creation to point at
// another tenant's resources without the create-time check running again.
// ValidateUpdate re-verifies authorization on update: a rule's VPC reference
// could otherwise be changed after creation to point at another tenant's VPC
// without the create-time check running again.
func (v *NetworkRuleValidator) ValidateUpdate(
ctx context.Context, _, newRule *networkv1alpha1.NetworkRule,
) (admission.Warnings, error) {
Expand Down Expand Up @@ -74,8 +74,8 @@ func (v *NetworkRuleValidator) authorize(ctx context.Context, rule *networkv1alp
}
if !ok {
return fmt.Errorf(
"%s is not authorized for vpcRef %q / vpcAttachmentRef %q (%s)",
req.UserInfo.Username, rule.Spec.VPCRef, rule.Spec.VPCAttachmentRef,
"%s is not authorized for vpcRef %q (%s)",
req.UserInfo.Username, rule.Spec.VPCRef,
networkv1alpha1.AcceptedReasonOwnershipDenied)
}
return nil
Expand Down
3 changes: 1 addition & 2 deletions internal/webhook/networkrule_webhook_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,7 @@ func contextWithRequester(username string) context.Context {
func testRule() *networkv1alpha1.NetworkRule {
return &networkv1alpha1.NetworkRule{
Spec: networkv1alpha1.NetworkRuleSpec{
VPCRef: "vpc-1",
VPCAttachmentRef: "attach-1",
VPCRef: "vpc-1",
},
}
}
Expand Down
Loading