diff --git a/docs/agents/ARCHITECTURE-GATEWAY.md b/docs/agents/ARCHITECTURE-GATEWAY.md index 258e5e01..460b8fe8 100644 --- a/docs/agents/ARCHITECTURE-GATEWAY.md +++ b/docs/agents/ARCHITECTURE-GATEWAY.md @@ -759,7 +759,7 @@ that tag into `config/galactic-gateway/base`. - **No `NetworkRule` admission webhook is deployed in this repo.** The CRD's own doc comment and `NetworkRuleReconciler`'s doc comment both describe an admission webhook that verifies VPC/VPCAttachment ownership before setting the `Accepted` condition — no `ValidatingWebhookConfiguration` exists in `config/` yet (`config/webhook/` doesn't exist). `updateAcceptedCondition` currently sets `Accepted=True` unconditionally once gateway nodes exist for the namespace, not gated on any ownership check. - **The uSID backend resolver's tenant-ownership check depends on `BGPVRFInstance` naming staying deterministic.** `verifyTenantOwnership` closes the ambiguous-match gap an earlier version of this resolver had, but it is only as strong as `crdnames.BGPVRFInstanceName(vpc, nodeName)` staying the exact name `galactic-bgp` writes — a divergence between the two would fail closed (a real backend never resolving) rather than open (a wrong-tenant resolve), which is the safer failure direction but still worth knowing about when either side of that naming contract changes. - **`GALACTIC_GATEWAY_SRV6_ADDRESS` has no in-cluster derivation mechanism.** It is operator-supplied per gateway node today; nothing in this repo yet computes it automatically from a node's own `BGPRouter` locator/node-ID. See [SRv6 encap-source address](#srv6-encap-source-address) above. -- **The uSID TC-BPF/XDP FIB-lookup PMTUD gap applies here too.** When `bpf_fib_lookup()` returns `BPF_FIB_LKUP_RET_FRAG_NEEDED`, `edgedsr.c` counts `DROP_REASON_FIB_FRAG_NEEDED` and drops rather than emitting an ICMPv6 Packet Too Big — the same gap `internal/plumbing/ebpf/prog/usid.c`'s `usid_ingress` has for its FIB lookup (see [ARCHITECTURE-CNI.md#known-constraints](ARCHITECTURE-CNI.md#known-constraints)). `usid_egress` and the egress shard both send Packet Too Big now; closing it here is #700. +- **Only the forward path sends Packet Too Big.** When the route to a backend refuses a client's packet for size once the 40-byte outer header is on (`BPF_FIB_LKUP_RET_FRAG_NEEDED`), `edge_lb` counts the drop as `fib_frag_needed` and answers the client with an ICMPv6 Packet Too Big from the VIP, reporting the route's MTU less 40 and quoting the packet's IPv6 header and first 8 transport bytes (#700). A route MTU under 1320 gets no Packet Too Big, since it would report less than the 1280 an IPv6 sender never goes below. The messages draw on a per-CPU token bucket (`icmp_rate_bucket`, 1000 a second with a burst of 100 per CPU, the egress shard's limits), and a refusal is counted as `icmp_rate_limited`. `edge_return` still drops a reply too big for its route as `fib_frag_needed` without telling the backend, and sends no Time Exceeded for an expired hop limit. - **`bpf_fib_lookup()` requires IPv6 forwarding sysctls on the public uplink, not just XDP driver support.** `setupGatewayDatapath` calls `sysctl.ConfigureFIBLookupUplinkSysctls` before attaching the datapath — without `net.ipv6.conf..forwarding` and `net.ipv6.conf.all.forwarding` both set, the kernel returns `BPF_FIB_LKUP_RET_NOT_FWDED` for every lookup regardless of anything this program does, which `edgedsr.c`'s own drop-reason accounting cannot distinguish from a generic FIB lookup failure. Found via live-kernel investigation of a pre-existing containerlab veth/XDP_TX blocker: the sysctl gap, not `XDP_TX` itself, was the actual cause. A related, lab-only characteristic the same investigation turned up: native `XDP_TX` on a veth pair only promotes a frame into the peer's normal receive stack (visible to `tcpdump`) if the peer *also* runs an XDP program — otherwise delivery uses a raw fast-path invisible to normal tools. This does not apply to a real physical NIC uplink in production, where there is no "peer's own XDP program" question to begin with. - **A bonded public uplink attaches per-slave, with per-slave FIB-lookup sysctls to match.** Native-mode XDP against a Linux bonding master is not reliable: confirmed failing outright with "operation not supported" on a real gateway node (802.3ad over an igb/tg3 slave pair). Not every kernel's bonding driver categorically lacks `ndo_bpf` — some do implement it by forwarding the attach to every slave — but that still requires each slave's own driver to support native XDP itself, which not every NIC driver does (tg3 is a commonly cited example that doesn't), so this codebase never relies on attaching to the master working, on any kernel. `edgeattach.ResolveTargets` expands a bonding-master `GALACTIC_GATEWAY_PUBLIC_INTERFACE` to its slave interfaces (never the master itself — see `internal/plumbing/bond`, shared with `internal/plumbing/ebpf/attach`'s TC-BPF path, which attaches to the master *and* its slaves instead), and `setupGatewayDatapath` attaches to and configures FIB-lookup sysctls on every one of them. This is not cosmetic: `edgedsr.c`'s `push_outer_header` calls `bpf_fib_lookup()` with `ctx->ingress_ifindex` — confirmed against the source, not assumed — which for a native XDP program attached to a bond slave is that slave's own ifindex, not the bond master's, so `net.ipv6.conf..forwarding` (not `net.ipv6.conf..forwarding`) is what the kernel actually checks. A bonding master with no resolvable slaves is a hard startup error, not a silent fallback to attaching the master (which would only risk repeating the same failure). `edgeattach.Attach` is all-or-nothing across every resolved slave in one call — if any single slave's driver can't accept a native XDP attach (a real possibility per the tg3 note above, not independently confirmed against that node specifically), the whole datapath startup fails rather than running in a degraded, missing-that-slave's-traffic state. The attach step itself lives in `internal/plumbing/ebpf/xdpattach`, shared with `natattach`: every target is checked for native XDP support through the netdev generic netlink family before any is touched, and each bond slave is waited back into its aggregate (MII up, plus LACP collecting|distributing on an 802.3ad bond) before the next is attached (#583); this has not been exercised against real igb/tg3 hardware, only veth in tests, so whether all of a real bonded pair's slaves actually accept native XDP on the affected class of hardware is still open. Attaching per-slave rather than to the bond as a whole means the slave set has to be kept current after startup: `watchTargets` (`cmd/galactic-gateway/gateway.go`) re-runs `ResolveTargets` on every netlink link or route change, and every 30s with none, and hands the result to an `xdpattach.Set` per program (#647). The set attaches a member enslaved or replaced (new ifindex under the same name) since startup through the same gate, one at a time, and skips one whose bounce would leave its bond with no member carrying traffic; it never detaches a member that still exists, since detaching bounces the link as attaching does. Unlike the startup attach, one member failing there does not undo the rest: it is retried on the next pass, and while any resolved target lacks the program the gateway's `readiness` gRPC health service (`config.GRPCReadinessService`, the readinessProbe's `service`) reports NOT_SERVING, without failing liveness. - **`XDP_REDIRECT` into a veth needs NAPI enabled on the *peer*, which is a lab-only concern.** `edgedsr.c` returns `XDP_REDIRECT` whenever the route to the backend egresses an interface other than the ingress one. In native mode that calls the egress device's `ndo_xdp_xmit`, and veth's implementation silently discards the frame unless the peer end has NAPI enabled — which for a veth means the peer runs its own XDP program or has GRO turned on. A containerlab peer inside an FRR/transit container has neither by default, so a redirect there can fail exactly the way this datapath's original bug did: no drop counter moves, because `bpf_redirect()` itself succeeded and the discard happens later in `xdp_do_redirect`. Both programs are affected, and `edge_return` more so: it attaches to compute-facing links, which are veth pairs in the lab and may be veth or a plain NIC in production. `bpftool prog tracelog` and the `xdp:xdp_redirect_err` tracepoint are the diagnostics; `ethtool -K gro on` is the lab fix. A production uplink (physical NIC, or a bond slave per the bullet above) implements `ndo_xdp_xmit` natively and has no peer to ask about. This is the redirect-side analogue of the `XDP_TX`-on-veth observability quirk noted above. diff --git a/internal/plumbing/ebpf/edgeprog/dropreason.go b/internal/plumbing/ebpf/edgeprog/dropreason.go index 6a525c1e..b2b3425e 100644 --- a/internal/plumbing/ebpf/edgeprog/dropreason.go +++ b/internal/plumbing/ebpf/edgeprog/dropreason.go @@ -25,7 +25,8 @@ const ( DropReasonNoEgressIfindex uint32 = 7 DropReasonRedirectFailed uint32 = 8 DropReasonReturnHopLimit uint32 = 9 - DropReasonCount uint32 = 10 + DropReasonICMPRateLimited uint32 = 10 + DropReasonCount uint32 = 11 ) // DropReasonNames maps each index to a short, stable, metrics-friendly name, @@ -42,4 +43,5 @@ var DropReasonNames = map[uint32]string{ DropReasonNoEgressIfindex: "no_egress_ifindex", DropReasonRedirectFailed: "redirect_failed", DropReasonReturnHopLimit: "return_hop_limit", + DropReasonICMPRateLimited: "icmp_rate_limited", } diff --git a/internal/plumbing/ebpf/edgeprog/edgedsr.c b/internal/plumbing/ebpf/edgeprog/edgedsr.c index 5ba9b0ea..69418fc8 100644 --- a/internal/plumbing/ebpf/edgeprog/edgedsr.c +++ b/internal/plumbing/ebpf/edgeprog/edgedsr.c @@ -51,6 +51,9 @@ // ingress interface, bpf_redirect and XDP_REDIRECT where it is not. The // two differ on every gateway that reaches its clients and its compute // nodes over different links, which is this role's normal shape. +// 7. A packet the route refuses for size, too big once the outer header is +// on, is answered with an ICMPv6 Packet Too Big to the client instead of +// vanishing; see send_too_big6. // // The verifier's bounds-narrowing behavior described at EDGE_BARRIER_VAR // applies to the backend-index lookup below, a plain byte read from a map @@ -81,6 +84,9 @@ static long (*bpf_xdp_adjust_head)(void *ctx, int delta) = (void *) BPF_FUNC_xdp static long (*bpf_fib_lookup)(void *ctx, struct bpf_fib_lookup *params, __s32 plen, __u32 flags) = (void *) BPF_FUNC_fib_lookup; static __u64 (*bpf_ktime_get_ns)(void) = (void *) BPF_FUNC_ktime_get_ns; +static long (*bpf_xdp_adjust_tail)(void *ctx, int delta) = (void *) BPF_FUNC_xdp_adjust_tail; +static __s64 (*bpf_csum_diff)(__be32 *from, __u32 from_size, __be32 *to, __u32 to_size, + __wsum seed) = (void *) BPF_FUNC_csum_diff; // bpf_redirect, not bpf_redirect_peer: the latter is TC-only, and every egress // interface here is in this node's own namespace anyway. static long (*bpf_redirect)(__u32 ifindex, __u64 flags) = (void *) BPF_FUNC_redirect; @@ -94,6 +100,29 @@ static long (*bpf_redirect)(__u32 ifindex, __u64 flags) = (void *) BPF_FUNC_redi #define EDGE_IPPROTO_TCP 6 #define EDGE_IPPROTO_UDP 17 #define EDGE_IPPROTO_IPV6 41 // this program's own pushed outer header's Next Header, always +#define EDGE_IPPROTO_ICMPV6 58 +#define EDGE_ICMPV6_PACKET_TOO_BIG 2 + +// EDGE_IP6HDR_LEN is the IPv6 fixed header size, and also exactly what +// push_outer_header adds to every packet. +#define EDGE_IP6HDR_LEN 40 + +// EDGE_FRAG_QUOTE6 is how much of the offending packet a Packet Too Big quotes: +// its IPv6 header and first 8 transport bytes, all a sender's stack matches an +// error on. +#define EDGE_FRAG_QUOTE6 (EDGE_IP6HDR_LEN + 8) + +// EDGE_IPV6_MIN_MTU is the smallest MTU an IPv6 sender ever uses (RFC 8200 +// section 5). A Packet Too Big reporting less cannot help: the sender keeps +// sending 1280-byte packets, still too big once encapsulated. +#define EDGE_IPV6_MIN_MTU 1280 + +// The Packet Too Big this program sends is limited by one token bucket per CPU: +// EDGE_ICMP_RATE messages a second with a burst of EDGE_ICMP_BURST, the same +// limits the egress shard uses. See icmp_rate_bucket. +#define EDGE_ICMP_RATE 1000 +#define EDGE_ICMP_BURST 100 +#define EDGE_ICMP_COST_NS (1000000000ULL / EDGE_ICMP_RATE) // EDGE_MAX_BACKENDS matches the CRD's own maximum backend count. It must stay a // power of two for the masking below to be equivalent to a range check. @@ -222,7 +251,8 @@ enum edge_drop_reason { DROP_REASON_NO_EGRESS_IFINDEX = 7, DROP_REASON_REDIRECT_FAILED = 8, DROP_REASON_RETURN_HOP_LIMIT = 9, - DROP_REASON_COUNT = 10, + DROP_REASON_ICMP_RATE_LIMITED = 10, + DROP_REASON_COUNT = 11, }; // --------------------------------------------------------------------- @@ -275,6 +305,28 @@ struct { __type(value, __u64); } drop_reasons SEC(".maps"); +// struct icmp_bucket is one CPU's token bucket, held as nanoseconds of credit +// rather than whole tokens so a refill is an addition, with no division on the +// packet path: each message costs EDGE_ICMP_COST_NS, and credit accrues one +// nanosecond per nanosecond up to EDGE_ICMP_BURST messages' worth. +struct icmp_bucket { + __u64 last_ns; + __u64 credit_ns; +}; + +// icmp_rate_bucket limits the ICMP this program emits, with one independent +// bucket per CPU and no locking. The effective limit for the node is the +// per-CPU rate times the number of CPUs receiving VIP traffic, and it is not +// per client: one client draining a CPU's bucket suppresses the errors every +// other client hashed to that CPU would get until it refills. Every refusal is +// counted as icmp_rate_limited. +struct { + __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); + __uint(max_entries, 1); + __type(key, __u32); + __type(value, struct icmp_bucket); +} icmp_rate_bucket SEC(".maps"); + // The node's XDP dispatcher (internal/plumbing/ebpf/xdpdispatch). edge_lb and // edge_return run from its gateway slots, and a packet either does not claim // goes on to the later slots, in practice the egress translation shard, which @@ -311,6 +363,41 @@ static EDGE_ALWAYS_INLINE void count_claimed_drop(__u32 reason, struct vip_stats __sync_fetch_and_add(&stats->dropped_packets, 1); } +// take_icmp_token reports whether this CPU's bucket can pay for one more +// message, and spends it if so. A clock reading behind last_ns earns nothing +// rather than underflowing into a full bucket. +static EDGE_ALWAYS_INLINE int take_icmp_token(void) +{ + __u32 key = 0; + struct icmp_bucket *b = bpf_map_lookup_elem(&icmp_rate_bucket, &key); + if (!b) + return 0; + + __u64 now = bpf_ktime_get_ns(); + __u64 credit = b->credit_ns + (now > b->last_ns ? now - b->last_ns : 0); + if (credit > EDGE_ICMP_BURST * EDGE_ICMP_COST_NS) + credit = EDGE_ICMP_BURST * EDGE_ICMP_COST_NS; + if (now > b->last_ns) + b->last_ns = now; + + if (credit < EDGE_ICMP_COST_NS) { + b->credit_ns = credit; + return 0; + } + b->credit_ns = credit - EDGE_ICMP_COST_NS; + return 1; +} + +// csum_fold_add folds a bpf_csum_diff result into a ones'-complement checksum. +static EDGE_ALWAYS_INLINE __be16 csum_fold_add(__be16 check, __s64 diff) +{ + __s64 sum = (__u16) ~check; + sum += diff; + sum = (sum & 0xffff) + (sum >> 16); + sum = (sum & 0xffff) + (sum >> 16); + return (__be16) ~((__u16) sum); +} + // stats_row returns key's row in a statistics map, creating a zeroed one on // first use. A row that cannot be created (map full) yields NULL, which every // caller of the counting helpers already tolerates: losing a counter must never @@ -368,10 +455,13 @@ static EDGE_ALWAYS_INLINE __u32 fnv1a_flow(const __u8 addr[16], __be16 port) // overwrites fib_params.ifindex with it on success. Discarding that value is // what made every encapsulated packet leave on the wrong wire before, since the // MACs written below belong to the egress link and nothing else. +// +// A lookup refused for size reports the route's MTU through frag_mtu, when the +// caller passes one. static EDGE_ALWAYS_INLINE long resolve_fib_and_write_eth(void *ctx, __u32 ifindex, const __u8 src[16], const __u8 dst[16], __u16 tot_len, struct edge_ethhdr *eth, - __u32 *egress_ifindex) + __u32 *egress_ifindex, __u16 *frag_mtu) { struct bpf_fib_lookup fib_params; __builtin_memset(&fib_params, 0, sizeof(fib_params)); @@ -382,6 +472,8 @@ static EDGE_ALWAYS_INLINE long resolve_fib_and_write_eth(void *ctx, __u32 ifinde fib_params.tot_len = tot_len; long fib_rc = bpf_fib_lookup(ctx, &fib_params, sizeof(fib_params), BPF_FIB_LOOKUP_DIRECT); + if (fib_rc == BPF_FIB_LKUP_RET_FRAG_NEEDED && frag_mtu) + *frag_mtu = fib_params.mtu_result; if (fib_rc != BPF_FIB_LKUP_RET_SUCCESS) return fib_rc; @@ -408,9 +500,13 @@ static EDGE_ALWAYS_INLINE void count_fib_drop(long fib_rc, struct vip_stats_valu // Every failure here is a claimed packet this gateway then failed to deliver, // so each one counts against the VIP's own dropped_packets as well as its // reason bucket -- a gateway that cannot deliver must not read as one that did. +// +// A packet the route refuses for size also reports the route's MTU through +// frag_mtu, so the caller can tell the client; see send_too_big6. static EDGE_ALWAYS_INLINE int push_outer_header(struct xdp_md *ctx, const __u8 src[16], const __u8 dst[16], __be16 inner_payload_len_plus_ip6hdr, - struct vip_stats_value *stats, __u32 *egress_ifindex) + struct vip_stats_value *stats, __u32 *egress_ifindex, + __u16 *frag_mtu) { if (bpf_xdp_adjust_head(ctx, -40) != 0) { count_claimed_drop(DROP_REASON_ADJUST_HEAD_FAILED, stats); @@ -443,7 +539,7 @@ static EDGE_ALWAYS_INLINE int push_outer_header(struct xdp_md *ctx, const __u8 s long fib_rc = resolve_fib_and_write_eth(ctx, ctx->ingress_ifindex, src, dst, (__u16) (sizeof(*outer) + __builtin_bswap16(inner_payload_len_plus_ip6hdr)), - eth, egress_ifindex); + eth, egress_ifindex, frag_mtu); if (fib_rc != BPF_FIB_LKUP_RET_SUCCESS) { count_fib_drop(fib_rc, stats); return -1; @@ -480,6 +576,93 @@ static EDGE_ALWAYS_INLINE int leave_via(struct xdp_md *ctx, __u32 egress_ifindex return XDP_REDIRECT; } +// send_too_big6 answers a client whose packet is too big for the fabric once +// push_outer_header has added its 40 bytes, the way a router would: with an +// ICMPv6 Packet Too Big, so its path MTU discovery adapts instead of every +// large packet to the VIP vanishing. +// +// It runs after push_outer_header has failed, so the frame holds the 54 bytes +// it wrote and then the client's packet, untouched, which is what gets quoted. +// The message comes from the VIP, the address the client was talking to and +// the one announced publicly, rather than this node's SRv6 source, which +// upstream source filtering may drop. mtu is the route's, less the +// encapsulation, so the client's next packet fits once wrapped. +// +// The packet itself is already counted as a fib_frag_needed drop against the +// VIP, so nothing here counts against the VIP's dropped_packets again; a +// failure only moves its reason bucket. edge_lb claims only TCP and UDP, so +// this never answers an ICMP error with another. +static EDGE_ALWAYS_INLINE int send_too_big6(struct xdp_md *ctx, __u32 mtu) +{ + void *data = (void *) (long) ctx->data; + void *data_end = (void *) (long) ctx->data_end; + const __u8 *inner = data + sizeof(struct edge_ethhdr) + EDGE_IP6HDR_LEN; + if ((void *) (inner + EDGE_FRAG_QUOTE6) > data_end) + return XDP_DROP; + + if (!take_icmp_token()) { + count_drop(DROP_REASON_ICMP_RATE_LIMITED); + return XDP_DROP; + } + + __be32 msg[(8 + EDGE_FRAG_QUOTE6) / 4]; + __builtin_memset(msg, 0, 8); + ((__u8 *) msg)[0] = EDGE_ICMPV6_PACKET_TOO_BIG; + msg[1] = __builtin_bswap32(mtu); + __builtin_memcpy(&msg[2], inner, EDGE_FRAG_QUOTE6); + + // The client is the quoted packet's source, and the VIP its destination. + __u8 client[16]; + __builtin_memcpy(client, &inner[8], 16); + __u8 vip[16]; + __builtin_memcpy(vip, &inner[24], 16); + + { + __be32 pseudo[10]; + __builtin_memcpy(&pseudo[0], vip, 16); + __builtin_memcpy(&pseudo[4], client, 16); + pseudo[8] = __builtin_bswap32(sizeof(msg)); + pseudo[9] = __builtin_bswap32(EDGE_IPPROTO_ICMPV6); + __s64 sum = bpf_csum_diff(0, 0, pseudo, sizeof(pseudo), 0); + sum = bpf_csum_diff(0, 0, msg, sizeof(msg), (__wsum) sum); + __be16 check = csum_fold_add(0xFFFF, sum); + __builtin_memcpy(&((__u8 *) msg)[2], &check, 2); + } + + int cur = (int) ((long) data_end - (long) data); + int want = (int) (sizeof(struct edge_ethhdr) + EDGE_IP6HDR_LEN + sizeof(msg)); + if (bpf_xdp_adjust_tail(ctx, want - cur) != 0) { + count_drop(DROP_REASON_ADJUST_HEAD_FAILED); + return XDP_DROP; + } + data = (void *) (long) ctx->data; + data_end = (void *) (long) ctx->data_end; + if (data + sizeof(struct edge_ethhdr) + EDGE_IP6HDR_LEN + sizeof(msg) > data_end) { + count_drop(DROP_REASON_ADJUST_HEAD_FAILED); + return XDP_DROP; + } + + struct edge_ethhdr *eth = data; + struct edge_ip6hdr *ip6 = (void *) (eth + 1); + __builtin_memset(ip6->vtc_flow, 0, sizeof(ip6->vtc_flow)); + ip6->vtc_flow[0] = 0x60; + ip6->payload_len = __builtin_bswap16(sizeof(msg)); + ip6->nexthdr = EDGE_IPPROTO_ICMPV6; + ip6->hop_limit = 64; + __builtin_memcpy(ip6->saddr, vip, 16); + __builtin_memcpy(ip6->daddr, client, 16); + __builtin_memcpy((void *) (ip6 + 1), msg, sizeof(msg)); + + __u32 egress_ifindex = 0; + long fib_rc = resolve_fib_and_write_eth(ctx, ctx->ingress_ifindex, vip, client, + (__u16) (EDGE_IP6HDR_LEN + sizeof(msg)), eth, &egress_ifindex, 0); + if (fib_rc != BPF_FIB_LKUP_RET_SUCCESS) { + count_fib_drop(fib_rc, 0); + return XDP_DROP; + } + return leave_via(ctx, egress_ifindex, 0); +} + // --------------------------------------------------------------------- // Entry points. // --------------------------------------------------------------------- @@ -571,9 +754,16 @@ int edge_lb(struct xdp_md *ctx) __builtin_bswap16((__u16) sizeof(struct edge_ip6hdr) + __builtin_bswap16(ip6->payload_len)); __u32 egress_ifindex = 0; + __u16 frag_mtu = 0; if (push_outer_header(ctx, cfg->encap_src, b->usid, inner_payload_len_plus_ip6hdr, stats, - &egress_ifindex) != 0) + &egress_ifindex, &frag_mtu) != 0) { + // A fabric route under 1320 bytes cannot carry even a minimum-MTU + // packet once encapsulated, so no Packet Too Big would help; the drop + // is already counted as fib_frag_needed. + if (frag_mtu >= EDGE_IPV6_MIN_MTU + EDGE_IP6HDR_LEN) + return send_too_big6(ctx, frag_mtu - EDGE_IP6HDR_LEN); return XDP_DROP; + } return leave_via(ctx, egress_ifindex, stats); } @@ -636,8 +826,8 @@ int edge_return(struct xdp_md *ctx) // Forwarding in XDP means the kernel never sees this packet, so nothing // else decrements the hop limit. Omitting it would forward a looping // packet forever and make this node invisible to traceroute. No ICMPv6 - // Time Exceeded is emitted, the same accepted gap this datapath already - // has for Packet Too Big. + // Time Exceeded is emitted, and a reply too big for its route is dropped + // as fib_frag_needed without a Packet Too Big; only edge_lb sends one. if (ip6->hop_limit <= 1) { count_claimed_drop(DROP_REASON_RETURN_HOP_LIMIT, stats); return XDP_DROP; @@ -646,7 +836,7 @@ int edge_return(struct xdp_md *ctx) __u32 egress_ifindex = 0; long fib_rc = resolve_fib_and_write_eth(ctx, ctx->ingress_ifindex, ip6->saddr, ip6->daddr, (__u16) (sizeof(*ip6) + __builtin_bswap16(ip6->payload_len)), - eth, &egress_ifindex); + eth, &egress_ifindex, 0); if (fib_rc != BPF_FIB_LKUP_RET_SUCCESS) { // Including BPF_FIB_LKUP_RET_NOT_FWDED, which covers both a // locally destined reply and forwarding being disabled on this diff --git a/internal/plumbing/ebpf/edgeprog/edgedsr_ptb_test.go b/internal/plumbing/ebpf/edgeprog/edgedsr_ptb_test.go new file mode 100644 index 00000000..c26aee77 --- /dev/null +++ b/internal/plumbing/ebpf/edgeprog/edgedsr_ptb_test.go @@ -0,0 +1,333 @@ +// Copyright 2026 Datum Cloud, Inc. +// +// SPDX-License-Identifier: AGPL-3.0-or-later + +package edgeprog + +import ( + "encoding/binary" + "math" + "net" + "net/netip" + "os" + "runtime" + "testing" + + "github.com/cilium/ebpf" + "github.com/vishvananda/netlink" + "golang.org/x/sys/unix" +) + +const ( + xdpRedirect = 4 + + ipprotoICMPv6 = 58 + icmpv6PacketTooBig = 2 + icmpLen = 8 + + // fabricMTU is the link every test here routes both the backend's uSID and + // the client over. + fabricMTU = 1500 + // bigPayload makes a TCP packet of exactly fabricMTU bytes: it fits the + // client's path, and not the fabric once the gateway adds its 40 bytes. + bigPayload = fabricMTU - ip6Len - tcpLen +) + +// fabricLinkMAC is the next hop every route in narrowFabric resolves to. +var fabricLinkMAC = net.HardwareAddr{0x02, 0, 0, 0, 0, 0x01} + +// icmpBucket mirrors the C struct icmp_bucket, which bpf2go is not asked to +// generate a type for. +type icmpBucket struct { + LastNs uint64 + CreditNs uint64 +} + +// narrowFabric moves the calling test's thread into a fresh network namespace +// whose routes reach each of dsts over a link of mtu bytes. Every FIB +// lookup a test run makes resolves there: an encapsulated packet bigger than +// the link is refused for size with the route's MTU, and the Packet Too Big the +// gateway answers with is routed to the client. +// +// The thread stays locked for the test, since a namespace belongs to a thread, +// and is moved back afterwards. A thread that cannot be is left locked, which +// makes the runtime discard it rather than reuse it. +func narrowFabric(t *testing.T, mtu int, dsts ...netip.Addr) { + t.Helper() + runtime.LockOSThread() + orig, err := unix.Open("/proc/thread-self/ns/net", unix.O_RDONLY|unix.O_CLOEXEC, 0) + if err != nil { + t.Fatalf("open thread network namespace: %v", err) + } + t.Cleanup(func() { + if err := unix.Setns(orig, unix.CLONE_NEWNET); err != nil { + t.Errorf("restore original network namespace: %v", err) + return + } + _ = unix.Close(orig) + runtime.UnlockOSThread() + }) + if err := unix.Unshare(unix.CLONE_NEWNET); err != nil { + t.Fatalf("unshare network namespace: %v", err) + } + + lo, err := netlink.LinkByName("lo") + if err != nil { + t.Fatalf("look up lo: %v", err) + } + if err := netlink.LinkSetUp(lo); err != nil { + t.Fatalf("set lo up: %v", err) + } + // A test run's ingress device is this namespace's loopback, and + // bpf_fib_lookup refuses to route for one that does not forward. + if err := os.WriteFile("/proc/sys/net/ipv6/conf/all/forwarding", []byte("1"), 0o644); err != nil { + t.Fatalf("enable IPv6 forwarding: %v", err) + } + + if err := netlink.LinkAdd(&netlink.Dummy{LinkAttrs: netlink.LinkAttrs{Name: "fab0", MTU: mtu}}); err != nil { + t.Fatalf("add fabric link: %v", err) + } + link, err := netlink.LinkByName("fab0") + if err != nil { + t.Fatalf("look up fabric link: %v", err) + } + if err := netlink.LinkSetUp(link); err != nil { + t.Fatalf("set fabric link up: %v", err) + } + + for _, dst := range dsts { + ip := net.IP(dst.AsSlice()) + route := &netlink.Route{ + LinkIndex: link.Attrs().Index, + Dst: &net.IPNet{IP: ip, Mask: net.CIDRMask(128, 128)}, + Scope: netlink.SCOPE_LINK, + } + if err := netlink.RouteAdd(route); err != nil { + t.Fatalf("route %v over the fabric link: %v", dst, err) + } + neigh := &netlink.Neigh{ + LinkIndex: link.Attrs().Index, + Family: netlink.FAMILY_V6, + State: netlink.NUD_PERMANENT, + IP: ip, + HardwareAddr: fabricLinkMAC, + } + if err := netlink.NeighAdd(neigh); err != nil { + t.Fatalf("neighbor %v on the fabric link: %v", dst, err) + } + } +} + +// ptbFixture is one VIP with one backend, the backend's uSID and the client +// both reachable over narrowFabric's link. +type ptbFixture struct { + objs *EdgedsrObjects + key EdgedsrVipKey + vip netip.Addr + client netip.Addr +} + +func newPTBFixture(t *testing.T) *ptbFixture { + t.Helper() + return newPTBFixtureMTU(t, fabricMTU) +} + +func newPTBFixtureMTU(t *testing.T, mtu int) *ptbFixture { + t.Helper() + requireRoot(t) + + f := &ptbFixture{ + vip: netip.MustParseAddr("2001:db8::100"), + client: netip.MustParseAddr("2001:db8:ffff::1"), + } + usid := netip.MustParseAddr("fc00:1:2::a1b2") + narrowFabric(t, mtu, usid, f.client) + + f.objs = loadObjects(t) + f.key = vipKey(ipprotoTCP, 8080, f.vip) + var value EdgedsrVipValue + value.BackendCount = 1 + value.Backends[0] = EdgedsrBackend{ + Addr: netip.MustParseAddr("fd20:60::5").As16(), + Port: bswap16(30080), + Usid: usid.As16(), + } + if err := f.objs.VipTable.Put(f.key, value); err != nil { + t.Fatalf("populate vip_table: %v", err) + } + if err := f.objs.EncapConfigTable.Put(uint32(0), EdgedsrEncapConfig{ + EncapSrc: netip.MustParseAddr("fc00:0:2::1").As16(), + }); err != nil { + t.Fatalf("populate encap_config_table: %v", err) + } + return f +} + +func (f *ptbFixture) run(t *testing.T, payload []byte) (uint32, []byte) { + t.Helper() + pkt := buildL4Packet(t, ipprotoTCP, f.vip, f.client, 41000, 8080, payload) + ret, out, err := f.objs.EdgeLb.Test(pkt) + if err != nil { + t.Fatalf("program test-run: %v", err) + } + return ret, out +} + +func (f *ptbFixture) vipStats(t *testing.T) EdgedsrVipStatsValue { + t.Helper() + var stats EdgedsrVipStatsValue + if err := f.objs.VipStatsTable.Lookup(f.key, &stats); err != nil { + t.Fatalf("lookup vip_stats_table: %v", err) + } + return stats +} + +// TestEdgeLB_PacketTooBigForTheFabric covers a client packet that fits the +// client's path but not the fabric once the gateway adds its 40-byte outer +// header. The client used to hear nothing, so its path MTU discovery never +// adapted and every large packet to the VIP vanished. It now receives a Packet +// Too Big from the VIP, with an MTU that fits once encapsulated, quoting the +// packet it sent. The packet itself is still counted as a fib_frag_needed drop. +// +// A small packet on the same flow still reaches the backend: the check is the +// route's own, not a size the gateway picks. +func TestEdgeLB_PacketTooBigForTheFabric(t *testing.T) { + f := newPTBFixture(t) + + if ret, _ := f.run(t, []byte("small")); ret != xdpRedirect { + t.Fatalf("small packet verdict = %d, want XDP_REDIRECT (%d) over the fabric link", ret, xdpRedirect) + } + + big := buildL4Packet(t, ipprotoTCP, f.vip, f.client, 41000, 8080, make([]byte, bigPayload)) + ret, out := f.run(t, make([]byte, bigPayload)) + if ret != xdpRedirect { + t.Fatalf("verdict = %d, want XDP_REDIRECT (%d): the Packet Too Big routed to the client", ret, xdpRedirect) + } + if got := sumPerCPU(t, f.objs.DropReasons, DropReasonFibFragNeeded); got != 1 { + t.Errorf("drop_reasons[fib_frag_needed] = %d, want 1 -- the packet is still dropped", got) + } + if stats := f.vipStats(t); stats.Packets != 2 || stats.DroppedPackets != 1 { + t.Errorf("vip_stats packets/dropped = %d/%d, want 2/1", stats.Packets, stats.DroppedPackets) + } + + if got := net.HardwareAddr(out[0:6]); got.String() != fabricLinkMAC.String() { + t.Errorf("destination MAC = %v, want the route's next hop %v", got, fabricLinkMAC) + } + if got := binary.BigEndian.Uint16(out[12:14]); got != 0x86DD { + t.Errorf("EtherType = %#04x, want 0x86DD", got) + } + ip6 := out[ethLen:] + if ip6[0]>>4 != 6 { + t.Errorf("IP version = %d, want 6", ip6[0]>>4) + } + if ip6[6] != ipprotoICMPv6 || ip6[7] != 64 { + t.Errorf("next header/hop limit = %d/%d, want %d/64", ip6[6], ip6[7], ipprotoICMPv6) + } + if got := addrAt(ip6, 8); got != f.vip { + t.Errorf("source = %v, want the VIP %v", got, f.vip) + } + if got := addrAt(ip6, 24); got != f.client { + t.Errorf("destination = %v, want the client %v", got, f.client) + } + msg := ip6[ip6Len:] + if want := icmpLen + ip6Len + 8; len(msg) != want || int(binary.BigEndian.Uint16(ip6[4:6])) != want { + t.Fatalf("message length = %d, payload length = %d, want both %d", len(msg), + binary.BigEndian.Uint16(ip6[4:6]), want) + } + if msg[0] != icmpv6PacketTooBig || msg[1] != 0 { + t.Errorf("type/code = %d/%d, want %d/0", msg[0], msg[1], icmpv6PacketTooBig) + } + if got := binary.BigEndian.Uint32(msg[4:8]); got != fabricMTU-ip6Len { + t.Errorf("MTU = %d, want %d (the fabric's, less the gateway's encapsulation)", got, fabricMTU-ip6Len) + } + if string(msg[icmpLen:]) != string(big[ethLen:ethLen+ip6Len+8]) { + t.Errorf("quote is not the packet the client sent:\n got % x\nwant % x", + msg[icmpLen:], big[ethLen:ethLen+ip6Len+8]) + } + assertICMP6Checksum(t, f.vip, f.client, msg) +} + +// TestEdgeLB_PacketTooBigRateLimited covers the bucket refusing the error: the +// packet is dropped as before, and the refusal is counted. +func TestEdgeLB_PacketTooBigRateLimited(t *testing.T) { + f := newPTBFixture(t) + + cpus, err := ebpf.PossibleCPU() + if err != nil { + t.Fatalf("possible CPUs: %v", err) + } + empty := make([]icmpBucket, cpus) + for i := range empty { + empty[i] = icmpBucket{LastNs: math.MaxUint64} + } + if err := f.objs.IcmpRateBucket.Put(uint32(0), empty); err != nil { + t.Fatalf("empty icmp_rate_bucket: %v", err) + } + + if ret, _ := f.run(t, make([]byte, bigPayload)); ret != xdpDrop { + t.Errorf("verdict = %d, want XDP_DROP (%d)", ret, xdpDrop) + } + if got := sumPerCPU(t, f.objs.DropReasons, DropReasonFibFragNeeded); got != 1 { + t.Errorf("drop_reasons[fib_frag_needed] = %d, want 1", got) + } + if got := sumPerCPU(t, f.objs.DropReasons, DropReasonICMPRateLimited); got != 1 { + t.Errorf("drop_reasons[icmp_rate_limited] = %d, want 1", got) + } +} + +// TestEdgeLB_NoPacketTooBigBelowTheIPv6Minimum covers a fabric route too small +// to carry even a 1280-byte packet once encapsulated. A Packet Too Big would +// report less than 1280, which an IPv6 sender never goes below, so its next +// packet would fail the same way. The packet is dropped and counted, and no +// message is sent or charged to the rate limit. +func TestEdgeLB_NoPacketTooBigBelowTheIPv6Minimum(t *testing.T) { + const mtu = 1300 + f := newPTBFixtureMTU(t, mtu) + + if ret, _ := f.run(t, make([]byte, mtu-ip6Len-tcpLen)); ret != xdpDrop { + t.Errorf("verdict = %d, want XDP_DROP (%d)", ret, xdpDrop) + } + if got := sumPerCPU(t, f.objs.DropReasons, DropReasonFibFragNeeded); got != 1 { + t.Errorf("drop_reasons[fib_frag_needed] = %d, want 1", got) + } + if got := sumPerCPU(t, f.objs.DropReasons, DropReasonICMPRateLimited); got != 0 { + t.Errorf("drop_reasons[icmp_rate_limited] = %d, want 0 (no message attempted)", got) + } +} + +func addrAt(b []byte, off int) netip.Addr { + return netip.AddrFrom16([16]byte(b[off : off+16])) +} + +// assertICMP6Checksum recomputes msg's checksum over the IPv6 pseudo-header +// independently of the datapath and compares. +func assertICMP6Checksum(t *testing.T, src, dst netip.Addr, msg []byte) { + t.Helper() + var sum uint32 + add := func(b []byte) { + for i := 0; i+1 < len(b); i += 2 { + sum += uint32(b[i])<<8 | uint32(b[i+1]) + } + if len(b)%2 == 1 { + sum += uint32(b[len(b)-1]) << 8 + } + } + s, d := src.As16(), dst.As16() + add(s[:]) + add(d[:]) + var lenProto [8]byte + binary.BigEndian.PutUint32(lenProto[0:4], uint32(len(msg))) + lenProto[7] = ipprotoICMPv6 + add(lenProto[:]) + zeroed := append([]byte{}, msg...) + zeroed[2], zeroed[3] = 0, 0 + add(zeroed) + for sum>>16 != 0 { + sum = sum&0xffff + sum>>16 + } + want := ^uint16(sum) + if got := binary.BigEndian.Uint16(msg[2:4]); got != want { + t.Errorf("ICMPv6 checksum = %#04x, want %#04x (independent full recompute over %v -> %v)", + got, want, src, dst) + } +}