Repository navigation
fix: Stop the gateway and NAT shard running without forwarding - #751
Conversation
The gateway and egress shard need IPv6 forwarding on every interface their XDP datapath runs on, or bpf_fib_lookup refuses every lookup and the datapath drops every packet. Both set the sysctls at startup, but an unprivileged pod gets /proc/sys read-only, so the writes failed. ConfigureFIBLookupUplinkSysctls logged each failure and returned nil, so the error branches in both binaries never ran and the pod reported healthy. The FIB-lookup sysctl helpers now read each value back and return an error unless it reads 1. A node where something else already enabled forwarding still passes with a read-only /proc/sys. The gateway and the shard now fail startup instead, and an interface found later stays without the datapath until its sysctls hold. Both base DaemonSets mount the host's /proc/sys/net read-write at /host/proc/sys/net, and the new GALACTIC_GATEWAY_PROC_SYS_PATH and GALACTIC_NAT_PROC_SYS_PATH settings point the helpers there. hostNetwork puts the writes in the node's own network namespace, and the pods stay unprivileged. The containerlab lab never showed this because gvpc.clab.yaml enables forwarding on every node. Fixes #586 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
go-sysctl turns every dot in a key into a path separator, so net.ipv6.conf.bond0.100.forwarding resolved to conf/bond0/100/forwarding, which does not exist. Before, that failure was only logged. Now that the FIB-lookup helpers return it, a gateway or NAT shard on a VLAN uplink would fail startup or leave the interface uncovered. The helpers now take each sysctl as path segments and write and read the file directly, and name it in errors the way sysctl(8) does, with the interface's dot written as a slash. The tests cover both helpers, add a dotted interface name, and skip only the read-only cases as root. The docs note that SELinux policy can still refuse the write. Related to #586 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Both reviewers returned VERDICT: merge on the first pass, with no blocker. pr-conventions-reviewer: VERDICT: merge. pr-adversary: VERDICT: merge. The launcher's brief did not carry either report's RAN list, so this comment does not repeat them. No code changed after review. Findings handled: the body now names the new startup failure and the staging rollout, and it mentions that a missing sysctl path is now fatal. The nit about squashing the two commits was left, and both commits stay. The overlap with open PRs #597, #613, #580 and #738 (docs/nat/configuration.md, internal/config/nat.go, docs/agents/ARCHITECTURE-GATEWAY.md) is textual and only affects rebase order. The adversary's live check ran on the containerlab dfw cluster. With eth1 forwarding off, the NAT shard set it back to 1 through the host mount and started healthy. Pointed at the read-only /proc/sys, it crash-looped naming the sysctl, while a node with forwarding already on started. The lab was restored afterwards. CI is green on head eedac7e: all 16 checks passed. Unit Tests (root) failed once in TestPMTU_VethPodLearnsPathMTU, an eBPF PMTU test that also fails on main at fffb635, and passed on re-run. Auto-merge is enabled with the merge method, the only one the ruleset allows. The ruleset requires 1 approving review, code owner review and approval of the last push, and it dismisses stale reviews on push, so the PR waits for a human approval. |
2 participants
Summary
The edge gateway and the egress NAT shard need IPv6 forwarding on every interface their datapath runs on, or the kernel refuses every route lookup and the datapath drops all traffic.
Both set it at startup, but the pod's kernel settings are read-only, and the failure was logged and ignored, so the pod reported healthy while forwarding nothing.
Both now check the setting after writing it and fail startup if forwarding is still off, and both DaemonSets mount the node's network settings writable so the write can succeed without a privileged pod.
A node that still refuses the write, such as one with IPv6 or SELinux blocking it, now crash-loops the shard in staging on the next sync instead of silently dropping its traffic.
Test plan
Fixes #586
🤖 Generated with Claude Code