Repository navigation
fix(cni): Cut galactic-cni eBPF map memory at startup - #841
Conversation
The private service route maps added about 52 MiB of eBPF maps, and the kernel charges a map's memory to the memory cgroup of the process that creates it. galactic-cni's credential-refresh container creates them, so a node upgrading from v0.5.4, which has none of them pinned, charges the new pod about 59 MiB of maps at start. Five of those maps are plain hashes preallocated at max_entries, about 28 MiB, although production holds no service policy at all. On the busiest production compute node that puts the upgraded container at about 87% of its 128Mi limit. service_route_table, service_access_table, service_deny_table, service_remote_grant_table and attachment_identity_table are now BPF_F_NO_PREALLOC. Only userspace writes them and the datapath only looks rows up, so the per-packet path never allocates. An empty map costs only its bucket array, about 3.5 MiB for the five, and the maps created at start drop from about 59.6 MiB to about 35.3 MiB (the kernel's sizing formula applied to the compiled spec, which reproduces the 59 MiB #808 measured). Rows are still charged to galactic-cni's cgroup as they are written (galactic-router writes the four policy maps, galactic-cni the identity table), so a full set costs what it did before; a row that does not fit now fails that write, which its writer retries, instead of the container's start. service_reverse_table keeps its preallocation: the datapath writes it, and the kernel requires preallocation for an LRU hash. Resizing it is a capacity decision left for later. The flag change makes the five pinned maps incompatible on nodes that already have them, which today means staging only. galactic-cni unpins and recreates them at start, and galactic-router sees the new map IDs and rebuilds its rows, the path the service route maps were built for. Related to #808 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The root unit test measured the uSID maps galactic-cni creates at 35 MiB, against a 128 MiB container limit. The comment on the policy maps ran long and repeated the reasoning, so it now says only why they are not preallocated and why the reverse table is. The README states the measured size. Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
privateip
changed the title
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Second-pass review of this PR is settled. pr-conventions-reviewer at 6a05012 returned VERDICT merge, with nits only. pr-rereviewer at 6a05012 confirmed all four first-pass findings fixed or settled, with only nits remaining. The wrap-width nits were applied in d338b57, a formatting-only commit on top of the reviewed head. The squash nit was declined because it would need a force-push and the PR is merged as a single unit anyway. CI is green on d338b57: Build, Lint, Unit Tests, Unit Tests (root), E2E Tests, and every image and kustomize bundle publish job passed. Two warnings were carried knowingly. The two staging test-plan checks stay unchecked and run after merge, because staging tracks main. PR #835 also adds maps to the same datapath source, so it should land after this one and re-run the map memory test (TestUsid_MapMemoryFitsGalacticCNILimit) in its own CI. Auto-merge is enabled with the merge method, the only one the main branch ruleset allows. The ruleset requires one approving review from a code owner and dismisses stale approvals on push, so this lands in front of a person. |
2 participants
Summary
Upgrading a node from the last release makes the new CNI pod pay for about 59 MiB of new eBPF maps at start, close to the memory limit on the busiest production compute node.
Five of those maps are policy tables that only userspace writes, so they now allocate rows on demand instead of reserving their full size up front.
CI measured the maps at 35 MiB against the 128 MiB limit, and a full set of policy rows costs what it did before.
Nodes that already have these maps, which is only staging today, recreate them once at start while the router rebuilds their rows.
Test plan
Related to #808
🤖 Generated with Claude Code