Repository navigation
docs(egress): Document hashed egress with alerts and a lab check - #837
Merged
Merged
Conversation
3 of 4 tasks
privateip
force-pushed
the
feat/egress-aa-cni
branch
from
October 12, 2026 16:05
90d6a75 to
9c13287
Compare
…st and lab check docs/nat/configuration.md gains an Active/active egress section: how a shard is chosen, the per-class groups, pins, draining with spec.drain, what happens with no reachable shard, turning hashed mode on and off, the snapshot publication and generation rules, and the metrics. It lists the prerequisites (the EgressShard CRD with spec.drain and a read-back check, the RBAC, a distinct identity per shard), says that placement is per node because the hash includes each node's own VRF table ID, and states the limits of address-based balancing, pins and fragment handling. Deleting an EgressShard is not a drain: set spec.drain, wait for its sessions to end, then delete; a busy tenant can keep a shard draining indefinitely. docs/nat/active-active-rollout.md is the staging acceptance and rollback checklist, ending with: switch to ordered mode and wait for galactic_cni_egress_group_routes to read 0 before rolling the image back. docs/cni/environment-variables.md, ARCHITECTURE-CNI.md, ARCHITECTURE-GATEWAY.md and the repository guide describe both modes. config/monitoring/ adds a galactic-cni-egress rule group with promtool tests, per translation class: GalacticEgressPoolEmpty, GalacticEgressPoolBelowMinimum, GalacticEgressShardGroupDropping, GalacticEgressClassUnserved, GalacticEgressShardWatchNotSynced, GalacticEgressSweepFailing and GalacticCNIMetricsDown. Infra must install them in its central evaluator. verify:nat-active-active checks hashed mode in the lab's dfw site in ten stages. Every probe fails unless every request succeeds, the off-fabric host sees every probe, and every source is an expected masquerade address, and a self-test blackholes a fresh tenant and requires the probe to fail. Fresh tenant addresses added inside the tenant's network namespace test drain and failure under default pins. The stages cover TCP, UDP and ICMP over NAT66 and NAT64, a galactic-cni restart during a download, a local prohibit-route failure, a real BGP withdrawal of a shard's SID, drain, every shard unavailable, a revoked EgressShard watch, and rollback to ordered mode. Cleanup restores what it changed. docs/feature_reviews/ keeps the feature's architecture review and re-review as written, and a resolution record mapping each finding to its change and validation. Three re-review findings (R02, R03, R05) are still open and block enabling hashed mode; the rollout checklist says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
privateip
marked this pull request as ready for review
October 12, 2026 16:19
privateip
force-pushed
the
feat/egress-aa-docs
branch
from
October 12, 2026 16:19
4156785 to
814dcef
Compare
privateip
enabled auto-merge
October 12, 2026 16:19
scotwells
approved these changes
Oct 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Third of three stacked PRs for active/active egress, on top of #836. Documentation, alerts and a lab check only; no Go code.
docs/nat/configuration.mdexplains how a shard is chosen, the per-class groups, pins, prerequisites, limits, and how to turn hashed mode on and off. It states that deleting anEgressShardis not a drain: setspec.drain, wait for its sessions to end, then delete.docs/nat/active-active-rollout.mdcovers staging acceptance and rollback. Its first gate is that the open re-review findings are fixed.config/monitoring/adds agalactic-cni-egressrule group of seven alerts with promtool tests: pool empty, below minimum, group dropping, class unserved, watch not synced, sweep failing, and metrics down. Infra has to install them in its central evaluator.task verify:nat-active-activeruns ten stages in the containerlab dfw site: TCP, UDP and ICMP over NAT66 and NAT64, a CNI restart, a local failure, a real BGP withdrawal, drain, all shards down, a revoked watch, and rollback. Probes fail unless every request succeeds and every source is an expected public address, and a self-test confirms the probe can fail.docs/feature_reviews/holds the feature's review and re-review as written, plus a resolution record.Open findings
The resolution record maps every finding to its change and validation. Three re-review findings are still open and block enabling hashed mode:
R01 is partly addressed: a late CNI ADD write is repaired within one sweep, but there is no barrier. Ordered mode is unaffected by all of them.
Test plan
task test:alertspassesbash -non the lab script passverify:nat-active-activepassed all ten stages in the local containerlab lab. A real BGP withdrawal and re-origination each reached the group in about 2s. One earlier run failed once in the drain stage and was not reproduced.🤖 Generated with Claude Code