Skip to content

Build local service-route intent for VPC attachments - #634

Merged
scotwells merged 16 commits into
mainfrom
feat/service-route-local-attachments
Oct 7, 2026
Merged

scotwells merged 16 commits into
mainfrom
feat/service-route-local-attachments

Conversation

@scotwells

@scotwells scotwells commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Implement private service connectivity entirely in Galactic's TC-eBPF dataplane. PreferNodeLocal uses a ready producer on the consumer node when available and otherwise tunnels to a deterministic ready remote producer. Consumer instances use their ordinary default route and receive no service-specific Linux routes.

Dataplane

  • Preserve the original consumer source IP across remote request and reply paths with a versioned, direction-tagged service shim over the SRv6 underlay.
  • Key routes and authorization by attachment ifindex, service IP, protocol, and port; multiple services may share a VIP.
  • Validate remote requests against a 128-bit grant and the expected consumer SID, then revalidate replies on both producer and consumer nodes so revocation is immediate.
  • Reject ambiguous overlapping five-tuples fail-closed and replace stale reverse state immediately after its prior grant is revoked.
  • Account for the full 60-byte tunnel overhead in MSS clamping and PMTU errors.
  • Use the live shared uplink map, reject uninitialized source SIDs, and remove reverse state when delivery fails.
  • Skip NPT/VIP translation for service tunnels so the inner source and destination remain unchanged.

Control plane and rollout

  • Select ready and programmed producer attachments deterministically, preferring node-local replicas and falling back remotely only for PreferNodeLocal.
  • Resolve consumer and producer End.DT46 SIDs from BGPVRFInstance and BGPRouter state.
  • Reconcile routing dependencies without publishing node-local readiness failures as shared policy invalidity.
  • Recreate only incompatible pinned maps, preserve unrelated live maps, and periodically detect map replacement to reconstruct desired service state.

Network PR #27 provides the API contract.

Validation

go test ./internal/plumbing/ebpf/prog ./internal/plumbing/ebpf/serviceroutemap ./internal/plumbing/ebpf/attach ./internal/serviceroute ./internal/controller ./cmd/galactic-router -count=1
go vet ./internal/plumbing/ebpf/prog ./internal/plumbing/ebpf/serviceroutemap ./internal/plumbing/ebpf/attach ./internal/serviceroute ./internal/controller ./cmd/galactic-router

The tests run on Linux with root capabilities and exercise verifier loading, source-preserving remote encapsulation, tuple-scoped map writes, state rollback/revocation, deterministic fallback, SID resolution, and selective pinned-map replacement.

@scotwells scotwells changed the title Index VPC attachments by observed node Build local service-route intent for VPC attachments Sep 30, 2026
@scotwells
scotwells force-pushed the feat/service-route-local-attachments branch from 9b8d7ff to 5d4c848 Compare October 6, 2026 17:09
@privateip

Copy link
Copy Markdown
Collaborator

Two changes on main now overlap this PR's changes to attach.Start and attach.Load, so a rebase will conflict there.

When you rebase, please swap the single-program replace in ReattachEgress for your two-program attach instead of adding a second re-attach pass. If you want a failed re-attach to stop startup, raise it on #776's follow-up so the two policies don't diverge.

Context: a datapath map change on 2026-10-07 blacked out every staging VM (#774).

@privateip

Copy link
Copy Markdown
Collaborator

One more overlap: #777 moves the attach-time map registration out of internal/cnibgp/bgp.go into a new internal/plumbing/ebpf/attachreg package, so this PR's change to attachUsidEgress will conflict on rebase. The function moves unchanged to attachreg/register.go, and the same edit applies there.

The move also means CNI ADD and the new startup repair pass share that code, so loading the service program in attachUsidEgress covers both. We plan to land #777 without waiting on this PR, so please build on it once it merges.

@scotwells
scotwells force-pushed the feat/service-route-local-attachments branch from d49a7cd to e8c8158 Compare October 7, 2026 13:40
@scotwells
scotwells marked this pull request as ready for review October 7, 2026 19:58
@scotwells
scotwells requested a review from a team as a code owner October 7, 2026 19:58
@scotwells
scotwells merged commit 6d428ce into main Oct 7, 2026
14 checks passed
@scotwells
scotwells deleted the feat/service-route-local-attachments branch October 7, 2026 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants