Repository navigation
Implement /96 connector addressing and aggregate return routing #978
Description
Parent: #970
Deliverable
Connector /96 allocation and routing integrated with the gVPC endpoint model, including return traffic to addresses and subdivisions behind a connector.
Dependencies
#972, #975, #976, foundation #998; integrated BGP/packet acceptance uses #1000/#1002 evidence.
Galactic Edge integration (#992)
Implement this milestone as a consumer of the Galactic Edge provider contract. #992 owns provider authorization, host attachment lifecycle, shared-state fencing and maintenance, independently enforced forwarding expiry, prefix conflict arbitration/publication, and gVPC traffic policy. MASQUE owns client authentication, connector /96 assignment and retention, session packet checks, transport behavior and its provider adapter. Use the foundation's versioned contract and existing provider helpers; do not introduce a second host-networking controller, lease store or BGP publisher. Any required foundation change must be tracked and qualified under #992 before this milestone claims integrated acceptance.
Work
- Implement the selected allocation and lifetime design so each connector receives one /96 consistent with the gVPC addressing plan.
- Expose the authorized prefix to the protocol layer using the agreed CONNECT-IP assignment contract.
- Publish an aggregate /96 route toward the active attachment only after readiness and ownership checks; demultiplex return traffic to the authorized tunnel.
- Support client-managed addresses and longer subprefixes inside the /96 without allocating or publishing a separate gVPC route for each one.
- Implement the agreed retention behavior across reconnects and owner replacement; keep allocation release separate from withdrawal of forwarding reachability.
Acceptance criteria
- Tests verify non-conflicting allocations within a gVPC and bidirectional traffic to multiple addresses in multiple subdivisions of one /96.
- The gVPC route table contains the aggregate route to the correct active attachment, and local and remote workloads reach those addresses.
- Reconnect and replacement behavior matches the documented retention decision; stale cleanup cannot remove the new owner's route.
- Allocation, route withdrawal and eventual release tests cover retries, failure and restart without leaking address space or forwarding authority.
Additional integration acceptance
- Keep durable connector-to-/96 allocation/retention distinct from EdgeService client-pool attachment addresses and transient EdgeRoute reachability; core route exclusivity is not durable connector IPAM.
- Request the aggregate /96 only inside authorized service prefixes/client pool and outside attachment addresses, conflicting routes, other reserved client pools and VPC subnets. Include concurrent allocation and subnet-created-after-route cases.
- Use the provider EdgeRoute contract; do not write BGPAdvertisement, kernel routes or eBPF maps from MASQUE. Wait for accepted ownership/readiness and observe publication status separately from remote convergence.
- Prove local and remote reachability to subdivisions, including a local node with NAT egress. Do not depend on advertising the attachment's own address.
- Persist allocation across the selected reconnect lifetime without retaining active reachability; late release cannot free a replacement connector's allocation or route.
Scope follows #970: CONNECT-IP with HTTP/2 fallback; no public API development, plugin/portal integration, product-client development or production rollout.
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Activity
[-]MASQUE: implement /96 connector addressing and aggregate return routing[/-][+]Implement /96 connector addressing and aggregate return routing[/+]privateip commentedon Oct 28, 2026
Verdict: valid, partly done. I checked this against
mainat 6b333af. Most of the Work list shipped with #975, #976 and #977. The acceptance evidence for rows M11 and M12 of the acceptance matrix is still missing.What's already on
main:ledger.go).server.go#L270-L271).route.go).tun.go#L116).TestTUNPrefixDemultiplexingcovers this.conflict.go,TestSubnetConflict,TestPoolAndGrantConflict,TestOwnAddressRefused).What's missing:
TestMASQUEalways sends from a single source address, so no test sends from several addresses in more than one subdivision of a /96. No test reaches those addresses from a local workload, a remote workload and the off-fabric host through NAT, and no test checks that exactly one aggregate /96 advertisement exists with nothing more specific inside it.privateip commentedon Oct 28, 2026
Plan (branch
fix/issue-978)Leak found while checking "without leaking address space".
Grants.Createallocates the connector's /96 in the ledger, then writes the grant record (grants.go#L92-L101). A unit test whose record write fails once reproduces it onmain:Retireof that connector fails withnot found, because it reads the record first.grant createwithout--connectorallocates a second /96.A ledger holds at most 256 entries, so each failed write costs one entry for good.
Fix
Create: when the record write fails, the error names the connector and says to retry with--connector. A retry with the same connector reuses its ledger entry, becauseAllocateis idempotent. If the record already exists after a lost reply, the error says to usegrant rotate, which issues a new credential for the same /96.Retire: when the connector has no record, find its entry in the authority's ledgers and tombstone it.Tests
TestMASQUErow,M12-aggregate-routing. The probe client sends from four addresses in two subdivisions of one /96 to a local workload, a remote workload and the off-fabric host through NAT. The row then checks that exactly one advertisement carries the /96 and none carries a longer prefix inside it.edgeprobe connectiptakes several-srcaddresses for this.Risk. Behavior changes only on the failure paths of
grant createandgrant retire. Nothing changes in the API, the ledger format or the data plane.