MASQUE Connector for gVPC #970

Goal

Provide IPv6 connectivity from tenant sites and clients to their VPC workloads using CONNECT-IP, served from Galactic edge nodes.

Support HTTP/3 over QUIC on UDP/443 and an HTTP/2 over TLS fallback on TCP/443. The fallback carries packets in capsules and establishes a new CONNECT-IP tunnel. Address retention and session continuity across reconnects are defined during design. Both transports use the same authentication, VPC selection, policy and ownership rules.

Implement MASQUE as a provider on the Galactic Edge foundation (#992), using its authorized packet attachments and prefix-reachability contract. Reuse its tenant isolation, lifecycle, policy, exclusive ownership and failure enforcement throughout startup, failure and shutdown.

Allocate a /96 to each connector, following the gVPC tenant addressing plan. Clients may subdivide that prefix and manage addresses behind the tunnel; gVPC routes the aggregate /96 to the connector's active attachment.

Done when

1. Independent clients can connect, send, receive, disconnect and reconnect over both transports, carrying 1280-byte IPv6 packets in both directions. HTTP/2 passes with UDP/443 blocked. Path-MTU changes and oversized packets produce correct ICMPv6 Packet Too Big or explicit tunnel failure behavior.
2. Both transports enforce VPC authorization and traffic policy in both directions, including rejection of unauthorized requests. Credential expiry or access revocation stops existing forwarding within a defined bound.
3. Two tenants with identical addresses on the same node remain isolated.
4. Each connector has one active forwarding owner. A failed instance or one cut off from the API stops forwarding and its routes are withdrawn within defined limits; ownership transfer never permits old and new owners to forward concurrently for the same connector.
5. Each connector receives a /96. Addresses in client-managed subdivisions are reachable through its aggregate gVPC route, and traffic using source addresses outside the delegated prefix is rejected.
6. Supported connector counts and throughput are measured for both transports, resource limits are enforced under overload, and operators can observe connection failures, policy drops and route status.

Foundation dependency and responsibility split

The provider contract from #993 is authoritative. MASQUE requests PacketAccess/v1 and PrefixReachability/v1 through an approved EdgeProvider, authority-managed EdgeService, pod-bound EdgeInstance and Lease, EdgeAttachment, and per-prefix EdgeRoute. The provider only sets fields the contract assigns to it. It receives packet access in its own network namespace, with no shared host networking, hostPath, NET_ADMIN or BPF access.

#992 owns provider admission (#994), attachment lifecycle (#995), maintenance integration (#996), instance leases and independent prefix gates (#997), exclusive route acceptance/publication (#998), gVPC policy (#999), independent-provider conformance (#1000), foundation diagnostics (#1001), and target-platform qualification (#1002). MASQUE supplies client authentication and authorization, durable connector-to-/96 assignments, session-level exclusivity and packet checks, transport adapters and integration evidence. A connector credential is not a foundation grant, an instance lease is not a connector session lease, and a successful claim write is not forwarding permission.

Use the contract's timing parameters and clock assumptions: 15 s instance lease, 5 s renewal, 15.5 s local cutoff on the holder's clock (15.52 s real-time test threshold), 15.53 s takeover wait unless prior closure is confirmed, and local route withdrawal within 1 s of gate expiry while the router runs. The ≤9 s reflector hold-time bound requires deployment configuration; 2 s connected revocation and 5 s remote convergence are measured targets. #972 must additionally specify client credential/revocation deadlines and prove how they remain effective for a single connector while other connectors keep the provider instance alive.

Protocol exploration may proceed before foundation qualification. Integrated completion requires executed #998/#1000/#1001/#1002 evidence for the pinned foundation artifacts and MASQUE-specific evidence in #985. Reuse that evidence without treating a merged implementation, an issue checkbox, or readiness as proof of acceptance. Foundation defects remain foundation work; an integration that needs a contract change must track and qualify it explicitly. The independent provider remains independent of MASQUE.

Out of scope

• Public API development and plugin/portal integration.
• Client development, including changes to datum-cloud/connect or datum-cloud/app for transport, automatic fallback, enrollment, local networking or UI. Service authentication contracts and independent-client testing remain in scope.

Phases

1. Design and prove

• Map connector/session identity to the existing Edge provider, service, instance, attachment and route claim model; define session exclusivity within one instance as well as cross-instance replacement
• Define address retention, session continuity and ownership transfer across reconnects, including HTTP/2 fallback
• Define client authentication, VPC authorization, credential expiry and access revocation, including bounds for terminating existing forwarding
• Adopt the foundation claim, forwarding-expiry and route-withdrawal bounds; specify additional client-authority deadlines and per-request resource limits
• Define connector-count and throughput targets, resource budgets and overload acceptance criteria for both transports
• Select libraries and independent test clients for both transports
• Prove tenant isolation on Talos/Cilium with overlapping addresses
• Prove CONNECT-IP interoperability, minimum packet size and HTTP/2 fallback against the acceptance criteria above

Done when decisions are documented and reviewed, and the prototypes pass. A failed prototype requires a design revision and a passing retest.

2. Galactic Edge provider integration

• Register and run MASQUE as an unprivileged PacketAccess/PrefixReachability provider; consume foundation attachment status and readiness
• Integrate instance renewal and accepted per-/96 route claims using the provider contract, preserving shared attachment users on cancellation
• Enforce connector-specific deadlines and test foundation-owned gates and withdrawal during API outages, provider/agent stalls, restart and replacement

Done when crash, API-partition and replacement tests meet the defined expiry bounds and prevent concurrent forwarding by old and new owners for the same connector.

3. CONNECT-IP service

• Implement client admission and VPC selection
• Enforce credential expiry and access revocation for active sessions within the defined bounds
• Serve both transports with shared policy and ownership enforcement
• Verify TLS certificate validation and HTTP/2 fallback with UDP/443 blocked
• Implement bounded capsule parsing and buffering, backpressure and independent request cancellation
• Each connector receives a /96 consistent with gVPC addressing and may subdivide it locally
• Publish the aggregate /96 route to the active attachment and deliver return traffic to client-managed addresses and subdivisions
• Enforce source-address ownership within each connector's delegated /96
• Enforce traffic policy in both directions
• Verify ICMPv6 Packet Too Big handling and path-MTU changes, including paths that cannot carry the required 1280-byte packets
• Expose connection failures, policy drops, active connector counts, resource usage and route status
• Measure connector capacity and throughput on both transports and verify resource limits under overload

Done when the connectivity, prefix-allocation, routing, authorization, isolation, MTU, capacity and observability acceptance criteria pass on a staging edge node.

Milestone coverage

Additional completion gates

• MASQUE works against pinned, qualified core artifacts through the published provider contract, without MASQUE-specific core dispatch or direct shared-host programming.
• Credential expiry/revocation and same-instance HTTP/3-to-HTTP/2 replacement stop only the affected connector; unrelated sessions continue, and old buffered traffic cannot enter a replacement session.
• All Edge pod attachments: unchecked caller inputs and weak isolation tests #989/CONNECT-IP server lacks overload limits and tenant isolation checks #991 findings and applicable Private service routes can fail silently after a roll or on IPv4 #986 findings have linked fixes or evidence-backed dispositions before the affected integration ships.
• Add reproducible end-to-end feature acceptance tests #985 records foundation and MASQUE evidence separately, including negative packet observations, actual privileges, compatibility rejection, independent forwarding cutoff and local/remote route timing.

Protocol references

• CONNECT-IP: RFC 9484 — Proxying IP in HTTP.
• Datagrams and capsules: RFC 9297 — HTTP Datagrams and the Capsule Protocol, and RFC 9221 — QUIC DATAGRAM extension.
• QUIC and encryption: RFC 9000 — QUIC transport, RFC 9001 — Using TLS to Secure QUIC, and RFC 8446 — TLS 1.3.
• HTTP/3: RFC 9114, with the Extended CONNECT mechanism defined in RFC 9220.
• HTTP/2 fallback: RFC 9113, with the Extended CONNECT mechanism defined in RFC 8441. CONNECT-IP uses this mechanism as specified in RFC 9484.