Implement HTTP/2 CONNECT-IP fallback over TLS #981

Activity

1. privateip

   added

   enhancementNew feature or requestNew feature or request

   on Oct 22, 2026
2. privateip

   added a parent issue on Oct 22, 2026

   • MASQUE Connector for gVPC #970
3. privateip

   changed the title [-]MASQUE: implement HTTP/2 CONNECT-IP fallback over TLS[/-] [+]Implement HTTP/2 CONNECT-IP fallback over TLS[/+] on Oct 22, 2026
4. privateip

   added the

   Task

   issue type on Oct 22, 2026
5. privateip

   self-assigned this

   on Oct 28, 2026
6. privateip commented on Oct 28, 2026

   

   privateip

   on Oct 28, 2026

   CollaboratorAuthor

   More actions

   Verdict: valid. No production HTTP/2 CONNECT-IP listener exists.

   Checked against main at d902d55:

   • connectip.Server only serves HTTP/3. Its only listener entry point is Serve(net.PacketConn), and doc.go calls HTTP/2 fallback "a separate implementation".
   • Reproduction: a unit test passed Server.ServeHTTP an Extended CONNECT shaped the way net/http's HTTP/2 server delivers it (CONNECT, HTTP/2.0, :protocol: connect-ip, a valid bearer credential and VPC). The server answered 501 and opened no backend session. connect-ip-go parses only HTTP/3 requests, and its proxy needs http3.HTTPStreamer.
   • The only HTTP/2 server is the test-only prototype in tests/connectipinterop (Prove CONNECT-IP interoperability over HTTP/3 and HTTP/2 #973). It has its own admission path, takes no CONNECT-IP server lacks overload limits and tenant isolation checks #991 limits, and has no packetPolicy (Enforce delegated-prefix ownership and bidirectional traffic policy #979). docs/edge/connectip-interop.md lists the six items still needed before a listener ships.

   Dependency: the HTTP/3 service from #980 (cmd/masque-provider, service.Run, config/masque) is still an open stack. The HTTP/2 listener has to run inside that service, so the fix stacks on it.
7. privateip commented on Oct 28, 2026

   

   privateip

   on Oct 28, 2026

   CollaboratorAuthor

   More actions

   Approved plan. Two PRs stacked on #980's stack (#1202 → #1203 → #1204), because the listener runs inside the service.Run that #1203 adds. Both PRs say "Related to #981". This issue stays open for the lab acceptance rows: real gVPC workloads with UDP/443 blocked, and replacement across nodes.

   Root cause. connectip.Server serves only HTTP/3. Its request path runs through connect-ip-go, which parses only HTTP/3 requests. The one HTTP/2 server is the #973 prototype in tests/connectipinterop.

   PR 1, internal/edge/connectip:

   • Shared request path. ServeHTTP becomes a transport parse followed by one shared path: admission, lease watch, backend, tunnel and proxySession. Both transports share the same Admission, the session, admission and per-credential caps, and packetPolicy. HTTP/2 cannot get a second admission path or forwarding owner.
   • Capsule codec. Parses incrementally, so a capsule may be split across reads or share a read with others. Oversized DATAGRAM capsules and unknown capsules are discarded without being buffered. Control capsules are capped in size. A malformed capsule ends only its own request.
   • Toward the client. The HTTP/2 tunnel checks and decrements the hop limit. A packet over 1500 bytes gets an ICMPv6 Packet Too Big with MTU 1280, which goes back through packetPolicy. Each capsule write has a 5 s deadline, so a client that stops reading ends only its own session.
   • Server.ServeTLS(net.Listener). HTTP/2 only, ALPN h2.
     • A listener wrapper applies MaxConnections and MaxConnectionsPerSource.
     • Limits sets the stream, header and handshake limits.
     • Authentication runs before any per-request work.
     • It refuses to start unless GODEBUG contains http2xconnect=1.

   PR 2, service:

   • --listen-tcp (default :8443). Startup fails if Extended CONNECT is off, and /readyz needs both listeners.
   • GODEBUG is set in the image and the Deployment. The Service gets TCP/443.
   • Docs are updated.
   • The interop harness's HTTP/2 path moves to the production server, and the prototype is deleted.

   Tests:

   • HTTP/2 round trip with 1280-byte packets.
   • HTTP/2 repeats the HTTP/3 authorization, spoofing, isolation and expiry cases.
   • Capsule input that is truncated, malformed, oversized, split or coalesced.
   • A client that stops reading, and RST_STREAM on one stream while another stream keeps forwarding.
   • The TCP limits.
   • QUIC to HTTP/2 replacement under a single-owner admission: the old session ends, and two backend sessions are never open at once.

   Risk. The stack merges after #980's stack. Extended CONNECT is process-wide, which only matters to masque-provider. The Service exposes mixed UDP and TCP ports.
8. privateip commented on Oct 28, 2026

   

   privateip

   on Oct 28, 2026

   CollaboratorAuthor

   More actions

   The fix is open as two drafts on top of #980's stack, in stack #1205:

   1. feat(connectip): Serve CONNECT-IP over HTTP/2 with TLS #1206 adds the HTTP/2 listener to connectip.Server, sharing one request path with HTTP/3.
   2. feat(masque): Serve the HTTP/2 CONNECT-IP fallback from masque-provider #1207 runs it in masque-provider on TCP/443 with GODEBUG=http2xconnect=1, and moves the interop harness onto the production server.

   One change from the plan: the HTTP/2 tunnel clears its write deadline after each write. net/http resets an HTTP/2 stream whose write deadline passes even when no write is pending, so without this every session idle for 5 s would end. A regression test covers it.

   task test:connectip-interop passed all 33 tests against the pinned Xray client over both transports, including HTTP/2 with UDP/443 blocked. The acceptance rows that need real gVPC workloads and replacement across nodes are not run. This issue stays open for them.
9. privateip

   added 2 commits that reference this issue on Oct 28, 2026

   feat(masque): Serve the HTTP/2 CONNECT-IP fallback from masque-provider

   ...

   c86a40d

   feat(connectip): Serve CONNECT-IP over HTTP/2 with TLS

   ...

   d4e4a3f
10. privateip commented on Oct 29, 2026

    

    privateip

    on Oct 29, 2026

    CollaboratorAuthor

    More actions

    Every acceptance criterion now has evidence, so I'm closing this. The HTTP/2 listener merged in #1206, and #1207 runs it in the service on TCP/443. #1228 adds an HTTP/2 transport to the lab probe, and #1240 adds the HTTP/2 rows to the containerlab lab suite. All of those rows passed.

    Criterion	Evidence
    With UDP/443 blocked, 1280-byte packets both ways with real gVPC workloads over TCP/443	h2-lifecycle, with UDP to the service port dropped by an nft rule: two VPCs, connect and reconnect, 500/500 1280-byte replies from each workload, and 40/40 1280-byte pings from the workloads to the connector. The independent Xray client also passed with UDP/443 blocked in the interop suite (#1207)
    HTTP/2 over TLS/TCP with no QUIC dependency	h2-lifecycle negotiated h2. No UDP datagrams to the service port reached the provider pod, and the block dropped none during the run. An HTTP/3 control attempt, by contrast, had 2 dropped
    Same authorization, spoofing, isolation and expiry cases on both transports	h2-authz-isolation: an unknown credential got 401 and a credential for another VPC got 403. The server counted 500/500 spoofed packets as refused, and none was delivered. The expiring credential stopped 2.02 s before expiry, and the other connector answered 2701/2701. The HTTP/3 rows from #1238 cover the same cases
    Malformed, truncated and oversized capsules are bounded, and slow readers and cancellation stay isolated	Unit tests cover capsule framing, split reads, malformed input that ends only its own request, a cancelled request that leaves its sibling running, and a slow reader that ends only its own request. A stalled HTTP/2 client ends after 5.5 s, within the 6.25 s bound (#1230)
    QUIC-to-HTTP/2 reconnect follows the continuity contract with no dual ownership	h3-to-h2-same-pod: a 19 ms gap and no stale packets in the new request. The claim epoch did not change
    Same foundation-backed path as HTTP/3, with no second authority store and readiness not bypassed	Both transports share one request path, the session limit, the adapter and the session generation (#1206)
    Replacement within one instance and across nodes while the old QUIC path still works	The same-pod row as above. h3-to-h2-cross-node: the old HTTP/3 request was still answering when the HTTP/2 request reached the other node. The server closed it, and the new request got 503 until the authority released the old authorization. The two never forwarded together: the gap was 32.6 s, then 500/500
    #991 limits and authentication before probes on TCP	Unit tests cover connection, per-source, handshake and session limits on HTTP/2, and that no probe runs before admission
11. privateip

    closed this as completedon Oct 29, 2026