Repository navigation
CONNECT-IP server lacks overload limits and tenant isolation checks #991
Description
The CONNECT-IP server from #954 has no binary or deployment yet. Before anything exposes it to the internet, it needs four fixes that did not block the review.
- Any client can open unlimited sessions and admission calls, because no limit sits ahead of admission.
- Unauthenticated clients get a probe packet before admission runs.
- One full tunnel device tears down other tenants' sessions.
- Routes are checked against each other but not against other tenants' ranges.
Lease fencing on the datapath is tracked in #976, and overload limits are a #970 criterion. Done when each item is fixed or the wrapper that exposes the server handles it. The detail is in the first comment.
Galactic Edge integration (#992)
Implement this milestone as a consumer of the Galactic Edge provider contract. #992 owns provider authorization, host attachment lifecycle, shared-state fencing and maintenance, independently enforced forwarding expiry, prefix conflict arbitration/publication, and gVPC traffic policy. MASQUE owns client authentication, connector /96 assignment and retention, session packet checks, transport behavior and its provider adapter. Use the foundation's versioned contract and existing provider helpers; do not introduce a second host-networking controller, lease store or BGP publisher. Any required foundation change must be tracked and qualified under #992 before this milestone claims integrated acceptance.
Integration disposition and acceptance
This issue is an exposure gate for #980/#981, with #984 supplying subsequent load qualification. #992 provides foundation gates and route arbitration; it does not automatically fix CONNECT-IP admission or queue behavior.
- Implement the HTTP/3 CONNECT-IP edge service #980/Implement HTTP/2 CONNECT-IP fallback over TLS #981/Enforce resource limits and qualify connector capacity #984 bound unauthenticated connections, streams, in-flight admission, sessions and queued bytes before allocating work; authentication precedes MTU probes on both transports.
- Integrate MASQUE with Galactic Edge provider attachments #975/Enforce delegated-prefix ownership and bidirectional traffic policy #979 isolate backend state by tenant attachment and session; TUN backpressure cannot terminate unrelated sessions or serialize unbounded work behind a shared lock.
- Implement admission, credential expiry and access revocation #977/Implement /96 connector addressing and aggregate return routing #978 validate client route authority against trusted grants and use Implement authorized prefix routing #998 for foundation prefix conflicts. Enforce delegated-prefix ownership and bidirectional traffic policy #979 verifies library packet checks and explicitly handles unsupported IPv4.
- Integrate connector ownership with Edge claims and independent expiry #976 integrates independent foundation gates and session authority before the raw TUN backend is reachable. Session cancellation alone is insufficient fencing.
- Address all six findings in the first comment with linked fixes/tests or evidence-backed current-path dispositions; rerun them for HTTP/2 as well as HTTP/3 and retain negative packet evidence in Add reproducible end-to-end feature acceptance tests #985.
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Activity
ecv commentedon Oct 22, 2026
Detail, as of #954 at c6c48bd:
server.gorunsAdmitwith no per-connection, per-client or global cap, andConfigsets no QUIC stream or connection limits. Each admitted session holds two 64KB buffers and up to 256 queued packets inTUNBackend. Add a bounded in-flight and session semaphore ahead ofAdmit, set explicit QUIC limits, and cap queued bytes per session.probeInnerMTUsends a 1.3KB datagram to any request with a well-formed bearer token, beforeAdmit. Move it after admission.tun.go, anyunix.Writeerror, EAGAIN on a full nonblocking TUN included, ends the session, and all sessions sharebackend.mu. Count EAGAIN as a dropped packet instead.routeRangeschecks routes against each other only, not against other tenants' /96s or the assigned prefix. Route authority belongs to the admission adapter, which does not exist yet.TUNBackendnever checks lease state, so forwarding stops only whenlease.Done()closes the session. KeepNewTUNBackendunexported until a fencing wrapper exists (Integrate connector ownership with Edge claims and independent expiry #976).connect-ip-gov0.6.0. Confirm the library enforces both in both directions for IPv4 and IPv6. TUN return traffic demultiplexes on the destination /96 only, so IPv4 packets never match a session.privateip commentedon Oct 28, 2026
Verdict: valid for items 1 and 2. Checked against
mainat 15a1b5a.Item 2 reproduced. A test sent a CONNECT-IP request over HTTP/3 with an unknown bearer token. The server answered 401 and also sent the client a 1,281-byte probe datagram.
ServeHTTPrunsprobeInnerMTUbeforeAdmit.Item 1 confirmed from source.
Configsets no connection, stream, admission or session limit.ServeHTTPcallsAdmitfor every request that carries a bearer token and a VPC header. EachTUNBackendsession can queue 256 packets of up to 64 KB, with no byte cap.The other items are already handled on the path that serves the server today, which is the MASQUE adapter used by
tests/edgeconformance/masqueprovider:packetSession.WritePacketcountsEAGAINas a drop and keeps the session. The shared lock inTUNBackendcovers one nonblocking write.forwardingcheck fences every packet on the instance lease epoch and the grant deadline (Integrate connector ownership with Edge claims and independent expiry #976).HTTP/2 has no server yet. #981 must probe after admission and apply the same limits.
privateip commentedon Oct 28, 2026
Approved fix plan for items 1 and 2.
Root cause.
ServeHTTPininternal/edge/connectip/server.goprobes the inner MTU before it callsAdmit.Configsets no limit on connections, streams, admissions or sessions.TUNBackendqueues up to 256 packets per session with no byte cap.Fix.
ServeHTTPprobes only afterAdmitsucceeds and the lease checks pass, and beforebackend.Open. A failed probe answers 503 and releases the lease.Config.Limitsadds four caps. A zero field takes its default.MaxConnections, default 1024: QUIC connections over the cap are closed withH3_EXCESSIVE_LOAD.MaxStreamsPerConn, default 16: sets QUICMaxIncomingStreams.MaxAdmissions, default 64: requests insideAdmitat once.MaxSessions, default 1024: admitted requests at once.Both request caps are taken before
Admitwithout waiting. A request over either cap gets 503.Each
TUNBackendsession also caps its queue at 1 MiB and drops packets past it.Tests. A request with an unknown credential receives no datagram. Requests over the admission and session caps get 503 before
Admitruns. A connection over the connection cap is closed. The queue byte cap has a unit test that does not need root.Risk. A client that opens more than 16 requests on one connection now waits for stream credit. These defaults are starting points, and #984 qualifies capacity. Items 3 to 6 need no code here, as the verdict comment explains.
privateip commentedon Oct 28, 2026
Change to the approved plan: the default limits now match the initial per-pod targets in the MASQUE contract, not the numbers in the plan.
MaxConnectionsMaxStreamsPerConnMaxAdmissionsMaxSessionsThis change does not add the contract's per-source-IP cap, per-credential cap, handshake and admission deadlines, or the 32 MiB aggregate queue cap. #984 qualifies the numbers, and those limits stay with #980 and #984.
fix(connectip): Bound CONNECT-IP work before admission