You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The residual gap from #451 (its fix covered only the bridge-polled path): both inline owners — arcbox-net-inject's readv InlineConn and splicetcp::direct_rx's tokio ConnSink — cap the honored window at 256 KiB but have no retransmission. The 2026-07 cross-repo audit re-confirmed it.
Why window gating alone is not enough
inline_conn.rs claims "the inject path is lossless — so staying inside [the window] means no gap can form". That holds only to guest eth0. For container flows the TCP endpoint is inside the container netns, and #451 established that the guest-internal bridge → veth → container-netns backlog drops frames under burst before TCP buffering — the advertised window bounds the container socket's buffer, not the veth hop. One dropped frame = permanent hole = wedged flow (and #451 measured window gating alone making it worse: every drop became a full-window deadlock).
Today the exposure is limited: inline is HV-only in practice and gated on peer_mss ≥ 1460. But it blocks#262 (promote all established flows to inline) and matters for #250 (HV as default backend).
Design (assessed 2026-07-21)
The zero-copy read (socket → guest descriptor buffer directly) is fundamentally at odds with retransmission, which needs the sent bytes host-side. Options considered:
Bound in-flight to a "lossless" budget — no such budget exists past eth0 (above).
Only promote root-netns flows to inline — the 4-tuple can't distinguish veth-backed destinations.
Tee-on-send (chosen): after each zero-copy read into the guest buffer, memcpy the payload into a per-flow retransmit ring (bounded by the same 256 KiB honored-window cap) shared with the bridge's FastPathConn (Arc<Mutex<…>> beside the existing shared atomics). poll_fast_path — which the shared arcbox_net loop already drives for inline flows (fix(net): add a zero-window persist probe to the download fast path #477 relies on this) — runs its existing dup-ACK/RTO machinery against the shared ring and re-emits via the normal frame path (emit_data_frames → guest_tx), draining the ring as guest_acked advances. The extra memcpy is noise next to the syscall+datapath cost at single-digit Gbps; the 1-syscall hot loop is untouched.
Applies identically to both owners (inject.rs, direct_rx.rs). Acceptance: the #450 network-workload suite (W1–W14) green on the HV backend, which currently cannot pass with inline flows under burst loss.
The residual gap from #451 (its fix covered only the bridge-polled path): both inline owners —
arcbox-net-inject's readvInlineConnandsplicetcp::direct_rx's tokio ConnSink — cap the honored window at 256 KiB but have no retransmission. The 2026-07 cross-repo audit re-confirmed it.Why window gating alone is not enough
inline_conn.rsclaims "the inject path is lossless — so staying inside [the window] means no gap can form". That holds only to guest eth0. For container flows the TCP endpoint is inside the container netns, and #451 established that the guest-internal bridge → veth → container-netns backlog drops frames under burst before TCP buffering — the advertised window bounds the container socket's buffer, not the veth hop. One dropped frame = permanent hole = wedged flow (and #451 measured window gating alone making it worse: every drop became a full-window deadlock).Today the exposure is limited: inline is HV-only in practice and gated on
peer_mss ≥ 1460. But it blocks #262 (promote all established flows to inline) and matters for #250 (HV as default backend).Design (assessed 2026-07-21)
The zero-copy read (socket → guest descriptor buffer directly) is fundamentally at odds with retransmission, which needs the sent bytes host-side. Options considered:
Bound in-flight to a "lossless" budget— no such budget exists past eth0 (above).Only promote root-netns flows to inline— the 4-tuple can't distinguish veth-backed destinations.FastPathConn(Arc<Mutex<…>>beside the existing shared atomics).poll_fast_path— which the shared arcbox_net loop already drives for inline flows (fix(net): add a zero-window persist probe to the download fast path #477 relies on this) — runs its existing dup-ACK/RTO machinery against the shared ring and re-emits via the normal frame path (emit_data_frames→ guest_tx), draining the ring asguest_ackedadvances. The extra memcpy is noise next to the syscall+datapath cost at single-digit Gbps; the 1-syscall hot loop is untouched.Applies identically to both owners (
inject.rs,direct_rx.rs). Acceptance: the #450 network-workload suite (W1–W14) green on the HV backend, which currently cannot pass with inline flows under burst loss.Refs: #451 (bridge-path fix + residual-gap note), #262 (blocked), #250,
common/splicetcp/src/tcp_bridge/fast_path.rs(RTO machinery),virt/arcbox-net-inject/src/inline_conn.rs("lossless" comment to correct).