Repository navigation
microVM: publish guest listeners to the host's private networks - #133
Merged
Merged
Conversation
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
3 open findings
other_addrbefore the subsequent code path can adjust/normalize it (e.g., via… · NewIN_ADDR.S_un.S_addris in network byte order; converting it withto_ne_bytes()will… · New
What changed in this PR
Implements private-network ingress for the microVM portable network profile by allowing the guest to announce listeners and having the host publish them (with flow-based reply admission under egress deny).
Changes:
- Add listener announcement format + recognition and wire ingress allow into egress policy (including new canonical digest encoding).
- Implement Consomme-side private-network publisher + inbound admission rules, and integrate announcement consumption + flow-based reply checks in TX filtering.
- Extend OpenVMM CLI/command line + docs to support
--network-ingress allowand thevirtnet_ingress=allowbootstrap token (with snapshot restrictions).
| File | Description |
|---|---|
| vm/devices/net/net_tap/src/lib.rs | Reject ingress-allowing egress policies for TAP backend. |
| vm/devices/net/net_consomme/src/tests.rs | Add endpoint-level ingress tests and helpers for synthetic frames. |
| vm/devices/net/net_consomme/src/resolver/addressing.rs | Plumb new private_network_ingress field through resolver addressing tests. |
| vm/devices/net/net_consomme/src/resolver.rs | Apply private_network_ingress to endpoint construction. |
| vm/devices/net/net_consomme/src/lib.rs | Add publisher state and public API to enable private-network ingress. |
| vm/devices/net/net_consomme/src/ingress.rs | New: publisher that binds host listeners for announced guest ports and manages updates. |
| vm/devices/net/net_consomme/src/egress.rs | Consume announcements and use flow-aware egress authorization for reply candidates. |
| vm/devices/net/net_consomme/consomme/src/windows.rs | Add host unicast address listing (IPv4+IPv6) and reuse it for IPv6 detection. |
| vm/devices/net/net_consomme/consomme/src/unix.rs | Add host unicast address listing (IPv4+IPv6) and reuse it for IPv6 detection. |
| vm/devices/net/net_consomme/consomme/src/udp.rs | Add inbound admission for UDP listeners, published-peer tracking, and published-port reply path. |
| vm/devices/net/net_consomme/consomme/src/tcp/tests.rs | Add tests for published TCP listeners (coexistence + host admission/reset). |
| vm/devices/net/net_consomme/consomme/src/tcp.rs | Add inbound admission to TCP listeners; publish/unpublish APIs; carries-flow helper. |
| vm/devices/net/net_consomme/consomme/src/lib.rs | Introduce published listener keying + host address cache; expose carries-flow APIs. |
| vm/devices/net/net_consomme/consomme/src/inbound.rs | New: inbound admission logic and private-network address classification + host detection. |
| vm/devices/net/net_consomme/Cargo.toml | Add tracelimit dependency for rate-limited logging. |
| vm/devices/net/net_backend_resources/src/lib.rs | Export new ingress module and add PrivateNetworkIngress to Consomme handle. |
| vm/devices/net/net_backend_resources/src/ingress.rs | New: announcement encoding/parsing and announcement frame recognition utilities. |
| vm/devices/net/net_backend_resources/src/egress/ipv6/tests.rs | Add tests for ingress reply admission and neighbor discovery behavior under IPv6. |
| vm/devices/net/net_backend_resources/src/egress/ipv6.rs | Allow ingress-enabled reply candidates + ND behavior in non-rules modes. |
| vm/devices/net/net_backend_resources/src/egress.rs | Add ingress action to policy, reply-candidate plumbing, and v5 canonical encoding. |
| petri/src/vm/openvmm/modify.rs | Initialize new private_network_ingress field in OpenVMM resources. |
| openvmm/openvmm_entry/src/ttrpc/mod.rs | Initialize new private_network_ingress field for ttrpc NIC backends. |
| openvmm/openvmm_entry/src/microvm/network.rs | Pass private_network_ingress into Consomme handle; extend discovery tuple. |
| openvmm/openvmm_entry/src/microvm/config.rs | Thread ingress flag into virtio discovery tuple + pass ingress config to endpoint. |
| openvmm/openvmm_entry/src/lib.rs | Initialize new private_network_ingress field when parsing endpoints. |
| openvmm/openvmm_entry/src/cli_args/microvm.rs | Implement CLI validation + policy building for --network-ingress allow and host-loopback interactions. |
| openvmm/openvmm_defs/src/microvm.rs | Add virtnet_ingress=allow token support and command-line validation for host-owned ingress bootstrap. |
| Guide/src/reference/openvmm/management/cli.md | Document --network-ingress allow semantics and interactions with snapshots/host-loopback. |
| Guide/src/reference/backends/consomme.md | Document Consomme portable-profile ingress allow behavior in detail. |
🧠 Review effort: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+369
to
+391
| if self.admission.publishes_guest_port() { | ||
| // A published port admits its private-network peers, | ||
| // which stay its peers while they keep exchanging | ||
| // datagrams with the guest. | ||
| let now = Instant::now(); | ||
| if recent_peer(&self.peers, &other_addr, now, timeout).is_none() | ||
| && !self.admission.admits( | ||
| other_addr.ip(), | ||
| None, | ||
| &state.params, | ||
| &mut state.host_addresses, | ||
| ) | ||
| { | ||
| tracelimit::info_ratelimited!( | ||
| peer = %other_addr, | ||
| guest_port = self.guest_port, | ||
| "dropping an inbound UDP datagram from a peer that the listener does not admit" | ||
| ); | ||
| self.stats.rx_refused.increment(); | ||
| continue; | ||
| } | ||
| remember_peer(&mut self.peers, other_addr, now, timeout); | ||
| } |
| let address = unsafe { | ||
| match row.Address.si_family { | ||
| AF_INET => Some(IpAddr::V4(Ipv4Addr::from( | ||
| row.Address.Ipv4.sin_addr.S_un.S_addr.to_ne_bytes(), |
Comment on lines
+15
to
+19
| //! An announcement's payload is the magic `NVXL`, a version byte, a flags | ||
| //! byte, a big-endian entry count, and that many four-byte entries in strictly | ||
| //! increasing order of transport and port: the IP protocol number (6 or 17), a | ||
| //! family mask (1 for IPv4, 2 for IPv6, or 3 for both), and the big-endian | ||
| //! port. Flag bit 0 marks an announcement that omits ports beyond |
ppenna
added a commit
to microsoft/nvx
that referenced
this pull request
Oct 11, 2026
The portable profile rejected --network-ingress allow before launch, because its user-mode NAT had no way to reach a guest listener: every inbound connection needs a host socket, and Consomme bound only the ports that explicit --host-loopback-forward options named. MXC's network.ingress.default: allow means that clients on the host's private networks reach every port on which the workload listens, without a port list, while the internet never reaches it and host loopback stays a separate decision (#275). fc2de0506 (nanvix/openvmm#133) implements that contract. With ingress allow, OpenVMM appends the host-owned virtnet_ingress=allow token, which callers may not supply, and the guest is to announce the TCP and UDP ports on which it listens in UDP datagrams to port 1021 of the IPv4 gateway, from a privileged source port so that only guest root can publish. For each announced port and family, OpenVMM listens on that port of every host address and relays only peers whose addresses are not globally reachable, lie outside the guest's own networks, and are not IPv6 link-local; it resets the TCP connections and drops the datagrams of other peers, so internet clients never reach the guest. The host itself, at loopback or one of its own addresses, is admitted only with explicit --host-loopback allow, which ingress allow now accepts without forwards. Replies to admitted peers pass whatever the egress policy, which admits a TCP segment other than an initial SYN, or a UDP datagram, only when the endpoint carries its flow. Ingress allow records the ingress action in canonical policy encoding version 5, leaving the digests of existing policies unchanged, and snapshot capture and restore reject it, because its listeners are live host attachments. The run guide gains a private-network ingress section, the usage reference describes --network-ingress and --host-loopback for run and sandbox, the cold-boot and machine ABI designs document the token, the announcement, and the policy, and the current limits and the implementation status describe what ingress admits and refuses. The directional-network-policy scenario no longer expects OpenVMM to reject ingress allow, and the validation design no longer lists that check. The guest does not announce its listeners until the next commit adds its announcer, so ingress allow publishes nothing yet. nvx.py verify passes with this pin. Part of #275. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Part of microsoft/nvx#275. --network-ingress allow was rejected because the portable profile's user-mode NAT had no way to reach a guest listener: a host socket must accept each inbound connection, and Consomme bound only the ports that explicit --host-loopback-forward options named. MXC's ingress default means that clients on the host's private networks reach every port on which the workload listens, without a port list, while the internet never reaches it and host loopback stays a separate decision. Implement that contract: - The guest announces its listeners. With ingress allow, OpenVMM appends the host-owned virtnet_ingress=allow token, which callers may not supply, and the guest sends the "NVXL" announcement that net_backend_resources::ingress defines in UDP datagrams to port 1021 of the IPv4 gateway from a privileged source port, so only guest root can publish. An announcement lists at most 64 TCP or UDP ports with their address families and replaces the published set. - net_consomme consumes announcements before the egress policy and Consomme. A publisher keeps one host listener per announced port and family, bound to every host address at the guest's port number; it binds new ports, unbinds withdrawn ones, and retries ports that the host could not bind with the next announcement. A publication has its own listener key, so it coexists with an explicit forward to the same guest port. - Consomme admits a peer on a publishing listener only when its address is not globally reachable: the IPv4 private-use, shared, link-local, benchmarking, and documentation ranges and the IPv6 unique local, site-local, documentation, and benchmarking ranges. It refuses peers inside the guest's own subnets, which the guest would take for its gateway or its neighbors, and IPv6 link-local peers, whose replies would leave from the guest's link-local address. A peer at loopback or at one of the host's own addresses, which the new host_unicast_addresses lists on Linux and Windows, is host loopback and is admitted only with explicit --host-loopback allow; when the host's addresses cannot be listed, a private-network peer is admitted only if the host would be, and a globally reachable one never. Refused TCP connections are reset, and refused datagrams dropped. - Replies pass whatever the egress policy, without widening it. The policy gains an ingress action, which the new canonical encoding version 5 records only for policies that allow ingress, so existing digests do not change. Under ingress allow, the policy admits ARP requests and Neighbor Solicitations for the guest's subnet, which the guest needs to answer at all under egress deny, Neighbor Solicitations for the link-local addresses at which the gateway presents host peers, and treats a TCP segment other than an initial SYN, or a UDP datagram, as a reply candidate. The virtio-net prefilter accepts candidates statelessly, and the endpoint's final check, authorize_frame_with_flows, accepts one only when Consomme carries its flow: a connection in its TCP table, or the association of a published UDP port with a peer that recently sent to it. - A UDP reply to a recent peer of a published port leaves through that port's listener, to the peer's complete host-side address, before the gateway interprets its destination, so the client sees it come from the address and port that it reached. --host-loopback allow without forwards is now accepted with ingress allow, whose listeners then admit the host. Snapshot capture and restore reject ingress allow, because its listeners are live host attachments, and the TAP backend rejects ingress-allow policies. Add unit tests for the announcement format and its recognition, the policy's ingress encoding, neighbor resolution, and reply admission over IPv4 and IPv6, peer classification and admission, TCP admission and resets, UDP publication and replies, publications beside explicit forwards, the command-line token, and the CLI, and an endpoint test that publishes announced ports and admits their replies under egress deny. Update the Guide's Consomme and CLI pages. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ppenna
force-pushed
the
fix/private-network-ingress-275
branch
from
October 13, 2026 11:46
fc2de05 to
dd8def1
Compare
ppenna
added a commit
to microsoft/nvx
that referenced
this pull request
Oct 13, 2026
The portable profile rejected --network-ingress allow before launch, because its user-mode NAT had no way to reach a guest listener: every inbound connection needs a host socket, and Consomme bound only the ports that explicit --host-loopback-forward options named. MXC's network.ingress.default: allow means that clients on the host's private networks reach every port on which the workload listens, without a port list, while the internet never reaches it and host loopback stays a separate decision (#275). dd8def143 (nanvix/openvmm#133) implements that contract. With ingress allow, OpenVMM appends the host-owned virtnet_ingress=allow token, which callers may not supply, and the guest is to announce the TCP and UDP ports on which it listens in UDP datagrams to port 1021 of the IPv4 gateway, from a privileged source port so that only guest root can publish. For each announced port and family, OpenVMM listens on that port of every host address and relays only peers whose addresses are not globally reachable, lie outside the guest's own networks, and are not IPv6 link-local; it resets the TCP connections and drops the datagrams of other peers, so internet clients never reach the guest. The host itself, at loopback or one of its own addresses, is admitted only with explicit --host-loopback allow, which ingress allow now accepts without forwards. Replies to admitted peers pass whatever the egress policy, which admits a TCP segment other than an initial SYN, or a UDP datagram, only when the endpoint carries its flow. Ingress allow records the ingress action in canonical policy encoding version 5, leaving the digests of existing policies unchanged, and snapshot capture and restore reject it, because its listeners are live host attachments. The run guide gains a private-network ingress section, the usage reference describes --network-ingress and --host-loopback for run and sandbox, the cold-boot and machine ABI designs document the token, the announcement, and the policy, and the current limits and the implementation status describe what ingress admits and refuses. The directional-network-policy scenario no longer expects OpenVMM to reject ingress allow, and the validation design no longer lists that check. The guest does not announce its listeners until the next commit adds its announcer, so ingress allow publishes nothing yet. nvx.py verify passes with this pin. Part of #275. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR implements private-network ingress for the microVM portable network profile, for microsoft/nvx#275; microsoft/nvx#495 pins it. It is based on
main(1feb1f4a3), whichdevof microsoft/nvx pins; merge it by fast-forwardingmain, as #126 and #130 were, so that the pinned revision stays onmain.Why
--network-ingress allowwas rejected before VM resources were opened, because the portable profile's user-mode NAT had no way to reach a guest listener: every inbound connection needs a host socket, and Consomme bound only the ports that explicit--host-loopback-forwardoptions named. MXC'snetwork.ingress.default: allowmeans that clients on the host's private networks reach every port on which the workload listens, without a port list, while the internet never reaches it and host loopback stays a separate, explicit decision. Replies must pass even when the egress policy denies everything.Changes
net_backend_resources::ingressdefines the announcement: the magicNVXL, version 1, a flags byte whose bit 0 marks omitted listeners, a big-endian count of at most 64, and entries in strictly increasing order of protocol (6 or 17), family mask (1 for IPv4, 2 for IPv6), and nonzero port. The guest sends it in UDP datagrams to port 1021 of the IPv4 gateway from a source port below 1024, so only guest root can publish.listener_announcement_payloadrecognizes one from its untagged, unfragmented IPv4 and UDP headers with a valid IPv4 checksumvirtnet_ingress=allowtoken after the other network tokens. Callers may not supplyvirtnet_ingress=, and the machine contract accepts the exact token at most once and only with virtio-netnet_consommeconsumes announcements before the egress policy and Consomme.PrivateNetworkPublisherkeeps one host listener per announced port and family, bound to every host address (0.0.0.0, or[::]withIPV6_V6ONLY) at the guest's port number; it binds new ports, unbinds withdrawn ones, and retries ports that the host could not bind with the next announcement. A publication has its own listener key, so it coexists with an explicit forward to the same guest portconsomme::InboundAdmission::PrivateNetworkadmits a peer only whenis_private_network_addressaccepts it: the IPv4 private-use, shared, link-local, benchmarking, and documentation ranges and the IPv6 unique local, site-local, documentation, and benchmarking ranges, none of which is globally reachable. It refuses peers inside the guest's own subnets, which the guest would take for its gateway or its neighbors, and IPv6 link-local peers, whose replies would leave from the guest's link-local address. A peer at loopback, at the address that it connected to, or at one of the host's own addresses (host_unicast_addresses, fromgetifaddrsorGetUnicastIpAddressTable, listed again after a second) is the host and is admitted only with explicit--host-loopback allow; when the host's addresses cannot be listed, a private-network peer is admitted only if the host would be, and a globally reachable one never. A refused TCP connection is reset withSO_LINGER0, and a refused datagram droppedauthorize_frame_with_flows, accepts it only when Consomme carries its flow (carries_tcp_flow,carries_udp_flow): a connection in its TCP table, or the association of a published UDP port with a peer that recently sent to it. A UDP reply to such a peer leaves through the publishing listener, to the peer's complete host-side address, before the gateway interprets its destination, so the client sees it come from the address and port that it reachedEgressPolicygains an ingress action. Canonical encoding version 5 appends it and is used only for policies that allow ingress, so existing digests do not change--network-ingress allowis accepted, and so is--host-loopback allowwithout forwards beside it, whose published listeners then also admit the host. Snapshot capture and restore reject ingress allow, because its listeners are live host attachments, and the TAP backend rejects ingress-allow policiesValidation
fc2de0506:cargo nextestfornet_backend_resources,consomme,net_consomme,net_tap,openvmm_defs,openvmm_entry, andvirtio_net: 502 passed, 1 skipped, with Rust 1.95.0 on Windows, and 567 passed, 1 skipped, on Linux on a bare-metal MSHV host. The new tests cover the announcement format and its recognition, the policy's ingress encoding, neighbor resolution, and reply admission over IPv4 and IPv6, peer classification and admission (including the guest's own subnets, IPv6 link-local peers, and unlisted host addresses), TCP admission and resets, UDP publication and replies, publications beside explicit forwards, the command-line token, and the CLI, plus an endpoint test that publishes announced ports and admits their replies under egress deny.clippy --all-targetson Windows and Linux and rustdoc with-D warningsare clean for every modified package, andxtask fmtpasses.fc2de0506before the rebase, on bare-metal KVM, MSHV, and WHP hosts: the newprivate-network-ingressscenario and the other network scenarios, theaci_edge_sandboxeshypervisor tests, and a cross-host test in which a client on another host of a benchmarking (198.18.0.0/15) lab network exchanged TCP and UDP with a guest on a KVM host through that host's LAN address under egress deny, while the host itself was refused. See that PR for the observations.main(1feb1f4a3) asdd8def143, which applied without conflicts and whosegit range-diffagainstfc2de0506shows the same patch:cargo nextestfor the same packages, 503 passed, 1 skipped, with Rust 1.95.0 on Windows;clippy --all-targetswith-D warningsand rustdoc with-D warningsare clean for every modified package on Windows, andcargo xtask fmtpasses. network: publish guest listeners to the host's private networks microsoft/nvx#495 now pinsdd8def143.