openvmm: answer guest UDP to host loopback from the gateway

doc/design/3.5-network.md

@@ -140,7 +140,9 @@ admits the guest's replies on the flows that the endpoint carries, as
 
 By default, and subject to the egress policy, the gateway maps guest TCP and
 UDP flows onto host loopback: the IPv4 gateway maps to `127.0.0.1`, and the
-IPv6 gateway maps to `::1`. The guest's own loopback remains guest loopback.
+IPv6 gateway maps to `::1`. Replies on these flows come from the gateway
+address and port that the guest addressed, as connected UDP sockets require.
+The guest's own loopback remains guest loopback.
 Other host-local destinations, such as link-local addresses, also reach host
 sockets unless host loopback is denied.
 
@@ -202,9 +204,10 @@ endpoint carries: a TCP segment other than an initial SYN, or a UDP datagram
 from a published port to a peer that recently sent one to it, which leaves
 through the port's host listener.
 
-The endpoint presents host peers at synthetic addresses inside the guest's
-subnet of each family, allocated downward from its last usable address, such
-as `10.0.0.254` and `fd00::a00:ff` for the identity in Table 3.12. Explicit
+The endpoint presents other host peers, such as the host clients of published
+listeners, at synthetic addresses inside the guest's subnet of each family,
+allocated downward from its last usable address, such as `10.0.0.254` and
+`fd00::a00:ff` for the identity in Table 3.12. Explicit
 host-loopback access therefore requires an IPv4 network of /29 or larger: a
 /30 holds only the gateway and the guest and leaves no address for host peers.
 The guest routes its replies to a synthetic address through its NIC rather

doc/openvmm-upstream-roadmap.md

@@ -5,7 +5,7 @@
 - OpenVMM fork base: `e1cdbd916d3aacb216309c8e7779b832546759a1`.
 - Mapped fork tip: `2728f33ea9a44d2d0fb956d9f01768ae2100fb62`, position 166. The graph, PR
   summary, and commit mapping below cover positions 001-166.
-- NVX's pinned OpenVMM revision: `c6d97c529986c0b3b1bf1fd7f00406654cdeee67`, 152 commits past the
+- NVX's pinned OpenVMM revision: `5ab8a84621524f1777b2cb5e65a70dab42c8b453`, 154 commits past the
   mapped tip on the same first-parent chain. These commits are not yet assigned to PRs; see
   [Commits after the mapped tip](#commits-after-the-mapped-tip).
 - Upstream main: `1fd455b19e7c69c72a2d375d0ed1cb35d901c688` (2026-10-01), 18 commits past
@@ -199,7 +199,7 @@ e1cdbd916
   -> PR18/PR23 split [162]
   -> PR18 fixup [163] -> PR25 fixup [164] -> PR27 [165-166]
   -> 2728f33ea (mapped tip)
-  -> unassigned [167-318] -> c6d97c529 (NVX pin)
+  -> unassigned [167-320] -> 5ab8a8462 (NVX pin)
 ```
 
 For upstream review, PR01-PR08 and PR14 can be rebased independently onto current upstream main.
@@ -215,7 +215,7 @@ can be submitted independently if preferred.
 
 ## Commits after the mapped tip
 
-The NVX pin extends the chain past position 166 without merges. The 152 later commits are not
+The NVX pin extends the chain past position 166 without merges. The 154 later commits are not
 yet classified or assigned to PRs. This table groups them by topic in chain order; ranges are
 inclusive, as in the [PR summary](#pr-summary).
 
@@ -243,6 +243,8 @@ inclusive, as in the [PR summary](#pr-summary).
 | 316 | `a44fcc2aa` | 1 | Bidirectional host-loopback access: `--host-loopback allow` publishes the guest's announced listeners on host loopback for the host alone |
 | 317 | `7f225e2cd` | 1 | IPv6 host peers appear inside the static IPv6 subnet, so the guest answers them from its address there |
 | 318 | `c6d97c529` | 1 | Standalone time ABI documentation: comments describe the contract without citing NVX's removed specification |
+| 319 | `eae7da6f3` | 1 | Replies of the host-loopback services that the guest reaches at its gateway come from the gateway, so connected guest UDP sockets receive them |
+| 320 | `5ab8a8462` | 1 | The VM worker stops its devices before it tears down the partition and guest memory, so no device worker reads guest memory that teardown has unmapped |
 
 Some of these commits change mapped work rather than add to it. Positions 255-259 remove the
 legacy microVM clock paths and change files that PR10 and PR17 also change. Position 261 makes

doc/run.md

@@ -450,10 +450,12 @@ to host loopback, without a port list:
 
 The guest reaches host loopback services through its gateway, subject to the
 egress policy: a host service on `127.0.0.1:8080` at `10.0.0.1:8080`, and one
-on `[::1]:8080` at `[fd00::a00:1]:8080`. The guest sees host peers at
-addresses that OpenVMM assigns inside its own network, and the egress policy
-applies to its traffic to them as it applies at the gateway, so a rule that
-denies a gateway port denies it at those addresses too. In the other
+on `[::1]:8080` at `[fd00::a00:1]:8080`. Their replies come from the gateway
+address and port that the guest addressed, so connected UDP sockets receive
+them too. The guest sees other host peers, such as clients of its published
+ports, at addresses that OpenVMM assigns inside its own network, and the egress
+policy applies to its traffic to them as it applies at the gateway, so a rule
+that denies a gateway port denies it at those addresses too. In the other
 direction, OpenVMM appends `virtnet_ingress=host-loopback` to the kernel
 command line unless ingress is `allow`, and the guest's `nvx-ingress` reports
 its listeners as it does for

scripts/nvx_tools/microvm_test_scripts/host-loopback-policy.sh.in

@@ -68,6 +68,15 @@ allow)
     general=$(wget -T 5 -qO- http://@GATEWAY_IPV4@:@GENERAL_PORT@/general) ||
         fail 93
     [ "$general" = NVX-HOST-LOOPBACK-GENERAL ] || fail 94
+    # A connected UDP socket, as nc's, receives the answers of host services
+    # on IPv4 and IPv6 loopback only from the gateway address and port that it
+    # sent its datagram to (#560).
+    udp_echo=$(printf 'NVX-HOST-LOOPBACK-UDP-ECHO\n' |
+        nc -u -w 2 @GATEWAY_IPV4@ @UDP_ECHO_PORT@ 2>/dev/null) || true
+    [ "$udp_echo" = NVX-HOST-LOOPBACK-UDP-ECHO-REPLY ] || fail 84
+    udp_echo=$(printf 'NVX-HOST-LOOPBACK-UDP-ECHO\n' |
+        nc -u -w 2 @GATEWAY_IPV6@ @UDP_ECHO_IPV6_PORT@ 2>/dev/null) || true
+    [ "$udp_echo" = NVX-HOST-LOOPBACK-UDP-ECHO-REPLY ] || fail 85
     rm -f /tmp/nvx-host-loopback-tcp-ipv4 /tmp/nvx-host-loopback-tcp-ipv6 \
         /tmp/nvx-host-loopback-tcp-forward /tmp/nvx-host-loopback-udp \
         /tmp/nvx-host-loopback-udp-ipv6 \

scripts/nvx_tools/microvm_tests.py

@@ -210,6 +210,8 @@
 HOST_LOOPBACK_TCP_REQUEST = b"NVX-HOST-LOOPBACK-INBOUND\n"
 HOST_LOOPBACK_UDP_REQUEST = b"NVX-HOST-LOOPBACK-UDP\n"
 HOST_LOOPBACK_UDP_REPLY = b"NVX-HOST-LOOPBACK-UDP-REPLY\n"
+HOST_LOOPBACK_UDP_ECHO_REQUEST = b"NVX-HOST-LOOPBACK-UDP-ECHO\n"
+HOST_LOOPBACK_UDP_ECHO_REPLY = b"NVX-HOST-LOOPBACK-UDP-ECHO-REPLY\n"
 HOST_LOOPBACK_PORT_BIND_ATTEMPTS = 16
 # IANA's dynamic port range, which is also Windows' default. The host-loopback
 # scenario draws the port of each TCP and UDP listener pair from it.
@@ -3171,6 +3173,57 @@ def _http_server(
         errors.append(error)
 
 
+def _udp_echo_server(
+    server: socket.socket, stop: threading.Event, errors: list[Exception]
+) -> None:
+    """Answer each echo request that ``server`` receives until ``stop`` is set."""
+    try:
+        server.settimeout(0.05)
+        while not stop.is_set():
+            try:
+                request, peer = server.recvfrom(4096)
+            except (TimeoutError, ConnectionResetError):
+                # Windows reports a datagram that a closed port refused as a
+                # reset on the next receive.
+                continue
+            if request != HOST_LOOPBACK_UDP_ECHO_REQUEST:
+                raise RuntimeError(
+                    f"unexpected host-loopback UDP echo request: {request!r}"
+                )
+            server.sendto(HOST_LOOPBACK_UDP_ECHO_REPLY, peer)
+    except Exception as error:
+        errors.append(error)
+
+
+def _bind_host_loopback_udp_echo(
+    family: int, address: str, published: Sequence[int]
+) -> socket.socket:
+    """Bind a UDP socket to an ephemeral port of ``address`` outside ``published``.
+
+    The probes that drew the guest ports that OpenVMM publishes released them,
+    so an ephemeral bind can take one, keep OpenVMM from publishing it, and
+    receive the host's datagrams for the guest. Each rejected socket holds its
+    port until the search ends, so no later attempt draws that port again.
+    """
+    attempts: list[socket.socket] = []
+    try:
+        for _ in range(HOST_LOOPBACK_PORT_BIND_ATTEMPTS):
+            udp_echo = socket.socket(family, socket.SOCK_DGRAM)
+            attempts.append(udp_echo)
+            if family == socket.AF_INET6:
+                udp_echo.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)
+            udp_echo.bind((address, 0))
+            if int(udp_echo.getsockname()[1]) not in published:
+                return attempts.pop()
+    finally:
+        for rejected in attempts:
+            rejected.close()
+    raise RuntimeError(
+        f"failed to bind a UDP port on {address} outside guest ports "
+        f"{sorted(published)} after {HOST_LOOPBACK_PORT_BIND_ATTEMPTS} attempts"
+    )
+
+
 def _host_loopback_port_candidates() -> list[int]:
     """Draw distinct ports uniformly from ``HOST_LOOPBACK_PORT_RANGE``.
 
@@ -3278,11 +3331,14 @@ def run_host_loopback_policy(
                 "host-loopback-policy.sh.in",
                 MODE="udp-control",
                 GATEWAY_IPV4=DIRECTIONAL_NETWORK_GATEWAY_IPV4,
+                GATEWAY_IPV6=DIRECTIONAL_NETWORK_GATEWAY_IPV6,
                 GENERAL_PORT=str(denied_general_port),
                 PROXY_PORT=str(proxy_port),
                 GUEST_PORT="0",
                 PRIVATE_GUEST_PORT="0",
                 UDP_IPV6_GUEST_PORT="0",
+                UDP_ECHO_PORT="0",
+                UDP_ECHO_IPV6_PORT="0",
             ),
             HOST_LOOPBACK_UDP_CONTROL_MARKER,
             timeout=timeout,
@@ -3313,11 +3369,14 @@ def run_host_loopback_policy(
                 "host-loopback-policy.sh.in",
                 MODE="deny",
                 GATEWAY_IPV4=DIRECTIONAL_NETWORK_GATEWAY_IPV4,
+                GATEWAY_IPV6=DIRECTIONAL_NETWORK_GATEWAY_IPV6,
                 GENERAL_PORT=str(denied_general_port),
                 PROXY_PORT=str(proxy_port),
                 GUEST_PORT="0",
                 PRIVATE_GUEST_PORT="0",
                 UDP_IPV6_GUEST_PORT="0",
+                UDP_ECHO_PORT="0",
+                UDP_ECHO_IPV6_PORT="0",
             ),
             HOST_LOOPBACK_DENY_MARKER,
             timeout=timeout,
@@ -3382,7 +3441,34 @@ def run_host_loopback_policy(
         daemon=True,
     )
     allow_server.start()
+    udp_echoes: list[socket.socket] = []
+    udp_echo_servers: list[threading.Thread] = []
+    udp_echo_errors: list[Exception] = []
+    udp_echo_stop = threading.Event()
     try:
+        # Host UDP services on IPv4 and IPv6 loopback, which the guest reaches
+        # at its gateway. The guest's UDP servers are dual-stack, so OpenVMM
+        # publishes both of their ports on both loopback addresses, and
+        # neither service may hold either port.
+        for family, address in (
+            (socket.AF_INET, "127.0.0.1"),
+            (socket.AF_INET6, "::1"),
+        ):
+            udp_echo = _bind_host_loopback_udp_echo(
+                family, address, (guest_port, udp_ipv6_guest_port)
+            )
+            udp_echoes.append(udp_echo)
+            udp_echo_server = threading.Thread(
+                target=_udp_echo_server,
+                args=(udp_echo, udp_echo_stop, udp_echo_errors),
+                name="nvx-host-loopback-udp-echo",
+                daemon=True,
+            )
+            udp_echo_server.start()
+            udp_echo_servers.append(udp_echo_server)
+        udp_echo_port, udp_echo_ipv6_port = (
+            int(udp_echo.getsockname()[1]) for udp_echo in udp_echoes
+        )
         allow_command = workload_boot_command(
             executable,
             backend,
@@ -3393,8 +3479,9 @@ def run_host_loopback_policy(
             network=DIRECTIONAL_NETWORK_CIDR,
         )
         # Egress admits only the guest's own connection to the general host
-        # service, so the guest's replies to the host pass because host
-        # loopback admits them, not because egress does.
+        # service and its datagrams to the UDP services, so the guest's
+        # replies to the host pass because host loopback admits them, not
+        # because egress does.
         allow_command.extend(
             (
                 "--host-loopback",
@@ -3405,6 +3492,10 @@ def run_host_loopback_policy(
                 "deny",
                 "--network-egress-allow",
                 f"{DIRECTIONAL_NETWORK_GATEWAY_IPV4}/32:tcp:{allowed_general_port}",
+                "--network-egress-allow",
+                f"{DIRECTIONAL_NETWORK_GATEWAY_IPV4}/32:udp:{udp_echo_port}",
+                "--network-egress-allow",
+                f"{DIRECTIONAL_NETWORK_GATEWAY_IPV6}/128:udp:{udp_echo_ipv6_port}",
                 "--host-loopback-forward",
                 f"tcp:{host_forward_port}:{guest_port}",
             )
@@ -3422,11 +3513,14 @@ def run_host_loopback_policy(
                     "host-loopback-policy.sh.in",
                     MODE="allow",
                     GATEWAY_IPV4=DIRECTIONAL_NETWORK_GATEWAY_IPV4,
+                    GATEWAY_IPV6=DIRECTIONAL_NETWORK_GATEWAY_IPV6,
                     GENERAL_PORT=str(allowed_general_port),
                     PROXY_PORT="0",
                     GUEST_PORT=str(guest_port),
                     PRIVATE_GUEST_PORT=str(private_guest_port),
                     UDP_IPV6_GUEST_PORT=str(udp_ipv6_guest_port),
+                    UDP_ECHO_PORT=str(udp_echo_port),
+                    UDP_ECHO_IPV6_PORT=str(udp_echo_ipv6_port),
                 ),
             )
             process.wait_for_line(HOST_LOOPBACK_INGRESS_READY_MARKER, timeout)
@@ -3451,9 +3545,18 @@ def run_host_loopback_policy(
             raise RuntimeError("general host-loopback server failed") from allow_errors[
                 0
             ]
+        if udp_echo_errors:
+            raise RuntimeError(
+                "host-loopback UDP echo service failed"
+            ) from udp_echo_errors[0]
     finally:
         allowed_general.close()
         allow_server.join(timeout=NETWORK_SERVER_JOIN_TIMEOUT_SECONDS)
+        udp_echo_stop.set()
+        for udp_echo_server in udp_echo_servers:
+            udp_echo_server.join(timeout=NETWORK_SERVER_JOIN_TIMEOUT_SECONDS)
+        for udp_echo in udp_echoes:
+            udp_echo.close()
 
     run_host_loopback_rejections(
         executable,