Skip to content

openvmm: let a platform restore install a new network policy on a NIC template - #931

Merged
Pedro Henrique Penna (ppenna) merged 4 commits into
devfrom
agents/issue-536-logical-commits-pr-guidelines
Oct 27, 2026
Merged

Pedro Henrique Penna (ppenna) merged 4 commits into
devfrom
agents/issue-536-logical-commits-pr-guidelines

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #536. Pins microsoft/openvmm#4767 at 57d0a18b3b542dce3f0787e60f3c48296b40d30e, which lets a microVM platform restore install a network policy of its own on a NIC template, a platform snapshot whose NIC was captured with a complete deny-all policy, before any vCPU runs, including private-network ingress, explicit host loopback, and forwards. It adds a microVM scenario that restores one NIC template with four policies on KVM, MSHV, and WHP.

Based on dev at 675190a30. microsoft/openvmm#4767 is two commits directly on e68871cc6, the head of feature/microvm/dev and the revision that dev pins, so this branch's four commits are its whole diff.

Change

Commit Files Change
cbedd5bca openvmm openvmm e68871cc6 → 57d0a18b3: the two commits of microsoft/openvmm#4767
7adba0772 doc doc/openvmm-upstream-roadmap.md Records them as positions 346 and 347, unassigned like positions 341-343 because they change PR21's position 129 and the unassigned positions 306 and 307, moves the NVX pin, and records that microsoft/openvmm#4762 merged positions 344 and 345 into feature/microvm/dev
355e03a2d doc doc/run.md A Platform NIC templates section for direct OpenVMM restores; the ingress and host-loopback refusals link to it
bc971187d scripts scripts/nvx_tools/microvm_tests.py, scripts/nvx_tools/microvm_test_scripts/platform-network-policy.sh.in, scripts/test_microvm_tests.py, scripts/nvx_tools/microvm_shards.json, doc/ci.md, doc/usage.md New default-suite scenario platform-network-policy; shards replanned

OpenVMM changes

  • A NIC template (3e3a454c5). A NIC captured with the complete deny-all policy, --network-egress deny and --host-loopback deny without egress rules, a proxy, or ingress, holds no flows, listeners, or forwards, and its guest learned no DNS server and announces no listeners, so nothing in the snapshot depends on its policy beyond the NIC's identity. OpenVMM recognizes it from the network record that every networked snapshot already has: the record must be the complete deny-all's for its own identity, in digest encoding 3, or 4 with IPv6. Encodings 0 to 2 record neither the host-loopback action nor the proxy endpoint, so they never identify a template. The manifest format does not change.
  • --restore-network-policy (57d0a18b3). A platform restore of a NIC template installs the network policy of its own options in place of the template's deny-all. The same code as at a cold boot validates it, it is bound to the NIC's captured identity, which the clone keeps, and the device installs it, with its forwards bound, before any vCPU runs, so a forward that cannot bind its port fails the restore. The restore contract still compares the whole network record, against the template's deny-all. Ingress, explicit host loopback, and forwards, which every snapshot refused, are admitted at such a restore only; captures, other tiers, and restores without the option keep the same-policy rule. The guest keeps its captured command line, so it learns its DNS server and whether to announce its listeners from its host after the restore (nvx-rs: a canonical activation record for platform clones #838, nvx-rs: per-clone network policies on deny-all templates #850).

Tests

  • platform-network-policy captures a NIC template with sandbox blocks and restores it four times with --restore-network-policy: under the template's own deny-all, under egress rules that admit one host-loopback service, with allowed egress, explicit host loopback, a forward, and private-network ingress, and with deny-all egress and host loopback and a proxy at the other service. Each clone's guest reports its NIC's MAC, IPv4, and IPv6 addresses, which must be the template's, probes both services through its gateway, and serves one TCP client until the host releases it. Each service must receive exactly the requests that the clone's policy admits, and the host reaches the guest through the forward only in the clone that has one. Before the guest runs, OpenVMM must refuse a restore of the template that keeps its policy but passes another or explicit host loopback, a new address, a proxy that is not the gateway, a forward whose host port another socket holds, the option on a platform snapshot captured with plain --network-egress deny, which leaves host loopback allowed, and the option on a cold boot; no restore may change the template. It runs with the Alpine sandbox-control guest only, like platform-zero-fill, and joins shard 2 on every backend, each planned with the scenario's time in the bare-metal default suite below.
  • Unit tests cover the guest's checks, the clones' policies and reports, the commands, the refusals, a failure that the guest reports, the services' request counts, and the template's integrity, simulating each clone against the scenario's real host services. [DONT REVIEW] microVM: let a platform restore install a new network policy on a NIC template openvmm#4767 adds unit tests for the template rule, the CLI, the restore's tier check, and restores of one template with eight policies.

Of #536's acceptance cases, the scenario covers a restore with no connectivity and three with policies of their own from one template without a change of the NIC's identity; egress rules, host loopback, a proxy, a forward, and private-network ingress at restore; a listener bind failure; and the refusal of a template that is not a complete deny-all. Packets cannot escape before the vCPUs run because the policy is installed at device construction and no restored vCPU runs under another, which the deny-all clone checks from both sides. Ingress publication itself needs the guest's announcer, which only #850's activation starts in a clone, so the scenario checks that the restore admits ingress and leaves publication to #850.

Evidence

On the bare-metal hosts prometheus30 (Linux/MSHV), prometheus32 (Linux/KVM), and prometheus28 (Windows/WHP), each in a separate worktree with OpenVMM built there from 57d0a18b3 and the guest of development release v0.1.0-dev.239efb959e70:

Check MSHV KVM WHP
platform-network-policy with the fix Passes (16.2 s) Passes (13.6 s) Passes (37.6 s)
Default microVM suite with the fix 40/40 (46 executions) 40/40 (46 executions) 40/40 (46 executions)
platform-network-policy in the default suite 16.5 s 13.7 s 24.8 s

With that release's unpatched OpenVMM e68871cc6 on KVM, the template's capture succeeds and the first clone's restore fails at unexpected argument '--restore-network-policy'.

Every clone reported the template's mac=52:54:00:00:00:02 ipv4=10.0.0.2/24 ipv6=fd00::a00:2/120 on every backend, and the same outcomes:

Clone Policy at restore Allowed service Other service Through the forward
deny-all --network-egress deny --host-loopback deny denied denied none
egress-rules --network-egress deny --network-egress-allow 10.0.0.1/32:tcp:ALLOWED reached denied none
forward --network-egress allow --host-loopback allow --host-loopback-forward tcp:HOST:18080 --network-ingress allow reached reached reached
proxy --network-egress deny --host-loopback deny --network-proxy 10.0.0.1:OTHER denied reached none

The refusals failed before the guest ran, with: restore-time egress policy does not match the snapshot contract (another policy without the option); microVM snapshots do not support --host-loopback allow, ..., except at a restore with --restore-network-policy; microVM restore takes network addressing from saved state; do not pass --net; network proxy address 10.0.0.9 must match the guest gateway 10.0.0.1; failed to launch vm worker caused by failed to resolve resource of type net_endpoint_handle:consomme and failed to create socket for port forward (Tcp 127.0.0.1:PORT) (the busy forward); snapshot network policy is not a complete deny-all, so a restore cannot replace it; ... (plain --network-egress deny); and --restore-network-policy requires --restore-snapshot (cold boot).

Equivalence and validation

  • Hosts. The hosts ran the default suite at ad723317a, on dev at 675190a30, and platform-network-policy again at this head, where it passes in 14.3 s on KVM, 16.4 s on MSHV, and 24.6 s on WHP. This head differs from ad723317a in the scenario only by the two review fixes: the release of the guest's server says whether the host exchanged a line through a forward, so that the guest waits for a request that the release may overtake, and the refusals forbid every line that a restored guest prints, from its platform repair's report on. It also differs in the shard partition, which the hosts' unsharded runs don't read and which is planned with their in-suite times, and in the scenario commit's message.
  • OpenVMM, on [DONT REVIEW] microVM: let a platform restore install a new network policy on a NIC template openvmm#4767's head, on Windows. cargo clippy --all-targets and cargo doc --no-deps for openvmm_helpers and openvmm_entry, cargo nextest run --profile agent for both (335 passed), and cargo xtask fmt --fix (clean). The protocol vectors that nvx-rs/tests/vectors/openvmm/ copies are unchanged.
  • NVX, at this head, on Windows. nvx.py verify, ruff check, ruff format --check, pyright for Linux and Windows, the Markdown link check, the validate-nvx unittest list (1,323 tests, 48 skipped; the 18 shell cases of test_setup_azure.py, which need a Linux bash, fail the same way on dev here), test_adversarial.py, test_hosts.py, and the CLI --help checks.
  • Not run locally. Those shell cases; the Specula wiring tests; shellcheck and shfmt, which run in Docker, although no checked shell script changed; and the OpenVMM VMM tests, which cannot reach a NIC snapshot through the management RPC. CI runs all of them.

Notes

Exact NVX head submitted to CI: bc971187dafaf5f25776a178d10a34b5d9881b7b.

Pin microsoft/openvmm#4767 at 57d0a18b3b542dce3f0787e60f3c48296b40d30e,
two commits on e68871cc6, the previous pin, which is the head of
feature/microvm/dev.

The first recognizes a NIC template: a platform snapshot whose NIC was
captured with the complete deny-all policy, --network-egress deny and
--host-loopback deny without egress rules, a proxy, or ingress. Its
guest learned no DNS server and announces no listeners, and its
endpoint held no flows, listeners, or forwards, so nothing in the
snapshot depends on its policy beyond the NIC's identity. A snapshot's
network record identifies a template only in digest encodings from
version 3 on, since earlier ones record neither the host-loopback
action nor the proxy endpoint. The manifest format does not change.

The second adds --restore-network-policy, with which a platform restore
installs the network policy of its own options in place of the
template's deny-all, including private-network ingress, explicit host
loopback, and host-loopback forwards, which snapshots refused. The same
code as at a cold boot validates the policy, which is bound to the
NIC's captured identity, and the device installs it, with its forwards
bound, before any vCPU runs; a forward that cannot bind its port fails
the restore. The restore contract still compares the whole network
record, against the template's deny-all. Captures, other tiers, and
restores without the option keep the same-policy rule.

Part of #536

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Record the two commits that the previous commit pins as positions 346
and 347 of the microVM chain, and move the NVX pin to 57d0a18b3.

Position 346 recognizes a NIC template, a platform snapshot whose NIC
was captured with the complete deny-all policy, from the policy digest
that every networked snapshot records. Position 347 adds
--restore-network-policy, which installs a restore's own policy on such
a template. They add behavior rather than fix a mapped commit, and they
change the snapshot network policy and the forward refusal that PR21's
position 129 added and the snapshot refusals of the unassigned
positions 306 and 307, so they stay unassigned, like positions 341-343.
Note in the table's remarks that assigning them means revisiting that
work. The pin is now 190 commits past the mapped tip.

Positions 344 and 345 are no longer only a proposal:
microsoft/openvmm#4762 merged them, so position 345, e68871cc6, is the
head of feature/microvm/dev. Say so in the scope, and name
microsoft/openvmm#4767 as the proposal of positions 346 and 347.

Part of #536

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
doc/run.md said that snapshot capture and restore reject private-network
ingress and explicit host-loopback access, whose listeners are live
host attachments. The pinned OpenVMM admits them, and forwards, at a
restore that installs a new policy on a NIC template with
--restore-network-policy.

Add a Platform NIC templates section for direct OpenVMM users: what a
NIC template is, an example restore with its own policy, that OpenVMM
validates the policy as a cold boot's, binds it to the captured
addresses, and installs it with its forwards before any vCPU runs, that
a busy forward port fails the restore, and that the guest keeps its
captured command line, so it learns its DNS server and whether to
report its listeners from its host after the restore. It also names
the refusals, and that nvx-rs run does not pass the option. The two
refusal sentences link to it.

Part of #536

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A listener-cleanup race can discard valid forwarded requests and intermittently fail the default microVM suite.

1 open issue
What changed in this PR

This PR integrates OpenVMM support for installing per-clone network policies on deny-all platform NIC templates, enabling later NVX clone activation work while preserving NIC identity.

Changes:

  • Advances the OpenVMM pin and documents restore-time policy support and restrictions.
  • Adds a default-suite scenario covering four clone policies, rejected restores, and template integrity.
  • Adds unit coverage and rebalances scenario shards across KVM, MSHV, and WHP.
File Description
scripts/​test_microvm_tests.py Tests scenario commands, policy outcomes, failures, and template integrity.
scripts/​nvx_tools/​microvm_tests.py Implements and registers the network-policy restore scenario.
scripts/​nvx_tools/​microvm_test_scripts/​platform-network-policy.sh.in Checks guest NIC identity, connectivity, and forwarded traffic.
scripts/​nvx_tools/​microvm_shards.json Adds scenario timings and rebalances backend shards.
doc/​usage.md Lists the new scenario's guest restrictions.
doc/​run.md Documents NIC templates and restore-time network options.
doc/​openvmm-upstream-roadmap.md Records the new pin and upstream commits.
doc/​ci.md Describes the scenario's coverage.

Review effort: Balanced

@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-536-logical-commits-pr-guidelines branch from 6e40a8c to 50e4117 Compare October 27, 2026 18:17
Copilot AI balanced review requested due to automatic review settings October 27, 2026 18:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Security-sensitive pre-vCPU policy enforcement needs final human review, and the refusal checks still leave early guest execution insufficiently verified.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread scripts/nvx_tools/microvm_tests.py Outdated
No test covered a platform restore that installs a network policy of
its own, which OpenVMM refused until the pinned change. Add the
platform-network-policy microVM scenario to the default suite.

The scenario captures a NIC template: a platform snapshot with sandbox
blocks whose NIC, 10.0.0.2/24, was captured with the complete deny-all
policy --network-egress deny --host-loopback deny. Two HTTP services on
host loopback, the allowed one and the other one, count the requests
they receive. Each clone restores the template with
--restore-network-policy and a policy of its own:

- the template's own deny-all, which reaches neither service;
- egress rules that admit the allowed service's port at the gateway;
- allowed egress, explicit host loopback with a forward from a free host
  port to the guest, and private-network ingress, which reaches both
  services and serves the host through the forward;
- deny-all egress and host loopback with a proxy at the other service.

Each clone's guest reports its NIC's MAC, IPv4, and IPv6 addresses,
which must be the template's, probes both services through its gateway,
and serves one TCP client until the host releases it. The host
exchanges a line with it through the forward, when the clone has one,
and its release then tells the guest to wait for that line, which the
release may overtake. Each service must receive exactly the requests
that the clone's policy admits, so a clone that reaches more than its
policy admits fails even if its guest reports otherwise. The template's
capture must reach neither service.

Before the guest runs, OpenVMM must refuse a restore of the template
that keeps its policy but passes another or explicit host loopback, a
new address, a proxy that is not the gateway, a forward whose host
port another socket holds, --restore-network-policy on a platform
snapshot captured with plain --network-egress deny, which leaves host
loopback allowed, and the option on a cold boot: a refused restore must
fail before its guest prints any line, from its platform repair's report
to the restore's end. No restore may change the template.

The scenario needs sandbox blocks, so it runs only with the Alpine
sandbox-control guest, like platform-zero-fill. In the default suite,
it took 13.7 s on bare-metal KVM, 16.5 s on MSHV, and 24.8 s on WHP, so
each backend's shards are planned again with its time, which CI's
duration reports replace once CI has run it; it joins shard 2 on each.
The default suite passed on each of those hosts. With the unpatched
OpenVMM of the latest development release, e68871cc6, the template's
capture succeeds and the first clone's restore fails at unexpected
argument '--restore-network-policy'. Unit tests cover the guest's
checks, the clones' policies and reports, the capture and restore
commands, the refusals, a failure that the guest reports, the
services' request counts, and the template's integrity, with each
clone simulated against the scenario's real host services.

doc/ci.md describes the scenario, and doc/usage.md lists it among the
scenarios that need the sandbox-control guest.

Part of #536

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-536-logical-commits-pr-guidelines branch from 50e4117 to bc97118 Compare October 27, 2026 18:30
Copilot AI balanced review requested due to automatic review settings October 27, 2026 18:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The OpenVMM pin changes network isolation before restored vCPUs run, requiring final human review of policy enforcement and restore-barrier guarantees.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@ppenna
Pedro Henrique Penna (ppenna) merged commit b65df47 into dev Oct 27, 2026
45 checks passed
@ppenna
Pedro Henrique Penna (ppenna) deleted the agents/issue-536-logical-commits-pr-guidelines branch October 27, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenVMM prerequisite: fixed NIC with pre-vCPU mutable policy on platform restore

2 participants