Repository navigation
microVM: share several host directories through one virtio-fs slot - #127
Merged
Merged
Conversation
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
4 open findings
What changed in this PR
Enables exporting up to 256 host directories through a single microVM virtio-fs slot by introducing an “aggregate” share with a synthetic root and per-child access mode/policy, and updates snapshot/restore + docs accordingly.
Changes:
- Adds aggregate share modeling (children, per-child policy) and schema v8 saved-state/snapshot support.
- Updates microVM slot semantics to a single virtio-fs slot (removes the second slot) and adjusts CLI/resource/profile plumbing.
- Extends policy handling (hidden root + spaces in policy paths) and adds targeted tests + documentation updates.
| File | Description |
|---|---|
| vm/devices/virtio/virtiofs/src/microvm/state.rs | Adds aggregate schema handling and per-volume policy validation during restore validation. |
| vm/devices/virtio/virtiofs/src/microvm/saved_state.rs | Introduces schema version 8 and persisted aggregate-children metadata. |
| vm/devices/virtio/virtiofs/src/microvm/resolver.rs | Resolves new MicrovmAggregate profile/backend pairing and validates child layout match. |
| vm/devices/virtio/virtiofs/src/microvm/profile.rs | Adds aggregate profile/child types and policy-path parsing changes (spaces + hidden root support). |
| vm/devices/virtio/virtiofs/src/microvm/policy_tests.rs | Exposes helpers to share across new aggregate tests; adds tests for hidden root + spaces. |
| vm/devices/virtio/virtiofs/src/microvm/policy.rs | Allows denying the root (empty path) when allowed paths expose parts of it. |
| vm/devices/virtio/virtiofs/src/microvm/mod.rs | Adds aggregate-specific test module. |
| vm/devices/virtio/virtiofs/src/microvm/fs.rs | Implements microVM aggregate attachment creation and per-child volume restore plumbing. |
| vm/devices/virtio/virtiofs/src/microvm/device.rs | Adds microVM aggregate device constructor. |
| vm/devices/virtio/virtiofs/src/microvm/aggregate_fs_tests.rs | New tests covering synthetic root behavior, per-child policies, EXDEV, and snapshot restore. |
| vm/devices/virtio/virtiofs/src/lib.rs | Enforces EXDEV for hard links across aggregated volumes. |
| vm/devices/virtio/virtiofs/src/aggregate.rs | Adds fixed-children aggregate state constructor + microVM-specific root permissions. |
| vm/devices/virtio/virtio_resources/src/fs/microvm.rs | Extends resource profile with MicrovmAggregate and child policy/identity fields. |
| openvmm/openvmm_helpers/src/snapshot/microvm.rs | Records aggregate children in snapshot policy and derives aggregate attachment identity. |
| openvmm/openvmm_entry/src/vm_controller/microvm.rs | Switches snapshot contract construction from “filesystems list” to optional single filesystem source (aggregate-aware). |
| openvmm/openvmm_entry/src/ttrpc/microvm.rs | Explicitly rejects restoring aggregate filesystems via ttrpc API. |
| openvmm/openvmm_entry/src/microvm/restore.rs | Updates restore contract shape to match single optional filesystem source. |
| openvmm/openvmm_entry/src/microvm/network.rs | Updates test harness calls to new filesystem contract signature. |
| openvmm/openvmm_entry/src/microvm/mod.rs | Introduces MicrovmFilesystemHost to represent single/aggregate host roots for snapshot/restore. |
| openvmm/openvmm_entry/src/microvm/launch.rs | Validates private-storage constraints across all aggregate child roots. |
| openvmm/openvmm_entry/src/microvm/config.rs | Plumbs aggregate backend/profile creation into virtio device construction and resources. |
| openvmm/openvmm_entry/src/cli_args/microvm.rs | Reworks CLI: --mount single-use; adds --mount-aggregate + repeatable --mount-child; validates dependencies. |
| openvmm/openvmm_defs/src/microvm.rs | Collapses to one virtio-fs slot; adds aggregate config types, constraints, and bootstrap token virtfs_aggregate=1. |
| Guide/src/user_guide/openvmm/snapshots.md | Documents aggregate snapshot behavior and one-slot compatibility break. |
| Guide/src/reference/openvmm/management/grpc.md | Documents that gRPC cannot attach/restore aggregate filesystems. |
| Guide/src/reference/openvmm/management/cli.md | Updates CLI reference for aggregate mounts and one-slot semantics. |
| Guide/src/reference/devices/virtio/virtio-fs.md | Updates virtio-fs microVM docs for aggregate mode, policies, bootstrap tokens, and compatibility. |
🧠 Review effort: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Part of microsoft/nvx#281. A microVM could attach two host directories, each to its own virtio-fs slot, which used the last free ISA interrupt, IRQ 13. A sandbox that maps directories on several volumes, or with different access modes, needs more than two, and each must be enforced by the host. Remove the second slot and let one slot serve several directories: - One slot. MICROVM_FILESYSTEM_SLOTS keeps only fs:microvm0, with tag microvm, MMIO 0xd0001000, and IRQ 6. The second slot's MMIO window 0xd0008000, IRQ 13, shared-status word, and tag are gone, and so are its discovery token and device. `--mount` may appear once. A snapshot that recorded a second virtio-fs slot fails validation with a clear error, and the contract's microvm_additional_filesystems field is retired. - Aggregate. `--mount-aggregate GUEST_TARGET` attaches a synthetic, read-only root whose named children are host directories, and each `--mount-child NAME,HOST_PATH[,ro|rw]` adds one. The mode follows the last comma, so a host path may contain commas. Names are 1 to 64 ASCII letters, digits, '.', '_', or '-', other than '.' and '..', and unique, and an aggregate has 1 to 256 children. Child roots are canonical plain directories without link components, and must not overlap by path, root identity, or, on Linux, mount source. - Policy. Each child has its own access mode and its own denied, allowed, and writable paths, which must be absolute and apply to the child that contains them, with today's per-kind bounds and a 128-KiB bound across the aggregate. `--mount-owner` applies to every child; with `caller`, every child root must have the same owner. - Device. The new MicrovmAggregate profile builds each child volume as the microVM HostFs device builds its volume: strict paths, the subtree policy, pinned identities, and the root identity check. It applies the configuration's child-name rule, whose 64-byte bound MICROVM_AGGREGATE_MAX_CHILD_NAME mirrors. The synthetic root belongs to the guest's root user with mode 0500, rejects every modification with EROFS, and lists only the children. A rename across children fails with EXDEV. A hard link first requires a writable destination and fails with EROFS otherwise, as Linux's linkat does; a link into a writable directory of another child then fails with EXDEV. A microVM aggregate's children are fixed. - Bootstrap. The guest gets one virtfs_dir/tag/mode triplet, whose mode is rw when any child is, and virtfs_aggregate=1. - Snapshots. An aggregate cannot be captured yet; its attachment identity is the SHA-256 of the children's names and root identities. The ttrpc API still binds only a single directory. Update the Guide's virtio-fs, CLI, gRPC, and snapshot pages. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Part of microsoft/nvx#281. A microVM whose host directories share the virtio-fs slot as an aggregate could not capture a snapshot, so sharing several directories cost the snapshots that two slots allowed. Record and restore the aggregate: - Device state. Aggregate states use the new schema version 8, which records each child's name, root identity, access mode, and denied, allowed, and writable paths, in child order. Inodes already carry their volume ID; restore now rebuilds each child's inodes in that child's volume, rather than requiring volume 0. Node 1 is the synthetic root, which has no saved inode, and each child's root is the inode with the empty alias in its volume. Writable handles are checked against the mode and policy of their child. The synthetic root's directory handle is stateless and its cookies are child positions, so restoring the same children in the same order keeps them valid. Releases that predate aggregates reject version 8. - Contract. SnapshotMicrovmFilesystem gains the aggregate's children, with each child's name, canonical host path, root identity, access mode, and policy paths, and the attachment's identity is the SHA-256 of the children's names and root identities, which validation recomputes. - Restore. A snapshot of an aggregate requires `--mount-aggregate` with the same guest target and the same `--mount-child` names, order, canonical host paths, modes, policy paths, and owner mode. A dormant-slot snapshot still binds only a single `--mount`, and the ttrpc API rejects an aggregate snapshot. Update the Guide's virtio-fs, CLI, gRPC, and snapshot pages. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Part of microsoft/nvx#281. A sandbox that maps single files of a directory exports the directory, but every other entry of it must stay hidden. Denying each sibling cannot hide entries that appear later, and a denied path could not be the export's root. Let `--mount-deny` name the root of a `--mount` or `--mount-child` when `--mount-allow` paths expose parts of it: - Policy. The empty denied path is the root. It is valid only with allowed paths, and it makes the root traverse-only: the guest sees only the allowed paths and the directories that lead to them, and can modify nothing in the root. Any other denied path must then lie inside an allowed path, and writable paths must be visible, as before. - Pinning. The root's object is pinned to the root, so a bind mount or link that reaches it at another name is hidden, as for any traverse-only directory. - Validation. The profile, the machine contract, and the snapshot contract accept the empty denied path, which sorts first, only with allowed paths. The command line resolves a `--mount-deny` that names an export root to it, and still rejects a root as an allowed or writable path. Update the Guide's virtio-fs and CLI pages. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Part of microsoft/nvx#281. Policy paths rejected every whitespace character, so a sandbox could not hide, expose, or narrow writes to a path such as "Program Files/My Tool". Nothing in the virtio-fs server or the snapshot contract needs that rule: policy paths travel as separate command-line arguments and protobuf strings, never as kernel command-line tokens. Accept U+0020 inside a name of a denied, allowed, or writable path, in the profile, the machine contract, and the command line. A name still may not begin or end with a space, which Windows would not preserve, and other whitespace, NUL, backslashes, and colons are still rejected. Update the Guide's virtio-fs page. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ppenna
force-pushed
the
fix/aggregate-microvm-mounts-281
branch
from
October 10, 2026 04:03
14c1fbd to
7bf0ee2
Compare
This was referenced Oct 13, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR lets a microVM share several host directories through its one virtio-fs slot, each with its own access mode and policy, for microsoft/nvx#281.
Why
A microVM could attach at most two host directories, one per virtio-fs slot, and the second slot used IRQ 13, the last free ISA interrupt. NVX's sandbox backend for MXC therefore exported the deepest directory that contained every mapped path. Paths on different volumes, or under nothing narrower than a volume's root, could not be mapped, and read-only mappings and unmapped siblings were protected only by the guest kernel.
Changes
3072d7a91One slot, aggregate sharesfs:microvm1, tagmicrovm1, MMIO0xd0008000, IRQ 13), so--mountmay appear once.--mount-aggregate GUEST_TARGETwith a repeatable--mount-child NAME,HOST_PATH[,ro|rw]shares 1 to 256 host directories as named children of a synthetic root, which is read-only and lists only the children. A child's name is 1 to 64 ASCII letters, digits,.,_, or-, other than.and..; the configuration and the virtio-fs device apply the same rule. Each child has its own mode and its own--mount-deny,--mount-allow, and--mount-writepaths; these are absolute and belong to the child that contains them. Children must not overlap by path, root identity, or, on Linux, mount source. Renames across children fail withEXDEV. Hard links across children do too, except that a link into a read-only destination fails withEROFSfirst, as Linux reports it. The guest gets one bootstrap triplet andvirtfs_aggregate=1.3bb911803Aggregate snapshots--mount, and ttrpc rejects an aggregate snapshot.efdf5df20Hidden share root--mount-denymay name the root of a--mountor--mount-childwhen--mount-allowexposes paths inside it. The root becomes traverse-only and is pinned like any traverse-only directory.7bf0ee28bSpaces in policy paths\, and:are still rejected.The Guide's virtio-fs, CLI, gRPC, and snapshot pages describe the new behavior.
Compatibility
This change is intentionally breaking:
--mountis rejected; use the aggregate instead.microvm1tag and its discovery token are gone.A single
--mountand its snapshots behave as before. IRQ 13 and the0xd0008000window are now free for other devices.Validation
The branch sits on
mainate5bf79776, which includes #126 (IPv6). Each commit builds withcargo check --all-targets.On Windows,
cargo clippy --all-targets,cargo doc --no-deps, andcargo nextest run --profile agentforvirtiofs,virtio_resources,openvmm_defs,openvmm_helpers,openvmm_entry,net_backend_resources,consomme, andnet_consomme: clean, with 644 tests passed.cargo xtask fmt --fixmakes no changes.filesystem: map host paths from any volume, each with its own access microsoft/nvx#459, which pins this head (
7bf0ee28b), passed on local WHP:test-microvmsuite, 39 scenarios including IPv6 egress. Itsfilesystem-sharesscenario covers three shares through the aggregate, a hidden root, a name with spaces, host enforcement through the aggregate root,EXDEVandEROFSfor links across children, rejected boots, snapshot capture, and mismatched, renamed-child, and full restores;aci_edge_sandboxeslifecycle tests, 14 of 14.Its CI covers KVM, MSHV, and WHP. Merge this PR before filesystem: map host paths from any volume, each with its own access microsoft/nvx#459.