Skip to content

filesystem: share single host files without their directories #475

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Pedro Henrique Penna (ppenna) merged 6 commits into from
Oct 11, 2026
Merged

filesystem: share single host files without their directories #475

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4afaa6a
openvmm: pin single-file aggregate virtio-fs children
ppenna Oct 10, 2026
ad2aa25
doc: index the OpenVMM commits that the file-children pin adds
ppenna Oct 10, 2026
d8acaf0
aci_edge_sandboxes: export mapped files without their directories
ppenna Oct 10, 2026
a787963
run, sandbox: share single host files at their targets
ppenna Oct 10, 2026
4db152f
test-microvm: share single files through the aggregate virtio-fs device
ppenna Oct 10, 2026
32d3cb3
ci: share single files in the concurrent sandbox shares smoke test
ppenna Oct 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 73 additions & 21 deletions .github/workflows/run-nvx-microvm-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -453,6 +453,7 @@ jobs:
scratch="build/ubuntu-shares-${{ inputs.backend }}.ext4"
workspace="build/ubuntu-shares-workspace-${{ inputs.backend }}"
toolcache="build/ubuntu-shares-toolcache-${{ inputs.backend }}"
files="build/ubuntu-shares-files-${{ inputs.backend }}"
outside="$(dirname "${workspace}")/nvx-outside-marker"
logs="build/test-results/ubuntu-${{ inputs.backend }}"
# Keep the control socket path short.
Expand All @@ -466,19 +467,29 @@ jobs:
}
trap cleanup EXIT
prepare_shares() {
rm -rf "${workspace}" "${toolcache}"
rm -rf "${workspace}" "${toolcache}" "${files}"
mkdir -p "${workspace}/nvx-denied" "${workspace}/nvx-links" \
"${toolcache}/nvx-denied"
"${toolcache}/nvx-denied" "${files}/config" "${files}/results"
printf 'host-to-guest\n' >"${workspace}/nvx-host-marker"
printf 'host-to-guest\n' >"${toolcache}/nvx-host-marker"
printf 'secret\n' >"${workspace}/nvx-denied/secret"
printf 'secret\n' >"${toolcache}/nvx-denied/secret"
printf 'host-outside\n' >"${outside}"
# Each shared file sits beside a file that no share exposes. Every
# guest identity may write both shared files, so only the read-only
# share can refuse a write.
printf 'host-to-guest\n' >"${files}/config/settings.json"
printf 'secret\n' >"${files}/config/secret.json"
printf 'host-to-guest\n' >"${files}/results/output.txt"
printf 'other\n' >"${files}/results/other.txt"
chmod 0666 "${files}/config/settings.json" "${files}/results/output.txt"
chmod 0777 "${workspace}" "${workspace}/nvx-links"
cp --reflink=auto build/ubuntu-smoke-scratch.ext4 "${scratch}"
}
# The guest writes to the read-write share, and neither it nor a link
# in the read-write share changes the read-only share.
# in the read-write share changes the read-only share. It rewrites
# only the read-write file, and the files beside the shared files
# stay unchanged.
check_shares() {
test "$(cat "${workspace}/nvx-guest-marker")" = guest-to-host
test -d "${workspace}/nvx-guest-directory"
Expand All @@ -488,14 +499,24 @@ jobs:
test "$(cat "${toolcache}/nvx-denied/secret")" = secret
test "$(cat "${workspace}/nvx-denied/secret")" = secret
test "$(cat "${outside}")" = host-outside
test "$(ls -A "${files}/config" | sort | tr '\n' ' ')" = "secret.json settings.json "
test "$(ls -A "${files}/results" | sort | tr '\n' ' ')" = "other.txt output.txt "
test "$(cat "${files}/config/settings.json")" = host-to-guest
test "$(cat "${files}/config/secret.json")" = secret
test "$(cat "${files}/results/output.txt")" = guest-to-host
test "$(cat "${files}/results/other.txt")" = other
}
share_options=(
--mount "/workspace,${workspace},rw" --mount-deny nvx-denied
--mount "/opt/hostedtoolcache,${toolcache},ro" --mount-deny nvx-denied
--mount "/config/settings.json,${files}/config/settings.json,ro"
--mount "/results/output.txt,${files}/results/output.txt,rw"
)
smoke_arguments=(
--entrypoint /sbin/nvx-sandbox-smoke
--arg /workspace --arg rw --arg /opt/hostedtoolcache --arg ro
--arg /config/settings.json --arg file-ro
--arg /results/output.txt --arg file-rw
)
prepare_shares
python3 scripts/nvx.py sandbox \
Expand All @@ -507,7 +528,7 @@ jobs:
--workload-user 65534:65534 \
--memory-mib 256
check_shares
# A managed sandbox persists both shares and reattaches them on start.
# A managed sandbox persists every share and reattaches them on start.
prepare_shares
python3 scripts/nvx.py sandbox provision \
--state-dir "${state}" \
Expand All @@ -523,21 +544,28 @@ jobs:
with open(sys.argv[1], encoding="utf-8") as stream:
config = json.load(stream)
shares = [
(mount["guest_target"], mount["access"], mount["denied_paths"])
(
mount["guest_target"],
mount["access"],
mount["denied_paths"],
mount["kind"],
)
for mount in config["mounts"]
]
expected = [
("/workspace", "rw", ["nvx-denied"]),
("/opt/hostedtoolcache", "ro", ["nvx-denied"]),
("/workspace", "rw", ["nvx-denied"], "directory"),
("/opt/hostedtoolcache", "ro", ["nvx-denied"], "directory"),
("/config/settings.json", "ro", [], "file"),
("/results/output.txt", "rw", [], "file"),
]
if config["format"] != 4 or "mount" in config or shares != expected:
sys.exit(f"managed sandbox did not persist both shares: {config}")
if config["format"] != 6 or "mount" in config or shares != expected:
sys.exit(f"managed sandbox did not persist every share: {config}")
EOF
python3 scripts/nvx.py sandbox start --state-dir "${state}"
python3 scripts/nvx.py sandbox exec --state-dir "${state}" \
"${smoke_arguments[@]}"
python3 scripts/nvx.py sandbox stop --state-dir "${state}"
# A clean ext4 journal shows that stop unmounted both shares, the
# A clean ext4 journal shows that stop unmounted every share, the
# overlay, and scratch in order.
python3 - "${state}/outcome.json" "${scratch}" <<'EOF'
import json
Expand All @@ -556,7 +584,7 @@ jobs:
EOF
check_shares
python3 scripts/nvx.py sandbox deprovision --state-dir "${state}"
rm -rf "${workspace}" "${toolcache}" "${scratch}" "${outside}"
rm -rf "${workspace}" "${toolcache}" "${files}" "${scratch}" "${outside}"

- name: Run Ubuntu sandbox share access policy on Linux
if: runner.os != 'Windows' && !inputs.debug-kernel
Expand Down Expand Up @@ -901,32 +929,46 @@ jobs:
$scratch = Join-Path $PWD "build\ubuntu-shares-${{ inputs.backend }}.ext4"
$workspace = Join-Path $PWD "build\ubuntu-shares-workspace-${{ inputs.backend }}"
$toolcache = Join-Path $PWD "build\ubuntu-shares-toolcache-${{ inputs.backend }}"
$files = Join-Path $PWD "build\ubuntu-shares-files-${{ inputs.backend }}"
$state = Join-Path $PWD "build\ubuntu-shares-state-${{ inputs.backend }}"
$outside = Join-Path (Split-Path $workspace) "nvx-outside-marker"
$logs = "build\test-results\ubuntu-${{ inputs.backend }}"
$settings = Join-Path $files "config\settings.json"
$output = Join-Path $files "results\output.txt"
$shareOptions = @(
"--mount", "/workspace,$workspace,rw", "--mount-deny", "nvx-denied",
"--mount", "/opt/hostedtoolcache,$toolcache,ro", "--mount-deny", "nvx-denied"
"--mount", "/opt/hostedtoolcache,$toolcache,ro", "--mount-deny", "nvx-denied",
"--mount", "/config/settings.json,$settings,ro",
"--mount", "/results/output.txt,$output,rw"
)
$smokeArguments = @(
"--entrypoint", "/sbin/nvx-sandbox-smoke",
"--arg", "/workspace", "--arg", "rw",
"--arg", "/opt/hostedtoolcache", "--arg", "ro"
"--arg", "/opt/hostedtoolcache", "--arg", "ro",
"--arg", "/config/settings.json", "--arg", "file-ro",
"--arg", "/results/output.txt", "--arg", "file-rw"
)
function Initialize-Shares {
foreach ($path in $workspace, $toolcache, $state) {
foreach ($path in $workspace, $toolcache, $files, $state) {
if (Test-Path $path) { Remove-Item -Recurse -Force $path }
}
New-Item -ItemType Directory -Force (Join-Path $workspace "nvx-denied"), (Join-Path $workspace "nvx-links"), (Join-Path $toolcache "nvx-denied") | Out-Null
New-Item -ItemType Directory -Force (Join-Path $workspace "nvx-denied"), (Join-Path $workspace "nvx-links"), (Join-Path $toolcache "nvx-denied"), (Join-Path $files "config"), (Join-Path $files "results") | Out-Null
foreach ($share in $workspace, $toolcache) {
[IO.File]::WriteAllText((Join-Path $share "nvx-host-marker"), "host-to-guest`n")
[IO.File]::WriteAllText((Join-Path $share "nvx-denied\secret"), "secret`n")
}
[IO.File]::WriteAllText($outside, "host-outside`n")
# Each shared file sits beside a file that no share exposes.
[IO.File]::WriteAllText($settings, "host-to-guest`n")
[IO.File]::WriteAllText((Join-Path $files "config\secret.json"), "secret`n")
[IO.File]::WriteAllText($output, "host-to-guest`n")
[IO.File]::WriteAllText((Join-Path $files "results\other.txt"), "other`n")
Copy-Item -Force build\ubuntu-smoke-scratch.ext4 $scratch
}
# The guest writes to the read-write share, and neither it nor a link
# in the read-write share changes the read-only share.
# in the read-write share changes the read-only share. It rewrites
# only the read-write file, and the files beside the shared files
# stay unchanged.
function Test-Shares {
if ([IO.File]::ReadAllText((Join-Path $workspace "nvx-guest-marker")) -ne "guest-to-host`n") {
throw "guest write did not reach the read-write share"
Expand All @@ -944,16 +986,26 @@ jobs:
}
if ([IO.File]::ReadAllText((Join-Path $toolcache "nvx-host-marker")) -ne "host-to-guest`n") { throw "guest changed the read-only share" }
if ([IO.File]::ReadAllText($outside) -ne "host-outside`n") { throw "a guest link reached host data outside the shares" }
foreach ($directory in "config", "results") {
$entries = (Get-ChildItem -Force -Name (Join-Path $files $directory) | Sort-Object) -join " "
$expected = @{ config = "secret.json settings.json"; results = "other.txt output.txt" }[$directory]
if ($entries -ne $expected) { throw "guest changed the directory of a shared file: $entries" }
}
if ([IO.File]::ReadAllText($settings) -ne "host-to-guest`n") { throw "guest changed the read-only file" }
if ([IO.File]::ReadAllText($output) -ne "guest-to-host`n") { throw "guest write did not reach the read-write file" }
if ([IO.File]::ReadAllText((Join-Path $files "config\secret.json")) -ne "secret`n") { throw "guest changed a file beside a shared file" }
if ([IO.File]::ReadAllText((Join-Path $files "results\other.txt")) -ne "other`n") { throw "guest changed a file beside a shared file" }
}
Initialize-Shares
Invoke-Nvx sandbox --hypervisor "${{ inputs.backend }}" --layer $layer --scratch $scratch @shareOptions @smokeArguments --workload-user 65534:65534 --memory-mib 256
Test-Shares
# A managed sandbox persists both shares and reattaches them on start.
# A managed sandbox persists every share and reattaches them on start.
Initialize-Shares
Invoke-Nvx sandbox provision --state-dir $state --hypervisor "${{ inputs.backend }}" --layer $layer --scratch $scratch @shareOptions --workload-user 65534:65534
$config = Get-Content -Raw (Join-Path $state "config.json") | ConvertFrom-Json
if ($config.format -ne 4 -or $config.mounts.Count -ne 2 -or $config.mounts[0].guest_target -ne "/workspace" -or $config.mounts[1].access -ne "ro") {
throw "managed sandbox did not persist both shares: $($config | ConvertTo-Json -Compress -Depth 5)"
$shares = ($config.mounts | ForEach-Object { "$($_.guest_target),$($_.access),$($_.kind)" }) -join " "
if ($config.format -ne 6 -or $shares -ne "/workspace,rw,directory /opt/hostedtoolcache,ro,directory /config/settings.json,ro,file /results/output.txt,rw,file") {
throw "managed sandbox did not persist every share: $($config | ConvertTo-Json -Compress -Depth 5)"
}
try {
Invoke-Nvx sandbox start --state-dir $state
Expand All @@ -971,7 +1023,7 @@ jobs:
if ($outcome.operation -ne "managed" -or $outcome.category -ne "success" -or $outcome.status_code -ne 0) {
throw "managed sandbox outcome is invalid: $($outcome | ConvertTo-Json -Compress)"
}
# A clean ext4 journal shows that stop unmounted both shares, the
# A clean ext4 journal shows that stop unmounted every share, the
# overlay, and scratch in order.
$image = [IO.File]::OpenRead($scratch)
try {
Expand All @@ -986,7 +1038,7 @@ jobs:
}
Test-Shares
Invoke-Nvx sandbox deprovision --state-dir $state
Remove-Item -Recurse -Force $workspace, $toolcache, $scratch, $outside
Remove-Item -Recurse -Force $workspace, $toolcache, $files, $scratch, $outside

- name: Run Ubuntu sandbox share access policy on Windows
if: runner.os == 'Windows' && !inputs.debug-kernel
Expand Down
60 changes: 33 additions & 27 deletions aci_edge_sandboxes/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -622,29 +622,33 @@ translation, for example to turn a host working directory into a

- OpenVMM offers one virtio-fs device, which the backend attaches as an
aggregate: a synthetic, read-only root that lists one exported host
directory per child, mounted at `/run/nvx/hostfs/root`, a guest directory
that only the guest's root can enter. The exported directories are the
outermost mapped directories and the parents of the outermost mapped files,
so mapped paths may lie anywhere, on any volume. The guest agent then
bind-mounts each mapped path at its guest path, read-only or read-write, so
workloads see only the mapped paths.
- The host enforces each mapping's access. An exported directory is
read-write only if it holds a read-write mapping, and OpenVMM then limits
writes to its read-write mappings, so only a read-only path inside a
read-write mapping is read-only through the guest's bind mount alone. The
parent of mapped files exposes only its mapped paths: OpenVMM hides
everything else in it, so unmapped siblings are not exported at all.
Mapping a whole volume (`/` or `C:\`) is refused; map directories inside it.
So is mapping a file directly in a volume's root, such as `C:\notes.txt` or
`/notes.txt`, because its parent would be the volume's root; move the file
into a directory, and map the file or the directory.
directory or file per child, mounted at `/run/nvx/hostfs/root`, a guest
directory that only the guest's root can enter. The children are the
outermost mapped directories and the mapped files outside them, so mapped
paths may lie anywhere, on any volume. The guest agent then bind-mounts
each mapped path at its guest path, read-only or read-write, so workloads
see only the mapped paths.
- The host enforces each mapping's access. A child is read-write only if it
holds a read-write mapping, and OpenVMM then limits writes to its
read-write mappings, so only a read-only path inside a read-write mapping
is read-only through the guest's bind mount alone. OpenVMM exports a mapped
file alone: the guest reaches neither its siblings nor its directory, which
it cannot list, inspect, or change, so a read-write file can be rewritten
in place but not replaced, renamed, or removed. OpenVMM follows the file's
name, so the guest sees the file that a host program, such as an editor,
saves over it, and the file's accesses fail with `EACCES` while anything
but a regular file holds the name. Start refuses to run a sandbox whose
mapped file a directory replaced, or whose mapped directory a file
replaced, and OpenVMM refuses such a child too, so a file grant never
becomes a directory grant. Mapping a whole volume (`/` or `C:\`) is
refused; map directories inside it.
- Denied paths inside an exported directory are hidden by OpenVMM itself: they
are absent from listings and inaccessible through any name, except a host
hard link to a file inside a denied directory, which OpenVMM cannot tell
from any other file. Below its exported directory, a path that OpenVMM hides
or singles out, such as a mapped file or a read-write path inside a
read-only one, must not contain colons, backslashes, links, or whitespace
other than spaces, and no name in it may begin or end with a space.
or singles out, such as a read-write path inside a read-only one, must not
contain colons, backslashes, links, or whitespace other than spaces, and no
name in it may begin or end with a space.
- Mapped paths must exist and be directories or regular files. A path listed
both read-only and read-write is mapped read-only. A denied path must not
contain a mapped path, and a denied path that does not exist yet must not
Expand All @@ -653,16 +657,18 @@ translation, for example to turn a host working directory into a
read-only file inside a read-write directory is refused. Mapping over the
guest's own system directories (`/usr`, `/etc`, and so on) is refused.
Violations are reported as `policy_validation`.
- OpenVMM exposes at most 128 paths in each exported directory, so mapping
more than 128 individual files from one directory fails at provision with
`policy_validation`; map the directory instead. Likewise, OpenVMM hides at
most 128 denied paths, and singles out at most 128 read-write paths inside
read-only ones, in each exported directory.
- A sandbox maps at most 4096 paths from at most 256 exported directories.
OpenVMM's command line must fit the 32,767 characters of a Windows process
- A sandbox maps at most 4096 paths through at most 256 children. A policy
whose mapped directories, and mapped files outside them, would need more
children is refused rather than served through the directories that hold
its files, which would grant more than the files; map directories that
contain the files instead. OpenVMM hides at most 128 denied paths, and
singles out at most 128 read-write paths inside read-only ones, in each
exported directory.
- OpenVMM's command line must fit the 32,767 characters of a Windows process
command line, with 1,024 held in reserve, on every host, which typically
leaves room for several hundred paths. Provision rejects a policy beyond
these limits with `policy_validation`.
these limits with `policy_validation`, as it does one whose children would
export one object twice, such as two mapped host hard links to one file.
- Provision records the identity of every mapped and denied object inside a
read-write mapping, and start refuses to run if one changed, so a workload
cannot rename a denied or read-only object and leave a decoy at its path for
Expand Down
Loading
Loading