Repository navigation
nvx-rs: validate GPT disks against the kernel's partitions in the guest - #715
Closed
Enrique Saurez (esaurez) wants to merge 8 commits into
Closed
Enrique Saurez (esaurez) wants to merge 8 commits into
Enrique Saurez (esaurez) wants to merge 8 commits into
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Concurrent partition recreation can mix sysfs attributes from different generations and incorrectly pass validation.
1 open issue
What changed in this PR
Adds guest-side GPT device validation on top of #673’s reader, supporting future layered-root assembly without changing release guest images.
Changes:
- Validates whole disks and compares GPT entries with kernel partition extents.
- Adds unit tests, privileged guest tests, and a test-program build command.
- Runs the new opt-in scenario across CI backends and documents its usage.
| File | Description |
|---|---|
| scripts/test_nvx_tools.py | Checks guest-test CI wiring. |
| scripts/test_microvm_tests.py | Tests scenario selection and disk fixtures. |
| scripts/test_guest_binaries.py | Tests guest-test program builds. |
| scripts/nvx.py | Adds build-guest-tests. |
| scripts/nvx_tools/microvm_tests.py | Adds GPT fixtures and scenario execution. |
| scripts/nvx_tools/microvm_test_scripts/gpt-device.sh.in | Runs privileged tests inside the guest. |
| scripts/nvx_tools/guest_test_programs.py | Builds static guest-test executables. |
| scripts/nvx_tools/gpt_image.py | Generates GPT and MBR test images. |
| nvx-rs/tests/gpt_device.rs | Adds privileged device integration tests. |
| nvx-rs/src/lib.rs | Documents guest GPT validation. |
| nvx-rs/src/guest/sys/mod.rs | Exposes block-device helpers internally. |
| nvx-rs/src/guest/sys/block.rs | Wraps block-device ioctls. |
| nvx-rs/src/guest/mod.rs | Exports the GPT guest library. |
| nvx-rs/src/guest/gpt/tests.rs | Tests comparison, waiting, and sysfs parsing. |
| nvx-rs/src/guest/gpt/mod.rs | Implements device and partition validation. |
| nvx-rs/src/guest/gpt/error.rs | Defines validation errors. |
| nvx-rs/src/gpt/tests.rs | Shares a standard disk fixture. |
| nvx-rs/src/gpt/mod.rs | Makes test helpers available internally. |
| nvx-rs/README.md | Documents the API and guest tests. |
| doc/usage.md | Documents build and scenario commands. |
| doc/ci.md | Describes guest-library CI coverage. |
| .github/workflows/run-nvx-microvm-tests.yml | Runs the scenario and collects logs. |
| .github/workflows/ci.yml | Builds and uploads guest-test programs. |
Review effort: Balanced
Enrique Saurez (esaurez)
force-pushed
the
esaurez/a3-gpt-device-validation-20261020
branch
from
October 22, 2026 03:09
de3ffde to
42922ea
Compare
Enrique Saurez (esaurez)
force-pushed
the
esaurez/a3-gpt-validation-20261020
branch
from
October 22, 2026 04:31
2a43977 to
7265b64
Compare
Enrique Saurez (esaurez)
force-pushed
the
esaurez/a3-gpt-device-validation-20261020
branch
from
October 22, 2026 04:44
efc2fed to
c85d39d
Compare
Enrique Saurez (esaurez)
force-pushed
the
esaurez/a3-gpt-device-validation-20261020
branch
from
October 22, 2026 16:29
c85d39d to
c34bbd7
Compare
Enrique Saurez (esaurez)
marked this pull request as ready for review
October 22, 2026 16:35
Enrique Saurez (esaurez)
requested a review
from Pedro Henrique Penna (ppenna)
October 22, 2026 16:40
Enrique Saurez (esaurez)
force-pushed
the
esaurez/a3-gpt-device-validation-20261020
branch
from
October 22, 2026 17:43
c34bbd7 to
ba08cc6
Compare
Add nvx::guest::gpt, which checks a whole block device in the guest before its partitions are used. It reads the disk's GPT with nvx::gpt and the caller's Expected values, has the kernel reread the partition table, and waits, up to the caller's timeout, until the partitions that sysfs reports match the table's entries in number, start, and size. The first difference fails with the partition that differs, so a disk that the kernel reads as another table, or that changed after it was read, never reaches the code that would mount its partitions. BlockDevice::new accepts only a whole disk whose device number sysfs confirms, and reads its logical block size and size with the block ioctls that nvx::guest::sys now wraps, along with the rereading of the partition table. The privileged checks run in a guest. build-guest-tests builds nvx-rs/tests/gpt_device.rs as a static test program, and the gpt-device scenario of test-microvm, which runs only when named, shares it with GPT images that the host writes and runs it as root. It checks a valid disk, a rewritten one, late and missing partitions, extra and moved ones, a disk that the kernel reads as an MBR, and a reread while a partition is open. CI builds the program in a job of its own beside the guest artifacts and runs the scenario in every backend's guest and sandbox job. Part of #621. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1b4b569c-a481-43aa-bfd8-c14a9762c371
The kernel may drop a partition and create another under its name between two reads of its sysfs attributes, so reading them by path could combine the start of one partition with the size of the other, and accept an extent that neither has. Open each partition's sysfs directory once, and read its attributes relative to it: once the kernel drops the partition, its directory holds no attributes, so the partition counts as gone and the comparison tries again. Read the disk's own description the same way, and add sys::open_at for both. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1b4b569c-a481-43aa-bfd8-c14a9762c371
nvx::gpt takes and returns nvx::uuid::Uuid in place of its own Guid, so the privileged guest tests build the disk GUID that must fail to match with it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1b4b569c-a481-43aa-bfd8-c14a9762c371
A reading of the kernel's partitions could combine partitions from before and after a reread of the table: one read before the kernel dropped them all, and the others read after it created the new table's. Together they could match the validated table when neither table does. After each reading, read the partitions again, and read each partition again through the sysfs directory that the first reading opened, which no longer describes it once the kernel dropped it. The partitions that pass all existed together, so they come from one reading of the table. A reading that changed counts as a mismatch, and the comparison repeats until the deadline, as for any other. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1b4b569c-a481-43aa-bfd8-c14a9762c371
The confirmation of a reading of the kernel's partitions read every partition a second time, so that a comparison took about two and a half times as long as a single reading. Instead, list the disk's sysfs directories again, which must be those that the first reading opened, and check each of them again through the descriptor that the first reading opened: a partition's number must be the same, which shows that the partition still exists, and another directory must still describe no partition. A reread still can't tear a reading, since it drops the partitions that the first reading holds, and the listing shows a partition that the kernel created under a new name. A directory that described no partition and now describes one shows a partition that the kernel finished creating after the first reading. A partition keeps its number, first byte, and device number while it exists, so only a program that resizes partitions one at a time could change a description meanwhile, as it can after the comparison anyway. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1b4b569c-a481-43aa-bfd8-c14a9762c371
List nvx::gpt, the guest library that validates a GPT disk's block device, the guest GPT device tests, and the two CLI modules of the gpt-device scenario, gpt_image.py and guest_test_programs.py, in the project structure. Part of #621. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1b4b569c-a481-43aa-bfd8-c14a9762c371
"p1" was shorthand that each reader had to decode. Name the partition explicitly throughout: in the API and its errors, the documentation, and the gpt-device scenario. - `Disk::p1` and `Disk::read_p1` become `Disk::first_partition` and `Disk::read_first_partition`, and `GptError::P1TooLarge` becomes `GptError::FirstPartitionTooLarge`. - `ValidatedDisk::p1` becomes `ValidatedDisk::first_partition`. - The scenario passes the first partition's bytes to the guest's tests as `NVX_GPT_DEVICE_FIRST_PARTITION`. Tests and comments name the other partitions by their numbers too. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1b4b569c-a481-43aa-bfd8-c14a9762c371
The kernel creates a disk's partitions from its table when it finds the disk, so at boot they already match the table that the validation reads, and the reread only dropped them and created them again. It took 0.48 to 2.2 ms of the 0.57 to 2.4 ms that a validation took on KVM and WHP. Compare the partitions first, as systemd's partition-table reread leaves partitions that already match alone, and reread only when they differ. A disk whose partitions match therefore also validates while one of them is open, which a reread refuses. The pauses between comparisons, which a wait takes only while the partitions differ, now grow from 1 ms to 4 ms rather than 50 ms: a comparison of a few partitions takes well under a millisecond. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1b4b569c-a481-43aa-bfd8-c14a9762c371
Enrique Saurez (esaurez)
force-pushed
the
esaurez/a3-gpt-device-validation-20261020
branch
from
October 22, 2026 19:02
28b7446 to
e88506f
Compare
This was referenced Oct 22, 2026
Contributor
Author
|
Superseded by #771, which reduces this to a first iteration: it reads a GPT disk's table through the guest's block device and checks the expected disk GUID. The comparison with the kernel's partitions, the reread, and the microVM tests move to #770, with this pull request and its branch, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Adds
nvx::guest::gpt, which validates a GPT disk that the guest's kernel attached as a block device against the partitions that the kernel created from its table. It reads the table withnvx::gpt(#673) and checks that the kernel's partitions are exactly the table's; only when they differ does it ask the kernel to reread the table, and wait until they match. A guest therefore never uses a partition that the table doesn't describe: not when the kernel read another table, such as an MBR, in its place, and not when the table changed after the guest read it. Resolving the disk by its role and mounting its layers come in later pull requests.What changed
BlockDevice::new(&File)accepts only a whole disk: a block device whose sysfs directory,/sys/dev/block/MAJOR:MINOR, describes the same device number and has nopartitionattribute. It reads the device's logical block size (BLKSSZGET) and size (BLKGETSIZE64) through ioctls thatnvx::guest::sysnow wraps, beside the reread (BLKRRPART) andopen_at. The module calls the kernel only throughsys.BlockDevice::validate(&Expected, timeout)runs these steps, which are also public for a caller that needs them apart:read_diskreads and checks the table and the caller'sExpectedvalues through the device, at its logical block size and size, asDisk::readdescribes.await_partitions, without waiting, compares the kernel's partitions, as sysfs lists them, with the table's: one for each entry, with its number, first byte, and size, and no other. Sysfs gives starts and sizes in 512-byte sectors whatever the logical block size, so the comparison is in bytes. The kernel created the partitions from the table when it found the disk, so at boot they match, and the validation ends here.reread_partitionsasks the kernel to reread the table, which it does before it returns, andawait_partitionscompares the partitions again until they match or the timeout passes. systemd's partition-table reread likewise leaves the partitions that already match alone.Mismatch::Missing,Unexpected, orExtent, for the lowest partition number that differs, or asChangedif the partitions changed while they were read. The same wait bounds a caller whose partitions appear late.Changed, and the comparison tries again. In the unoptimized test program, a comparison of a three-partition disk takes about 56 µs on WHP and 41 µs on KVM, against 50 to 57 µs and 30 µs without the confirmation.ValidatedDiskreturns the table and, for each partition, its table entry, the kernel's name of its block device (such asvda2), and its device number:first_partition()for the configuration that the caller reads withDisk::read_first_partition, andlowers()for the layers.nvx::gpt'sDisk::p1,Disk::read_p1, andGptError::P1TooLarge, from nvx-rs: add a strict GPT reader for layered root disks #673, becomeDisk::first_partition,Disk::read_first_partition, andGptError::FirstPartitionTooLarge, so that no name or document uses the shorthand "p1". Nothing else calls them yet.DeviceErrornames the device operation or the sysfs path that failed, the table'sGptError, or theMismatch.CAP_SYS_ADMIN, and fails while any partition of the disk is open. A validation whose partitions already match needs neither, so it also succeeds while a layer is mounted. Nothing is hotplugged: the disk is attached when the VM is created.NotWholeDiskCAP_SYS_ADMIN, for a rereadRunning the privileged tests in a guest
nvx-rs/tests/gpt_device.rsholds nine tests that need root and a guest's disks, so they're ignored on a host.python3 scripts/nvx.py build-guest-testsbuilds them on Linux as one static musl test program,build/guest-tests/gpt-device(1.7 MB, without debug information).gpt-devicescenario ofnvx.py test-microvmis an explicit scenario, likesmp-lapic: it runs only when named, so neither the default suite nor its shards run it. The host writes GPT images withscripts/nvx_tools/gpt_image.py, attaches a read-only disk and a writable scratch disk, and shares the program and the scratch disk's images over virtio-fs. The guest copies the program to/tmpand runs it as root with--ignored. The release initramfs is unchanged.EBUSY, only when the validation must reread the table.guest-test-programs, builds the program beside the guest artifact jobs, under their condition, and uploads it as theguest-test-programsartifact. Only each backend'snvx-microvm-guests-*job, which runs the guest and sandbox steps beside the default suite, waits for it, downloads it intobuild/guest-tests, and runs the scenario; the default suites and the debug-kernel jobs don't. The required status check expects the job whenever the guest artifacts build, and the development release and performance persistence require it, as they do the debug kernel's build, because they accept skipped microVM jobs.gpt_image.pyandguest_test_programs.pyrun only in the microVM jobs and in that job, so the change classifier doesn't count them as inputs of the platform jobs.doc/project-structure.mdlistsnvx::gpt, the guest library, the guest GPT device tests, and the two CLI modules.Guest binaries
The release guest binaries keep their sizes and their instructions, against those built from
devat 58a4a8f:nvx-console-pendingnvx-device-ionvx-exitnvx-ingressnvx-managed-agentnvx-mmio-writenvx-reseednvx-timenvx-time-probeTheir instructions are unchanged. Seven of them differ in one byte, the line of a panic location in
nvx-rs/src/guest/sys/mod.rs, whose lines moved, andnvx-ingressandnvx-timeplace some of their functions in another order. The commits that rename the first partition and compare the partitions first change none of them.Part of #621. Refs #622 and #535.
Validation
This head, e88506f, is on
devat 58a4a8f, after #764 and #713. #713 adds itsguest-artifact-reusejob where this pull request addsguest-test-programs, to the jobs and results that the required status check expects, so both stay, and its test of the guest artifacts that adevpush reuses now also listsguest-test-programsamong the handoffs that no package ships, beside the debug kernel. The Rust gates ran at 838cba70, which differs from this head only in that test, on Windows 11 (MSVC, Rust 1.95.0) and on an Azure Linux 3.0 virtual machine as a non-root user (Rust 1.95.0), innvx-rswithNVX_BUNDLE=skip:cargo fmt --all --checkcargo clippy --locked --all-targets --all-features -- -D warningscargo clippy --locked --lib --no-default-features -- -D warningscargo test --locked --all-features, thencargo test --locked. The new unit tests compare tables with kernel partitions, wait against injected observations (late, changing, at the deadline, with a zero timeout, and failing), read a fake sysfs tree with vanished and malformed attributes and with a partition that is created again between two reads of its attributes, confirm readings against partitions that are dropped, created, or created again after them, against a partition whose attributes the kernel finishes creating after them, from each partial state, and against a reading torn by a reread between two partitions, and check the ioctl request numbers and the splitting of device numbers. Without the second listing, without the check of the partitions' directories, or without that of the other directories, some of the confirmation tests fail.cargo doc --locked --no-deps --all-featureswithRUSTDOCFLAGS=-D warningscargo +1.89 check --locked --all-targets --all-features, the minimum supported Rust versiontests/runtime-adapterin its protocol-only and host-only configurations, and on Linux its musl guest-only one: rustfmt, Clippy, build, tests, and the 1.89 checkaci_edge_sandboxes, which builds onnvx, as its CI checks it, withACI_EDGE_SANDBOXES_AGENT_BUNDLE=skip: rustfmt, Clippy, tests,nvx.py test-agent-bundling, rustdoc, its minimum supported Rust version, and on Linux the macOS checkOn Linux only, at 838cba70:
check-nvx-rsruns itcargo check --locked --all-targets --all-features --target aarch64-apple-darwinpython3 scripts/nvx.py check-guest-binaries,build-guest-binaries --output DIR, andtest-guest-binaries --binaries DIR; and the comparison above against those built fromdevat 58a4a8fpython3 scripts/nvx.py build-guest-testsscripts/test_microvm_tests.py, Specula's tests, which needfcntl,.github/skills/nvx-host-connect/scripts/test_hosts.py, and thenvx.pyhelp commands. All the validate-nvx modules andscripts/test_adversarial.pypassed there at ba08cc6, exceptdev'stest_sleeper_subshell_records_and_reaps_its_sleepers, which requires that nosleep 611.25process exist on the host: it failed in the full run while another checkout ran it, and passed when it ran alone.On Windows, at this head: the validate-nvx test modules,
scripts/test_adversarial.py,test_hosts.py, thenvx.pyhelp commands,python -m pyright --pythonplatform Linuxand--pythonplatform Windows,ruff checkandruff format --check, at the versions inrequirements-dev.txt,python scripts/nvx.py verify, andpython scripts/check_markdown_links.py.On real VMs at 838cba70, with the OpenVMM, kernel, and initramfs of
dev's development release at a6785cd, whose OpenVMM pin this head keeps,nvx.py test-microvm --scenario gpt-devicepassed all nine guest tests three times on each of:--cpu-profile host, since no built-in CPU profile serves its CPU.The duration report times the scenario as its own span, 3.6 s on MSHV and 3.5 s on WHP. The timings of the validation above come from the same scenario at 28b7446, the same code before the rebase, with a test program that reports them, built once without and once with optimization.
The full default
nvx.py test-microvmsuite, its 36 scenarios, passed on Windows/WHP at ba08cc6; the commits since change no scenario butgpt-device, which the suite doesn't run.CI runs the hypervisor matrix at this head.