nvx-rs: validate GPT disks against the kernel's partitions in the guest

.github/workflows/ci.yml

@@ -609,6 +609,54 @@ jobs:
           if-no-files-found: error
           retention-days: 1
 
+  # The test-only programs that the microVM guest library scenarios, such as
+  # gpt-device, share into an Alpine guest rather than grow its initramfs.
+  # Beside the guest artifacts, so that their build delays only the microVM
+  # guest and sandbox jobs, which run them.
+  guest-test-programs:
+    name: Build guest test programs
+    needs: [openvmm-changes]
+    if: >-
+      ${{
+        always() &&
+        !cancelled() &&
+        (
+          github.event_name != 'pull_request' ||
+          github.event.pull_request.draft == false
+        ) &&
+        (
+          needs.openvmm-changes.result != 'success' ||
+          needs.openvmm-changes.outputs.run-guest-artifacts == 'true'
+        )
+      }}
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v5
+        with:
+          persist-credentials: false
+
+      - name: Build guest test programs
+        shell: bash
+        run: |
+          set -euo pipefail
+          toolchain=$(sed -n 's/^channel = "\([^"]*\)".*/\1/p' rust-toolchain.toml)
+          test -n "${toolchain}"
+          rustup toolchain install "${toolchain}" --profile minimal --no-self-update \
+            --target x86_64-unknown-linux-musl
+          python3 scripts/nvx.py build-guest-tests --output build/guest-tests
+
+      # A one-file artifact keeps its file at its root, so the jobs that run
+      # the programs download it into build/guest-tests.
+      - name: Upload guest test programs
+        uses: actions/upload-artifact@v7
+        with:
+          name: guest-test-programs
+          path: build/guest-tests/gpt-device
+          if-no-files-found: error
+          retention-days: 1
+
   build-openvmm-linux-gnu:
     name: OpenVMM binary / Linux GNU
     needs: [openvmm-changes]
@@ -1444,11 +1492,12 @@ jobs:
   # The steps that follow each backend's default suite, from the public
   # managed sandbox acceptance to the runtime adapter scenarios, depend on
   # none of its results, so a second job runs them beside it, on the same
-  # runner pool. They alone read the Ubuntu and Azure Linux artifacts, so
-  # only this job waits for distro-artifacts.
+  # runner pool. They alone read the Ubuntu and Azure Linux artifacts and the
+  # guest test programs, so only this job waits for distro-artifacts and
+  # guest-test-programs.
   nvx-microvm-guests-kvm:
     name: NVX microVM tests / Linux / KVM / Guests and sandboxes
-    needs: [artifacts, distro-artifacts, build-openvmm-linux-gnu, nvx-cli-linux, openvmm-changes]
+    needs: [artifacts, distro-artifacts, guest-test-programs, build-openvmm-linux-gnu, nvx-cli-linux, openvmm-changes]
     if: >-
       ${{
         always() &&
@@ -1463,6 +1512,7 @@ jobs:
         ) &&
         needs.artifacts.result == 'success' &&
         needs.distro-artifacts.result == 'success' &&
+        needs.guest-test-programs.result == 'success' &&
         needs.build-openvmm-linux-gnu.result == 'success' &&
         needs.nvx-cli-linux.result == 'success' &&
         (
@@ -1483,7 +1533,7 @@ jobs:
 
   nvx-microvm-guests-mshv:
     name: NVX microVM tests / Linux / MSHV / Guests and sandboxes
-    needs: [artifacts, distro-artifacts, build-openvmm-linux-gnu, nvx-cli-linux, openvmm-changes]
+    needs: [artifacts, distro-artifacts, guest-test-programs, build-openvmm-linux-gnu, nvx-cli-linux, openvmm-changes]
     if: >-
       ${{
         always() &&
@@ -1498,6 +1548,7 @@ jobs:
         ) &&
         needs.artifacts.result == 'success' &&
         needs.distro-artifacts.result == 'success' &&
+        needs.guest-test-programs.result == 'success' &&
         needs.build-openvmm-linux-gnu.result == 'success' &&
         needs.nvx-cli-linux.result == 'success' &&
         (
@@ -1518,7 +1569,7 @@ jobs:
 
   nvx-microvm-guests-whp:
     name: NVX microVM tests / Windows / WHP / Guests and sandboxes
-    needs: [artifacts, distro-artifacts, build-openvmm-windows-msvc, nvx-cli-windows, openvmm-changes]
+    needs: [artifacts, distro-artifacts, guest-test-programs, build-openvmm-windows-msvc, nvx-cli-windows, openvmm-changes]
     if: >-
       ${{
         always() &&
@@ -1533,6 +1584,7 @@ jobs:
         ) &&
         needs.artifacts.result == 'success' &&
         needs.distro-artifacts.result == 'success' &&
+        needs.guest-test-programs.result == 'success' &&
         needs.build-openvmm-windows-msvc.result == 'success' &&
         needs.nvx-cli-windows.result == 'success' &&
         (
@@ -1780,6 +1832,7 @@ jobs:
     name: Publish development release
     needs:
       - debug-kernel
+      - guest-test-programs
       - quality
       - aci-edge-sandboxes
       - nvx-rs
@@ -1811,6 +1864,7 @@ jobs:
         github.ref == 'refs/heads/dev' &&
         !cancelled() &&
         needs.debug-kernel.result == 'success' &&
+        needs.guest-test-programs.result == 'success' &&
         needs.quality.result == 'success' &&
         needs.aci-edge-sandboxes.result == 'success' &&
         needs.nvx-rs.result == 'success' &&
@@ -1930,6 +1984,7 @@ jobs:
       - distro-artifacts
       - debug-kernel
       - guest-artifact-reuse
+      - guest-test-programs
       - build-openvmm-linux-gnu
       - build-openvmm-linux-musl
       - build-openvmm-windows-msvc
@@ -1984,6 +2039,7 @@ jobs:
           DISTRO_ARTIFACTS_RESULT: ${{ needs.distro-artifacts.result }}
           DEBUG_KERNEL_RESULT: ${{ needs.debug-kernel.result }}
           GUEST_ARTIFACT_REUSE_RESULT: ${{ needs.guest-artifact-reuse.result }}
+          GUEST_TEST_PROGRAMS_RESULT: ${{ needs.guest-test-programs.result }}
           BUILD_LINUX_GNU_RESULT: ${{ needs.build-openvmm-linux-gnu.result }}
           BUILD_LINUX_MUSL_RESULT: ${{ needs.build-openvmm-linux-musl.result }}
           BUILD_WINDOWS_MSVC_RESULT: ${{ needs.build-openvmm-windows-msvc.result }}
@@ -2019,6 +2075,7 @@ jobs:
     name: Persist performance baseline
     needs:
       - debug-kernel
+      - guest-test-programs
       - nvx-cli-linux
       - nvx-cli-windows
       - openvmm-changes
@@ -2047,6 +2104,7 @@ jobs:
         github.ref == 'refs/heads/dev' &&
         !cancelled() &&
         needs.debug-kernel.result == 'success' &&
+        needs.guest-test-programs.result == 'success' &&
         needs.nvx-cli-linux.result == 'success' &&
         needs.nvx-cli-windows.result == 'success' &&
         (

.github/workflows/run-nvx-microvm-tests.yml

@@ -35,10 +35,10 @@ on:
         description: >-
           Run the steps that follow the default suite instead of it, in a
           second job of the backend beside the default suite: the public
-          managed sandbox acceptance, the Ubuntu and Azure Linux guest tests,
-          the Ubuntu sandbox steps, the aci_edge_sandboxes lifecycle, and the
-          runtime adapter scenarios. Not for the debug-kernel jobs, which skip
-          these steps
+          managed sandbox acceptance, the guest library scenarios, the Ubuntu
+          and Azure Linux guest tests, the Ubuntu sandbox steps, the
+          aci_edge_sandboxes lifecycle, and the runtime adapter scenarios. Not
+          for the debug-kernel jobs, which skip these steps
         required: false
         type: boolean
         default: false
@@ -89,7 +89,8 @@ jobs:
       # qualifies its runner with the kernel and the Alpine initramfs, the
       # default suite boots only those, and the debug-kernel jobs run only
       # Alpine scenarios on the debug kernel, so only the guest and sandbox
-      # steps take the Ubuntu and Azure Linux artifacts.
+      # steps take the Ubuntu and Azure Linux artifacts and the guest test
+      # programs.
       - name: Download guest kernel
         uses: actions/download-artifact@v8
         with:
@@ -137,6 +138,15 @@ jobs:
           name: guest-azurelinux
           path: build
 
+      # The artifact holds the programs at its root, and test-microvm looks
+      # for them in build/guest-tests.
+      - name: Download guest test programs
+        if: ${{ inputs.guests-and-sandboxes && !inputs.debug-kernel }}
+        uses: actions/download-artifact@v8
+        with:
+          name: guest-test-programs
+          path: build/guest-tests
+
       - name: Download OpenVMM executable
         uses: actions/download-artifact@v8
         with:
@@ -215,6 +225,26 @@ jobs:
           --scenario sandbox-lifecycle
           --output-dir "build/test-results/public-exec-${{ inputs.backend }}"
 
+      # The guest library scenarios run the test-only guest programs that the
+      # guest-test-programs job builds.
+      - name: Run guest library scenarios on Linux
+        if: runner.os != 'Windows' && inputs.guests-and-sandboxes && !inputs.debug-kernel
+        shell: bash
+        run: >-
+          python3 scripts/nvx.py test-microvm
+          --backend "${{ inputs.backend }}"
+          --scenario gpt-device
+          --output-dir "build/test-results/guest-library-${{ inputs.backend }}"
+
+      - name: Run guest library scenarios on Windows
+        if: runner.os == 'Windows' && inputs.guests-and-sandboxes && !inputs.debug-kernel
+        shell: powershell
+        run: >-
+          python scripts\nvx.py test-microvm
+          --backend "${{ inputs.backend }}"
+          --scenario gpt-device
+          --output-dir "build/test-results/guest-library-${{ inputs.backend }}"
+
       - name: Run Ubuntu guest smoke tests on Linux
         if: runner.os != 'Windows' && inputs.guests-and-sandboxes && !inputs.debug-kernel
         shell: bash
@@ -1275,6 +1305,7 @@ jobs:
           name: nvx-microvm-tests-${{ inputs.backend }}${{ inputs.debug-kernel && '-debug' || '' }}${{ inputs.guests-and-sandboxes && '-guests' || '' }}${{ inputs.shard && format('-shard-{0}', inputs.shard) || '' }}
           path: |
             build/test-results/public-exec-${{ inputs.backend }}
+            build/test-results/guest-library-${{ inputs.backend }}
             build/test-results/microvm-${{ inputs.backend }}
             build/test-results/microvm-${{ inputs.backend }}-debug
             build/test-results/ubuntu-${{ inputs.backend }}

doc/ci.md

@@ -191,8 +191,9 @@ production kernels leave out. It selects the same-host restore scenarios
 `smp`, `smp-snapshot`, `restore-processors`, `restore-downtime`, and
 `snapshot-tiers`. The jobs skip every other test step, because those boot the
 production kernel, which the `nvx-microvm-guests-{kvm,mshv,whp}` jobs already
-cover: the public managed execution configuration, the Ubuntu and Azure Linux
-guest tests, the Ubuntu sandbox layer and live-share smoke tests, the
+cover: the public managed execution configuration, the guest library
+scenarios, the Ubuntu and Azure Linux guest tests, the Ubuntu sandbox layer
+and live-share smoke tests, the
 caller-owned live-share test, and the `aci_edge_sandboxes` lifecycle test. Any
 RCU stall, soft lockup, or hung task makes the guest's time ABI watcher power
 off with status 194, which fails the run. Every guest those scenarios restore
@@ -294,7 +295,16 @@ all. Ubuntu keys also include the Canonical archive pin, supplemental
 package lock, common guest sources, shared download and guest descriptor
 modules, and converter implementation. Each component is handed to the jobs
 that read it as a workflow artifact of its own, under its build filenames (see
-[Guest artifact handoff](#guest-artifact-handoff)). Each
+[Guest artifact handoff](#guest-artifact-handoff)). Beside them, the
+GitHub-hosted `guest-test-programs` job builds the test-only guest programs
+with `nvx.py build-guest-tests`, and each backend's
+`nvx-microvm-guests-{kvm,mshv,whp}` job runs the guest library scenario
+`gpt-device` with them: as root in an Alpine guest, the `gpt-device` program
+validates a crafted GPT disk through the `nvx-rs` guest library, and rewrites
+a scratch disk so that the kernel's partitions after a reread are missing,
+extra, moved, or late, or come from an MBR that the kernel reads in place of
+the table, all of which the validation must refuse or wait for. Its logs are
+part of the backend's failure logs. Each
 backend also boots the Ubuntu initramfs and runs
 `/sbin/nvx-sandbox-smoke` from the Ubuntu EROFS layer as UID/GID 65534 over a
 fresh ext4 scratch copy, with a 32 MiB `--memory-max` and an 8-process
@@ -490,7 +500,8 @@ GitHub-hosted runners, and `openvmm-changes`, which classifies a push or pull
 request and selects every suite for a full run (see
 [Change classification](#change-classification)). When it finishes, the jobs
 that it selects start: `artifacts`, `distro-artifacts`, and `debug-kernel`
-restore or build the guest artifacts on GitHub-hosted runners (see
+restore or build the guest artifacts, and `guest-test-programs` builds the
+guest test programs, on GitHub-hosted runners (see
 [Guest artifact handoff](#guest-artifact-handoff)), the three OpenVMM binary
 producers restore or build OpenVMM on a KVM, an MSHV, and a WHP runner,
 `build-openvmm-vmm-tests-linux` builds the Linux VMM tests on a KVM runner, and
@@ -500,7 +511,8 @@ Alpine initramfs, and the Linux VMM-test build, and `openvmm-vmm-tests-windows`
 for `artifacts`. Each microVM test and platform job waits for `artifacts`, its
 OpenVMM producer, and the NVX CLI job of its operating system. The microVM
 guest and sandbox jobs and the platform jobs also wait for `distro-artifacts`,
-and the debug-kernel jobs for `debug-kernel`. `Performance regression gate`,
+the microVM guest and sandbox jobs for `guest-test-programs`, and the
+debug-kernel jobs for `debug-kernel`. `Performance regression gate`,
 `Publish development release`, `Persist performance baseline`, and `Required
 status check` wait for the jobs whose results they check. Required handoffs
 between these jobs use same-run workflow artifacts; caches only skip
@@ -706,10 +718,13 @@ each with the
 action, whose `components` input selects what it builds: `artifacts` the
 kernel and the Alpine initramfs, `distro-artifacts` the Ubuntu initramfs, its
 EROFS distro layer, and the sandbox scratch template, together with the Azure
-Linux initramfs, and `debug-kernel` the CI debug kernel. Each job uploads each
-of its components as a one-day workflow artifact of its own. Each job that
-boots or reads guest artifacts downloads, by name and into `build/`, only the
-components that its steps read, and waits only for the jobs that upload them:
+Linux initramfs, and `debug-kernel` the CI debug kernel. Beside them,
+`guest-test-programs` builds the test-only guest programs with
+`nvx.py build-guest-tests`. Each job uploads each of its components as a
+one-day workflow artifact of its own. Each job that boots or reads guest
+artifacts downloads, by name and into the directory of `build/` that holds
+its files, only the components that its steps read, and waits only for the
+jobs that upload them:
 
 | Artifact | Files in `build/` | Producer | Jobs that download it |
 | --- | --- | --- | --- |
@@ -720,14 +735,16 @@ components that its steps read, and waits only for the jobs that upload them:
 | `guest-ubuntu-scratch` | `ubuntu-smoke-scratch.ext4`, the sandbox scratch template | `distro-artifacts` | microVM guest and sandbox jobs |
 | `guest-azurelinux` | `initramfs-azurelinux.cpio.gz` and its `.packages.json` | `distro-artifacts` | microVM guest and sandbox jobs, platform jobs |
 | `guest-debug-kernel` | `vmlinux-debug`, `vmlinux-debug.config`, `vmlinux-debug.provenance.json` | `debug-kernel` | debug-kernel jobs |
+| `guest-test-programs` | `guest-tests/gpt-device` | `guest-test-programs` | microVM guest and sandbox jobs |
 
 The OpenVMM VMM tests boot only the kernel and the Alpine initramfs, and their
 Linux build boots no guest, so it downloads no component. The
 debug-kernel jobs run the Alpine scenarios on the debug kernel and qualify
 their runner, through the doctor's preflight (H3), with the kernel and the
 Alpine initramfs. The microVM default suites boot only those two, and the
-microVM guest and sandbox jobs boot every guest and run sandboxes from the
-Ubuntu distro layer over copies of the scratch template. The platform
+microVM guest and sandbox jobs boot every guest, run sandboxes from the
+Ubuntu distro layer over copies of the scratch template, and run the guest
+test programs in the Alpine guest. The platform
 jobs boot the kernel and the Alpine initramfs to qualify their runner and to
 benchmark, and package every guest image but the scratch template into their
 release. A missing artifact fails its download step, so no job runs without a
@@ -944,8 +961,9 @@ other path selects each suite that takes it as an input:
   of the guest binaries, which `GuestRustBuildConstants.SOURCE_PATTERN`
   matches, so a change to its host side or tests runs the NVX microVM tests
   and the platform jobs but not the VMM tests.
-- `run-guest-artifacts` enables `artifacts`, `distro-artifacts`, and
-  `debug-kernel`, which build the guest artifacts and the CI debug kernel, and
+- `run-guest-artifacts` enables `artifacts`, `distro-artifacts`,
+  `debug-kernel`, and `guest-test-programs`, which build the guest artifacts,
+  the CI debug kernel, and the guest test programs, and
   `run-openvmm-binaries` the three OpenVMM binary producers. Neither has a
   use of its own, so the classifier sets each exactly when a job that uses its
   artifacts runs: the guest artifacts for the OpenVMM VMM tests, the NVX
@@ -954,17 +972,18 @@ other path selects each suite that takes it as an input:
   of each job that uses it. Every input of the VMM tests is also an input of
   the microVM tests, which read the Ubuntu and Azure Linux images of
   `distro-artifacts`, so the three guest artifact jobs share the output. The
-  debug kernel is built with the guest artifacts, as before, because the
-  development release and performance persistence require its build too.
+  debug kernel and the guest test programs are built with the guest
+  artifacts, because the development release and performance persistence
+  require their builds too.
 
 Every CLI module is an input of the microVM tests, and of the platform jobs,
 unless none of that suite's jobs runs it, so a new module selects both until
 the classifier excludes it. `microvm_tests.py` and the modules that only it,
-`nvx.py sandbox`, or `nvx.py test-aci-edge-sandboxes` use, with the
-`microvm_test_scripts/` that it runs in guests and the partition of its
-[shards](#microvm-test-shards), `microvm_shards.json`, are inputs of the
-microVM tests alone, and `performance.py` is an input of the platform jobs
-alone. The
+`nvx.py sandbox`, `nvx.py test-aci-edge-sandboxes`, or
+`nvx.py build-guest-tests` use, with the `microvm_test_scripts/` that it runs
+in guests and the partition of its [shards](#microvm-test-shards),
+`microvm_shards.json`, are inputs of the microVM tests alone, and
+`performance.py` is an input of the platform jobs alone. The
 modules that both kinds of job run, such as `benchmark.py`, which
 `microvm_tests.py` imports, `doctor.py`, which every runner validation runs,
 and `release.py`, whose `verify` command the OpenVMM builds run, select both.