Skip to content

runtime-adapter: prove two clone-restore invariants from the host side - #1068

Merged
Pedro Henrique Penna (ppenna) merged 2 commits into
devfrom
agents/fix-issue-965-draft-pr-review-comments
Oct 31, 2026
Merged

Pedro Henrique Penna (ppenna) merged 2 commits into
devfrom
agents/fix-issue-965-draft-pr-review-comments

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #965.

Adds host-side checks for the two clone-restore invariants that #954's vm_clone checked only from the guest, each with a positive control, on KVM, MSHV, and WHP:

  1. OpenVMM reads no host input before the guest's repair acknowledges the restore.
  2. A clone's restore reads no large content of its disks, the scratch included.

What changed

nvx (nvx-rs): two small public host primitives

The adapter forbids unsafe and reaches the platform only through nvx's public API, so the checks need these:

  • Process::bytes_read and process::own_bytes_read: the bytes that a process read through read calls, cached reads included. This is rchar of /proc/<pid>/io on Linux, opened before the process's identity is re-checked so the count is the process's own. On Linux it also includes the reads of children the process reaped, since the kernel adds a reaped child's I/O to its parent's. On Windows it is ReadTransferCount of GetProcessIoCounters, through the handle's existing PROCESS_QUERY_LIMITED_INFORMATION access, and counts the process's own reads only.
  • Transport::unread, a provided method that defaults to None: what the endpoint has not read of the transport's writes. On Linux it is SIOCOUTQ of the Unix socket, through a safe wrapper in host::sys: the kernel's charge for each write's buffers until the peer reads each buffer whole. A partial read of a buffer leaves the charge unchanged, so only data written a byte at a time shows the read of every byte. On Windows it is None, since a named pipe tells its writer nothing.

Runtime adapter

  • Test-only repair delay. create/refresh take RUNTIME_ADAPTER_REPAIR_DELAY (positive milliseconds, up to 30 s; anything else is a usage error). They write the delay into the capture's initramfs at /etc/runtime-adapter-repair-delay and record it as the class's one invariant setting, runtime-adapter.repair-delay-ms, so a delayed template is a class of its own. In template mode the guest reads the file before the capture, and each clone sleeps that long before clone::repair. A malformed file ends the template's VM with status 20. The activation probe reports the wait as repair-delay.
  • stage and a successful start print read-bytes=BYTES as they exit. run's preflight prints nothing new, so run's exact outputs are unchanged.
  • New scenario openvmm_reads_none_of_the_host_s_input_until_the_repair_acknowledges_the_restore. It starts a clone of an 8-second-delayed template through NVX's public API with the host program's CloneOrder, over a Transport wrapper that watches the session's attachment and writes it one byte per write. It checks that:
    • the attachment is written within half the delay of the launch;
    • the broker's first answer and the attachment come only after the earliest possible acknowledgement (launch + delay). The broker answers an authenticated host at once with Wait, so an early read would show here;
    • on Linux, SIOCOUTQ rises by one constant, positive step with each byte written, so no byte was read during the write. It then stays at that charge at every sample (every 50 ms) until the earliest acknowledgement, and is zero once the broker answered. Each byte has a buffer of its own, so a read of any prefix would lower the charge;
    • a local positive control for that check: a listener in the test process, connected through nvx::host::session::connect, receives an attachment written the same way. The charge falls by one step after it reads one byte, and reaches zero after the rest;
    • the positive control for the gate: a reattach_clone attachment, which no gate holds, is answered within a quarter of the delay;
    • the guest reports repair-delay = 8 s, and its timeline stays in order;
    • the in-process start writes the host program's CloneRecord, so probe and stop work as after runtime-adapter-host start.
  • Extended a_clone_reads_no_more_of_its_disks_than_its_root_needs with host-side counts:
    • staging's own count covers at least the image, which it hashed. This is the positive control for the host program's counter;
    • the start reads less than 1/8 of the image;
    • the VMM reads less than 1/8 of the disks whose files back the clone, by the time the clone is ready;
    • after the guest reads each disk to its end, the VMM's count rises by at least what the guest asked of those files on Windows. On Linux it rises by less than 1/8 of it, a documented limitation explained below.
  • README, scenarios.txt, module docs, and unit tests updated. That includes a refusal test for the variable, the template archive entry, the delay file's format, the invariant on the class, and stage's output.

Deviations from the issue (please review)

  • The delay travels in the template's initramfs, not its kernel command line. OpenVMM's capture of a platform snapshot refuses any command-line token outside the platform tier's allowlist (platform_command_line_token_is_invariant), as the adapter's template::boot does. NVX also renders a capture's command line itself. The host program already writes the capture's initramfs, so the delay rides there, and its class invariant keeps delayed and undelayed templates apart.
  • The gate scenario runs the restore in-process, as refuses_early already does, so the test owns the transport. Measuring through runtime-adapter-host start would need another process's socket.
  • Linux cannot observe the VMM's disk reads with rchar. OpenVMM serves sandbox blocks through disk_blockdevice, which reads via io_uring, and rchar doesn't count io_uring reads. On Linux the VMM check therefore covers its read calls only, which would still catch a hash or copy through them. The positive control there pins down this limitation instead of passing vacuously. It is documented in tests/runtime-adapter/README.md, as the issue's done-condition allows. On Windows the positive control is exact.

Validation

  • Windows (local): nvx-rs clippy (all targets/features; also cross-checked for x86_64-unknown-linux-gnu) and the full --all-features suite. Adapter clippy for the host config and for the guest config (musl), the host-config tests, test-runtime-adapter --check-only, the adapter-suite Python tests (README ↔ scenarios.txt), and the markdown link check.
  • nvx-mshv-test (Azure Linux 3, kernel 6.18): every check-nvx-rs step except MSRV/macOS/bundling. That is nvx-rs fmt, three clippy variants, the --all-features and no-default-features tests, and docs, plus the adapter's protocol/host/guest configurations with clippy, tests, and docs.
  • Both new scenarios pass on KVM (nvx-kvm-test, kernel 6.6), MSHV (nvx-mshv-test), and WHP (nvx-whp-test, Windows Server 2025), with release artifacts v0.1.0-dev.525c8bed3101. That release has no artifact changes since, and the guest agent was built from this branch.
  • After the review fixes (head ed5e24bcf): local clippy (Windows and Linux cross-check), docs with warnings denied, fmt, --check-only, the adapter-suite Python tests, and the markdown link check. The nvx-rs test a_unix_transport_reports_what_its_peer_has_not_read passes on kernels 6.6 and 6.18. Both scenarios pass again on KVM, MSHV, and WHP.

Measurements from instrumented runs (the instrumentation is not committed). KVM's attachment rows come from a run at ed5e24bcf, which writes the attachment a byte at a time. The other rows come from a run at the first revision, which wrote it whole.

KVM WHP
Attachment written after launch 57 ms 74 ms
Broker's first answer after launch 8.08 s 8.17 s
SIOCOUTQ while written / held / after answer 0, then +768 per byte to 58,368 (76 B) / 58,368 at all 144 samples / 0 n/a (named pipe)
Local control: whole / after a 1-byte read / after the rest 49,152 (64 B) / 48,384 / 0 n/a
Reattach answered after its last byte 7 µs 83 µs
Staging read / image 59,983,035 / 59,940,864 59,975,604 / 59,940,864
Start read ~54 KB ~309 KB
VMM read by ready ~31 KB ~300–330 KB
VMM rise after full disk read / guest asked (scratch) 288 B / 127 MB 127,001,732 / 127,000,576

Copilot AI max review requested due to automatic review settings October 31, 2026 02:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The attachment test can pass despite an early prefix read, and the read-counter documentation omits Linux’s inclusion of reads from reaped child processes.

2 open issues
What changed in this PR

Adds host-side checks for runtime-adapter clone restores, extending #954 toward #965’s requirements for input gating and limited disk reads on KVM, MSHV, and WHP.

Changes:

  • Adds process read counters and optional transport queue reporting to NVX’s public host API.
  • Introduces test-only repair delays and a host-observed input-gating scenario.
  • Extends disk-read checks with host measurements and positive controls, documenting platform limitations.
File Description
tests/​runtime-adapter/​tests/​vm_clone.rs Adds input-gating checks and host-side disk-read measurements.
tests/​runtime-adapter/​tests/​template_capture.rs Tests delay validation and template class separation.
tests/​runtime-adapter/​tests/​staged_images.rs Checks staging’s read-counter output.
tests/​runtime-adapter/​tests/​initramfs.rs Tests delay-file archive entries and initramfs output.
tests/​runtime-adapter/​tests/​guest_session.rs Verifies activation reports no delay by default.
tests/​runtime-adapter/​tests/​common/​mod.rs Allows template creation with environment overrides.
tests/​runtime-adapter/​src/​worker/​capture.rs Delays clone repair when configured.
tests/​runtime-adapter/​src/​worker/​activation.rs Reports repair delay in activation facts.
tests/​runtime-adapter/​src/​worker.rs Passes repair delay into clone session state.
tests/​runtime-adapter/​src/​template.rs Defines and parses the repair-delay file format.
tests/​runtime-adapter/​src/​initramfs.rs Includes optional repair-delay files in archives.
tests/​runtime-adapter/​src/​capture.rs Reads delay settings and records the class invariant.
tests/​runtime-adapter/​src/​bin/​runtime-adapter-host.rs Configures delayed captures and prints read counters.
tests/​runtime-adapter/​src/​bin/​runtime-adapter-guest.rs Loads and validates the template’s delay file.
tests/​runtime-adapter/​scenarios.txt Registers the new validation scenarios.
tests/​runtime-adapter/​README.md Documents delays, host observations, and platform limits.
nvx-rs/​src/​host/​sys/​socket.rs Adds a safe wrapper for Linux socket queue reporting.
nvx-rs/​src/​host/​sys/​mod.rs Exports the socket queue helper internally.
nvx-rs/​src/​host/​process.rs Exposes process read counters and adds tests.
nvx-rs/​src/​host/​platform/​windows.rs Implements counters using Windows process handles.
nvx-rs/​src/​host/​platform/​other.rs Provides unsupported-platform counter behavior.
nvx-rs/​src/​host/​platform/​mod.rs Adds optional queue reporting to Transport.
nvx-rs/​src/​host/​platform/​linux.rs Implements Linux counters and transport queue reporting.
nvx-rs/​README.md Documents the new public host primitives.

Review effort: Max

Comment thread nvx-rs/src/host/process.rs Resolved
Comment thread tests/runtime-adapter/tests/vm_clone.rs Resolved
Process::bytes_read and own_bytes_read report the bytes that a process
read through the operating system's read calls, the cache's included:
rchar of /proc/<pid>/io on Linux, read only while the process still runs
with its reference's identity, and ReadTransferCount of
GetProcessIoCounters on Windows. Transport::unread reports what the
endpoint has not read of the transport's writes, where the platform
tells: SIOCOUTQ of the Unix socket on Linux, and None on Windows, whose
named pipes tell their writer nothing of the kind.

An adapter, which reaches NVX through its public API alone, needs both
to check from the host that a clone's restore reads no large content,
and that OpenVMM reads no host input before the guest's repair
acknowledges the restore.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A template may delay the repair of each of its clones, for tests: the
host program's create and refresh take the delay from
RUNTIME_ADAPTER_REPAIR_DELAY, write it into the capture's initramfs, at
/etc/runtime-adapter-repair-delay, and name it as the class's one
invariant setting. The kernel command line cannot carry it, since
OpenVMM's capture of a platform snapshot refuses any parameter beyond
the tier's allowlist. The guest reads the file before the capture, waits
that long before each clone's repair, and reports the wait in its
activation probe.

vm_clone then checks from the host that OpenVMM reads none of the host's
input until the repair acknowledges the restore: it starts a clone of a
delayed template through NVX's public API, over transports that watch
the session's attachment, which the broker answers only after the delay,
and which waits unread at the control socket meanwhile on Linux, as
SIOCOUTQ reports it. A named pipe tells its writer nothing of the kind,
so on Windows the check is the timing alone. A reattach, which no gate
holds, is answered at once, as the positive control.

The scenario of a clone's disk reads also counts, from the host, the
bytes that the start and the clone's VMM read, which stage and start now
print as they exit: the start reads less than an eighth of the image,
which the staging's hash reads whole, and the VMM less than an eighth of
the disks' files once the clone is ready. The guest's read of each disk
to its end raises the VMM's count by what it asked of the files on
Windows, while on Linux, where OpenVMM reads them through io_uring,
rchar misses those reads, as the README documents.

Fixes #965

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/fix-issue-965-draft-pr-review-comments branch from ad41db2 to ed5e24b Compare October 31, 2026 02:32
Copilot AI balanced review requested due to automatic review settings October 31, 2026 02:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the host-side clone-restore checks and supporting APIs.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

@ppenna
Pedro Henrique Penna (ppenna) marked this pull request as ready for review October 31, 2026 02:37
@ppenna
Pedro Henrique Penna (ppenna) added this pull request to the merge queue Oct 31, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 31, 2026
@ppenna
Pedro Henrique Penna (ppenna) added this pull request to the merge queue Oct 31, 2026
Merged via the queue into dev with commit 2d4cd20 Oct 31, 2026
83 checks passed
@ppenna
Pedro Henrique Penna (ppenna) deleted the agents/fix-issue-965-draft-pr-review-comments branch October 31, 2026 05:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

runtime-adapter: prove two clone-restore invariants from the host side

2 participants