From 7f4de94cf75293ec0f59da733ffc6e5eaace1ada Mon Sep 17 00:00:00 2001 From: Enrique Saurez Date: Sun, 25 Oct 2026 19:11:08 -0700 Subject: [PATCH 1/3] nvx-rs: restore clones of GPT disks A clone's plan refused a GPT disk as its image before any VMM started: the kernel of a clone's guest had created the block devices of the partitions of the template's placeholder, and no clone rebuilt them. The guest of a GPT clone now asks the kernel to reread the disk's partition table before it assembles the root, so the plan takes a GPT disk as it takes an EROFS layer. The record of a GPT clone names the disk: the identity of its binding, the GUID that the adapter's plan expects of it, and the bound of its first partition, which CloneRequest gains, as first_partition_bound, beside the journal policy. An EROFS layer has no partitions, so the record of a clone of one leaves the bound out. A bound of 0 is a setting that no record holds, and the plan refuses it before it opens anything, as it refuses the request's other Activate settings, with the evidence InvalidActivate. InvalidFormat now names only an image that is not of the format and identity that its binding names. The plan checks a GPT disk as it checks an EROFS layer, from its metadata alone: the check of its binding, and the geometry of the template's placeholder, whatever the placeholder's format. As at a cold launch, the guest reads and checks the partition table with nvx::gpt before it assembles the root, so the host reads none of the disk, and hashes nothing. The test adapter's clone requests bound a GPT disk's first partition by 64 KiB, the default of a launch's guest. Part of #845 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 80932e09-af82-4b79-b417-70a241303cda --- nvx-rs/README.md | 2 +- nvx-rs/src/host/template/clone.rs | 75 +++++---- nvx-rs/src/host/template/clone/failure.rs | 3 +- nvx-rs/src/host/template/clone/record.rs | 32 ++-- nvx-rs/src/host/template/clone/tests.rs | 150 +++++++++++++++--- tests/runtime-adapter/README.md | 28 ++-- .../src/bin/runtime-adapter-host.rs | 9 +- tests/runtime-adapter/src/clone.rs | 14 +- 8 files changed, 220 insertions(+), 93 deletions(-) diff --git a/nvx-rs/README.md b/nvx-rs/README.md index 5f8048ed9..dfa10032f 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -94,7 +94,7 @@ nvx-rs/ | `nvx::host::nvxs` | `HostConnection`: the host's role of the NVXS session over a platform transport, on which `nvx::host::session` and other host applications run their protocols. It attaches with the launch capability, waiting while the broker asks it to, sends application bytes in as many DATA records as they take, and returns each DATA payload from the guest, with deadlines; a receive that times out keeps a partial record | | `nvx::host::process` | Verified process references. A `ProcessRef` names one process by its ID, its start time, and the `BootId` of the host boot during which it started. `Process::open` checks a reference and holds its process through a pidfd on Linux or a process handle on Windows, whether or not the caller started it, so that waits sleep until the process exits and termination never reaches a process that reuses its ID. A reference from an earlier boot names an exited process, and a process whose identity this user cannot check, such as another user's, makes `Process::open` fail rather than report an exit. Linux hosts without pidfds poll the reference instead, and cannot terminate the process | | `nvx::host::supervisor` | The supervisor of a sandbox's VMM, beneath the CLI, `Instance`, and external orchestrators, which keep the configuration, the authorization, and the application protocol. `launch` runs the store's launch steps under the sandbox's lock, as a launch whose `LaunchId` the caller reserved beforehand, a fresh one for each launch, or that a clone's lease reserves: it writes the journal's marker, which names the launch, spawns OpenVMM with the claimed log, records its `ProcessRef` right after, and returns a `Starting` launch that the caller commits once the VMM serves its control endpoint, or abandons. A launch that fails never leaves an unrecorded VMM running. `VmmCommand::inherit` lists the descriptors (Linux) or handles (Windows) that the VMM inherits under their numbers, besides its standard streams, and it inherits nothing else of the caller's: on Linux, they must be close-on-exec, a flag that only the VMM's own process clears; on Windows, they join the VMM's explicit handle list. `Vmm::stop` runs one graceful step of the caller's and waits for the VMM's exit event until a deadline, and takes a bounded forced step through the VMM's verified identity only on request. `Vmm::terminate` takes that forced step alone, within the caller's bound, for a VM whose guest the caller must not ask anything, such as one of another runtime identity. `reattach` recovers an interrupted launch, and holds the recorded VMM again after the orchestrator restarts within the same host boot: `Live`, `Mismatched` with another runtime identity, `Exited`, or `Indeterminate`. A clone's launch takes its locks in one order, the template store's, then the sandbox's, which guards its clone binding and launch journal, then the scratch image's claim, and holds none for the VM's lifetime; its lease lasts through crashes and reattaches, until a release ends it once a stop, a termination, or a reattach has verified the VM's exit | -| `nvx::host::template` | The metadata of templates, captures of a microVM from which independent clones are restored. `TemplatePlan::new` lowers an adapter's `SandboxPlan` into the plan of a template's capture, without I/O: the placeholder `distro`, which its `DistroBinding` binds, and whose length is the capacity of every clone's image, over a dormant scratch device, the sparse file `SCRATCH_NAME` in the capture's transaction directory, with the dormant filesystem slot, and no NIC or one under the template's policy, which allows neither ingress nor host-loopback access both ways; `TemplateResources`, with every processor online at boot, and RAM and a `--memory-capacity` in blocks of `MEMORY_BLOCK_SIZE`; the kernel command line `nvx_sandbox=1`, so that the capture's whole command line holds tokens of OpenVMM's platform allowlist alone, and no tenant data, which the tests check with a copy of the allowlist's source; and the capture's options, `--snapshot-tier platform`, a `--snapshot-destination`, `SNAPSHOT_NAME` in the transaction directory, and `--snapshot-block-identity generation` with the capture's bound-block `StorageGeneration`, so that the capture hashes no block device. `TemplateManifest::read` reads the `manifest.bin` that OpenVMM wrote for a capture, at most `MAX_MANIFEST_LEN` bytes of a regular file that is not a link or other reparse point, and refuses another snapshot format version, an unknown, mistyped, or repeated field, truncated bytes, and a capture that is not a template's: a platform-tier capture for clones, with a read-only `distro` whose identity is unbound, a fresh scratch, and the dormant filesystem slot. `CaptureClass` holds the invariant inputs of a capture, known before it: the runtime, the guest's build, the backend, the requested CPU profile, the processors and RAM, the geometries of the disks, the network, and the adapter's other `Invariants`. `TemplateKey::new` checks that the capture made the machine that its class requested, and derives the template's compatibility key, from metadata alone: the runtime and contract versions through which its clones are restored, and the backend, CPU profile and effective CPUID, topology, rates, resources, kernel command line, disks, and devices of its machine. A capture's own metadata, such as its time anchor and host, stays out of the key. A key and a class each have one canonical, versioned encoding, whose domain-separated SHA-256 digest is its identity, a `KeyIdentity`, which converts to and from the `nvx::activation::Identity` by which a clone's activation record names its template's key, or a `ClassIdentity`; the [conformance vectors](tests/vectors/README.md) of version 1, `tests/vectors/template-key-v1.txt`, pin the key's encoding and identity. `Store` keeps the generations of templates that a host published, apart from every sandbox's store, at a location that the adapter chooses: a private directory with an identity of its own, a `StoreId`, the immutable payload of each generation, its key's canonical encoding beside the snapshot that OpenVMM published, and the versioned `Index` of them, which lists each generation with the identities of its key and capture class and its own metadata, such as its time contract. `Store::create` creates a store, or opens the one that its directory holds, crash-safely and once between concurrent creations, and refuses a directory with foreign entries; `Store::open` opens one. Its records are JSON in one format, replaced atomically, and records of another format fail rather than being migrated. Its directories must be private to the current user, and it refuses links and, on Windows, other reparse points among its directories and files. Four namespaces stay apart: the `nvx::activation::StoreGeneration`, the host's publication ID, which the index assigns in the order of publication and never twice, and by which a clone's activation record names its template; the bound-block `StorageGeneration` of a capture, which belongs to the storage owner; the time lineage counter of the time contract; and the VM generation ID, which the store never records. `TimeContract::eligibility` decides, apart from compatibility, whether a clone may restore a capture at a host's time, a `HostTime`, which `HostTime::now` samples as OpenVMM does: within the boot of the capture, by the host's monotonic clock, and otherwise by UTC, the downtime must not be negative, and the interval by which a restore advances the clone's VM time at most `MAX_DOWNTIME_NS`, 30 days. `Store::select` selects the latest eligible generation of a key that no tombstone quarantines. `Store::acquire` leases a generation to a clone in one transaction, under the store's lock and then the sandbox's: it checks the generation's eligibility, writes the lease's record in the store and the clone binding, `clone.json`, beside the sandbox's launch journal, and returns the lease with the sandbox's lock, the only one under which the caller launches the clone, once, as the launch that the lease reserves, `Lease::launch`. A lease never expires: `Store::release` ends it once the sandbox records no VM and no launch. `ClonePlan::new` plans a clone under the lock that acquired its lease, before the launch that the lease reserves, from metadata alone: the leased generation, its key, which must speak the current contracts, the boot and control contracts of the runtime that captured it included, and its snapshot; a network exactly when the template has a NIC; a memory target in whole blocks of `MEMORY_BLOCK_SIZE` from the template's RAM to its capacity; the clone's image, which the check of its `DistroBinding` opens; its upper, either the caller's scratch image, which the check of its `ScratchBinding` claims, and which nothing formats, copies, or reads, or for an upper in RAM a fresh sparse file, `clone-scratch.raw` in the sandbox's directory, which the release removes; and a fresh capability. It builds the clone's activation record, `ClonePlan::activation`, which the clone's guest applies in place of the configuration that the template captured, from the request's `WorkloadView`, overlay mode, and journal policy, and from the checked inputs: a restore section with a fresh transaction ID of 16 random bytes and the lease's `TemplateReference`; the image, an EROFS layer, with its binding's identity and the UUID that the adapter's plan expects of it, since a GPT image is refused; a tmpfs, or the scratch image's binding identity with the journal policy; no export; the template NIC's policy identity, as its key records it, unpublished listeners, and the guest's proxy URL; and the template's processors and the memory target. A binding's identity is the SHA-256 digest of a domain and of the binding's serde form, never of a file's content. The image and the scratch device must present the `BlockGeometry` of the template's, which a restore compares, and be, at the paths by which OpenVMM opens them, the regular files that the checks found, which are not links or other reparse points, and which no one else may change. Neither the image nor a scratch image may be a file that the sandbox's store owns, such as `clone-scratch.raw`, its log, or its records, by any path, since the store's transitions replace or remove them, and a link or a directory under one of the store's names is refused. `ClonePlan::launch` then launches the clone as the launch that its lease reserves, once it has checked that the sandbox's clone binding records the lease, and that the sandbox records no failure of the clone, which fails once under its lease, and returns a `Restoring`: it writes the clone's activation record in its canonical encoding, `clone-activation.bin`, from which a caller resends it unchanged; it listens for the VMM's restore-ready event, at the socket `restore.sock` on Linux, and at a named pipe of a random name on Windows; and it runs `supervisor::launch` with OpenVMM's restore of the clone's devices, `--restore-snapshot` with the generation's snapshot in place, which no clone copies or changes, so that clones of one generation restore it at once, `--restore-memory` with the memory target, and `--restore-ready-path`, under the template's backend, CPU profile, and processors, with a boot console that discards its output, as cold boots have, and the control console, but no kernel, initramfs, command line, or guest address, which the snapshot fixes. On Linux, where OpenVMM requires a restore's control socket in the snapshot's parent, the clone's VMM listens in the generation's payload, at `.sock`, named after the launch that the lease reserves, whose path must be shorter than 108 bytes with or without the payload's links resolved; on Windows, it listens at a named pipe, as every launch does. The VMM inherits the claim of a scratch image, on Windows through the image's guard, once the launch has closed the claim's handle, which OpenVMM's open of the image would refuse. A payload holds the sockets of its clones only while they lease it: the VMM removes its socket as it exits, a stop or a recovery removes the one that an abrupt exit leaves, and the release makes the socket's absence durable before the lease ends, and removes the record and `restore.sock`, as it removes `clone-scratch.raw`. `Restoring::activate` then runs the clone's restore sequence through the adapter's framing of its control session, a `CloneControl`, by a restore-to-Ready deadline that runs from the launch, `RESTORE_TO_READY_TIMEOUT`, 60 seconds by default: it reads the restore-ready event as soon as it comes, refusing another process's connection and other bytes, since OpenVMM on Windows waits until the host has read it; only then connects to the clone's control endpoint, whose console bounds the host's authentication from the connection on; attaches with the launch's capability; sends the record, read again from `clone-activation.bin`, which must still be byte for byte the launch's, in the adapter's Activate, as the session's first request; and checks that Ready reports the record's configuration identity and transaction ID. It replaces a session that the adapter lost, as `CloneControl::lost` tells, with a new one that sends the same record, in at most `MAX_ACTIVATION_SESSIONS` sessions by the same deadline, and retries nothing else. `Activated::start` then starts the sandbox by a Ready-to-Start deadline from the check of Ready, `READY_TO_START_TIMEOUT`, 30 seconds by default, never sending the start again, and `Started::commit` commits the launch, which writes its runtime record. A failure ends the launch as `Starting::abandon` does; a crash before the commit leaves the launch's journal, whose recovery ends the VMM, while the lease lasts until the release, and a crash after it leaves a VM that reattaches, and keeps its lease while it runs. A failure of a clone is a `CloneFailure`, whose `Evidence` attributes it to an `Outcome`, at a `Stage`, once the clone got as far as its `Progress`: the request, for a denied authorization, a stale binding, invalid targets, format, settings, Activate, or mappings, or an image, a scratch, or a root that the template cannot serve, and for a supplied file of the wrong identity, length, or geometry, which the host's staging takes instead once the caller proves it, `CloneFailure::with_staging_proven`; the host, for an endpoint that cannot be bound, resource exhaustion, a host that cannot restore the template, another runtime, or a failure of its own operations; unknown, for a lost status, a timeout, a reset, a forced kill, or a broken protocol; the template, for a malformed generation, or a defect of its capture or repair that a diagnosis attributes to it, the only outcome that quarantines the generation, through its tombstone, `CloneFailure::quarantine`; and the repair stage, for a status of the clone's repair alone. OpenVMM's outcome report gives the status with which the guest ended the clone's VM, which `status_evidence` attributes, and which prevails over what the host or the caller saw, but not over evidence of the template, which shows the provenance that a status alone leaves to establish: a status of the repair at any point, and one of the activation before the start was sent. A VMM that exits before its restore-ready event, as OpenVMM does once it cannot restore the snapshot, after it closed the event's endpoint, which the sequence then gives ten seconds to exit, leaves the line with which OpenVMM reported its fatal error in its log, which the failure's message keeps, and whose time ABI code, if it says that the host cannot present the template's CPU profile, TSC, or LAPIC timer, attributes the failure to the host. From the launch on, a failure ends the clone's VMM through its verified identity, and the sandbox keeps the clone's progress log, `clone-progress.log`, which records each step from the spawn to the commit, after a first line that names the launch and the boot of the host, a few lines that it appends without a sync, and the record of its failure, `clone-failure.json`, JSON in format `FAILURE_FORMAT`, which it writes before the teardown, with the teardown pending, so that a crash of the caller during the teardown leaves it for a reattach to finish, and again once the teardown ended, with the stage, the evidence and the status, the original outcome, the transaction, the generation, the progress, and the teardown: no VM, an exit or a termination that the host verified, or an unresolved one, which keeps the launch's journal or runtime record, and the lease; a release removes both, and a sandbox that boots cold writes neither. A step or a check of the caller's own that fails once the clone is ready, or once its sandbox started, fails the clone with the caller's evidence through `Activated::fail` or `Started::fail`, which end the launch and record the failure as the sequence's own failures do. `reattach_clone` takes a clone over after the caller of its launch restarted: it reattaches by the identity of its VMM's process, checks a committed clone in a new session, in which the clone answers the byte-equal Activate of `clone-activation.bin` with the Ready that it answered once, and ends a clone that another runtime identity launched, or that fails the check, through its verified identity; it records the failure of a launch that a crash interrupted, but not the journal that a commit may leave beside its runtime record, classifies a VM that exited by its outcome report, records a failure before it ends a VMM or clears the runtime record of one that exited, finishes a failure that a crash left pending as its teardown would have recorded it, keeping its transaction, resolves the teardown of an earlier failure with what the VMM left as it ended, its outcome report and OpenVMM's fatal error, attributes a status of the guest's as one that a workload may have given once the clone's progress log is lost, damaged, or was written during another boot of the host, whose crash may have lost its last lines, and reads nothing of the template store, so that a tombstone or a retirement of the generation leaves the clone its lease. `CloneRecords::read` reads what the sandbox keeps of a clone, for the operations that list, inspect, or remove clones. `Store::tombstone` quarantines a generation, which takes no new lease while its leases keep working, and `Store::retire` deletes it once its leases have drained, crash-safely, removing the records of leases that no clone binding names any more. `Store::create_template` serves a `Capture`, an adapter's plan for a capture class with the placeholder's binding and the runtime's artifacts, with a generation of its class that the store holds and that is eligible at the host's time, which it reads once it holds the class's slot, `Created::Existing`, or captures one, `Created::Captured`, while `Store::refresh_template` always captures, and leaves older generations and their leases untouched. A capture runs in its class's slot under `captures/`, a sandbox store whose lock serializes the class's captures, which comes before the store's lock, and which keeps the log and outcome report of the VMM of the class's latest capture: in a private transaction on the store's volume, which is also the sandbox store of its VM, since OpenVMM requires the control socket of a VM that captures a snapshot, on Linux, beside the snapshot's destination, it creates the dormant scratch, checks the placeholder through its binding, and both block devices at their paths, regular files that are not links or other reparse points, that belong to the user or an administrator, and that no one else may change, and launches the template's VM through `supervisor::launch` from the transaction, holding no handle that could write either file, since OpenVMM opens them itself by path, as it does a restore's. Once the VMM has exited, it checks the outcome report, the snapshot's manifest and artifacts, and the key, and publishes the transaction in two commits, a rename into `generations/` that replaces nothing, then the index. A capture that fails publishes nothing, and `Store::recover_captures` ends the captures that a crash interrupted, publishing those that were checked, and retiring the others | +| `nvx::host::template` | The metadata of templates, captures of a microVM from which independent clones are restored. `TemplatePlan::new` lowers an adapter's `SandboxPlan` into the plan of a template's capture, without I/O: the placeholder `distro`, which its `DistroBinding` binds, and whose length is the capacity of every clone's image, over a dormant scratch device, the sparse file `SCRATCH_NAME` in the capture's transaction directory, with the dormant filesystem slot, and no NIC or one under the template's policy, which allows neither ingress nor host-loopback access both ways; `TemplateResources`, with every processor online at boot, and RAM and a `--memory-capacity` in blocks of `MEMORY_BLOCK_SIZE`; the kernel command line `nvx_sandbox=1`, so that the capture's whole command line holds tokens of OpenVMM's platform allowlist alone, and no tenant data, which the tests check with a copy of the allowlist's source; and the capture's options, `--snapshot-tier platform`, a `--snapshot-destination`, `SNAPSHOT_NAME` in the transaction directory, and `--snapshot-block-identity generation` with the capture's bound-block `StorageGeneration`, so that the capture hashes no block device. `TemplateManifest::read` reads the `manifest.bin` that OpenVMM wrote for a capture, at most `MAX_MANIFEST_LEN` bytes of a regular file that is not a link or other reparse point, and refuses another snapshot format version, an unknown, mistyped, or repeated field, truncated bytes, and a capture that is not a template's: a platform-tier capture for clones, with a read-only `distro` whose identity is unbound, a fresh scratch, and the dormant filesystem slot. `CaptureClass` holds the invariant inputs of a capture, known before it: the runtime, the guest's build, the backend, the requested CPU profile, the processors and RAM, the geometries of the disks, the network, and the adapter's other `Invariants`. `TemplateKey::new` checks that the capture made the machine that its class requested, and derives the template's compatibility key, from metadata alone: the runtime and contract versions through which its clones are restored, and the backend, CPU profile and effective CPUID, topology, rates, resources, kernel command line, disks, and devices of its machine. A capture's own metadata, such as its time anchor and host, stays out of the key. A key and a class each have one canonical, versioned encoding, whose domain-separated SHA-256 digest is its identity, a `KeyIdentity`, which converts to and from the `nvx::activation::Identity` by which a clone's activation record names its template's key, or a `ClassIdentity`; the [conformance vectors](tests/vectors/README.md) of version 1, `tests/vectors/template-key-v1.txt`, pin the key's encoding and identity. `Store` keeps the generations of templates that a host published, apart from every sandbox's store, at a location that the adapter chooses: a private directory with an identity of its own, a `StoreId`, the immutable payload of each generation, its key's canonical encoding beside the snapshot that OpenVMM published, and the versioned `Index` of them, which lists each generation with the identities of its key and capture class and its own metadata, such as its time contract. `Store::create` creates a store, or opens the one that its directory holds, crash-safely and once between concurrent creations, and refuses a directory with foreign entries; `Store::open` opens one. Its records are JSON in one format, replaced atomically, and records of another format fail rather than being migrated. Its directories must be private to the current user, and it refuses links and, on Windows, other reparse points among its directories and files. Four namespaces stay apart: the `nvx::activation::StoreGeneration`, the host's publication ID, which the index assigns in the order of publication and never twice, and by which a clone's activation record names its template; the bound-block `StorageGeneration` of a capture, which belongs to the storage owner; the time lineage counter of the time contract; and the VM generation ID, which the store never records. `TimeContract::eligibility` decides, apart from compatibility, whether a clone may restore a capture at a host's time, a `HostTime`, which `HostTime::now` samples as OpenVMM does: within the boot of the capture, by the host's monotonic clock, and otherwise by UTC, the downtime must not be negative, and the interval by which a restore advances the clone's VM time at most `MAX_DOWNTIME_NS`, 30 days. `Store::select` selects the latest eligible generation of a key that no tombstone quarantines. `Store::acquire` leases a generation to a clone in one transaction, under the store's lock and then the sandbox's: it checks the generation's eligibility, writes the lease's record in the store and the clone binding, `clone.json`, beside the sandbox's launch journal, and returns the lease with the sandbox's lock, the only one under which the caller launches the clone, once, as the launch that the lease reserves, `Lease::launch`. A lease never expires: `Store::release` ends it once the sandbox records no VM and no launch. `ClonePlan::new` plans a clone under the lock that acquired its lease, before the launch that the lease reserves, from metadata alone: the leased generation, its key, which must speak the current contracts, the boot and control contracts of the runtime that captured it included, and its snapshot; a network exactly when the template has a NIC; a memory target in whole blocks of `MEMORY_BLOCK_SIZE` from the template's RAM to its capacity; the clone's image, which the check of its `DistroBinding` opens; its upper, either the caller's scratch image, which the check of its `ScratchBinding` claims, and which nothing formats, copies, or reads, or for an upper in RAM a fresh sparse file, `clone-scratch.raw` in the sandbox's directory, which the release removes; and a fresh capability. It builds the clone's activation record, `ClonePlan::activation`, which the clone's guest applies in place of the configuration that the template captured, from the request's `WorkloadView`, overlay mode, journal policy, and bound of a GPT disk's first partition, and from the checked inputs: a restore section with a fresh transaction ID of 16 random bytes and the lease's `TemplateReference`; the image, an EROFS layer or a GPT disk, as its binding names it, with its binding's identity and the filesystem UUID or disk GUID that the adapter's plan expects of it, and for a GPT disk the bound of its first partition, whose partition table the guest checks with `nvx::gpt` as at a cold launch; a tmpfs, or the scratch image's binding identity with the journal policy; no export; the template NIC's policy identity, as its key records it, unpublished listeners, and the guest's proxy URL; and the template's processors and the memory target. A binding's identity is the SHA-256 digest of a domain and of the binding's serde form, never of a file's content. The image and the scratch device must present the `BlockGeometry` of the template's, which a restore compares, and be, at the paths by which OpenVMM opens them, the regular files that the checks found, which are not links or other reparse points, and which no one else may change. Neither the image nor a scratch image may be a file that the sandbox's store owns, such as `clone-scratch.raw`, its log, or its records, by any path, since the store's transitions replace or remove them, and a link or a directory under one of the store's names is refused. `ClonePlan::launch` then launches the clone as the launch that its lease reserves, once it has checked that the sandbox's clone binding records the lease, and that the sandbox records no failure of the clone, which fails once under its lease, and returns a `Restoring`: it writes the clone's activation record in its canonical encoding, `clone-activation.bin`, from which a caller resends it unchanged; it listens for the VMM's restore-ready event, at the socket `restore.sock` on Linux, and at a named pipe of a random name on Windows; and it runs `supervisor::launch` with OpenVMM's restore of the clone's devices, `--restore-snapshot` with the generation's snapshot in place, which no clone copies or changes, so that clones of one generation restore it at once, `--restore-memory` with the memory target, and `--restore-ready-path`, under the template's backend, CPU profile, and processors, with a boot console that discards its output, as cold boots have, and the control console, but no kernel, initramfs, command line, or guest address, which the snapshot fixes. On Linux, where OpenVMM requires a restore's control socket in the snapshot's parent, the clone's VMM listens in the generation's payload, at `.sock`, named after the launch that the lease reserves, whose path must be shorter than 108 bytes with or without the payload's links resolved; on Windows, it listens at a named pipe, as every launch does. The VMM inherits the claim of a scratch image, on Windows through the image's guard, once the launch has closed the claim's handle, which OpenVMM's open of the image would refuse. A payload holds the sockets of its clones only while they lease it: the VMM removes its socket as it exits, a stop or a recovery removes the one that an abrupt exit leaves, and the release makes the socket's absence durable before the lease ends, and removes the record and `restore.sock`, as it removes `clone-scratch.raw`. `Restoring::activate` then runs the clone's restore sequence through the adapter's framing of its control session, a `CloneControl`, by a restore-to-Ready deadline that runs from the launch, `RESTORE_TO_READY_TIMEOUT`, 60 seconds by default: it reads the restore-ready event as soon as it comes, refusing another process's connection and other bytes, since OpenVMM on Windows waits until the host has read it; only then connects to the clone's control endpoint, whose console bounds the host's authentication from the connection on; attaches with the launch's capability; sends the record, read again from `clone-activation.bin`, which must still be byte for byte the launch's, in the adapter's Activate, as the session's first request; and checks that Ready reports the record's configuration identity and transaction ID. It replaces a session that the adapter lost, as `CloneControl::lost` tells, with a new one that sends the same record, in at most `MAX_ACTIVATION_SESSIONS` sessions by the same deadline, and retries nothing else. `Activated::start` then starts the sandbox by a Ready-to-Start deadline from the check of Ready, `READY_TO_START_TIMEOUT`, 30 seconds by default, never sending the start again, and `Started::commit` commits the launch, which writes its runtime record. A failure ends the launch as `Starting::abandon` does; a crash before the commit leaves the launch's journal, whose recovery ends the VMM, while the lease lasts until the release, and a crash after it leaves a VM that reattaches, and keeps its lease while it runs. A failure of a clone is a `CloneFailure`, whose `Evidence` attributes it to an `Outcome`, at a `Stage`, once the clone got as far as its `Progress`: the request, for a denied authorization, a stale binding, invalid targets, format, settings, Activate, or mappings, or an image, a scratch, or a root that the template cannot serve, and for a supplied file of the wrong identity, length, or geometry, which the host's staging takes instead once the caller proves it, `CloneFailure::with_staging_proven`; the host, for an endpoint that cannot be bound, resource exhaustion, a host that cannot restore the template, another runtime, or a failure of its own operations; unknown, for a lost status, a timeout, a reset, a forced kill, or a broken protocol; the template, for a malformed generation, or a defect of its capture or repair that a diagnosis attributes to it, the only outcome that quarantines the generation, through its tombstone, `CloneFailure::quarantine`; and the repair stage, for a status of the clone's repair alone. OpenVMM's outcome report gives the status with which the guest ended the clone's VM, which `status_evidence` attributes, and which prevails over what the host or the caller saw, but not over evidence of the template, which shows the provenance that a status alone leaves to establish: a status of the repair at any point, and one of the activation before the start was sent. A VMM that exits before its restore-ready event, as OpenVMM does once it cannot restore the snapshot, after it closed the event's endpoint, which the sequence then gives ten seconds to exit, leaves the line with which OpenVMM reported its fatal error in its log, which the failure's message keeps, and whose time ABI code, if it says that the host cannot present the template's CPU profile, TSC, or LAPIC timer, attributes the failure to the host. From the launch on, a failure ends the clone's VMM through its verified identity, and the sandbox keeps the clone's progress log, `clone-progress.log`, which records each step from the spawn to the commit, after a first line that names the launch and the boot of the host, a few lines that it appends without a sync, and the record of its failure, `clone-failure.json`, JSON in format `FAILURE_FORMAT`, which it writes before the teardown, with the teardown pending, so that a crash of the caller during the teardown leaves it for a reattach to finish, and again once the teardown ended, with the stage, the evidence and the status, the original outcome, the transaction, the generation, the progress, and the teardown: no VM, an exit or a termination that the host verified, or an unresolved one, which keeps the launch's journal or runtime record, and the lease; a release removes both, and a sandbox that boots cold writes neither. A step or a check of the caller's own that fails once the clone is ready, or once its sandbox started, fails the clone with the caller's evidence through `Activated::fail` or `Started::fail`, which end the launch and record the failure as the sequence's own failures do. `reattach_clone` takes a clone over after the caller of its launch restarted: it reattaches by the identity of its VMM's process, checks a committed clone in a new session, in which the clone answers the byte-equal Activate of `clone-activation.bin` with the Ready that it answered once, and ends a clone that another runtime identity launched, or that fails the check, through its verified identity; it records the failure of a launch that a crash interrupted, but not the journal that a commit may leave beside its runtime record, classifies a VM that exited by its outcome report, records a failure before it ends a VMM or clears the runtime record of one that exited, finishes a failure that a crash left pending as its teardown would have recorded it, keeping its transaction, resolves the teardown of an earlier failure with what the VMM left as it ended, its outcome report and OpenVMM's fatal error, attributes a status of the guest's as one that a workload may have given once the clone's progress log is lost, damaged, or was written during another boot of the host, whose crash may have lost its last lines, and reads nothing of the template store, so that a tombstone or a retirement of the generation leaves the clone its lease. `CloneRecords::read` reads what the sandbox keeps of a clone, for the operations that list, inspect, or remove clones. `Store::tombstone` quarantines a generation, which takes no new lease while its leases keep working, and `Store::retire` deletes it once its leases have drained, crash-safely, removing the records of leases that no clone binding names any more. `Store::create_template` serves a `Capture`, an adapter's plan for a capture class with the placeholder's binding and the runtime's artifacts, with a generation of its class that the store holds and that is eligible at the host's time, which it reads once it holds the class's slot, `Created::Existing`, or captures one, `Created::Captured`, while `Store::refresh_template` always captures, and leaves older generations and their leases untouched. A capture runs in its class's slot under `captures/`, a sandbox store whose lock serializes the class's captures, which comes before the store's lock, and which keeps the log and outcome report of the VMM of the class's latest capture: in a private transaction on the store's volume, which is also the sandbox store of its VM, since OpenVMM requires the control socket of a VM that captures a snapshot, on Linux, beside the snapshot's destination, it creates the dormant scratch, checks the placeholder through its binding, and both block devices at their paths, regular files that are not links or other reparse points, that belong to the user or an administrator, and that no one else may change, and launches the template's VM through `supervisor::launch` from the transaction, holding no handle that could write either file, since OpenVMM opens them itself by path, as it does a restore's. Once the VMM has exited, it checks the outcome report, the snapshot's manifest and artifacts, and the key, and publishes the transaction in two commits, a rename into `generations/` that replaces nothing, then the index. A capture that fails publishes nothing, and `Store::recover_captures` ends the captures that a crash interrupted, publishing those that were checked, and retiring the others | | `nvx::host::seal` | `FileSeal`: what the host reports about an open file from its metadata alone, without reading it: the platform scheme, the device or volume serial number, the file ID, the kind, the length, the modification time, and on Linux the change time, with a versioned serde form. A seal detects accidental change, not tampering; see [Distro bindings](#distro-bindings) | | `nvx::host::session`, `platform`, `report` | The authenticated control session, which carries NVX's control protocol, `NVXC`, one frame per DATA record of a `HostConnection`; Linux and Windows primitives; and OpenVMM's outcome report | | `nvx::host::sandbox` | The launch profile of `nvx.py sandbox`, ported from the one that `scripts/nvx_tools/sandbox.py` held: `SandboxLaunch`, with the distro `SandboxLayer`, an EROFS layer or a GPT disk, the one layer that a sandbox takes, the ext4 scratch disk, the entrypoint, arguments, hostname, workload identity, and resource limits, and the `SandboxPlan` that it makes of them. `plan`: `SandboxPlan`, the typed plan of a cold layered launch, which is persistable in a versioned serde form: one read-only `distro` block device, an EROFS filesystem or a GPT disk with the identity that the guest expects of it, if any, an upper, the ext4 scratch disk or RAM, the launch settings, the `ExportPlan` of its live shares, and optionally a `SandboxNetwork`, the guest network and its typed `NetworkPolicy`, whose proxy's URL as the guest sees it the plan reports. The plan of an adapter that boots its own guest, `SandboxPlan::adapter`, holds no launch settings and no export plan. A plan knows only the `distro` and `scratch` `BlockRole`s, and gives, for every combination, its `--microvm-sandbox-block` arguments, with `--microvm-workload-identity` when it has launch settings, with no scratch block device for a RAM upper, and those of its network, which `nvx::host::network` lowers, and its kernel command line, with the boot tokens of `nvx::protocol::root`, within the 1024-byte budget that OpenVMM leaves it, refusing one whose root tokens the guest's parser refuses. Both deliver the live shares as the managed lifecycle does: the arguments export them at `nvx::host::filesystem::GUEST_EXPORT`, the command line announces the number of their mappings with `nvx_maps=`, and `mapping_table` gives the ordered table that `start` sends once the guest agent is ready; the CLI's one-shot `run` still renders them as boot tokens. An adapter's plan renders the caller's tokens before those of its root, and refuses tokens that its root or OpenVMM's shares own. `mount`: `SandboxMount` requests of live shares of host directories and files, their guest targets and the reserved ones, their access policies and owner, and the checks of their host paths; `plan_mounts`, which plans them through `nvx::host::filesystem` for the container root, and `boot_shares`, which renders a plan as the OpenVMM arguments and boot tokens of one share or of the aggregate of several, as the one-shot lifecycle delivers them. `options`: the checks that `nvx.py sandbox` and `nvx.py run` apply to their options: the attribution of `--mount-deny`, `--mount-allow`, and `--mount-write` to shares, the systemd entrypoint check and the explicit verification of the distro layer, the targets of `nvx.py run` shares, and the managed exec environment; see [Python parity](#python-parity). `distro`: the `DistroBinding` of a distro layer, an EROFS layer or a GPT disk, which explicit verification makes or a staging caller imports, and its check from metadata alone, which returns the layer open; see [Distro bindings](#distro-bindings). `scratch`: the `ScratchBinding` of a scratch image, its identity, kind, and length, and its check from metadata alone, which claims the image for one sandbox at a time and returns it open. `CheckedDistro::geometry` and `CheckedScratch::geometry` read, through the checked files, the `BlockGeometry` that OpenVMM presents for them: their length, a nonzero multiple of 512 bytes, logical blocks of 512 bytes, and physical blocks of the I/O block size of their filesystem on Linux, and of 4096 bytes on Windows. `prepared`: `PreparedLaunch`, the launch of a plan with the checked files of its block devices, which renders the plan's OpenVMM arguments and kernel command line, naming the checked files as `/proc/self/fd/N` on Linux and by their final paths on Windows, and lists the descriptors that the VMM inherits through `VmmCommand::inherit`; see [Checked handoff](#checked-handoff) | diff --git a/nvx-rs/src/host/template/clone.rs b/nvx-rs/src/host/template/clone.rs index 93db06353..6faa474d8 100644 --- a/nvx-rs/src/host/template/clone.rs +++ b/nvx-rs/src/host/template/clone.rs @@ -69,11 +69,12 @@ const SPARSE_WHAT: &str = "scratch device of the clone"; /// scratch image of an upper on one, and the memory target, which [`ClonePlan::new`] checks /// against the generation, and the settings of the clone's activation that the template does not /// fix, which the clone's [activation record](ClonePlan::activation) names: the workload's view -/// of its VM, the overlay's mode, and the journal policy of a scratch image. +/// of its VM, the overlay's mode, the journal policy of a scratch image, and the bound of a GPT +/// disk's first partition. #[derive(Debug, Clone)] pub struct CloneRequest { /// The adapter's plan, which [`SandboxPlan::adapter`] makes, of the clone's image, its - /// `distro`, an EROFS layer, over its upper: in RAM, a tmpfs, as + /// `distro`, an EROFS layer or a GPT disk, over its upper: in RAM, a tmpfs, as /// [`SandboxUpper::RAM`](crate::host::sandbox::SandboxUpper::RAM) plans it, or on the /// caller's scratch image. Its network is the one of the template's NIC, which every restore /// of the template attaches, or none for a template without a NIC. @@ -98,6 +99,10 @@ pub struct CloneRequest { /// Whether the filesystem of the scratch image of an upper on one may have a journal. An /// upper in RAM has none, so the record of a clone in RAM leaves the policy out. pub journal: Journal, + /// The most bytes that the first partition of a GPT disk may hold, at least 1, which the + /// guest checks as it reads the disk's partition table. An EROFS layer has no partitions, so + /// the record of a clone of one leaves the bound out. + pub first_partition_bound: u64, } /// The upper of a clone's plan, with the file of the clone's `scratch` block device, which every @@ -129,13 +134,16 @@ pub enum CloneUpper { /// compares with those that the template recorded. The plan checks each of the clone's inputs as /// OpenVMM will find it, and holds what the checks opened: /// -/// - The image, the clone's `distro`, must pass the check of its binding, -/// [`DistroBinding::check`], and present the geometry of the template's placeholder, as its -/// key records it: the placeholder's length, and the same logical and physical block sizes, -/// which [`CheckedDistro::geometry`] reads through the checked file. At its path, it must be a -/// regular file, not a link, nor on Windows any other reparse point, that is the file that the -/// check found, that belongs to the current user or to an administrator, and that no one else -/// may change. The plan holds the file that the check opened, for reading alone. +/// - The image, the clone's `distro`, an EROFS layer or a GPT disk, as its binding names it, must +/// pass the check of its binding, [`DistroBinding::check`], and present the geometry of the +/// template's placeholder, as its key records it: the placeholder's length, and the same logical +/// and physical block sizes, which [`CheckedDistro::geometry`] reads through the checked file, +/// whatever the placeholder's format. At its path, it must be a regular file, not a link, nor on +/// Windows any other reparse point, that is the file that the check found, that belongs to the +/// current user or to an administrator, and that no one else may change. The plan holds the +/// file that the check opened, for reading alone. As at a cold launch, the guest reads and +/// checks a GPT disk's partition table, with `nvx::gpt`, before it assembles the root, so the +/// plan reads none of it. /// - Neither the image nor a scratch image may be a file that the sandbox's store owns, by any /// path, through a link among its directories, say, or a hard link: the store's transitions /// replace or remove its files, as a plan replaces the file of the scratch device of a clone in @@ -161,8 +169,8 @@ pub enum CloneUpper { /// - The activation record, which the clone's guest applies in place of the configuration that /// the template captured, names the restore, with a fresh transaction ID, the clone's image, /// upper, and settings, the template's network, and the resources that the clone must have, -/// as [`activation`](Self::activation) describes. Its image must be an EROFS layer, and its -/// settings must keep the limits of a record, as [`Configuration::check`] checks them. +/// as [`activation`](Self::activation) describes. Its settings must keep the limits of a +/// record, as [`Configuration::check`] checks them. /// - The capability is fresh: the plan draws it for the clone's launch, [`launch`](Self::launch). /// /// The plan's checks read the metadata of the clone's files, never their content. @@ -207,6 +215,7 @@ pub enum CloneUpper { /// }, /// overlay: OverlayMode::UntrustedLowers, /// journal: Journal::Allowed, +/// first_partition_bound: 64 * 1024, /// }; /// let deadline = Instant::now() + Duration::from_secs(5); /// let plan = ClonePlan::new(store, &mut acquired, &request, deadline)?; @@ -292,15 +301,13 @@ impl ClonePlan { /// /// - with [`ErrorKind::Unsupported`] for a request that no clone takes: a plan of the sandbox /// profile, with workload settings and its shares, [`Evidence::InvalidSettings`]; a plan - /// whose image is not the format and identity that the binding names, as for a binding - /// whose identity is not a UUID in its canonical text, and an image that is a GPT disk, of - /// which this build restores no clone, since a clone's guest assembles its root only from an - /// EROFS layer, [`Evidence::InvalidFormat`]; an image or a scratch image whose path is - /// relative, which OpenVMM would resolve against the directory in which it runs, holds a - /// comma or a semicolon, which OpenVMM's block device options would split, or whose name - /// ends in `.vhd`, `.vhdx`, `.vmgs`, or `.iso`, which OpenVMM would not attach as a raw - /// image, and an upper on a scratch image without the image's binding, or one in RAM with a - /// binding, [`Evidence::InvalidSettings`]; + /// whose image is not the format and identity that the binding names, as for a binding whose + /// identity is not a UUID in its canonical text, [`Evidence::InvalidFormat`]; an image or a + /// scratch image whose path is relative, which OpenVMM would resolve against the directory in + /// which it runs, holds a comma or a semicolon, which OpenVMM's block device options would + /// split, or whose name ends in `.vhd`, `.vhdx`, `.vmgs`, or `.iso`, which OpenVMM would not + /// attach as a raw image, and an upper on a scratch image without the image's binding, or one + /// in RAM with a binding, [`Evidence::InvalidSettings`]; /// - with [`ErrorKind::Other`] for a lease of another store, and a lock that is not the one /// that acquired the lease, before the launch that it reserves began, /// [`Evidence::HostFailure`]; @@ -322,7 +329,8 @@ impl ClonePlan { /// guest's resolver in another form than OpenVMM writes: more than once, without a NIC, or /// other than as one IP address in its canonical spelling, [`Evidence::MalformedGeneration`]; /// - with [`ErrorKind::Unsupported`] for settings that no activation record holds, as - /// [`Configuration::check`] finds them, such as a hostname that is empty, longer than + /// [`Configuration::check`] finds them, such as a GPT disk's first-partition bound of 0, a + /// hostname that is empty, longer than /// [`MAX_HOSTNAME_BYTES`](crate::activation::MAX_HOSTNAME_BYTES), or holds a NUL, and a /// workload identity whose UID or GID is 0, [`Evidence::InvalidActivate`], and with /// [`ErrorKind::Other`] if the transaction ID of the clone's restore cannot be drawn, the @@ -531,16 +539,18 @@ impl ClonePlan { /// [`ActivationRecord::identity`], the guest's Ready reports. /// /// The record holds the restore section, with the restore's transaction ID, 16 random bytes - /// that the plan drew, and the template that the [lease](Lease::template) names; the image, - /// an EROFS layer, with the identity of its binding and the UUID that the guest expects of its - /// filesystem, if the adapter's plan names one; the upper, a tmpfs, or the scratch image, with - /// the identity of its binding and the request's journal policy; the request's overlay mode - /// and workload view; no export; the network of the template's NIC, if it has one, with the - /// identity of the policy that OpenVMM enforces there, as the template's key records it, the - /// DNS server with which the template's guest configured its resolver at boot, if the - /// template's [command line](TemplateKey::command_line) names one, unpublished listeners, and - /// the URL of the policy's proxy as the guest sees it, if any; and the template's processors, - /// all online, and the memory target, as the resources that the clone must have. + /// that the plan drew, and the template that the [lease](Lease::template) names; the image, in + /// the format that its binding names, with the identity of its binding: an EROFS layer, with + /// the UUID that the guest expects of its filesystem, or a GPT disk, with the GUID that the + /// guest expects of the disk, each if the adapter's plan names one, and the request's bound of + /// the disk's first partition; the upper, a tmpfs, or the scratch image, with the identity of + /// its binding and the request's journal policy; the request's overlay mode and workload view; + /// no export; the network of the template's NIC, if it has one, with the identity of the + /// policy that OpenVMM enforces there, as the template's key records it, the DNS server with + /// which the template's guest configured its resolver at boot, if the template's + /// [command line](TemplateKey::command_line) names one, unpublished listeners, and the URL of + /// the policy's proxy as the guest sees it, if any; and the template's processors, all + /// online, and the memory target, as the resources that the clone must have. /// /// The identity of a binding is the SHA-256 digest of a domain of its kind and of its serde /// form, a few hundred bytes of metadata: the plan reads no content to compute it. @@ -612,9 +622,6 @@ fn check_request(request: &CloneRequest) -> Result<(), Fault> { )), )); } - if matches!(binding.format(), DistroFormat::Gpt) { - return Err(Fault::new(Evidence::InvalidFormat, record::gpt_refused())); - } check_disk_path(distro.pure_path(), "image").attribute(Evidence::InvalidSettings)?; match (adapter.upper().pure_scratch_path(), &request.scratch) { (Some(path), Some(_)) => { diff --git a/nvx-rs/src/host/template/clone/failure.rs b/nvx-rs/src/host/template/clone/failure.rs index b5a107bc6..1333df62c 100644 --- a/nvx-rs/src/host/template/clone/failure.rs +++ b/nvx-rs/src/host/template/clone/failure.rs @@ -82,8 +82,7 @@ pub enum Evidence { /// have, as the guest's /// [`ACTIVATION_RESOURCES`](crate::protocol::status::ACTIVATION_RESOURCES) tells. InvalidTargets, - /// The request's image is of a format that no clone restores, a GPT disk, or is not of the - /// format and identity that its binding names. + /// The request's image is not of the format and identity that its binding names. InvalidFormat, /// The request's settings are invalid: a plan that no clone takes, a path that OpenVMM would /// not open as checked, a network that the template's NIC does not match, an image or a diff --git a/nvx-rs/src/host/template/clone/record.rs b/nvx-rs/src/host/template/clone/record.rs index 9f04ab631..9883b9b85 100644 --- a/nvx-rs/src/host/template/clone/record.rs +++ b/nvx-rs/src/host/template/clone/record.rs @@ -38,8 +38,10 @@ const SCRATCH_DOMAIN: &[u8] = b"nvx scratch binding\0"; /// /// - the restore section: a transaction ID of 16 random bytes, and the template that `lease` /// names, its generation and the identity of its key; -/// - the image: an EROFS layer, the identity of its binding, and the UUID of its filesystem, -/// which the adapter's plan names, as the binding does; +/// - the image, in the format that its binding names: an EROFS layer, with the identity of its +/// binding and the UUID of its filesystem, or a GPT disk, with the identity of its binding, the +/// GUID of the disk, and the request's bound of its first partition; the adapter's plan names +/// the UUID or the GUID, as the binding does; /// - the upper: a tmpfs, or the scratch, with the identity of its binding and the request's /// journal policy; /// - the request's overlay mode and workload view; @@ -53,9 +55,9 @@ const SCRATCH_DOMAIN: &[u8] = b"nvx scratch binding\0"; /// - the expected resources: the template's processors, all of which a template's VM keeps /// online, and the request's memory target. /// -/// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for a request whose image -/// is not an EROFS layer, [`Evidence::InvalidFormat`], and for a configuration that no record -/// holds, as [`Configuration::check`] finds it, such as a hostname that is empty or longer than +/// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for a configuration that +/// no record holds, as [`Configuration::check`] finds it, such as a GPT disk's first-partition +/// bound of 0, a hostname that is empty or longer than /// [`MAX_HOSTNAME_BYTES`](crate::activation::MAX_HOSTNAME_BYTES), or a workload identity whose /// UID or GID is 0, [`Evidence::InvalidActivate`]; with [`ErrorKind::Io`](crate::ErrorKind::Io) /// for a key whose command line names its DNS server in a form that OpenVMM does not write, as @@ -67,12 +69,18 @@ pub(super) fn build( lease: &Lease, key: &TemplateKey, ) -> std::result::Result { + let binding = distro_identity(&request.binding).on_host()?; + let expected = request.adapter.distro().image().identity; let image = match request.binding.format() { DistroFormat::Erofs => Image::Erofs { - binding: distro_identity(&request.binding).on_host()?, - uuid: request.adapter.distro().image().identity, + binding, + uuid: expected, + }, + DistroFormat::Gpt => Image::Gpt { + binding, + disk_guid: expected, + first_partition_bound: request.first_partition_bound, }, - DistroFormat::Gpt => return Err(Fault::new(Evidence::InvalidFormat, gpt_refused())), }; let upper = match &request.scratch { None => Upper::Tmpfs, @@ -118,14 +126,6 @@ pub(super) fn build( }) } -/// Returns the error of a request whose image is a GPT disk, which no clone restores yet. -pub(super) fn gpt_refused() -> Error { - Error::unsupported( - "the image of a clone must be an EROFS layer: this build restores no clone of a GPT \ - image", - ) -} - /// Returns the identity of `binding`, the binding of a clone's image, as the [module](self) /// describes it. pub(super) fn distro_identity(binding: &DistroBinding) -> Result { diff --git a/nvx-rs/src/host/template/clone/tests.rs b/nvx-rs/src/host/template/clone/tests.rs index 778de4a6b..7d5b763de 100644 --- a/nvx-rs/src/host/template/clone/tests.rs +++ b/nvx-rs/src/host/template/clone/tests.rs @@ -44,6 +44,9 @@ const SCRATCH_LEN: u64 = 2 * MIB; const MEMORY: u64 = 512 * MIB; /// The template's RAM capacity, in bytes, as [`ManifestSpec::template`] records it. const MEMORY_CAPACITY: u64 = 2048 * MIB; +/// The bound of the first partition of the tests' GPT disks, which the record of a clone of an +/// EROFS layer leaves out. +const FIRST_PARTITION_BOUND: u64 = 64 * 1024; /// Nanoseconds in a second. const SECOND: u64 = 1_000_000_000; @@ -161,10 +164,15 @@ fn layer(path: &Path) -> SandboxLayer { SandboxLayer::new(LayerRole::Distro, path.to_str().unwrap(), UUID) } +/// Returns the distro GPT disk at `path`, whose disk GUID is [`UUID`]. +fn gpt_layer(path: &Path) -> SandboxLayer { + SandboxLayer::gpt(LayerRole::Distro, path.to_str().unwrap(), UUID) +} + /// Returns the request of a clone of the image at `image`, which `binding` binds, over an upper in /// RAM, with the template's RAM as its memory target, and the settings of [`workload`], for /// untrusted lowers, and with a journal policy, [`Journal::Forbidden`], that its record leaves -/// out. +/// out, and for a GPT disk, the bound [`FIRST_PARTITION_BOUND`]. fn tmpfs_request(image: &Path, binding: &DistroBinding) -> CloneRequest { CloneRequest { adapter: SandboxPlan::adapter( @@ -178,6 +186,7 @@ fn tmpfs_request(image: &Path, binding: &DistroBinding) -> CloneRequest { workload: workload(), overlay: OverlayMode::UntrustedLowers, journal: Journal::Forbidden, + first_partition_bound: FIRST_PARTITION_BOUND, } } @@ -350,6 +359,15 @@ impl Fixture { (path, binding) } + /// Stages the GPT disk `name` of `length` bytes, whose disk GUID is [`UUID`], as + /// [`image`](Self::image) stages an EROFS layer, and returns its path and its binding. + fn gpt_image(&self, name: &str, length: u64) -> (PathBuf, DistroBinding) { + let path = self.path("images", name); + stage(&path, length); + let binding = DistroBinding::import(&gpt_layer(&path), PolicyVerdict::ACCEPTED).unwrap(); + (path, binding) + } + /// Stages the scratch image `name` of `length` bytes, and returns its path and its binding. fn scratch(&self, name: &str, length: u64) -> (PathBuf, ScratchBinding) { let path = self.path("scratches", name); @@ -1128,6 +1146,25 @@ fn images_of_another_length_or_geometry_are_refused() { ) ); } + // A GPT disk has the placeholder's length too. + let (gpt, gpt_binding) = fixture.gpt_image("other.gpt", IMAGE_LEN + 512); + let plan = fixture.plan( + &mut acquired, + &tmpfs_request(&gpt, &gpt_binding), + NO_CONTENT, + ); + assert_eq!( + refused(plan, Evidence::SuppliedLength), + ( + ErrorKind::Unsupported, + format!( + "the image {} is {} bytes long, not the {IMAGE_LEN} bytes of the template's \ + placeholder", + shown(&gpt), + IMAGE_LEN + 512 + ) + ) + ); assert_untouched(&acquired); // A template whose placeholder presented physical blocks of another size serves no image of @@ -1439,6 +1476,39 @@ fn requests_that_no_clone_takes_are_refused_before_anything_is_opened() { shown(&image) ) ); + // An image of another format than the one that the binding names, either way. + let identity = Some(Uuid::parse(UUID).unwrap()); + let as_gpt = CloneRequest { + adapter: SandboxPlan::adapter( + SandboxDistro::new(image.to_str().unwrap(), RootImage::gpt(identity)), + SandboxUpper::RAM, + ), + ..tmpfs.clone() + }; + assert_eq!( + refusal(&as_gpt, Evidence::InvalidFormat), + format!( + "the image {} is not the erofs layer with identity {UUID} that its distro binding \ + names", + shown(&image) + ) + ); + let (gpt, gpt_binding) = fixture.gpt_image("distro.gpt", IMAGE_LEN); + let as_erofs = CloneRequest { + adapter: SandboxPlan::adapter( + SandboxDistro::new(gpt.to_str().unwrap(), RootImage::erofs(identity)), + SandboxUpper::RAM, + ), + binding: gpt_binding, + ..tmpfs.clone() + }; + assert_eq!( + refusal(&as_erofs, Evidence::InvalidFormat), + format!( + "the image {} is not the gpt layer with identity {UUID} that its distro binding names", + shown(&gpt) + ) + ); // Paths that OpenVMM would not open as the plan checks them. let at = |path: &str| CloneRequest { adapter: SandboxPlan::adapter(SandboxDistro::new(path, distro.image()), SandboxUpper::RAM), @@ -1794,6 +1864,47 @@ fn the_activation_record_of_a_clone_on_a_scratch_image_names_its_binding_and_jou ); } +#[test] +fn the_activation_record_of_a_gpt_clone_names_its_disk_and_the_bound_of_its_first_partition() { + let fixture = Fixture::new(); + let (image, binding) = fixture.gpt_image("distro.gpt", IMAGE_LEN); + let (scratch, scratch_binding) = fixture.scratch("scratch.ext4", SCRATCH_LEN); + let mut acquired = fixture.acquire("sandbox"); + let gpt = |first_partition_bound| Image::Gpt { + binding: digest_of(b"nvx distro binding\0", &binding), + disk_guid: Some(Uuid::parse(UUID).unwrap()), + first_partition_bound, + }; + // A GPT disk of the placeholder's geometry serves over RAM and over a scratch image, as an + // EROFS layer does, and its plan reads none of its content: the guest reads and checks the + // disk's partition table as it assembles the root. + for request in [ + tmpfs_request(&image, &binding), + scratch_request(&image, &binding, &scratch, &scratch_binding), + ] { + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); + assert_eq!(plan.image().path().to_str().unwrap(), shown(&image)); + assert_eq!( + plan.image().geometry().unwrap(), + geometry(IMAGE_LEN, fixture.physical) + ); + let mut byte = [0]; + assert!(plan.image().file().read(&mut byte).is_err()); + let configuration = plan.activation().configuration(); + assert_eq!(configuration.image, gpt(FIRST_PARTITION_BOUND)); + assert_eq!(configuration.workload, workload()); + } + // The record keeps the request's bound, whatever it is from 1 on. + for bound in [1, u64::MAX] { + let request = CloneRequest { + first_partition_bound: bound, + ..tmpfs_request(&image, &binding) + }; + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); + assert_eq!(plan.activation().configuration().image, gpt(bound)); + } +} + #[test] fn the_activation_record_of_a_clone_of_a_template_with_a_nic_names_its_policy_and_dns_server() { let fixture = Fixture::with(|class, spec, physical| { @@ -1920,27 +2031,28 @@ fn settings_that_no_activation_record_holds_are_refused_before_anything_is_creat ); assert_untouched(&acquired); } - // A GPT image, whose binding names it, is refused before anything is opened. - let path = fixture.path("images", "distro.gpt"); - stage(&path, IMAGE_LEN); - let gpt = SandboxLayer::gpt(LayerRole::Distro, path.to_str().unwrap(), UUID); - let gpt_binding = DistroBinding::import(&gpt, PolicyVerdict::ACCEPTED).unwrap(); - let gpt_request = CloneRequest { - adapter: SandboxPlan::adapter( - SandboxDistro::new(path.to_str().unwrap(), bound_image(&gpt_binding).unwrap()), - SandboxUpper::RAM, - ), - binding: gpt_binding, - ..request.clone() + // A GPT disk's first-partition bound of 0, which an EROFS layer's record leaves out. + let (path, gpt_binding) = fixture.gpt_image("distro.gpt", IMAGE_LEN); + let unbounded = CloneRequest { + first_partition_bound: 0, + ..tmpfs_request(&path, &gpt_binding) }; - let plan = fixture.plan(&mut acquired, &gpt_request, NO_CONTENT); - let (kind, message) = refused(plan, Evidence::InvalidFormat); - assert_eq!(kind, ErrorKind::Unsupported, "{message}"); - assert!( - message.starts_with("the image of a clone must be an EROFS layer"), - "{message}" + let plan = fixture.plan(&mut acquired, &unbounded, NO_CONTENT); + assert_eq!( + refused(plan, Evidence::InvalidActivate), + ( + ErrorKind::Unsupported, + "no clone activates with the settings of the request: the bound of a GPT disk's first \ + partition is 0" + .to_owned() + ) ); assert_untouched(&acquired); + let erofs = CloneRequest { + first_partition_bound: 0, + ..request.clone() + }; + drop(fixture.plan(&mut acquired, &erofs, NO_CONTENT).unwrap()); drop(fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap()); } diff --git a/tests/runtime-adapter/README.md b/tests/runtime-adapter/README.md index 6b03bb153..7fe717708 100644 --- a/tests/runtime-adapter/README.md +++ b/tests/runtime-adapter/README.md @@ -1377,16 +1377,20 @@ of the class's latest capture, when the class's slot keeps one. `runtime-adapter-host start` starts a clone of a generation of a template store in a sandbox store, through NVX's public API, `nvx::host::template`. Its library side, `clone`, holds what the command's options choose of the clone, -`clone::Request`: its image, an EROFS layer of the length of the template's -placeholder, its upper, on an ext4 scratch image or in RAM, its memory target, -the template's RAM unless given, the workload identity as which its executions -run, its overlay mode, and its scratch's journal policy. `Request::plan` plans -the image over the upper as a launch plans a layered root, and -`Request::clone_request` returns NVX's `CloneRequest` of it, with the bindings -that the sandbox's store stages for the images, as `stage` stages them, so the -start hashes nothing. The clone's activation record names the image's own -machine ID, the hostname `runtime-adapter`, and the guest's defaults of the -other settings, unless the request names them. +`clone::Request`: its image, an EROFS layer or a GPT disk of the length of the +template's placeholder, its upper, on an ext4 scratch image or in RAM, its +memory target, the template's RAM unless given, the workload identity as which +its executions run, its overlay mode, and its scratch's journal policy. +`Request::plan` plans the image over the upper as a launch plans a layered +root, and `Request::clone_request` returns NVX's `CloneRequest` of it, with the +bindings that the sandbox's store stages for the images, as `stage` stages +them, so the start hashes nothing. The clone's activation record names the +image's own machine ID, the hostname `runtime-adapter`, and the guest's +defaults of the other settings, unless the request names them; for a GPT disk, +it bounds the first partition by 64 KiB, as a launch's guest does by default. +The guest of a GPT clone rereads the disk's partition table before it +assembles the root, since its kernel created the partitions of the template's +placeholder. The command first reads the generation from the store's index, and its compatibility key, before it needs NVX's artifacts. The store then leases the @@ -1567,7 +1571,9 @@ the directory that `NVX_ARTIFACTS_DIR` names, and the guest agent's build from starts a [clone](#clones) of the generation G of the template store TEMPLATES in the sandbox store STORE, which it creates if needed, and whose `stage` staged its images, which no one else may change: the image PATH, an - EROFS file system, `erofs`, of which the guest expects the identity UUID, + EROFS file system, `erofs`, or a GPT disk, `gpt`, of which the guest expects + the identity UUID, the file system's UUID or the disk's GUID, of the length of + the template's placeholder, whatever the placeholder's format, over the ext4 scratch image PATH, or RAM, with a memory target of MIB mebibytes, the template's RAM by default, the workload identity UID:GID, if given, as which every execution runs, and whose account the guest checks before the clone is ready, and the overlay diff --git a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs index 4afb7ff20..f1f207c42 100644 --- a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs +++ b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs @@ -105,10 +105,11 @@ //! the sandbox, at the host's time, with `Store::acquire`, which refuses a generation that a //! tombstone quarantines, or that is not eligible then, and NVX's `ClonePlan` checks the clone //! against the generation, from the metadata of its files alone: its image, the raw file PATH, -//! an EROFS filesystem, `erofs`, of which the guest expects the identity UUID, of the length of -//! the template's placeholder, its upper, the ext4 scratch image PATH, of the geometry of the -//! template's scratch, or RAM, with the bindings that STORE stages for the images, as `stage` -//! stages them, and its memory target, in whole blocks, from the template's RAM to its capacity. +//! an EROFS filesystem, `erofs`, or a GPT disk, `gpt`, of which the guest expects the identity +//! UUID, of the length of the template's placeholder, whatever the placeholder's format, its +//! upper, the ext4 scratch image PATH, of the geometry of the template's scratch, or RAM, with +//! the bindings that STORE stages for the images, as `stage` stages them, and its memory target, +//! in whole blocks, from the template's RAM to its capacity. //! A request that the template cannot serve fails there, before any VMM starts. The command //! records the clone in STORE, with the template store and the generation; see //! `runtime_adapter::clone::CloneRecord`. OpenVMM then restores the generation's snapshot in diff --git a/tests/runtime-adapter/src/clone.rs b/tests/runtime-adapter/src/clone.rs index 64fd5dbf1..e877d50f7 100644 --- a/tests/runtime-adapter/src/clone.rs +++ b/tests/runtime-adapter/src/clone.rs @@ -2,10 +2,10 @@ //! NVX's template store leases to a sandbox, as `runtime-adapter-host start` asks. //! //! A [`Request`] holds what the host program's options choose of a clone: its image, an EROFS -//! layer of the length of the template's placeholder, its upper, on an ext4 scratch image or in -//! RAM, its memory target, the workload identity as which its executions run, its overlay mode, -//! and its scratch's journal policy. [`Request::plan`] plans the image over the upper as a launch -//! plans a layered root, and [`Request::clone_request`] returns NVX's +//! layer or a GPT disk of the length of the template's placeholder, its upper, on an ext4 scratch +//! image or in RAM, its memory target, the workload identity as which its executions run, its +//! overlay mode, and its scratch's journal policy. [`Request::plan`] plans the image over the +//! upper as a launch plans a layered root, and [`Request::clone_request`] returns NVX's //! `nvx::host::template::CloneRequest` of it, with the bindings that the sandbox's store stages //! for its images, as `layered::Staged` records them: staging hashes, and a start never does. //! NVX's `ClonePlan` then checks the request against the leased generation before any VMM starts, @@ -39,7 +39,7 @@ use nvx::{Error, Result}; use serde_json::{Value, json}; use crate::layered::{Bindings, RECORD_FORMAT, RECORD_NAME, invalid, plan, unstaged, write_record}; -use crate::root::{JournalPolicy, OverlayMode}; +use crate::root::{DEFAULT_FIRST_PARTITION_BOUND, JournalPolicy, OverlayMode}; /// The mode of a clone's [record](CloneRecord). pub const CLONE_MODE: &str = "clone"; @@ -102,7 +102,8 @@ impl Request { /// The record of the clone's activation names the workload identity, if any, the /// [`CLONE_HOSTNAME`], the image's own machine ID, the overlay's mode, and the scratch's /// journal policy, each the guest's default unless the request names it, as a root - /// configuration's are. + /// configuration's are, and for a GPT disk, the bound of its first partition that a launch's + /// guest takes by default, [`DEFAULT_FIRST_PARTITION_BOUND`]. /// /// Fails with [`ErrorKind::Unavailable`](nvx::ErrorKind::Unavailable) if the store stages no /// binding for the image, or for the scratch image of an upper on one, at its path. @@ -148,6 +149,7 @@ impl Request { }, overlay, journal, + first_partition_bound: DEFAULT_FIRST_PARTITION_BOUND, }) } } From 844e96009e19d075aeab8144bb1dfac99e3c0bab Mon Sep 17 00:00:00 2001 From: Enrique Saurez Date: Sun, 25 Oct 2026 18:15:36 -0700 Subject: [PATCH 2/3] runtime-adapter: start GPT clones, and refuse a used upper, in VMs A clone's guest rereads a GPT disk's partition table before it assembles the root, and refuses an upper that is not fresh, and the host now plans GPT clones. These VM tests start such clones through runtime-adapter-host start, on the hypervisor. start takes --first-partition-bound BYTES for a GPT disk, as launch does, and clone::Request takes the bound, 64 KiB unless given. In vm_clone, one template, whose placeholder is a GPT disk of two layers, serves four clones at once: the EROFS image over a scratch image and in RAM, a GPT disk of one layer in RAM, and one of three layers over a scratch image, with a bound that its first partition meets exactly. The disks' layer partitions start where none of the placeholder's do, nor each other's, so a clone whose kernel kept the placeholder's partitions would fail its assembly. Each GPT clone's root stacks the layers of its own disk's table, read-only, and returns its own first partition. A disk whose first partition is one byte larger than the bound ends the clone with status 54, as does a scratch image that an earlier clone used, whose upper is not fresh. The GPT writer moves from vm_layered into tests/common/gpt.rs, with the small ext2 layers and the CRC, and writes disks of any layout and length. vm_layered writes the same disks as before through it. Part of #845 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 80932e09-af82-4b79-b417-70a241303cda --- tests/runtime-adapter/README.md | 58 ++- .../src/bin/runtime-adapter-host.rs | 48 +- tests/runtime-adapter/src/clone.rs | 24 +- tests/runtime-adapter/tests/clone_start.rs | 17 +- tests/runtime-adapter/tests/common/gpt.rs | 382 ++++++++++++++++ tests/runtime-adapter/tests/common/mod.rs | 2 + tests/runtime-adapter/tests/vm_clone.rs | 419 +++++++++++++++--- tests/runtime-adapter/tests/vm_layered.rs | 337 ++------------ 8 files changed, 878 insertions(+), 409 deletions(-) create mode 100644 tests/runtime-adapter/tests/common/gpt.rs diff --git a/tests/runtime-adapter/README.md b/tests/runtime-adapter/README.md index 7fe717708..28a435453 100644 --- a/tests/runtime-adapter/README.md +++ b/tests/runtime-adapter/README.md @@ -1380,14 +1380,15 @@ library side, `clone`, holds what the command's options choose of the clone, `clone::Request`: its image, an EROFS layer or a GPT disk of the length of the template's placeholder, its upper, on an ext4 scratch image or in RAM, its memory target, the template's RAM unless given, the workload identity as which -its executions run, its overlay mode, and its scratch's journal policy. -`Request::plan` plans the image over the upper as a launch plans a layered -root, and `Request::clone_request` returns NVX's `CloneRequest` of it, with the -bindings that the sandbox's store stages for the images, as `stage` stages -them, so the start hashes nothing. The clone's activation record names the -image's own machine ID, the hostname `runtime-adapter`, and the guest's -defaults of the other settings, unless the request names them; for a GPT disk, -it bounds the first partition by 64 KiB, as a launch's guest does by default. +its executions run, its overlay mode, its scratch's journal policy, and the +bound of a GPT disk's first partition. `Request::plan` plans the image over the +upper as a launch plans a layered root, and `Request::clone_request` returns +NVX's `CloneRequest` of it, with the bindings that the sandbox's store stages +for the images, as `stage` stages them, so the start hashes nothing. The +clone's activation record names the image's own machine ID, the hostname +`runtime-adapter`, and the guest's defaults of the other settings, unless the +request names them; for a GPT disk, the bound of its first partition is 64 KiB +unless the request names one, as a launch's guest takes it by default. The guest of a GPT clone rereads the disk's partition table before it assembles the root, since its kernel created the partitions of the template's placeholder. @@ -1567,17 +1568,19 @@ the directory that `NVX_ARTIFACTS_DIR` names, and the guest agent's build from its record again, and releases the lease of a clone that ended, through the template store TEMPLATES, if given, in place of the one that the clone's record names. -- `start STORE --template TEMPLATES --generation G --distro FORMAT:PATH --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY]` +- `start STORE --template TEMPLATES --generation G --distro FORMAT:PATH --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES]` starts a [clone](#clones) of the generation G of the template store TEMPLATES in the sandbox store STORE, which it creates if needed, and whose `stage` staged its images, which no one else may change: the image PATH, an EROFS file system, `erofs`, or a GPT disk, `gpt`, of which the guest expects - the identity UUID, the file system's UUID or the disk's GUID, of the length of - the template's placeholder, whatever the placeholder's format, - over the ext4 scratch image PATH, or RAM, with a memory target of MIB mebibytes, the template's RAM by default, - the workload identity UID:GID, if given, as which every execution runs, and - whose account the guest checks before the clone is ready, and the overlay - mode MODE and the scratch's journal policy POLICY, as a launch takes them. + the identity UUID, the file system's UUID or the disk's GUID, of the length + of the template's placeholder, whatever the placeholder's format, over the + ext4 scratch image PATH, or RAM, with a memory target of MIB mebibytes, the + template's RAM by default, the workload identity UID:GID, if given, as which + every execution runs, and whose account the guest checks before the clone is + ready, the overlay mode MODE and the scratch's journal policy POLICY, as a + launch takes them, and for a GPT disk, the bound BYTES of its first + partition, 64 KiB by default, as a launch's guest takes it. Once the restore sequence started the clone's sandbox, it exchanges a `Ping` and the longest `Echo` with the guest agent, commits the launch, and exits, leaving the VM running, and the sandbox holding the lease. A clone that @@ -2043,9 +2046,10 @@ Windows, without a hypervisor: - the command lines of `start` that the host program refuses, each with status 2: missing options, a generation that is not a positive number, a scratch's - journal policy for an upper in RAM, invalid values, and the options of other - commands, and the options of `start` that other commands do not take, with a - clock fixture that is not a host's time too; + journal policy for an upper in RAM, a first-partition bound for an EROFS + image, invalid values, and the options of other commands, and the options of + `start` that other commands do not take, with a clock fixture that is not a + host's time too; - a start whose template store does not exist, or does not hold the generation, and whose image the plan refuses, each with status 1 before it needs NVX's artifacts or creates the sandbox's store; @@ -2343,9 +2347,10 @@ MBR, and the primary and backup tables of 128 entries, with the first partition, 4 KiB of bytes that are not UTF-8, then padding; the base layer, in partition 2, a copy of the template; and the top layer, in partition 3, a small file system in the ext2 format, which the kernel's ext4 driver mounts, with a -file and a directory. The test writes the top layer itself, since `mkfs.ext4` -is not available everywhere that the tests run, and checks the disk with -`nvx::gpt` before it launches the VM. For `corrupt_layer_fails_the_assembly`, +file and a directory. The test writes the disk, and the top layer itself, with +`tests/common/gpt.rs`, since `mkfs.ext4` is not available everywhere that the +tests run, and checks the disk with `nvx::gpt` before it launches the VM. For +`corrupt_layer_fails_the_assembly`, it corrupts the top layer: the inode of the layer's top directory loses its mode, an error that the kernel finds when it mounts the layer, and the error policy of its superblock asks the kernel to panic on an error, which the @@ -2399,14 +2404,21 @@ in the host-only configuration, through `runtime-adapter-host start`. Each test captures a template of its own, of 1 processor and 256 MiB of RAM with a capacity of 1 GiB, in a template store of its own, of a copy of NVX's Ubuntu EROFS image that a directory of the platform holds, which the clones stage as -their image, and starts its clones, each in a sandbox store of its own, over a -copy of the scratch template, which no one else may change either, or in RAM: +their image, or of a GPT disk of its length that the platform holds beside it, +and starts its clones, each in a sandbox store of its own, over a copy of the +scratch template, which no one else may change either, or in RAM. The GPT disks +of a template's placeholder and of the clones, which the test writes with +`tests/common/gpt.rs`, as `tests/vm_layered.rs` writes its own, hold their +first partition, then empty layers, at partitions that differ from one disk to +another: | Test | Clones | Expects | | --- | --- | --- | | `one_generation_serves_clones_of_each_upper_and_memory_target` | Two at once, over a scratch image at 256 MiB, and in RAM at 512 MiB with untrusted lowers, then one over a scratch image at 1 GiB | Each started, by the host's monotonic clock, with its own lease, which the store lists with the clone's sandbox, its memory target online, its upper, overlay mode, and image's contents, and the events of its activation in order; a stop releases each lease; the generation's snapshot unchanged, and its payload with nothing but the capture's files once the clones stopped | +| `one_generation_serves_clones_of_either_format_over_either_upper` | Of a template whose placeholder is a GPT disk of two layers, four at once: the EROFS image over a scratch image and in RAM, a GPT disk of one layer in RAM, and one of three layers over a scratch image, with a bound that its first partition meets exactly; then the disk of one layer, with a bound one byte smaller than its first partition | Each started, with its upper and the overlay's options; the EROFS image's root, without a first partition; each GPT disk's layers, ext4 file systems mounted read-only once each, from the partitions that its own table names, stacked in order, and the bytes of its own first partition; then status 8, and the record of a failure of the request, at the activation, once the VM exited, by the evidence of an unusable root, with the guest's status 54; nothing quarantined, and the lease released, with the records | | `a_start_refuses_what_the_template_cannot_serve_before_any_vmm_starts` | An image 4 KiB longer, one 4 KiB shorter, and one 100 bytes longer, than the placeholder, a scratch image of twice the template's scratch, memory targets of 2 GiB and of 320 MiB, unstaged images, and a generation that the store does not hold | For those that NVX's plan refuses, status 8, with NVX's reason, and the record of a failure of the request, at the plan, without a VM, by the evidence of a length that the request supplied, of the image's geometry, which OpenVMM cannot attach, or of invalid targets, which quarantines nothing; for the others, status 1, without a record; any lease released, and no VMM's log or record in the sandbox | | `a_clone_whose_activation_fails_ends_before_it_is_ready` | A scratch image of zeros, a journal that the record forbids, a workload identity whose account the image lacks, and, through NVX's restore sequence, a ping, a start, or the start of an execution before the `Activate` | Status 8, no `Ready`, and the record of a failure, at the activation, once the VM exited: of the request, by the evidence of an unusable root, with the guest's status 54, or of invalid settings, with 55, or, before the `Activate`, of an unknown cause, by the protocol's, with 51, which the sandbox records until the release; nothing quarantined, and the lease released, with the records | +| `a_clone_refuses_a_scratch_image_whose_upper_an_earlier_clone_used` | Over a scratch image, in whose root the `contents` probe writes a file, then over a copy of that scratch image once the first clone stopped | The first started; then status 8, since the copy's upper is not fresh, and the record of a failure of the request, at the activation, once the VM exited, by the evidence of an unusable root, with the guest's status 54; nothing quarantined, and the lease released, with the records | | `a_clone_runs_its_executions_as_its_workload_identity_once_it_is_ready` | Over a scratch image, as nobody | An execution that runs as nobody, one that names an identity refused with `EINVAL`, and the first execution after the clone's start | | `a_generation_is_eligible_by_the_clock_of_its_capture` | At the host's time, then with clock fixtures: UTC a day behind, another boot an hour after the capture by UTC, and the restore window passed by each clock, or UTC before the capture | The host's monotonic clock on the boot of the capture, whatever UTC says, UTC after a reboot, with the fixture's downtime, and the store's refusal without a lease outside the window | | `a_clone_reads_no_more_of_its_disks_than_its_root_needs` | Over a scratch image, and in RAM | Less than an eighth of each disk read once the clone is ready, and each disk's whole length once the guest read it to its end | diff --git a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs index f1f207c42..be453cbef 100644 --- a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs +++ b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs @@ -97,19 +97,20 @@ //! the session's next epoch, and exchanges a `Ping`, leaving the VM running. //! - `runtime-adapter-host start STORE --template TEMPLATES --generation G --distro FORMAT:PATH //! --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity -//! UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY]` starts a clone of the generation G -//! of the template store TEMPLATES, an `nvx::host::template::Store`, in the sandbox store STORE, -//! which it creates if needed; see `runtime_adapter::clone`. It reads the generation from the -//! store's index, and its compatibility key, whose RAM is the clone's memory target unless MIB -//! mebibytes are given, before it needs NVX's artifacts. The store then leases the generation to -//! the sandbox, at the host's time, with `Store::acquire`, which refuses a generation that a -//! tombstone quarantines, or that is not eligible then, and NVX's `ClonePlan` checks the clone -//! against the generation, from the metadata of its files alone: its image, the raw file PATH, -//! an EROFS filesystem, `erofs`, or a GPT disk, `gpt`, of which the guest expects the identity -//! UUID, of the length of the template's placeholder, whatever the placeholder's format, its -//! upper, the ext4 scratch image PATH, of the geometry of the template's scratch, or RAM, with -//! the bindings that STORE stages for the images, as `stage` stages them, and its memory target, -//! in whole blocks, from the template's RAM to its capacity. +//! UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES]` +//! starts a clone of the generation G of the template store TEMPLATES, an +//! `nvx::host::template::Store`, in the sandbox store STORE, which it creates if needed; see +//! `runtime_adapter::clone`. It reads the generation from the store's index, and its +//! compatibility key, whose RAM is the clone's memory target unless MIB mebibytes are given, +//! before it needs NVX's artifacts. The store then leases the generation to the sandbox, at the +//! host's time, with `Store::acquire`, which refuses a generation that a tombstone quarantines, +//! or that is not eligible then, and NVX's `ClonePlan` checks the clone against the generation, +//! from the metadata of its files alone: its image, the raw file PATH, an EROFS filesystem, +//! `erofs`, or a GPT disk, `gpt`, of which the guest expects the identity UUID, of the length of +//! the template's placeholder, whatever the placeholder's format, its upper, the ext4 scratch +//! image PATH, of the geometry of the template's scratch, or RAM, with the bindings that STORE +//! stages for the images, as `stage` stages them, and its memory target, in whole blocks, from +//! the template's RAM to its capacity. //! A request that the template cannot serve fails there, before any VMM starts. The command //! records the clone in STORE, with the template store and the generation; see //! `runtime_adapter::clone::CloneRecord`. OpenVMM then restores the generation's snapshot in @@ -118,10 +119,14 @@ //! adapter's clone order, `runtime_adapter::host::CloneOrder`, with the clone's activation //! record: the workload identity UID:GID, if given, as which every execution runs, whose account //! the guest checks in the clone's root before it is ready, and the overlay mode MODE and the -//! scratch's journal policy POLICY, as a launch takes them. Once the clone is ready, the command -//! starts its sandbox, exchanges a `Ping` and the longest `Echo` with the guest agent, commits -//! the launch, and exits, leaving the VM running, and the sandbox holding the lease, which a stop -//! releases. A clone that fails, from its plan to its commit, fails with NVX's +//! scratch's journal policy POLICY, as a launch takes them, and for a GPT disk, the bound BYTES +//! of its first partition, 64 KiB unless given, as a launch's guest takes it by default. The +//! guest refuses a first partition larger than BYTES, and an upper that is not fresh, whose +//! `upper` or `work` directory on the scratch image exists and is not empty, as an earlier +//! clone's would be, before the clone is ready. Once the clone is ready, the command starts its +//! sandbox, exchanges a `Ping` and the longest `Echo` with the guest agent, commits the launch, +//! and exits, leaving the VM running, and the sandbox holding the lease, which a stop releases. +//! A clone that fails, from its plan to its commit, fails with NVX's //! `nvx::host::template::CloneFailure`, which attributes the failure by its evidence, and records //! it in STORE from the clone's launch on: the command prints the failure's record, releases the //! lease once the failure's teardown is resolved, and nothing of the clone runs, or keeps it, and @@ -398,6 +403,7 @@ const USAGE: &str = concat!( " [--memory MIB]\n", " [--workload-identity UID:GID] [--overlay-mode MODE]", " [--scratch-journal POLICY]\n", + " [--first-partition-bound BYTES]\n", " runtime-adapter-host probe STORE PROBE... [--guest-build BUILD]\n", " runtime-adapter-host exec STORE [--guest-build BUILD] EXECUTION [';' EXECUTION]...\n", " EXECUTION: [--identity UID:GID [--groups GID,...] [--capabilities N,...]", @@ -899,6 +905,9 @@ fn parse(arguments: Vec) -> Result { if scratch.is_none() && options.scratch_journal.is_some() { return Err("--scratch-journal needs --upper scratch:PATH".to_owned()); } + if format == ImageFormat::Erofs && options.first_partition_bound.is_some() { + return Err("--first-partition-bound needs --distro gpt:PATH".to_owned()); + } Command::Start { store, guest_build, @@ -914,6 +923,7 @@ fn parse(arguments: Vec) -> Result { workload_identity: options.workload_identity, overlay_mode: options.overlay_mode, scratch_journal: options.scratch_journal, + first_partition_bound: options.first_partition_bound, }, }), time: host_time_from_env()?, @@ -1003,7 +1013,9 @@ fn parse_option( })?; options.scratch_journal = Some(policy); } - ("launch", "--first-partition-bound") if options.first_partition_bound.is_none() => { + ("launch" | "start", "--first-partition-bound") + if options.first_partition_bound.is_none() => + { let value = text(value)?; let bound = value.parse::().ok().filter(|bound| *bound > 0); options.first_partition_bound = Some(bound.ok_or_else(|| { diff --git a/tests/runtime-adapter/src/clone.rs b/tests/runtime-adapter/src/clone.rs index e877d50f7..57651a3a6 100644 --- a/tests/runtime-adapter/src/clone.rs +++ b/tests/runtime-adapter/src/clone.rs @@ -4,12 +4,12 @@ //! A [`Request`] holds what the host program's options choose of a clone: its image, an EROFS //! layer or a GPT disk of the length of the template's placeholder, its upper, on an ext4 scratch //! image or in RAM, its memory target, the workload identity as which its executions run, its -//! overlay mode, and its scratch's journal policy. [`Request::plan`] plans the image over the -//! upper as a launch plans a layered root, and [`Request::clone_request`] returns NVX's -//! `nvx::host::template::CloneRequest` of it, with the bindings that the sandbox's store stages -//! for its images, as `layered::Staged` records them: staging hashes, and a start never does. -//! NVX's `ClonePlan` then checks the request against the leased generation before any VMM starts, -//! from the metadata of the clone's files alone. +//! overlay mode, its scratch's journal policy, and the bound of a GPT disk's first partition. +//! [`Request::plan`] plans the image over the upper as a launch plans a layered root, and +//! [`Request::clone_request`] returns NVX's `nvx::host::template::CloneRequest` of it, with the +//! bindings that the sandbox's store stages for its images, as `layered::Staged` records them: +//! staging hashes, and a start never does. NVX's `ClonePlan` then checks the request against the +//! leased generation before any VMM starts, from the metadata of the clone's files alone. //! //! A clone's [record](CloneRecord) names the template store and the generation that the clone //! leases, beside the plan and the bindings, as the record of a launch names what it launched, so @@ -76,6 +76,9 @@ pub struct Request { /// Whether the scratch may have a journal, or `None` for the guest's default, which allows /// one. pub scratch_journal: Option, + /// The most bytes that the first partition of a GPT disk may hold, or `None` for + /// [`DEFAULT_FIRST_PARTITION_BOUND`], the default of a launch's guest. + pub first_partition_bound: Option, } impl Request { @@ -102,8 +105,9 @@ impl Request { /// The record of the clone's activation names the workload identity, if any, the /// [`CLONE_HOSTNAME`], the image's own machine ID, the overlay's mode, and the scratch's /// journal policy, each the guest's default unless the request names it, as a root - /// configuration's are, and for a GPT disk, the bound of its first partition that a launch's - /// guest takes by default, [`DEFAULT_FIRST_PARTITION_BOUND`]. + /// configuration's are, and the bound of a GPT disk's first partition, the default of a + /// launch's guest, [`DEFAULT_FIRST_PARTITION_BOUND`], unless the request names one. The record + /// of a clone of an EROFS layer leaves the bound out. /// /// Fails with [`ErrorKind::Unavailable`](nvx::ErrorKind::Unavailable) if the store stages no /// binding for the image, or for the scratch image of an upper on one, at its path. @@ -149,7 +153,9 @@ impl Request { }, overlay, journal, - first_partition_bound: DEFAULT_FIRST_PARTITION_BOUND, + first_partition_bound: self + .first_partition_bound + .unwrap_or(DEFAULT_FIRST_PARTITION_BOUND), }) } } diff --git a/tests/runtime-adapter/tests/clone_start.rs b/tests/runtime-adapter/tests/clone_start.rs index ab2202b45..fe0233d60 100644 --- a/tests/runtime-adapter/tests/clone_start.rs +++ b/tests/runtime-adapter/tests/clone_start.rs @@ -20,6 +20,7 @@ use runtime_adapter::clone::{ CLONE_MODE, CloneRecord, HOST_TIME_VARIABLE, Request, host_time_text, parse_host_time, }; use runtime_adapter::layered::{Bindings, LaunchRecord, RECORD_FORMAT, RECORD_NAME, Staged}; +use runtime_adapter::root::DEFAULT_FIRST_PARTITION_BOUND; use serde_json::{Value, json}; /// The identity that the tests' images have. @@ -123,6 +124,14 @@ fn the_host_program_refuses_options_that_do_not_make_a_start() { with(&["--upper", "ram", "--overlay-mode", "lowers"]), "--overlay-mode takes trusted-lowers or untrusted-lowers", ), + ( + with(&["--upper", "ram", "--first-partition-bound", "4096"]), + "--first-partition-bound needs --distro gpt:PATH", + ), + ( + with(&["--upper", "ram", "--first-partition-bound", "0"]), + "--first-partition-bound takes a positive number of bytes, not \"0\"", + ), ( with(&["--upper", "ram", "--template", "other"]), "unexpected option \"--template\" of start", @@ -153,11 +162,10 @@ fn the_host_program_refuses_options_that_do_not_make_a_start() { ), ]; // Each option that a start does not take. - let others: [(&str, &str); 8] = [ + let others: [(&str, &str); 7] = [ ("--guest-agent", "agent"), ("--hypervisor", "whp"), ("--runtime-dir", "/run/a"), - ("--first-partition-bound", "4096"), ("--share", "a:/b"), ("--network", "network.json"), ("--cpu-profile", "auto"), @@ -340,6 +348,7 @@ fn a_clone_s_record_names_its_template_store_and_generation_beside_its_plan() { workload_identity: Some((1000, 1000)), overlay_mode: None, scratch_journal: None, + first_partition_bound: None, }; let plan = request.plan().unwrap(); let staged = Staged::stage(&distro, ImageFormat::Erofs, identity(), Some(&scratch)).unwrap(); @@ -427,6 +436,7 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { workload_identity: None, overlay_mode: None, scratch_journal: None, + first_partition_bound: None, }; let plan = request.plan().unwrap(); let staged = Staged::stage(&distro, ImageFormat::Erofs, identity(), Some(&scratch)).unwrap(); @@ -451,12 +461,14 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { assert_eq!(clone.workload.identity, None); assert_eq!(clone.overlay, nvx::activation::OverlayMode::TrustedLowers); assert_eq!(clone.journal, nvx::activation::Journal::Allowed); + assert_eq!(clone.first_partition_bound, DEFAULT_FIRST_PARTITION_BOUND); // A request that names its settings. let named = Request { workload_identity: Some((65534, 65533)), overlay_mode: Some(runtime_adapter::root::OverlayMode::UntrustedLowers), scratch_journal: Some(runtime_adapter::root::JournalPolicy::Forbidden), + first_partition_bound: Some(4096), ..request.clone() }; let clone = named @@ -471,6 +483,7 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { ); assert_eq!(clone.overlay, nvx::activation::OverlayMode::UntrustedLowers); assert_eq!(clone.journal, nvx::activation::Journal::Forbidden); + assert_eq!(clone.first_partition_bound, 4096); // An image, or a scratch image, that the store does not stage. for (bindings, what) in [ diff --git a/tests/runtime-adapter/tests/common/gpt.rs b/tests/runtime-adapter/tests/common/gpt.rs new file mode 100644 index 000000000..58e112ba5 --- /dev/null +++ b/tests/runtime-adapter/tests/common/gpt.rs @@ -0,0 +1,382 @@ +//! The GPT disks of the VM tests' layered roots, which [`write_disk`] writes as UEFI lays out a +//! disk, and the small layers that they hold, file systems in the ext2 format that +//! [`small_layer`] writes, which the kernel's ext4 driver mounts, since `mkfs.ext4` is not +//! available everywhere that the tests run. + +use std::fs::{File, OpenOptions}; +use std::io::{self, Seek, SeekFrom, Write}; +use std::path::Path; + +use nvx::gpt::{Disk, Expected, ExpectedPartition}; +use nvx::uuid::Uuid; + +/// The size of a sector of a disk, its logical block, in bytes. +const SECTOR: u64 = 512; +/// The first sector that a partition may take: after the protective MBR, the primary header, and +/// its array of 128 entries of 128 bytes. +const FIRST_USABLE: u64 = 34; +/// The type GUID of every partition of the disks: Linux's file system data. +const LINUX_DATA: &str = "0fc63daf-8483-4772-8e79-3d69d8477de4"; +/// The size of a block of the layer that [`small_layer`] writes, in bytes. +pub(crate) const LAYER_BLOCK: usize = 1024; +/// The number of blocks of that layer: 128 KiB. +pub(crate) const LAYER_BLOCKS: usize = 128; +/// The number of inodes of that layer, of which the first ten are reserved, as in every ext2 file +/// system, and the second is its top directory. +const LAYER_INODES: usize = 16; + +/// The layout of a GPT disk that [`write_disk`] writes. +pub(crate) struct Layout<'a> { + /// The disk's GUID. + pub(crate) guid: &'a str, + /// The partitions, in the order of their entries: the first partition, then the layers. + pub(crate) partitions: &'a [Partition<'a>], + /// The number of sectors of which the first sector of each partition is a multiple: at least + /// [`FIRST_USABLE`]. + pub(crate) alignment: u64, + /// The disk's length, in bytes, whole sectors, or `None` for the partitions' and then + /// `alignment` sectors more, in which the backup array and header end the disk. + pub(crate) length: Option, +} + +/// A partition of a [`Layout`]. +pub(crate) struct Partition<'a> { + /// The partition's unique GUID. + pub(crate) guid: &'a str, + /// The partition's size, in bytes: whole sectors, at least what it holds. + pub(crate) size: u64, + /// What the partition holds from its start, before NUL bytes. + pub(crate) contents: Contents<'a>, +} + +/// What a [`Partition`] holds. +pub(crate) enum Contents<'a> { + /// These bytes. + Bytes(&'a [u8]), + /// The bytes of this file. + File(&'a Path), +} + +/// Writes the GPT disk of `layout` at `path`, as UEFI lays out a disk, with a protective MBR, the +/// primary GPT header and its array of 128 entries, and the backup array and header at the disk's +/// end, of its partitions in their order, each aligned to its `alignment`, after the end of the +/// one before. The file is a new one, which no one else may change: of the mode 0644 on Unix, +/// whatever the umask, and on Windows of its directory's access control list. +/// +/// Then checks the disk as the guest's assembly does, through `nvx::gpt`, with the GUIDs and +/// sizes that it wrote, and returns the bytes of its first partition, as the guest returns them, +/// without their trailing NUL bytes and whitespace. +pub(crate) fn write_disk(path: &Path, layout: &Layout<'_>) -> Vec { + let alignment = layout.alignment; + assert!( + alignment >= FIRST_USABLE, + "a partition starts after the table" + ); + let mut extents = Vec::new(); + let mut next = alignment; + for partition in layout.partitions { + assert_eq!(partition.size % SECTOR, 0, "a partition is whole sectors"); + let last = next + partition.size / SECTOR - 1; + extents.push((next, last)); + next = (last + 1).next_multiple_of(alignment); + } + // The backup array's 32 sectors and the backup header end the disk, after the usable sectors. + let sectors = match layout.length { + None => next + alignment, + Some(length) => { + assert_eq!(length % SECTOR, 0, "a disk is whole sectors"); + length / SECTOR + } + }; + let (first_usable, last_usable) = (FIRST_USABLE, sectors - 34); + assert!( + extents.last().is_some_and(|(_, last)| *last <= last_usable), + "the partitions end before the backup array" + ); + let (backup_entries, backup_header) = (sectors - 33, sectors - 1); + let guid = |text: &str| Uuid::parse(text).unwrap().to_bytes_le(); + + let mut entries = vec![0; 128 * 128]; + for (entry, ((first, last), partition)) in entries + .chunks_exact_mut(128) + .zip(extents.iter().zip(layout.partitions)) + { + entry[0..16].copy_from_slice(&guid(LINUX_DATA)); + entry[16..32].copy_from_slice(&guid(partition.guid)); + entry[32..40].copy_from_slice(&first.to_le_bytes()); + entry[40..48].copy_from_slice(&last.to_le_bytes()); + } + let entries_crc = crc32(&entries); + let header = |own: u64, alternate: u64, array: u64| { + let mut header = vec![0; 92]; + header[0..8].copy_from_slice(b"EFI PART"); + header[8..12].copy_from_slice(&0x0001_0000_u32.to_le_bytes()); + header[12..16].copy_from_slice(&92_u32.to_le_bytes()); + header[24..32].copy_from_slice(&own.to_le_bytes()); + header[32..40].copy_from_slice(&alternate.to_le_bytes()); + header[40..48].copy_from_slice(&first_usable.to_le_bytes()); + header[48..56].copy_from_slice(&last_usable.to_le_bytes()); + header[56..72].copy_from_slice(&guid(layout.guid)); + header[72..80].copy_from_slice(&array.to_le_bytes()); + header[80..84].copy_from_slice(&128_u32.to_le_bytes()); + header[84..88].copy_from_slice(&128_u32.to_le_bytes()); + header[88..92].copy_from_slice(&entries_crc.to_le_bytes()); + let crc = crc32(&header); + header[16..20].copy_from_slice(&crc.to_le_bytes()); + header + }; + // The protective MBR: one record of type 0xEE, from LBA 1 over the rest of the disk, with the + // addresses that UEFI gives it. + let mut mbr = vec![0; 512]; + let protective = u32::try_from(sectors - 1).unwrap_or(u32::MAX); + mbr[446..450].copy_from_slice(&[0x00, 0x00, 0x02, 0x00]); + mbr[450..454].copy_from_slice(&[0xee, 0xff, 0xff, 0xff]); + mbr[454..458].copy_from_slice(&1_u32.to_le_bytes()); + mbr[458..462].copy_from_slice(&protective.to_le_bytes()); + mbr[510..512].copy_from_slice(&[0x55, 0xaa]); + + let mut disk = OpenOptions::new() + .read(true) + .write(true) + .create(true) + .truncate(true) + .open(path) + .unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt as _; + disk.set_permissions(std::fs::Permissions::from_mode(0o644)) + .unwrap(); + } + disk.set_len(sectors * SECTOR).unwrap(); + let table = [ + (0, mbr), + (1, header(1, backup_header, 2)), + (2, entries.clone()), + (backup_entries, entries), + (backup_header, header(backup_header, 1, backup_entries)), + ]; + for (sector, bytes) in table { + disk.seek(SeekFrom::Start(sector * SECTOR)).unwrap(); + disk.write_all(&bytes).unwrap(); + } + for ((first, _), partition) in extents.iter().zip(layout.partitions) { + disk.seek(SeekFrom::Start(first * SECTOR)).unwrap(); + let written = match partition.contents { + Contents::Bytes(bytes) => { + disk.write_all(bytes).unwrap(); + u64::try_from(bytes.len()).unwrap() + } + Contents::File(file) => io::copy(&mut File::open(file).unwrap(), &mut disk).unwrap(), + }; + assert!(written <= partition.size, "a partition holds what it holds"); + } + disk.sync_all().unwrap(); + + let partitions = layout + .partitions + .iter() + .map(|partition| { + ExpectedPartition::new() + .unique_guid(Uuid::parse(partition.guid).unwrap()) + .size(partition.size) + }) + .collect(); + let expected = Expected::new() + .disk_guid(Uuid::parse(layout.guid).unwrap()) + .partitions(partitions); + let read = Disk::read(&disk, 512, sectors * SECTOR, &expected).unwrap(); + let size = layout.partitions[0].size; + read.read_first_partition(&disk, usize::try_from(size).unwrap()) + .unwrap() +} + +/// Returns a small file system in the ext2 format, of [`LAYER_BLOCKS`] blocks of [`LAYER_BLOCK`] +/// bytes, which the kernel's ext4 driver mounts as an ext4 file system without features: its top +/// directory holds the regular `files`, each a name and its bytes, of at most 12 blocks each, +/// then the empty `directories`, each with mode 0755, and at most [`LAYER_INODES`] inodes in all. +/// +/// Block 1 holds the superblock, 2 the group descriptor, 3 and 4 the block and inode bitmaps, 5 +/// and 6 the inode table, 7 the top directory, and the blocks after them the files' bytes and the +/// directories. +pub(crate) fn small_layer(files: &[(&str, &[u8])], directories: &[&str]) -> Vec { + /// Writes `value` at `offset` of `image`, in little-endian order. + fn put(image: &mut [u8], offset: usize, value: [u8; N]) { + image[offset..offset + N].copy_from_slice(&value); + } + /// Writes the inode `number` of `image`, of `mode`, `size` bytes, and `links` links, whose data + /// is in `blocks`, at most the 12 that an inode addresses directly. + fn inode( + image: &mut [u8], + number: usize, + mode: u16, + size: usize, + links: usize, + blocks: &[usize], + ) { + assert!( + blocks.len() <= 12, + "an inode of the layer addresses 12 blocks" + ); + let offset = 5 * LAYER_BLOCK + (number - 1) * 128; + put(image, offset, mode.to_le_bytes()); + put( + image, + offset + 4, + u32::try_from(size).unwrap().to_le_bytes(), + ); + put( + image, + offset + 26, + u16::try_from(links).unwrap().to_le_bytes(), + ); + // The number of 512-byte sectors that the data takes. + let sectors = u32::try_from(blocks.len() * LAYER_BLOCK / 512).unwrap(); + put(image, offset + 28, sectors.to_le_bytes()); + for (index, block) in blocks.iter().enumerate() { + let block = u32::try_from(*block).unwrap(); + put(image, offset + 40 + 4 * index, block.to_le_bytes()); + } + } + /// Writes the directory `entries` into `block` of `image`, each an inode, a name, and a file + /// type, the last of which spans the rest of the block. + fn directory(image: &mut [u8], block: usize, entries: &[(usize, &str, u8)]) { + let (mut offset, end) = (block * LAYER_BLOCK, (block + 1) * LAYER_BLOCK); + for (index, (inode, name, kind)) in entries.iter().enumerate() { + let length = if index + 1 == entries.len() { + end - offset + } else { + (8 + name.len()).next_multiple_of(4) + }; + put(image, offset, u32::try_from(*inode).unwrap().to_le_bytes()); + put( + image, + offset + 4, + u16::try_from(length).unwrap().to_le_bytes(), + ); + image[offset + 6] = u8::try_from(name.len()).unwrap(); + image[offset + 7] = *kind; + image[offset + 8..offset + 8 + name.len()].copy_from_slice(name.as_bytes()); + offset += length; + } + } + /// Sets the bits `bits` of the bitmap in `block` of `image`. + fn set(image: &mut [u8], block: usize, bits: std::ops::Range) { + for bit in bits { + image[block * LAYER_BLOCK + bit / 8] |= 1 << (bit % 8); + } + } + + let (top, regular, directory_type) = (2, 1, 2); + let mut image = vec![0; LAYER_BLOCK * LAYER_BLOCKS]; + // The first block and the first inode that are free. + let (mut block, mut number) = (8, 11); + let mut entries = vec![(top, ".", directory_type), (top, "..", directory_type)]; + for (name, bytes) in files { + let blocks = (block..block + bytes.len().div_ceil(LAYER_BLOCK)).collect::>(); + image[block * LAYER_BLOCK..block * LAYER_BLOCK + bytes.len()].copy_from_slice(bytes); + inode(&mut image, number, 0o100_644, bytes.len(), 1, &blocks); + entries.push((number, *name, regular)); + block += blocks.len(); + number += 1; + } + for name in directories { + directory( + &mut image, + block, + &[(number, ".", directory_type), (top, "..", directory_type)], + ); + inode(&mut image, number, 0o040_755, LAYER_BLOCK, 2, &[block]); + entries.push((number, *name, directory_type)); + block += 1; + number += 1; + } + directory(&mut image, 7, &entries); + inode( + &mut image, + top, + 0o040_755, + LAYER_BLOCK, + 2 + directories.len(), + &[7], + ); + assert!( + block <= LAYER_BLOCKS && number <= LAYER_INODES + 1, + "the layer is too small" + ); + + // The bitmaps of the used blocks, from block 1, and inodes, from inode 1, whose bits past the + // last block and inode are set, as `mke2fs` sets them. + set(&mut image, 3, 0..block - 1); + set(&mut image, 3, LAYER_BLOCKS - 1..LAYER_BLOCK * 8); + set(&mut image, 4, 0..number - 1); + set(&mut image, 4, LAYER_INODES..LAYER_BLOCK * 8); + let free_blocks = LAYER_BLOCKS - block; + let free_inodes = LAYER_INODES + 1 - number; + + // The superblock: one group of blocks, inodes of 128 bytes, the first of them not reserved, + // 11, and only the `filetype` feature. + let superblock = LAYER_BLOCK; + for (offset, value) in [ + (0x00, LAYER_INODES), + (0x04, LAYER_BLOCKS), + (0x0c, free_blocks), + (0x10, free_inodes), + (0x14, 1), + (0x20, LAYER_BLOCK * 8), + (0x24, LAYER_BLOCK * 8), + (0x28, LAYER_INODES), + (0x4c, 1), + (0x54, 11), + (0x60, 0x2), + ] { + let value = u32::try_from(value).unwrap(); + put(&mut image, superblock + offset, value.to_le_bytes()); + } + for (offset, value) in [ + (0x36, 0xffff), + (0x38, 0xef53), + (0x3a, 1), + (0x3c, 1), + (0x58, 128), + ] { + put(&mut image, superblock + offset, u16::to_le_bytes(value)); + } + let uuid = Uuid::parse("2d4e6f80-91a2-4b3c-8d5e-6f708192a3b4").unwrap(); + image[superblock + 0x68..superblock + 0x78].copy_from_slice(uuid.as_bytes()); + + // The group descriptor: where the bitmaps and the inode table are, and what is free. + let descriptor = 2 * LAYER_BLOCK; + for (offset, value) in [(0x00, 3), (0x04, 4), (0x08, 5)] { + put(&mut image, descriptor + offset, u32::to_le_bytes(value)); + } + for (offset, value) in [ + (0x0c, free_blocks), + (0x0e, free_inodes), + (0x10, 1 + directories.len()), + ] { + let value = u16::try_from(value).unwrap(); + put(&mut image, descriptor + offset, value.to_le_bytes()); + } + image +} + +/// Returns the CRC-32 of `bytes`, which GPT's headers and entry arrays hold: of the reflected +/// polynomial 0xEDB88320, from all ones, inverted. +fn crc32(bytes: &[u8]) -> u32 { + !bytes.iter().fold(!0, |crc, byte| { + (0..8).fold(crc ^ u32::from(*byte), |crc: u32, _| { + (crc >> 1) ^ (0xedb8_8320 & (crc & 1).wrapping_neg()) + }) + }) +} + +/// Returns `bytes` in lowercase hexadecimal, as the `first-partition` probe reports them. +pub(crate) fn hexadecimal(bytes: &[u8]) -> String { + const DIGITS: &[u8; 16] = b"0123456789abcdef"; + bytes + .iter() + .flat_map(|byte| [byte >> 4, byte & 0xf]) + .map(|digit| char::from(DIGITS[usize::from(digit)])) + .collect() +} diff --git a/tests/runtime-adapter/tests/common/mod.rs b/tests/runtime-adapter/tests/common/mod.rs index 061dd6c97..67604ddff 100644 --- a/tests/runtime-adapter/tests/common/mod.rs +++ b/tests/runtime-adapter/tests/common/mod.rs @@ -12,6 +12,8 @@ // Each test crate uses its own part of this module. #![allow(dead_code)] +pub(crate) mod gpt; + use std::collections::BTreeMap; use std::convert::Infallible; use std::env; diff --git a/tests/runtime-adapter/tests/vm_clone.rs b/tests/runtime-adapter/tests/vm_clone.rs index e8b1a67ab..aae54e246 100644 --- a/tests/runtime-adapter/tests/vm_clone.rs +++ b/tests/runtime-adapter/tests/vm_clone.rs @@ -5,15 +5,22 @@ //! //! Each scenario captures a [`Template`] of its own, with 256 MiB of RAM, of a capacity of //! 1 GiB, in a template store of its own, as `vm_template` does, of the platform's copy of the -//! fixtures' Ubuntu EROFS image, and starts its clones, each in a sandbox store of its own, which -//! stages the clone's images, as a launch's does: that copy, and for a clone over a scratch image, -//! a copy of the scratch template, which, as NVX requires of a clone's images, no one else may -//! change either. +//! fixtures' Ubuntu EROFS image, or of a GPT [`Disk`] of the image's length, and starts its +//! clones, each in a sandbox store of its own, which stages the clone's images, as a launch's +//! does: that copy, or a GPT disk, and for a clone over a scratch image, a copy of the scratch +//! template, which, as NVX requires of a clone's images, no one else may change either. //! //! - One generation serves clones over a scratch image and in RAM, at once and in turn, at each //! memory target from the template's RAM to its capacity, which the guest has online. The store //! leases the generation to each, a stop releases it, and the clones change nothing of the //! generation's snapshot, and leave nothing in its payload. +//! - One generation serves clones of either format over either upper, at once: the platform's +//! EROFS image, and GPT disks of other layouts than the template's placeholder, a GPT disk too, +//! and than each other's. The guest of a GPT clone rereads its disk's partition table before it +//! assembles the root, since its kernel created the partitions of the placeholder, so each +//! clone's root stacks its own disk's layers, from the partitions that its disk's table names. +//! A disk whose first partition is larger than the clone's bound ends the clone before it is +//! ready. //! - The store and NVX's plan refuse a clone that the template cannot serve before any VMM starts, //! each a failure of the request, as NVX attributes it by its evidence, with no VM: an image or a //! scratch image of another length than the template's, an image of a geometry that OpenVMM @@ -24,6 +31,8 @@ //! file system, a journal that the clone's record forbids, a workload identity whose account the //! image lacks, and a host that sends anything but the clone's Activate first, a ping, a start, or //! an execution. The sandbox records the failure until the release of its lease. +//! - A clone refuses a scratch image whose upper is not fresh, as an earlier clone left it, and +//! ends before it is ready. //! - A clone runs its executions as its record's workload identity, once it is ready: the repair //! acknowledged the restore before the clone took its reset, the host's first message came //! after that, and the clone's Ready, its start, and its first execution came in that order. @@ -91,8 +100,10 @@ use runtime_adapter::exec::{HostMessage, Start}; use runtime_adapter::frame::Message; use runtime_adapter::host::Client; use runtime_adapter::layered::Staged; +use runtime_adapter::root::DEFAULT_RUNTIME; use runtime_adapter::runtime::Runtime; +use common::gpt::{self, Contents, Layout, Partition, hexadecimal, small_layer}; use common::{Fixtures, Printed, Sandbox}; /// The exit status of the host program's command that failed, after a diagnostic. @@ -123,15 +134,106 @@ const HOUR_NS: u64 = 3_600_000_000_000; /// The error number with which the guest refuses an execution that names an identity in a /// sandbox that has its own: `EINVAL`. const EINVAL: i32 = 22; +/// The placeholder of the template of the scenarios of GPT disks: a disk of two layers, whose +/// partitions start at multiples of 1 MiB. +const PLACEHOLDER: Disk = Disk { + guid: "5c0f2a8e-3b71-4d96-8e04-7a1c9b2d6e30", + partitions: &[ + "5c0f2a8e-3b71-4d96-8e04-7a1c9b2d6e31", + "5c0f2a8e-3b71-4d96-8e04-7a1c9b2d6e32", + "5c0f2a8e-3b71-4d96-8e04-7a1c9b2d6e33", + ], + alignment: 2048, + first_size: 4096, + first: b"\xffthe placeholder's first partition", +}; +/// A clone's GPT disk of one layer, whose partitions start at multiples of 2 MiB, with a larger +/// first partition than the placeholder's: its layer's partition starts where none of the +/// placeholder's does. +const ONE_LAYER: Disk = Disk { + guid: "a17c3e90-6d24-4f5b-b8e1-02c9d4f6a8b0", + partitions: &[ + "a17c3e90-6d24-4f5b-b8e1-02c9d4f6a8b1", + "a17c3e90-6d24-4f5b-b8e1-02c9d4f6a8b2", + ], + alignment: 4096, + first_size: 8192, + first: b"\xfe\0one layer\x80", +}; +/// A clone's GPT disk of three layers, whose partitions start at multiples of 640 KiB, with the +/// largest first partition: its layers' partitions start where none of the placeholder's do, nor +/// [`ONE_LAYER`]'s. +const THREE_LAYERS: Disk = Disk { + guid: "b2e4f6a8-0c1d-4e3f-9a5b-6c7d8e9f0a10", + partitions: &[ + "b2e4f6a8-0c1d-4e3f-9a5b-6c7d8e9f0a11", + "b2e4f6a8-0c1d-4e3f-9a5b-6c7d8e9f0a12", + "b2e4f6a8-0c1d-4e3f-9a5b-6c7d8e9f0a13", + "b2e4f6a8-0c1d-4e3f-9a5b-6c7d8e9f0a14", + ], + alignment: 1280, + first_size: 12288, + first: b"\xfd\tthree layers\xc3", +}; + +/// A GPT disk of the scenarios, of the length of the platform's image, which +/// [`Template::of_disk`] captures as the template's placeholder, or [`Template::disk`] writes for +/// a clone. Its partitions follow each other, each at a multiple of its alignment: its first +/// partition, then a layer, an empty [`small_layer`], in each of the others. +struct Disk { + /// The disk's GUID, which the guest expects of it. + guid: &'static str, + /// The unique GUIDs of its partitions: the first partition's, then each layer's, from the base + /// up. + partitions: &'static [&'static str], + /// The number of sectors of which the first sector of each partition is a multiple. + alignment: u64, + /// The size of its first partition, in bytes. + first_size: u64, + /// The bytes that its first partition holds, before NUL bytes: those that the guest returns, + /// which end in neither a NUL byte nor whitespace. + first: &'static [u8], +} + +impl Disk { + /// Writes the disk at `path`, of `length` bytes, a file that no one else may change. + fn write(&self, path: &Path, length: u64) { + let layer = small_layer(&[], &[]); + let partitions = self + .partitions + .iter() + .enumerate() + .map(|(index, guid)| { + let (size, contents) = match index { + 0 => (self.first_size, self.first), + _ => (u64::try_from(layer.len()).unwrap(), layer.as_slice()), + }; + Partition { + guid, + size, + contents: Contents::Bytes(contents), + } + }) + .collect::>(); + let layout = Layout { + guid: self.guid, + partitions: &partitions, + alignment: self.alignment, + length: Some(length), + }; + assert_eq!(gpt::write_disk(path, &layout), self.first); + } +} /// The template of a scenario: the generation that a template store of its own captured, in the -/// scenario's sandbox, of the platform's copy of the fixtures' Ubuntu EROFS image, with -/// [`TEMPLATE_MEMORY`] MiB of RAM, of a capacity of [`TEMPLATE_CAPACITY`] MiB. +/// scenario's sandbox, of the platform's copy of the fixtures' Ubuntu EROFS image, or of a GPT +/// [`Disk`] of its length, with [`TEMPLATE_MEMORY`] MiB of RAM, of a capacity of +/// [`TEMPLATE_CAPACITY`] MiB. struct Template { /// The scenario's sandbox, whose store is the template store, until the template drops. sandbox: Option, - /// The platform's copy of the image, the template's placeholder, which the clones stage as - /// their image too. + /// The platform's copy of the image, which the clones stage as their image unless they name + /// another, and the template's placeholder, unless that is a GPT disk. image: PathBuf, /// The image's identity. identity: String, @@ -141,22 +243,42 @@ struct Template { impl Template { /// Creates the sandbox of the scenario `name`, stages the platform's copy of the image, and - /// captures the template in the sandbox's store. + /// captures the template of the image in the sandbox's store. /// /// On Linux, a clone's control socket, `generations/1/LAUNCH.sock` in the template store, /// after the 32 hexadecimal digits of the launch, must fit the path of a Unix socket, so the /// scenarios' names are short. fn create(name: &str) -> Self { + Self::capture(name, None) + } + + /// Creates the sandbox of the scenario `name`, stages the platform's copy of the image, and + /// captures the template of the GPT disk `placeholder`, of the image's length, which the + /// platform stages beside it, as [`create`](Self::create) does. + fn of_disk(name: &str, placeholder: &Disk) -> Self { + Self::capture(name, Some(placeholder)) + } + + /// Creates the sandbox of the scenario `name`, stages the platform's copy of the image, and + /// captures the template of the GPT disk `placeholder`, if any, and of the image otherwise. + fn capture(name: &str, placeholder: Option<&Disk>) -> Self { let mut sandbox = Sandbox::new(name); let fixtures = Fixtures::find(); let image = stage_platform(&sandbox.root.join("platform"), &fixtures); - let distro = erofs(&image); + let (distro, identity) = match placeholder { + None => (erofs(&image), fixtures.identity.as_str()), + Some(disk) => { + let path = image.with_file_name("placeholder.gpt"); + disk.write(&path, fs::metadata(&image).unwrap().len()); + (prefixed("gpt:", &path), disk.guid) + } + }; let (memory, capacity) = (TEMPLATE_MEMORY.to_string(), TEMPLATE_CAPACITY.to_string()); let options = [ OsStr::new("--distro"), &distro, OsStr::new("--distro-identity"), - OsStr::new(&fixtures.identity), + OsStr::new(identity), OsStr::new("--memory"), OsStr::new(&memory), OsStr::new("--memory-capacity"), @@ -218,6 +340,14 @@ impl Template { copy } + /// Writes the GPT disk `disk`, of the length of the platform's image, which a clone restores + /// in place of the template's placeholder, beside the image, at `name`, and returns its path. + fn disk(&self, name: &str, disk: &Disk) -> PathBuf { + let path = self.image.with_file_name(name); + disk.write(&path, fs::metadata(&self.image).unwrap().len()); + path + } + /// Stages the template's image in the store of `sandbox`, which it creates, for a clone in /// RAM. fn stage(&self, sandbox: &mut Sandbox) { @@ -235,12 +365,15 @@ impl Template { /// start exits with `status`, and returns the clone. fn start(&self, name: &str, start: StartOf<'_>, status: i32) -> Cloned { let mut sandbox = Sandbox::new(name); - let distro = erofs(start.image.unwrap_or(&self.image)); + let (distro, identity) = match start.image { + None => (erofs(&self.image), self.identity.as_str()), + Some(Image::Erofs(path)) => (erofs(path), self.identity.as_str()), + Some(Image::Gpt(disk, path)) => (prefixed("gpt:", path), disk.guid), + }; // NVX restores a clone only over images that no one else may change, so the scratch image // lies in a directory of the sandbox's own, as the platform's image lies in the platform's. - let images = sandbox.root.join("images"); - SandboxStore::create(&images).unwrap(); - let scratch = images.join("scratch.ext4"); + let scratch = scratch_image(&sandbox); + SandboxStore::create(scratch.parent().unwrap()).unwrap(); match start.upper { Upper::Ram => {} Upper::Scratch => { @@ -253,12 +386,15 @@ impl Template { Upper::Zeros => private_file(&scratch) .set_len(DEFAULT_SCRATCH_LENGTH) .unwrap(), + Upper::Used(used) => { + private_copy(used, &scratch); + } } let upper = match start.upper { Upper::Ram => OsString::from("ram"), _ => prefixed("scratch:", &scratch), }; - let identity = OsStr::new(&self.identity); + let identity = OsStr::new(identity); if !start.unstaged { let mut staging = vec![ OsStr::new("--distro"), @@ -332,7 +468,7 @@ impl Drop for Template { /// The upper of a clone. #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] -enum Upper { +enum Upper<'a> { /// RAM. #[default] Ram, @@ -343,15 +479,27 @@ enum Upper { Resized(u64), /// Zeros of the template's scratch length, which hold no file system. Zeros, + /// A copy of the scratch image at this path, which an earlier clone used, so that its upper is + /// not fresh. + Used(&'a Path), +} + +/// A clone's image other than the platform's. +#[derive(Clone, Copy)] +enum Image<'a> { + /// An EROFS image at this path, a variant of the platform's, with its identity. + Erofs(&'a Path), + /// A GPT disk of this layout at this path, which [`Template::disk`] wrote. + Gpt(&'a Disk, &'a Path), } /// What a scenario asks of the start of a clone. #[derive(Default)] struct StartOf<'a> { - /// The image, or the template's image by default. - image: Option<&'a Path>, + /// The image, or the platform's image by default. + image: Option>, /// The upper. - upper: Upper, + upper: Upper<'a>, /// The start's other options. options: &'a [&'a str], /// The host's time at which the store checks the generation's eligibility, as the clock @@ -435,6 +583,48 @@ impl Cloned { let store = SandboxStore::open(&self.sandbox.store).unwrap(); assert!(Lease::of(&store).unwrap().is_none()); } + + /// Checks that the start started the clone's VM, which ended before the clone was ready, with + /// `status`, the guest's status of its activation's failure, and that the start printed the + /// failure's record, of the request, by `evidence`, at the activation, once the VM exited, + /// quarantined nothing, and released the lease of `template`'s generation, which no other + /// clone holds, with the records of the clone's launch. + fn assert_unactivated(&self, template: &Template, evidence: &str, status: u8) { + let printed = &self.started; + assert!(printed.contains_key("pid"), "{printed:?}"); + assert!(!printed.contains_key("root"), "{printed:?}"); + let status = status.to_string(); + assert_eq!( + failure(printed), + [ + "request", + evidence, + "activation", + "activating", + status.as_str(), + "exited" + ], + "{printed:?}" + ); + assert_eq!(printed["quarantined"], "no"); + assert_eq!(printed["released"], printed["lease"], "{printed:?}"); + assert!(self.sandbox.record().is_none()); + // The release removed the records of the clone's launch. + let store = SandboxStore::open(&self.sandbox.store).unwrap(); + assert!(CloneRecords::read(&store).unwrap().is_none()); + assert!(template.leases().is_empty()); + } + + /// Returns the clone's scratch image, for a clone over one. + fn scratch(&self) -> PathBuf { + scratch_image(&self.sandbox) + } +} + +/// Returns the scratch image of a clone over one, in the clone's sandbox, `sandbox`, in a +/// directory of the sandbox's own. +fn scratch_image(sandbox: &Sandbox) -> PathBuf { + sandbox.root.join("images").join("scratch.ext4") } /// Stages the platform's copy of the fixtures' Ubuntu EROFS image in the directory `platform`, @@ -550,11 +740,60 @@ fn assert_memory(facts: &Printed, target: u64) { assert!(number("total") <= target * MIB, "{facts:?}"); } -/// Checks the root of a clone over `upper`, as its `mounts` and `contents` probes found it, in -/// the overlay mode that trusts the lowers, or not. -fn assert_root(facts: &Printed, upper: Upper, trusted: bool) { +/// Checks the root of a clone of the platform's image over `upper`, as its `mounts` and `contents` +/// probes found it, in the overlay mode that trusts the lowers, or not. +fn assert_root(facts: &Printed, upper: Upper<'_>, trusted: bool) { assert_eq!(facts["mounts.distro.count"], "1", "{facts:?}"); assert_eq!(facts["mounts.distro.type"], "erofs", "{facts:?}"); + assert_overlay(facts, upper, trusted, "ubuntu"); +} + +/// Checks the root of a clone of the GPT disk `disk` over `upper`, as its `devices`, `mounts`, +/// `contents`, and `first-partition` probes found it, in the overlay mode that trusts the lowers: +/// each of the disk's layers, an ext4 file system, mounted once, read-only, from the partition of +/// the clone's one read-only disk that the disk's table names, which the overlay stacks, the base +/// last; and the bytes of the disk's first partition. +fn assert_gpt_root(facts: &Printed, disk: &Disk, upper: Upper<'_>) { + let read_only = facts["devices.disks"] + .split(' ') + .filter_map(|device| device.strip_suffix(":ro")) + .filter_map(|device| device.split(':').next()) + .collect::>(); + let &[name] = read_only.as_slice() else { + panic!("the clone has one read-only disk: {facts:?}"); + }; + let numbers = 2..=disk.partitions.len(); + for number in numbers.clone() { + let fact = |key: &str| facts[&format!("mounts.p{number}.{key}")].as_str(); + let source = format!("/dev/{name}{number}"); + assert_eq!( + [fact("count"), fact("type"), fact("source")], + ["1", "ext4", source.as_str()], + "{facts:?}" + ); + let options = fact("options").split(',').collect::>(); + assert!(options.contains(&"ro"), "{facts:?}"); + } + let lowers = numbers + .rev() + .map(|number| format!("{DEFAULT_RUNTIME}/layers/p{number}")) + .collect::>(); + let stacked = format!("lowerdir={}", lowers.join(":")); + let overlay = facts["mounts.root.super"].split(',').collect::>(); + assert!( + overlay.contains(&stacked.as_str()), + "{stacked} in {facts:?}" + ); + assert_overlay(facts, upper, true, "absent"); + let (length, bytes) = (disk.first.len().to_string(), hexadecimal(disk.first)); + assert_eq!(facts["first-partition.length"], length, "{facts:?}"); + assert_eq!(facts["first-partition.bytes"], bytes, "{facts:?}"); +} + +/// Checks the upper, the overlay, and the contents of the root of a clone over `upper`, as its +/// `mounts` and `contents` probes found them, in the overlay mode that trusts the lowers, or not, +/// whose `os-release` names `identifier`, or `absent` for a root without one. +fn assert_overlay(facts: &Printed, upper: Upper<'_>, trusted: bool, identifier: &str) { let (used, unused, kind) = match upper { Upper::Ram => ("tmpfs", "scratch", "tmpfs"), _ => ("scratch", "tmpfs", "ext4"), @@ -575,7 +814,7 @@ fn assert_root(facts: &Printed, upper: Upper, trusted: bool) { .map(|key| &facts[&format!("contents.{key}")]); assert_eq!( contents, - ["ubuntu", "runtime-adapter", "runtime-adapter", "refused"], + [identifier, "runtime-adapter", "runtime-adapter", "refused"], "{facts:?}" ); } @@ -705,6 +944,7 @@ fn refuses_early(template: &Template, early: Message) { workload_identity: None, overlay_mode: None, scratch_journal: None, + first_partition_bound: None, }; let plan = request.plan().unwrap(); let staged = Staged::read(&sandbox.store).unwrap().unwrap(); @@ -866,6 +1106,76 @@ fn one_generation_serves_clones_of_each_upper_and_memory_target() { assert_payload(&template, false); } +#[test] +#[ignore = "boots VMs; see the module documentation for what they need"] +fn one_generation_serves_clones_of_either_format_over_either_upper() { + let template = Template::of_disk("cl-gpt", &PLACEHOLDER); + let one = template.disk("one.gpt", &ONE_LAYER); + let three = template.disk("three.gpt", &THREE_LAYERS); + + // At once: the platform's image over a scratch image and in RAM, and two GPT disks, of other + // layouts than the placeholder's and than each other's, one in RAM, and one over a scratch + // image, with a bound that its first partition meets exactly. + let bound = THREE_LAYERS.first_size.to_string(); + let exact = ["--first-partition-bound", bound.as_str()]; + let starts = [ + ("cl-e", None, Upper::Scratch, &[][..]), + ("cl-f", None, Upper::Ram, &[][..]), + ( + "cl-g", + Some(Image::Gpt(&ONE_LAYER, &one)), + Upper::Ram, + &[][..], + ), + ( + "cl-h", + Some(Image::Gpt(&THREE_LAYERS, &three)), + Upper::Scratch, + &exact[..], + ), + ]; + let mut clones = starts.map(|(name, image, upper, options)| { + let start = StartOf { + image, + upper, + options, + ..StartOf::default() + }; + let cloned = template.start(name, start, 0); + cloned.assert_started(); + (image, upper, cloned) + }); + assert_eq!(template.leases().len(), clones.len()); + assert_payload(&template, true); + for (image, upper, cloned) in &mut clones { + let facts = cloned.probe(&["devices", "mounts", "contents", "first-partition"]); + match image { + Some(Image::Gpt(disk, _)) => assert_gpt_root(&facts, disk, *upper), + _ => { + assert_root(&facts, *upper, true); + assert_eq!(facts["first-partition.length"], "none", "{facts:?}"); + } + } + } + for (_, _, cloned) in &mut clones { + cloned.stop(); + } + assert!(template.leases().is_empty()); + + // A disk whose first partition is larger than the clone's bound ends the clone before it is + // ready. + let under = (ONE_LAYER.first_size - 1).to_string(); + let options = ["--first-partition-bound", under.as_str()]; + let start = StartOf { + image: Some(Image::Gpt(&ONE_LAYER, &one)), + options: &options, + ..StartOf::default() + }; + let ended = template.start("cl-g", start, CLONE_FAILED); + ended.assert_unactivated(&template, "unusable-root", status::ACTIVATION_ROOT); + assert_payload(&template, false); +} + #[test] #[ignore = "boots VMs; see the module documentation for what they need"] fn a_start_refuses_what_the_template_cannot_serve_before_any_vmm_starts() { @@ -884,7 +1194,7 @@ fn a_start_refuses_what_the_template_cannot_serve_before_any_vmm_starts() { let cases = [ ( StartOf { - image: Some(&longer), + image: Some(Image::Erofs(&longer)), ..StartOf::default() }, "supplied-length", @@ -892,7 +1202,7 @@ fn a_start_refuses_what_the_template_cannot_serve_before_any_vmm_starts() { ), ( StartOf { - image: Some(&shorter), + image: Some(Image::Erofs(&shorter)), upper: Upper::Scratch, ..StartOf::default() }, @@ -901,7 +1211,7 @@ fn a_start_refuses_what_the_template_cannot_serve_before_any_vmm_starts() { ), ( StartOf { - image: Some(&ragged), + image: Some(Image::Erofs(&ragged)), ..StartOf::default() }, "supplied-geometry", @@ -1015,31 +1325,9 @@ fn a_clone_whose_activation_fails_ends_before_it_is_ready() { ]; for (start, (evidence, expected)) in cases { let ended = template.start("cl-u", start, CLONE_FAILED); - let printed = &ended.started; // The VM started, and ended before the clone was ready, with the status of the failure's // class, which attributes the failure to the request, and the start released the lease. - assert!(printed.contains_key("pid"), "{printed:?}"); - assert!(!printed.contains_key("root"), "{printed:?}"); - let status = expected.to_string(); - assert_eq!( - failure(printed), - [ - "request", - evidence, - "activation", - "activating", - status.as_str(), - "exited" - ], - "{printed:?}" - ); - assert_eq!(printed["quarantined"], "no"); - assert_eq!(printed["released"], printed["lease"], "{printed:?}"); - assert!(ended.sandbox.record().is_none()); - // The release removed the records of the clone's launch. - let store = SandboxStore::open(&ended.sandbox.store).unwrap(); - assert!(CloneRecords::read(&store).unwrap().is_none()); - assert!(template.leases().is_empty()); + ended.assert_unactivated(&template, evidence, expected); } // Until the clone is ready, it takes nothing but its Activate: a ping, a start, and the start @@ -1056,6 +1344,37 @@ fn a_clone_whose_activation_fails_ends_before_it_is_ready() { assert_payload(&template, false); } +#[test] +#[ignore = "boots VMs; see the module documentation for what they need"] +fn a_clone_refuses_a_scratch_image_whose_upper_an_earlier_clone_used() { + let template = Template::create("cl-fresh"); + // A clone over a scratch image, in whose root the `contents` probe writes a file, which + // reaches the upper. + let start = StartOf { + upper: Upper::Scratch, + ..StartOf::default() + }; + let mut earlier = template.start("cl-f", start, 0); + earlier.assert_started(); + let facts = earlier.probe(&["contents"]); + assert_eq!( + facts["contents.write.upper"], "runtime-adapter", + "{facts:?}" + ); + earlier.stop(); + + // A clone over a copy of its scratch image, whose upper is not fresh, ends before it is + // ready. + let used = earlier.scratch(); + let start = StartOf { + upper: Upper::Used(&used), + ..StartOf::default() + }; + let ended = template.start("cl-u", start, CLONE_FAILED); + ended.assert_unactivated(&template, "unusable-root", status::ACTIVATION_ROOT); + assert_payload(&template, false); +} + #[test] #[ignore = "boots VMs; see the module documentation for what they need"] fn a_clone_runs_its_executions_as_its_workload_identity_once_it_is_ready() { diff --git a/tests/runtime-adapter/tests/vm_layered.rs b/tests/runtime-adapter/tests/vm_layered.rs index 0bd6eccca..8fd2e2272 100644 --- a/tests/runtime-adapter/tests/vm_layered.rs +++ b/tests/runtime-adapter/tests/vm_layered.rs @@ -53,8 +53,6 @@ use std::thread; use std::time::{Duration, Instant}; use nvx::exec::{Key, Limits, Timers, Window}; -use nvx::gpt::{Disk, Expected, ExpectedPartition}; -use nvx::uuid::Uuid; use runtime_adapter::GUEST_BUILD; use runtime_adapter::exec::Start; use runtime_adapter::frame::MAX_BODY; @@ -64,6 +62,9 @@ use runtime_adapter::root::{ }; use serde_json::{Value, json}; +use common::gpt::{ + self, Contents, LAYER_BLOCK, LAYER_BLOCKS, Layout, Partition, hexadecimal, small_layer, +}; use common::{Fixtures, Printed, Sandbox}; /// The exit status of the host program's command that failed, after a diagnostic. @@ -89,8 +90,6 @@ const PARTITION_GUIDS: [&str; 3] = [ "0b7e4a2c-5d16-4f83-a9c0-1e2d3f4a5b62", "0b7e4a2c-5d16-4f83-a9c0-1e2d3f4a5b63", ]; -/// The type GUID of every partition of the scenarios' GPT disks: Linux's file system data. -const LINUX_DATA: &str = "0fc63daf-8483-4772-8e79-3d69d8477de4"; /// The size of the first partition of the scenarios' GPT disks, in bytes. const FIRST_PARTITION_SIZE: u64 = 4096; /// The bytes of the first partition that the guest returns: bytes that are not UTF-8, with a NUL @@ -108,13 +107,6 @@ const LAYER_FILE: &str = "layer-file"; const LAYER_FILE_SIZE: usize = 2500; /// The directory that the top layer of the scenarios' GPT disks holds. const LAYER_DIRECTORY: &str = "layer-directory"; -/// The size of a block of the layer that [`small_layer`] writes, in bytes. -const LAYER_BLOCK: usize = 1024; -/// The number of blocks of that layer: 128 KiB. -const LAYER_BLOCKS: usize = 128; -/// The number of inodes of that layer, of which the first ten are reserved, as in every ext2 file -/// system, and the second is its top directory. -const LAYER_INODES: usize = 16; /// The directory of a scenario's sandbox that holds the host directories and the file that a /// mapping table shares, as [`write_sources`] writes them. const SOURCES: &str = "sources"; @@ -2109,9 +2101,8 @@ fn tree(directory: &Path) -> BTreeMap> { found } -/// Writes the GPT disk of the scenarios at `path`, as UEFI lays out a disk, with a protective MBR, -/// the primary GPT header and its array of 128 entries, and the backup array and header at the -/// disk's end, of its partitions in this order, each aligned to [`ALIGNMENT`]: +/// Writes the GPT disk of the scenarios at `path`, with [`gpt::write_disk`], of its partitions in +/// this order, each aligned to [`ALIGNMENT`]: /// /// 1. the first partition, of [`FIRST_PARTITION_SIZE`] bytes, which holds [`FIRST_PARTITION`], /// then [`FIRST_PARTITION_PADDING`], then NUL bytes; @@ -2120,15 +2111,10 @@ fn tree(directory: &Path) -> BTreeMap> { /// 3. the top layer, a [`small_layer`] that holds [`LAYER_FILE`] and [`LAYER_DIRECTORY`], with /// the corruption or journal that `top_layer` names. /// -/// Then checks the disk as the guest's assembly does, through `nvx::gpt`, with the GUIDs and -/// sizes that it wrote. +/// Then checks that the first partition holds [`FIRST_PARTITION`] without its padding, as the +/// guest returns it. fn write_gpt_disk(path: &Path, template: &Path, top_layer: TopLayer) { let template_size = fs::metadata(template).unwrap().len(); - assert_eq!( - template_size % SECTOR, - 0, - "the scratch template is whole sectors" - ); let contents = b"runtime-adapter " .iter() .copied() @@ -2142,274 +2128,31 @@ fn write_gpt_disk(path: &Path, template: &Path, top_layer: TopLayer) { TopLayer::CyclicJournal => add_cyclic_journal(&mut top, false), TopLayer::NeedsRecovery => add_cyclic_journal(&mut top, true), } - let sizes = [ - FIRST_PARTITION_SIZE, - template_size, - u64::try_from(top.len()).unwrap(), + let first = [FIRST_PARTITION, FIRST_PARTITION_PADDING].concat(); + let partitions = [ + Partition { + guid: PARTITION_GUIDS[0], + size: FIRST_PARTITION_SIZE, + contents: Contents::Bytes(&first), + }, + Partition { + guid: PARTITION_GUIDS[1], + size: template_size, + contents: Contents::File(template), + }, + Partition { + guid: PARTITION_GUIDS[2], + size: u64::try_from(top.len()).unwrap(), + contents: Contents::Bytes(&top), + }, ]; - let mut extents = Vec::new(); - let mut next = ALIGNMENT; - for size in sizes { - let last = next + size / SECTOR - 1; - extents.push((next, last)); - next = (last + 1).next_multiple_of(ALIGNMENT); - } - // The backup array's 32 sectors and the backup header end the disk, after the usable sectors. - let sectors = next + ALIGNMENT; - let (first_usable, last_usable) = (34_u64, sectors - 34); - let (backup_entries, backup_header) = (sectors - 33, sectors - 1); - let guid = |text: &str| Uuid::parse(text).unwrap().to_bytes_le(); - - let mut entries = vec![0; 128 * 128]; - for (entry, ((first, last), unique)) in entries - .chunks_exact_mut(128) - .zip(extents.iter().zip(PARTITION_GUIDS)) - { - entry[0..16].copy_from_slice(&guid(LINUX_DATA)); - entry[16..32].copy_from_slice(&guid(unique)); - entry[32..40].copy_from_slice(&first.to_le_bytes()); - entry[40..48].copy_from_slice(&last.to_le_bytes()); - } - let entries_crc = crc32(&entries); - let header = |own: u64, alternate: u64, array: u64| { - let mut header = vec![0; 92]; - header[0..8].copy_from_slice(b"EFI PART"); - header[8..12].copy_from_slice(&0x0001_0000_u32.to_le_bytes()); - header[12..16].copy_from_slice(&92_u32.to_le_bytes()); - header[24..32].copy_from_slice(&own.to_le_bytes()); - header[32..40].copy_from_slice(&alternate.to_le_bytes()); - header[40..48].copy_from_slice(&first_usable.to_le_bytes()); - header[48..56].copy_from_slice(&last_usable.to_le_bytes()); - header[56..72].copy_from_slice(&guid(DISK_GUID)); - header[72..80].copy_from_slice(&array.to_le_bytes()); - header[80..84].copy_from_slice(&128_u32.to_le_bytes()); - header[84..88].copy_from_slice(&128_u32.to_le_bytes()); - header[88..92].copy_from_slice(&entries_crc.to_le_bytes()); - let crc = crc32(&header); - header[16..20].copy_from_slice(&crc.to_le_bytes()); - header + let layout = Layout { + guid: DISK_GUID, + partitions: &partitions, + alignment: ALIGNMENT, + length: None, }; - // The protective MBR: one record of type 0xEE, from LBA 1 over the rest of the disk, with the - // addresses that UEFI gives it. - let mut mbr = vec![0; 512]; - let protective = u32::try_from(sectors - 1).unwrap_or(u32::MAX); - mbr[446..450].copy_from_slice(&[0x00, 0x00, 0x02, 0x00]); - mbr[450..454].copy_from_slice(&[0xee, 0xff, 0xff, 0xff]); - mbr[454..458].copy_from_slice(&1_u32.to_le_bytes()); - mbr[458..462].copy_from_slice(&protective.to_le_bytes()); - mbr[510..512].copy_from_slice(&[0x55, 0xaa]); - - let mut disk = File::create(path).unwrap(); - disk.set_len(sectors * SECTOR).unwrap(); - let parts = [ - (0, mbr), - (1, header(1, backup_header, 2)), - (2, entries.clone()), - (backup_entries, entries), - (backup_header, header(backup_header, 1, backup_entries)), - ( - extents[0].0, - [FIRST_PARTITION, FIRST_PARTITION_PADDING].concat(), - ), - (extents[2].0, top), - ]; - for (sector, bytes) in parts { - disk.seek(SeekFrom::Start(sector * SECTOR)).unwrap(); - disk.write_all(&bytes).unwrap(); - } - disk.seek(SeekFrom::Start(extents[1].0 * SECTOR)).unwrap(); - io::copy(&mut File::open(template).unwrap(), &mut disk).unwrap(); - disk.sync_all().unwrap(); - drop(disk); - - let disk = File::open(path).unwrap(); - let partitions = PARTITION_GUIDS - .iter() - .zip(sizes) - .map(|(unique, size)| { - ExpectedPartition::new() - .unique_guid(Uuid::parse(unique).unwrap()) - .size(size) - }) - .collect(); - let expected = Expected::new() - .disk_guid(Uuid::parse(DISK_GUID).unwrap()) - .partitions(partitions); - let read = Disk::read(&disk, 512, sectors * SECTOR, &expected).unwrap(); - let first_partition = read - .read_first_partition(&disk, usize::try_from(FIRST_PARTITION_SIZE).unwrap()) - .unwrap(); - assert_eq!(first_partition, FIRST_PARTITION); -} - -/// Returns a small file system in the ext2 format, of [`LAYER_BLOCKS`] blocks of [`LAYER_BLOCK`] -/// bytes, which the kernel's ext4 driver mounts as an ext4 file system without features: its top -/// directory holds the regular `files`, each a name and its bytes, of at most 12 blocks each, -/// then the empty `directories`, each with mode 0755, and at most [`LAYER_INODES`] inodes in all. -/// -/// Block 1 holds the superblock, 2 the group descriptor, 3 and 4 the block and inode bitmaps, 5 -/// and 6 the inode table, 7 the top directory, and the blocks after them the files' bytes and the -/// directories. -fn small_layer(files: &[(&str, &[u8])], directories: &[&str]) -> Vec { - /// Writes `value` at `offset` of `image`, in little-endian order. - fn put(image: &mut [u8], offset: usize, value: [u8; N]) { - image[offset..offset + N].copy_from_slice(&value); - } - /// Writes the inode `number` of `image`, of `mode`, `size` bytes, and `links` links, whose data - /// is in `blocks`, at most the 12 that an inode addresses directly. - fn inode( - image: &mut [u8], - number: usize, - mode: u16, - size: usize, - links: usize, - blocks: &[usize], - ) { - assert!( - blocks.len() <= 12, - "an inode of the layer addresses 12 blocks" - ); - let offset = 5 * LAYER_BLOCK + (number - 1) * 128; - put(image, offset, mode.to_le_bytes()); - put( - image, - offset + 4, - u32::try_from(size).unwrap().to_le_bytes(), - ); - put( - image, - offset + 26, - u16::try_from(links).unwrap().to_le_bytes(), - ); - // The number of 512-byte sectors that the data takes. - let sectors = u32::try_from(blocks.len() * LAYER_BLOCK / 512).unwrap(); - put(image, offset + 28, sectors.to_le_bytes()); - for (index, block) in blocks.iter().enumerate() { - let block = u32::try_from(*block).unwrap(); - put(image, offset + 40 + 4 * index, block.to_le_bytes()); - } - } - /// Writes the directory `entries` into `block` of `image`, each an inode, a name, and a file - /// type, the last of which spans the rest of the block. - fn directory(image: &mut [u8], block: usize, entries: &[(usize, &str, u8)]) { - let (mut offset, end) = (block * LAYER_BLOCK, (block + 1) * LAYER_BLOCK); - for (index, (inode, name, kind)) in entries.iter().enumerate() { - let length = if index + 1 == entries.len() { - end - offset - } else { - (8 + name.len()).next_multiple_of(4) - }; - put(image, offset, u32::try_from(*inode).unwrap().to_le_bytes()); - put( - image, - offset + 4, - u16::try_from(length).unwrap().to_le_bytes(), - ); - image[offset + 6] = u8::try_from(name.len()).unwrap(); - image[offset + 7] = *kind; - image[offset + 8..offset + 8 + name.len()].copy_from_slice(name.as_bytes()); - offset += length; - } - } - /// Sets the bits `bits` of the bitmap in `block` of `image`. - fn set(image: &mut [u8], block: usize, bits: std::ops::Range) { - for bit in bits { - image[block * LAYER_BLOCK + bit / 8] |= 1 << (bit % 8); - } - } - - let (top, regular, directory_type) = (2, 1, 2); - let mut image = vec![0; LAYER_BLOCK * LAYER_BLOCKS]; - // The first block and the first inode that are free. - let (mut block, mut number) = (8, 11); - let mut entries = vec![(top, ".", directory_type), (top, "..", directory_type)]; - for (name, bytes) in files { - let blocks = (block..block + bytes.len().div_ceil(LAYER_BLOCK)).collect::>(); - image[block * LAYER_BLOCK..block * LAYER_BLOCK + bytes.len()].copy_from_slice(bytes); - inode(&mut image, number, 0o100_644, bytes.len(), 1, &blocks); - entries.push((number, *name, regular)); - block += blocks.len(); - number += 1; - } - for name in directories { - directory( - &mut image, - block, - &[(number, ".", directory_type), (top, "..", directory_type)], - ); - inode(&mut image, number, 0o040_755, LAYER_BLOCK, 2, &[block]); - entries.push((number, *name, directory_type)); - block += 1; - number += 1; - } - directory(&mut image, 7, &entries); - inode( - &mut image, - top, - 0o040_755, - LAYER_BLOCK, - 2 + directories.len(), - &[7], - ); - assert!( - block <= LAYER_BLOCKS && number <= LAYER_INODES + 1, - "the layer is too small" - ); - - // The bitmaps of the used blocks, from block 1, and inodes, from inode 1, whose bits past the - // last block and inode are set, as `mke2fs` sets them. - set(&mut image, 3, 0..block - 1); - set(&mut image, 3, LAYER_BLOCKS - 1..LAYER_BLOCK * 8); - set(&mut image, 4, 0..number - 1); - set(&mut image, 4, LAYER_INODES..LAYER_BLOCK * 8); - let free_blocks = LAYER_BLOCKS - block; - let free_inodes = LAYER_INODES + 1 - number; - - // The superblock: one group of blocks, inodes of 128 bytes, the first of them not reserved, - // 11, and only the `filetype` feature. - let superblock = LAYER_BLOCK; - for (offset, value) in [ - (0x00, LAYER_INODES), - (0x04, LAYER_BLOCKS), - (0x0c, free_blocks), - (0x10, free_inodes), - (0x14, 1), - (0x20, LAYER_BLOCK * 8), - (0x24, LAYER_BLOCK * 8), - (0x28, LAYER_INODES), - (0x4c, 1), - (0x54, 11), - (0x60, 0x2), - ] { - let value = u32::try_from(value).unwrap(); - put(&mut image, superblock + offset, value.to_le_bytes()); - } - for (offset, value) in [ - (0x36, 0xffff), - (0x38, 0xef53), - (0x3a, 1), - (0x3c, 1), - (0x58, 128), - ] { - put(&mut image, superblock + offset, u16::to_le_bytes(value)); - } - let uuid = Uuid::parse("2d4e6f80-91a2-4b3c-8d5e-6f708192a3b4").unwrap(); - image[superblock + 0x68..superblock + 0x78].copy_from_slice(uuid.as_bytes()); - - // The group descriptor: where the bitmaps and the inode table are, and what is free. - let descriptor = 2 * LAYER_BLOCK; - for (offset, value) in [(0x00, 3), (0x04, 4), (0x08, 5)] { - put(&mut image, descriptor + offset, u32::to_le_bytes(value)); - } - for (offset, value) in [ - (0x0c, free_blocks), - (0x0e, free_inodes), - (0x10, 1 + directories.len()), - ] { - let value = u16::try_from(value).unwrap(); - put(&mut image, descriptor + offset, value.to_le_bytes()); - } - image + assert_eq!(gpt::write_disk(path, &layout), FIRST_PARTITION); } /// Corrupts `layer`, a [`small_layer`]: its top directory's inode, inode 2, loses its mode, while @@ -2489,26 +2232,6 @@ fn add_cyclic_journal(layer: &mut Vec, needs_recovery: bool) { } } -/// Returns the CRC-32 of `bytes`, which GPT's headers and entry arrays hold: of the reflected -/// polynomial 0xEDB88320, from all ones, inverted. -fn crc32(bytes: &[u8]) -> u32 { - !bytes.iter().fold(!0, |crc, byte| { - (0..8).fold(crc ^ u32::from(*byte), |crc: u32, _| { - (crc >> 1) ^ (0xedb8_8320 & (crc & 1).wrapping_neg()) - }) - }) -} - -/// Returns `bytes` in lowercase hexadecimal, as the `first-partition` probe reports them. -fn hexadecimal(bytes: &[u8]) -> String { - const DIGITS: &[u8; 16] = b"0123456789abcdef"; - bytes - .iter() - .flat_map(|byte| [byte >> 4, byte & 0xf]) - .map(|digit| char::from(DIGITS[usize::from(digit)])) - .collect() -} - /// The capabilities that an entry allows, by their names in the cases of the `entries` probe, each /// with its number and the attempt of the probe that it alone allows, if any. const CAPABILITIES: [(&str, u8, Option<&str>); 13] = [ From 62b88de4995da02dafb3d5cf12e777e4ad30f422 Mon Sep 17 00:00:00 2001 From: Enrique Saurez Date: Sun, 25 Oct 2026 18:32:26 -0700 Subject: [PATCH 3/3] runtime-adapter: start clones with generated machine IDs, in VMs A clone's guest sets the hostname and the machine ID of the record's workload view for every execution: the image's own machine ID, or one that it generates once for the clone, which a read-only bind holds at /etc/machine-id. These VM tests check that view in clones that run at once. start takes --machine-id image|generated, the image's own by default, and clone::Request passes the mode to the clone's activation record. In vm_clone, one template serves three clones at once: over a scratch image and in RAM, each with a generated machine ID, and in RAM with the image's own. Two executions in each, as user ID 0, see the hostname, none of the markers that the other clones' executions wrote in their roots, and the clone's machine ID: a generated one, distinct between the clones, the same in both executions, held by one bind that is read-only, nosuid and nodev, which a write cannot change; or, with the image's own, none, since the image has none, and then the one that the first execution wrote. Each stop then tears its root down, binds included. A clone of a GPT disk whose layer holds a regular file etc starts with the image's own machine ID, which needs no bind, and ends with status 55 before it is ready with a generated one, which no bind can hold below a file. The template of that scenario is of the EROFS image, so it also restores a GPT clone of an EROFS placeholder. The scenarios' GPT disks may hold regular files in their top layer. Part of #845 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 80932e09-af82-4b79-b417-70a241303cda --- tests/runtime-adapter/README.md | 40 +-- .../src/bin/runtime-adapter-host.rs | 33 ++- tests/runtime-adapter/src/clone.rs | 14 +- tests/runtime-adapter/tests/clone_start.rs | 31 ++- tests/runtime-adapter/tests/vm_clone.rs | 233 +++++++++++++++++- 5 files changed, 311 insertions(+), 40 deletions(-) diff --git a/tests/runtime-adapter/README.md b/tests/runtime-adapter/README.md index 28a435453..bbe85bca3 100644 --- a/tests/runtime-adapter/README.md +++ b/tests/runtime-adapter/README.md @@ -1380,15 +1380,18 @@ library side, `clone`, holds what the command's options choose of the clone, `clone::Request`: its image, an EROFS layer or a GPT disk of the length of the template's placeholder, its upper, on an ext4 scratch image or in RAM, its memory target, the template's RAM unless given, the workload identity as which -its executions run, its overlay mode, its scratch's journal policy, and the -bound of a GPT disk's first partition. `Request::plan` plans the image over the -upper as a launch plans a layered root, and `Request::clone_request` returns -NVX's `CloneRequest` of it, with the bindings that the sandbox's store stages -for the images, as `stage` stages them, so the start hashes nothing. The -clone's activation record names the image's own machine ID, the hostname -`runtime-adapter`, and the guest's defaults of the other settings, unless the -request names them; for a GPT disk, the bound of its first partition is 64 KiB -unless the request names one, as a launch's guest takes it by default. +its executions run, its overlay mode, its scratch's journal policy, the bound +of a GPT disk's first partition, and the machine ID that its executions see. +`Request::plan` plans the image over the upper as a launch plans a layered +root, and `Request::clone_request` returns NVX's `CloneRequest` of it, with the +bindings that the sandbox's store stages for the images, as `stage` stages +them, so the start hashes nothing. The clone's activation record names the +hostname `runtime-adapter`, which every execution sees, the image's own machine +ID, unless the request asks for one that the guest generates once for the +clone, which a read-only bind holds at the root's `/etc/machine-id` for every +execution, and the guest's defaults of the other settings, unless the request +names them; for a GPT disk, the bound of its first partition is 64 KiB unless +the request names one, as a launch's guest takes it by default. The guest of a GPT clone rereads the disk's partition table before it assembles the root, since its kernel created the partitions of the template's placeholder. @@ -1568,7 +1571,7 @@ the directory that `NVX_ARTIFACTS_DIR` names, and the guest agent's build from its record again, and releases the lease of a clone that ended, through the template store TEMPLATES, if given, in place of the one that the clone's record names. -- `start STORE --template TEMPLATES --generation G --distro FORMAT:PATH --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES]` +- `start STORE --template TEMPLATES --generation G --distro FORMAT:PATH --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES] [--machine-id (image | generated)]` starts a [clone](#clones) of the generation G of the template store TEMPLATES in the sandbox store STORE, which it creates if needed, and whose `stage` staged its images, which no one else may change: the image PATH, an @@ -1580,7 +1583,10 @@ the directory that `NVX_ARTIFACTS_DIR` names, and the guest agent's build from every execution runs, and whose account the guest checks before the clone is ready, the overlay mode MODE and the scratch's journal policy POLICY, as a launch takes them, and for a GPT disk, the bound BYTES of its first - partition, 64 KiB by default, as a launch's guest takes it. + partition, 64 KiB by default, as a launch's guest takes it. Every execution + of the clone sees the hostname `runtime-adapter`, and the machine ID that + `--machine-id` chooses: the image's own, `image`, by default, or one that the + guest generates once for the clone, `generated`. Once the restore sequence started the clone's sandbox, it exchanges a `Ping` and the longest `Echo` with the guest agent, commits the launch, and exits, leaving the VM running, and the sandbox holding the lease. A clone that @@ -2047,9 +2053,9 @@ Windows, without a hypervisor: - the command lines of `start` that the host program refuses, each with status 2: missing options, a generation that is not a positive number, a scratch's journal policy for an upper in RAM, a first-partition bound for an EROFS - image, invalid values, and the options of other commands, and the options of - `start` that other commands do not take, with a clock fixture that is not a - host's time too; + image, invalid values, an option given twice, and the options of other + commands, and the options of `start` that other commands do not take, with a + clock fixture that is not a host's time too; - a start whose template store does not exist, or does not hold the generation, and whose image the plan refuses, each with status 1 before it needs NVX's artifacts or creates the sandbox's store; @@ -2409,8 +2415,8 @@ and starts its clones, each in a sandbox store of its own, over a copy of the scratch template, which no one else may change either, or in RAM. The GPT disks of a template's placeholder and of the clones, which the test writes with `tests/common/gpt.rs`, as `tests/vm_layered.rs` writes its own, hold their -first partition, then empty layers, at partitions that differ from one disk to -another: +first partition, then layers, empty but for one disk's regular file `etc`, at +partitions that differ from one disk to another: | Test | Clones | Expects | | --- | --- | --- | @@ -2419,6 +2425,8 @@ another: | `a_start_refuses_what_the_template_cannot_serve_before_any_vmm_starts` | An image 4 KiB longer, one 4 KiB shorter, and one 100 bytes longer, than the placeholder, a scratch image of twice the template's scratch, memory targets of 2 GiB and of 320 MiB, unstaged images, and a generation that the store does not hold | For those that NVX's plan refuses, status 8, with NVX's reason, and the record of a failure of the request, at the plan, without a VM, by the evidence of a length that the request supplied, of the image's geometry, which OpenVMM cannot attach, or of invalid targets, which quarantines nothing; for the others, status 1, without a record; any lease released, and no VMM's log or record in the sandbox | | `a_clone_whose_activation_fails_ends_before_it_is_ready` | A scratch image of zeros, a journal that the record forbids, a workload identity whose account the image lacks, and, through NVX's restore sequence, a ping, a start, or the start of an execution before the `Activate` | Status 8, no `Ready`, and the record of a failure, at the activation, once the VM exited: of the request, by the evidence of an unusable root, with the guest's status 54, or of invalid settings, with 55, or, before the `Activate`, of an unknown cause, by the protocol's, with 51, which the sandbox records until the release; nothing quarantined, and the lease released, with the records | | `a_clone_refuses_a_scratch_image_whose_upper_an_earlier_clone_used` | Over a scratch image, in whose root the `contents` probe writes a file, then over a copy of that scratch image once the first clone stopped | The first started; then status 8, since the copy's upper is not fresh, and the record of a failure of the request, at the activation, once the VM exited, by the evidence of an unusable root, with the guest's status 54; nothing quarantined, and the lease released, with the records | +| `clones_at_once_keep_their_own_uppers_and_machine_ids` | Three at once: over a scratch image and in RAM, each with a generated machine ID, and in RAM with the image's own; two executions in each, as user ID 0, that write a marker in the root, and try to write `/etc/machine-id` | In every execution, the hostname `runtime-adapter`, and no marker of another clone; a generated machine ID that differs between the clones, the same in each execution of a clone, held by one bind that is read-only, `nosuid`, and `nodev`, and that the write cannot change; with the image's own machine ID, none, since the image has none, no bind, and in the later execution, the ID that the first wrote; then each stop releases its lease | +| `a_clone_whose_view_cannot_be_set_ends_before_it_is_ready` | Of a template of the EROFS image, a GPT disk whose layer holds a regular file `etc`, with the image's own machine ID, then with a generated one | The first started, with the disk's root; then status 8, since no bind can hold the machine ID below a file, and the record of a failure of the request, at the activation, once the VM exited, by the evidence of invalid settings, with the guest's status 55; nothing quarantined, and the lease released, with the records | | `a_clone_runs_its_executions_as_its_workload_identity_once_it_is_ready` | Over a scratch image, as nobody | An execution that runs as nobody, one that names an identity refused with `EINVAL`, and the first execution after the clone's start | | `a_generation_is_eligible_by_the_clock_of_its_capture` | At the host's time, then with clock fixtures: UTC a day behind, another boot an hour after the capture by UTC, and the restore window passed by each clock, or UTC before the capture | The host's monotonic clock on the boot of the capture, whatever UTC says, UTC after a reboot, with the fixture's downtime, and the store's refusal without a lease outside the window | | `a_clone_reads_no_more_of_its_disks_than_its_root_needs` | Over a scratch image, and in RAM | Less than an eighth of each disk read once the clone is ready, and each disk's whole length once the guest read it to its end | diff --git a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs index be453cbef..42e67ff4e 100644 --- a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs +++ b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs @@ -97,10 +97,10 @@ //! the session's next epoch, and exchanges a `Ping`, leaving the VM running. //! - `runtime-adapter-host start STORE --template TEMPLATES --generation G --distro FORMAT:PATH //! --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity -//! UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES]` -//! starts a clone of the generation G of the template store TEMPLATES, an -//! `nvx::host::template::Store`, in the sandbox store STORE, which it creates if needed; see -//! `runtime_adapter::clone`. It reads the generation from the store's index, and its +//! UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES] +//! [--machine-id (image | generated)]` starts a clone of the generation G of the template store +//! TEMPLATES, an `nvx::host::template::Store`, in the sandbox store STORE, which it creates if +//! needed; see `runtime_adapter::clone`. It reads the generation from the store's index, and its //! compatibility key, whose RAM is the clone's memory target unless MIB mebibytes are given, //! before it needs NVX's artifacts. The store then leases the generation to the sandbox, at the //! host's time, with `Store::acquire`, which refuses a generation that a tombstone quarantines, @@ -120,7 +120,10 @@ //! record: the workload identity UID:GID, if given, as which every execution runs, whose account //! the guest checks in the clone's root before it is ready, and the overlay mode MODE and the //! scratch's journal policy POLICY, as a launch takes them, and for a GPT disk, the bound BYTES -//! of its first partition, 64 KiB unless given, as a launch's guest takes it by default. The +//! of its first partition, 64 KiB unless given, as a launch's guest takes it by default. Every +//! execution of the clone sees the hostname `runtime-adapter`, and the machine ID that +//! `--machine-id` chooses: the image's own, `image`, by default, or one that the guest generates +//! once for the clone, `generated`, which a read-only bind holds at `/etc/machine-id`. The //! guest refuses a first partition larger than BYTES, and an upper that is not fresh, whose //! `upper` or `work` directory on the scratch image exists and is not empty, as an earlier //! clone's would be, before the clone is ready. Once the clone is ready, the command starts its @@ -283,7 +286,7 @@ use std::path::{Path, PathBuf}; use std::process::ExitCode; use std::time::{Duration, Instant}; -use nvx::activation::StoreGeneration; +use nvx::activation::{MachineIdMode, StoreGeneration}; use nvx::exec::{Cause, Key, Limits, Outcome, Status, Stream, Timers, Window}; use nvx::host::artifacts::{Artifacts, absolute}; use nvx::host::instance::generate_capability; @@ -403,7 +406,7 @@ const USAGE: &str = concat!( " [--memory MIB]\n", " [--workload-identity UID:GID] [--overlay-mode MODE]", " [--scratch-journal POLICY]\n", - " [--first-partition-bound BYTES]\n", + " [--first-partition-bound BYTES] [--machine-id (image | generated)]\n", " runtime-adapter-host probe STORE PROBE... [--guest-build BUILD]\n", " runtime-adapter-host exec STORE [--guest-build BUILD] EXECUTION [';' EXECUTION]...\n", " EXECUTION: [--identity UID:GID [--groups GID,...] [--capabilities N,...]", @@ -653,6 +656,8 @@ struct Options { scratch_journal: Option, /// `--first-partition-bound`. first_partition_bound: Option, + /// `--machine-id`. + machine_id: Option, /// `--workload-identity`. workload_identity: Option<(u32, u32)>, /// `--share`, `--deny`, `--reserved`, and `--share-root`, in order. @@ -924,6 +929,7 @@ fn parse(arguments: Vec) -> Result { overlay_mode: options.overlay_mode, scratch_journal: options.scratch_journal, first_partition_bound: options.first_partition_bound, + machine_id: options.machine_id.unwrap_or(MachineIdMode::Image), }, }), time: host_time_from_env()?, @@ -1022,6 +1028,19 @@ fn parse_option( format!("--first-partition-bound takes a positive number of bytes, not {value:?}") })?); } + ("start", "--machine-id") if options.machine_id.is_none() => { + let value = text(value)?; + let mode = match value.as_str() { + "image" => MachineIdMode::Image, + "generated" => MachineIdMode::Generated, + _ => { + return Err(format!( + "--machine-id takes image or generated, not {value:?}" + )); + } + }; + options.machine_id = Some(mode); + } ("launch" | "start", "--workload-identity") if options.workload_identity.is_none() => { let value = text(value)?; let identity = ids(&value) diff --git a/tests/runtime-adapter/src/clone.rs b/tests/runtime-adapter/src/clone.rs index 57651a3a6..f2d639dc7 100644 --- a/tests/runtime-adapter/src/clone.rs +++ b/tests/runtime-adapter/src/clone.rs @@ -4,7 +4,8 @@ //! A [`Request`] holds what the host program's options choose of a clone: its image, an EROFS //! layer or a GPT disk of the length of the template's placeholder, its upper, on an ext4 scratch //! image or in RAM, its memory target, the workload identity as which its executions run, its -//! overlay mode, its scratch's journal policy, and the bound of a GPT disk's first partition. +//! overlay mode, its scratch's journal policy, the bound of a GPT disk's first partition, and the +//! machine ID that its executions see, the image's own or one that its guest generates. //! [`Request::plan`] plans the image over the upper as a launch plans a layered root, and //! [`Request::clone_request`] returns NVX's `nvx::host::template::CloneRequest` of it, with the //! bindings that the sandbox's store stages for its images, as `layered::Staged` records them: @@ -45,8 +46,8 @@ use crate::root::{DEFAULT_FIRST_PARTITION_BOUND, JournalPolicy, OverlayMode}; pub const CLONE_MODE: &str = "clone"; /// The hostname that a clone's activation record names. The guest leaves the hostname to the -/// activation of the workload's view, as it leaves the machine ID, which the record keeps as the -/// image's own. +/// activation of the workload's view, which sets it for every execution, as it sets the machine +/// ID that the record names. pub const CLONE_HOSTNAME: &str = "runtime-adapter"; /// The variable from which `runtime-adapter-host start` takes the host's time, for tests, in the @@ -79,6 +80,9 @@ pub struct Request { /// The most bytes that the first partition of a GPT disk may hold, or `None` for /// [`DEFAULT_FIRST_PARTITION_BOUND`], the default of a launch's guest. pub first_partition_bound: Option, + /// The machine ID that the clone's executions see: the image's own, as a launch's do, or one + /// that the clone's guest generates. + pub machine_id: MachineIdMode, } impl Request { @@ -103,7 +107,7 @@ impl Request { /// `layered::Staged::bindings` finds them, and the memory target `memory`, in bytes. /// /// The record of the clone's activation names the workload identity, if any, the - /// [`CLONE_HOSTNAME`], the image's own machine ID, the overlay's mode, and the scratch's + /// [`CLONE_HOSTNAME`], the request's machine ID, the overlay's mode, and the scratch's /// journal policy, each the guest's default unless the request names it, as a root /// configuration's are, and the bound of a GPT disk's first partition, the default of a /// launch's guest, [`DEFAULT_FIRST_PARTITION_BOUND`], unless the request names one. The record @@ -146,7 +150,7 @@ impl Request { memory, workload: WorkloadView { hostname: CLONE_HOSTNAME.as_bytes().to_vec(), - machine_id: MachineIdMode::Image, + machine_id: self.machine_id, identity: self .workload_identity .map(|(uid, gid)| WorkloadIdentity { uid, gid }), diff --git a/tests/runtime-adapter/tests/clone_start.rs b/tests/runtime-adapter/tests/clone_start.rs index fe0233d60..4d26729ad 100644 --- a/tests/runtime-adapter/tests/clone_start.rs +++ b/tests/runtime-adapter/tests/clone_start.rs @@ -11,7 +11,7 @@ use std::num::NonZeroU64; use std::path::{Path, PathBuf}; use std::process::{Command, Output}; -use nvx::activation::StoreGeneration; +use nvx::activation::{MachineIdMode, StoreGeneration}; use nvx::host::artifacts::Artifacts; use nvx::host::template::{HostClock, HostTime, Store}; use nvx::protocol::root::ImageFormat; @@ -132,6 +132,21 @@ fn the_host_program_refuses_options_that_do_not_make_a_start() { with(&["--upper", "ram", "--first-partition-bound", "0"]), "--first-partition-bound takes a positive number of bytes, not \"0\"", ), + ( + with(&["--upper", "ram", "--machine-id", "random"]), + "--machine-id takes image or generated, not \"random\"", + ), + ( + with(&[ + "--upper", + "ram", + "--machine-id", + "image", + "--machine-id", + "generated", + ]), + "unexpected option \"--machine-id\" of start", + ), ( with(&["--upper", "ram", "--template", "other"]), "unexpected option \"--template\" of start", @@ -148,6 +163,10 @@ fn the_host_program_refuses_options_that_do_not_make_a_start() { vec!["probe", "store", "memory", "--template", "templates"], "unexpected option \"--template\" of probe", ), + ( + vec!["launch", "store", "--machine-id", "generated"], + "unexpected option \"--machine-id\" of launch", + ), ( vec!["create", "store", "--generation", "1"], "unexpected option \"--generation\" of create", @@ -349,6 +368,7 @@ fn a_clone_s_record_names_its_template_store_and_generation_beside_its_plan() { overlay_mode: None, scratch_journal: None, first_partition_bound: None, + machine_id: MachineIdMode::Image, }; let plan = request.plan().unwrap(); let staged = Staged::stage(&distro, ImageFormat::Erofs, identity(), Some(&scratch)).unwrap(); @@ -437,6 +457,7 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { overlay_mode: None, scratch_journal: None, first_partition_bound: None, + machine_id: MachineIdMode::Image, }; let plan = request.plan().unwrap(); let staged = Staged::stage(&distro, ImageFormat::Erofs, identity(), Some(&scratch)).unwrap(); @@ -454,10 +475,7 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { nvx::host::sandbox::distro::DISTRO_POLICY_REVISION ); assert_eq!(clone.workload.hostname, b"runtime-adapter"); - assert_eq!( - clone.workload.machine_id, - nvx::activation::MachineIdMode::Image - ); + assert_eq!(clone.workload.machine_id, MachineIdMode::Image); assert_eq!(clone.workload.identity, None); assert_eq!(clone.overlay, nvx::activation::OverlayMode::TrustedLowers); assert_eq!(clone.journal, nvx::activation::Journal::Allowed); @@ -469,6 +487,7 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { overlay_mode: Some(runtime_adapter::root::OverlayMode::UntrustedLowers), scratch_journal: Some(runtime_adapter::root::JournalPolicy::Forbidden), first_partition_bound: Some(4096), + machine_id: MachineIdMode::Generated, ..request.clone() }; let clone = named @@ -481,6 +500,8 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { gid: 65533 }) ); + assert_eq!(clone.workload.hostname, b"runtime-adapter"); + assert_eq!(clone.workload.machine_id, MachineIdMode::Generated); assert_eq!(clone.overlay, nvx::activation::OverlayMode::UntrustedLowers); assert_eq!(clone.journal, nvx::activation::Journal::Forbidden); assert_eq!(clone.first_partition_bound, 4096); diff --git a/tests/runtime-adapter/tests/vm_clone.rs b/tests/runtime-adapter/tests/vm_clone.rs index aae54e246..871e63dbf 100644 --- a/tests/runtime-adapter/tests/vm_clone.rs +++ b/tests/runtime-adapter/tests/vm_clone.rs @@ -33,6 +33,14 @@ //! an execution. The sandbox records the failure until the release of its lease. //! - A clone refuses a scratch image whose upper is not fresh, as an earlier clone left it, and //! ends before it is ready. +//! - Clones at once each keep a root of their own, and every execution of a clone sees its +//! hostname and its machine ID: one that its guest generated, distinct in each clone, the same +//! in each of the clone's executions, which a read-only bind holds at `/etc/machine-id`, or the +//! image's own, which the guest leaves alone, so that what an execution writes there, the +//! clone's later executions see. +//! - A clone whose generated machine ID the guest cannot bind, since its image holds a regular +//! file `etc`, ends before it is ready, with the status of the workload's view, while a clone of +//! the image that keeps the image's own machine ID, which needs no bind, starts. //! - A clone runs its executions as its record's workload identity, once it is ready: the repair //! acknowledged the restore before the clone took its reset, the host's first message came //! after that, and the clone's Ready, its start, and its first execution came in that order. @@ -78,7 +86,7 @@ use std::path::{Path, PathBuf}; use std::thread; use std::time::{Duration, Instant, SystemTime}; -use nvx::activation::{ReadyActivation, StoreGeneration}; +use nvx::activation::{MachineIdMode, ReadyActivation, StoreGeneration}; use nvx::exec::{Key, Limits, Timers, Window}; use nvx::host::artifacts::Artifacts; use nvx::host::platform::Transport; @@ -95,7 +103,7 @@ use nvx::protocol::status; use nvx::uuid::Uuid; use runtime_adapter::GUEST_BUILD; use runtime_adapter::capture::DEFAULT_SCRATCH_LENGTH; -use runtime_adapter::clone::{HOST_TIME_VARIABLE, Request, host_time_text}; +use runtime_adapter::clone::{CLONE_HOSTNAME, HOST_TIME_VARIABLE, Request, host_time_text}; use runtime_adapter::exec::{HostMessage, Start}; use runtime_adapter::frame::Message; use runtime_adapter::host::Client; @@ -134,6 +142,26 @@ const HOUR_NS: u64 = 3_600_000_000_000; /// The error number with which the guest refuses an execution that names an identity in a /// sandbox that has its own: `EINVAL`. const EINVAL: i32 = 22; +/// The machine ID that each execution of [`VIEW`] writes to a clone's `/etc/machine-id`. +const WRITTEN_MACHINE_ID: &str = "0f1e2d3c4b5a69788796a5b4c3d2e1f0"; +/// The program of the executions that show a clone's view of its root, `/bin/sh -c VIEW sh +/// MARKER ID`, which prints, a line each: the hostname, `hostname NAME`; the machine ID at +/// `/etc/machine-id`, or `absent`, `machine-id ID`; the options of each mount there, `bind +/// OPTIONS`; whether a write of ID there succeeded, `write written`, or not, `write refused`; and +/// what the root's `/marker` holds, or `absent`, `marker TEXT`. Then it writes MARKER to `/marker`. +const VIEW: &str = r#"echo "hostname $(uname -n)" +echo "machine-id $(cat /etc/machine-id 2>/dev/null || echo absent)" +while read -r _ _ _ _ point options _; do + if [ "$point" = /etc/machine-id ]; then echo "bind $options"; fi +done < /proc/self/mountinfo +if { printf '%s\n' "$2" > /etc/machine-id; } 2>/dev/null; then + echo "write written" +else + echo "write refused" +fi +echo "marker $(cat /marker 2>/dev/null || echo absent)" +printf '%s' "$1" > /marker +"#; /// The placeholder of the template of the scenarios of GPT disks: a disk of two layers, whose /// partitions start at multiples of 1 MiB. const PLACEHOLDER: Disk = Disk { @@ -146,6 +174,7 @@ const PLACEHOLDER: Disk = Disk { alignment: 2048, first_size: 4096, first: b"\xffthe placeholder's first partition", + top: &[], }; /// A clone's GPT disk of one layer, whose partitions start at multiples of 2 MiB, with a larger /// first partition than the placeholder's: its layer's partition starts where none of the @@ -159,6 +188,7 @@ const ONE_LAYER: Disk = Disk { alignment: 4096, first_size: 8192, first: b"\xfe\0one layer\x80", + top: &[], }; /// A clone's GPT disk of three layers, whose partitions start at multiples of 640 KiB, with the /// largest first partition: its layers' partitions start where none of the placeholder's do, nor @@ -174,12 +204,26 @@ const THREE_LAYERS: Disk = Disk { alignment: 1280, first_size: 12288, first: b"\xfd\tthree layers\xc3", + top: &[], +}; +/// A clone's GPT disk of one layer whose top directory holds a regular file `etc`, below which +/// nothing can be bound at `/etc/machine-id`. +const ETC_FILE: Disk = Disk { + guid: "c3d5e7f9-1a2b-4c3d-8e4f-5a6b7c8d9e00", + partitions: &[ + "c3d5e7f9-1a2b-4c3d-8e4f-5a6b7c8d9e01", + "c3d5e7f9-1a2b-4c3d-8e4f-5a6b7c8d9e02", + ], + alignment: 2048, + first_size: 4096, + first: b"\xfcetc is a file", + top: &[("etc", b"not a directory\n")], }; /// A GPT disk of the scenarios, of the length of the platform's image, which /// [`Template::of_disk`] captures as the template's placeholder, or [`Template::disk`] writes for /// a clone. Its partitions follow each other, each at a multiple of its alignment: its first -/// partition, then a layer, an empty [`small_layer`], in each of the others. +/// partition, then a layer, a [`small_layer`], in each of the others, empty but the top one. struct Disk { /// The disk's GUID, which the guest expects of it. guid: &'static str, @@ -193,20 +237,27 @@ struct Disk { /// The bytes that its first partition holds, before NUL bytes: those that the guest returns, /// which end in neither a NUL byte nor whitespace. first: &'static [u8], + /// The regular files in the top directory of its top layer, each a name and its bytes. + top: &'static [(&'static str, &'static [u8])], } impl Disk { /// Writes the disk at `path`, of `length` bytes, a file that no one else may change. fn write(&self, path: &Path, length: u64) { - let layer = small_layer(&[], &[]); + let (layer, top) = (small_layer(&[], &[]), small_layer(self.top, &[])); let partitions = self .partitions .iter() .enumerate() .map(|(index, guid)| { - let (size, contents) = match index { - 0 => (self.first_size, self.first), - _ => (u64::try_from(layer.len()).unwrap(), layer.as_slice()), + let contents = match index { + 0 => self.first, + _ if index + 1 == self.partitions.len() => top.as_slice(), + _ => layer.as_slice(), + }; + let size = match index { + 0 => self.first_size, + _ => u64::try_from(contents.len()).unwrap(), }; Partition { guid, @@ -619,6 +670,56 @@ impl Cloned { fn scratch(&self) -> PathBuf { scratch_image(&self.sandbox) } + + /// Runs [`VIEW`] in the clone's root, as user ID 0, which writes `marker` to the root's + /// `/marker` last, and returns what it found. + fn view(&mut self, marker: &str) -> View { + let program = ["/bin/sh", "-c", VIEW, "sh", marker, WRITTEN_MACHINE_ID]; + let ran = self.exec(&[&["--identity", "0:0", "--"][..], &program].concat()); + let facts = ["launch", "status", "stderr"].map(|key| &ran[&format!("exec.1.{key}")]); + assert_eq!(facts, ["started", "exit 0", ""], "{ran:?}"); + // The host program escapes each line feed, and each backslash, which none of the lines + // holds. + let lines = ran["exec.1.stdout"] + .strip_suffix("\\n") + .filter(|lines| !lines.contains("\\\\")) + .unwrap_or_else(|| panic!("{ran:?}")) + .split("\\n") + .collect::>(); + let value = |line: &str, key: &str| { + let value = line + .strip_prefix(key) + .and_then(|rest| rest.strip_prefix(' ')); + value + .unwrap_or_else(|| panic!("{key} in {ran:?}")) + .to_owned() + }; + let [hostname, machine_id, binds @ .., write, marker] = lines.as_slice() else { + panic!("{ran:?}"); + }; + View { + hostname: value(hostname, "hostname"), + machine_id: value(machine_id, "machine-id"), + binds: binds.iter().map(|line| value(line, "bind")).collect(), + write: value(write, "write"), + marker: value(marker, "marker"), + } + } +} + +/// What an execution of [`VIEW`] found in a clone's root. +#[derive(Clone, Debug, PartialEq, Eq)] +struct View { + /// The hostname. + hostname: String, + /// The machine ID at `/etc/machine-id`, or `absent`. + machine_id: String, + /// The options of each mount at `/etc/machine-id`. + binds: Vec, + /// Whether a write of [`WRITTEN_MACHINE_ID`] at `/etc/machine-id` was `written` or `refused`. + write: String, + /// What the root's `/marker` held, or `absent`. + marker: String, } /// Returns the scratch image of a clone over one, in the clone's sandbox, `sandbox`, in a @@ -819,6 +920,23 @@ fn assert_overlay(facts: &Printed, upper: Upper<'_>, trusted: bool, identifier: ); } +/// Checks that an execution of [`VIEW`] found a machine ID that the clone's guest generated: 32 +/// lowercase hexadecimal digits, which one bind holds at `/etc/machine-id`, read-only, without +/// set-user-ID programs or devices, so that the execution's write there was refused. +fn assert_generated(view: &View) { + let id = &view.machine_id; + let digit = |byte: u8| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte); + assert!(id.len() == 32 && id.bytes().all(digit), "{view:?}"); + let [options] = view.binds.as_slice() else { + panic!("one bind holds the machine ID: {view:?}"); + }; + let options = options.split(',').collect::>(); + for flag in ["ro", "nosuid", "nodev"] { + assert!(options.contains(&flag), "{flag} in {view:?}"); + } + assert_eq!(view.write, "refused", "{view:?}"); +} + /// Checks the events of a clone's activation that its `activation` probe found: the repair /// acknowledged the restore, and the clone took its reset, the host's first message, its Ready, /// and its start, in that order, after the acknowledgement. Returns when each came, in @@ -945,6 +1063,7 @@ fn refuses_early(template: &Template, early: Message) { overlay_mode: None, scratch_journal: None, first_partition_bound: None, + machine_id: MachineIdMode::Image, }; let plan = request.plan().unwrap(); let staged = Staged::read(&sandbox.store).unwrap().unwrap(); @@ -1375,6 +1494,106 @@ fn a_clone_refuses_a_scratch_image_whose_upper_an_earlier_clone_used() { assert_payload(&template, false); } +#[test] +#[ignore = "boots VMs; see the module documentation for what they need"] +fn clones_at_once_keep_their_own_uppers_and_machine_ids() { + let template = Template::create("cl-view"); + // Three clones at once: two whose guests generate their machine IDs, over a scratch image and + // in RAM, and one in RAM that keeps the image's own. + let generated = ["--machine-id", "generated"]; + let starts = [ + ("cl-s", Upper::Scratch, &generated[..]), + ("cl-r", Upper::Ram, &generated[..]), + ("cl-i", Upper::Ram, &[][..]), + ]; + let mut clones = starts.map(|(name, upper, options)| { + let start = StartOf { + upper, + options, + ..StartOf::default() + }; + let cloned = template.start(name, start, 0); + cloned.assert_started(); + (name, cloned) + }); + assert_eq!(template.leases().len(), clones.len()); + + // Each clone's first execution sees the clone's hostname, and a root of the clone's own, + // without the marker that the executions before it left in the other clones' roots. + let firsts = clones.each_mut().map(|(name, cloned)| { + let view = cloned.view(name); + let seen = [view.hostname.as_str(), view.marker.as_str()]; + assert_eq!(seen, [CLONE_HOSTNAME, "absent"], "{view:?}"); + view + }); + let [scratch, ram, image] = &firsts; + assert_generated(scratch); + assert_generated(ram); + assert_ne!(scratch.machine_id, ram.machine_id); + // The guest leaves the image's own machine ID alone: the Ubuntu image has none, and an + // execution may write one. + let image_view = ( + image.machine_id.as_str(), + image.binds.len(), + image.write.as_str(), + ); + assert_eq!(image_view, ("absent", 0, "written"), "{image:?}"); + + // A later execution of each clone sees the machine ID that its first saw, or, with the image's + // own, the one that its first wrote, and its first's marker. + for ((name, cloned), first) in clones.iter_mut().zip(&firsts) { + let machine_id = match first.binds.as_slice() { + [] => WRITTEN_MACHINE_ID, + _ => first.machine_id.as_str(), + }; + let expected = View { + machine_id: machine_id.to_owned(), + marker: (*name).to_owned(), + ..first.clone() + }; + assert_eq!(cloned.view(name), expected); + } + for (_, cloned) in &mut clones { + cloned.stop(); + } + assert!(template.leases().is_empty()); + assert_payload(&template, false); +} + +#[test] +#[ignore = "boots VMs; see the module documentation for what they need"] +fn a_clone_whose_view_cannot_be_set_ends_before_it_is_ready() { + // The template of the EROFS image serves clones of a GPT disk of its length too. + let template = Template::create("cl-noview"); + let disk = template.disk("etc.gpt", &ETC_FILE); + let image = Some(Image::Gpt(&ETC_FILE, &disk)); + + // With the image's own machine ID, the guest binds nothing in the root, so a clone of a disk + // whose top layer holds a regular file `etc` starts. + let start = StartOf { + image, + ..StartOf::default() + }; + let mut cloned = template.start("cl-n", start, 0); + cloned.assert_started(); + let facts = cloned.probe(&["devices", "mounts", "contents", "first-partition"]); + assert_gpt_root(&facts, &ETC_FILE, Upper::Ram); + cloned.stop(); + + // A generated machine ID, which the guest binds at `/etc/machine-id`, cannot be bound below a + // file, so a clone of the disk that asks for one ends before it is ready, with the status of + // the workload's view, which attributes the failure to the request's invalid settings. + let options = ["--machine-id", "generated"]; + let start = StartOf { + image, + options: &options, + ..StartOf::default() + }; + let ended = template.start("cl-n", start, CLONE_FAILED); + ended.assert_unactivated(&template, "invalid-settings", status::ACTIVATION_WORKLOAD); + assert_payload(&template, false); +} + #[test] #[ignore = "boots VMs; see the module documentation for what they need"] fn a_clone_runs_its_executions_as_its_workload_identity_once_it_is_ready() {