Skip to content

nvx-rs: derive the compatibility keys of templates - #879

Merged
Pedro Henrique Penna (ppenna) merged 3 commits into
devfrom
esaurez/839-pr1-compatibility-key-20261024
Oct 27, 2026
Merged

Pedro Henrique Penna (ppenna) merged 3 commits into
devfrom
esaurez/839-pr1-compatibility-key-20261024

Conversation

@esaurez

@esaurez Enrique Saurez (esaurez) commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Based on dev, where #866, #882, #883, #876, #871, #873, and #874 have merged. Stacked above it: #881 → #885 → #886 → #884 → #887 → #888.

Which clones a template can serve, and on which hosts, depends on small metadata: the runtime that captured it, the contracts through which its clones are restored, and the machine that they restore. Stores, adapters, and restores need one key that holds exactly that metadata, whose canonical bytes and identity never vary between hosts or builds. This adds nvx::host::template: the reader of a capture's manifest, the invariant inputs of a capture, and the compatibility key that they derive, from metadata alone. It changes no behavior: nothing captures, stores, or restores a template yet.

Part of #839: the first of its four pull requests. The second adds the template store, the third eligibility, leases, tombstones, and retirement, and the fourth the supervisor seam and the lock order.

New public constants

nvx::contract gains the versions of the contracts through which OpenVMM restores a snapshot, which no runtime identity records, each a pub const ...: u64 beside BOOT_CONTRACT_VERSION, CONTROL_SESSION_PROTOCOL_VERSION, and MICROVM_ABI_VERSION:

Constant Value Must match, in the OpenVMM submodule
SNAPSHOT_FORMAT_VERSION 6 MANIFEST_VERSION in openvmm/openvmm_helpers/src/snapshot.rs
DEVICE_STATE_SCHEMA_VERSION 1 SAVED_STATE_SCHEMA_VERSION in openvmm/openvmm_helpers/src/snapshot/format.rs
TIME_ABI_VERSION 1 TIME_ABI_VERSION in vmm_core/virt/src/time_abi/mod.rs
RESTORE_PACKET_VERSION 4 RESTORE_PACKET_VERSION in vm/devices/chipset_resources/src/microvm_time.rs

The ignored test of tests/openvmm_vectors.rs, which the NVX CLI jobs run with the submodule checked out, now also checks each against the submodule's source. The guest's restore-packet decoder takes its version from RESTORE_PACKET_VERSION.

What changed

The manifest (manifest.rs, wire.rs). TemplateManifest::read reads the manifest.bin that OpenVMM wrote for a capture: at most MAX_MANIFEST_LEN (1 MiB) bytes of a regular file that is not a link or other reparse point, through a strict reader of the protobuf wire format that refuses unknown, mistyped, repeated, and truncated fields. It refuses a manifest of another snapshot format, saved-state schema, machine ABI, boot layout, or time ABI, and a capture that is not a template's: a platform-tier capture for clones, with a read-only distro whose identity is unbound, a fresh scratch, and the dormant filesystem slot. It reads the time contract, TimeContract, and never reads another artifact of the snapshot.

The class and the key (key.rs). CaptureClass holds the invariant inputs of a capture, known before it: the runtime, the guest's build, the backend, the requested CPU profile, the processors and RAM, the geometries of the disks, the network, and the adapter's other Invariants. TemplateKey::new checks that the capture made the machine that its class requested, and derives the key, which holds, and nothing else:

  • Bundle and contracts: the OpenVMM, kernel, and initramfs identities, the guest's build, and the versions of the key's encoding and of the boot, snapshot, device-state, time, restore-packet, control, and activation contracts. The activation version is nvx-rs: define the activation record of a layered root and its limits #871's nvx::activation::VERSION, widened to a u64.
  • CPU and time: the backend; the CPU profile's vendor, generation, revision, and digest, and the effective CPUID; the topology; and the TSC and LAPIC timer rates of the time contract, with its rate tolerance.
  • Resources: the processor capacity and those online at boot; the RAM at boot, its capacity, its block size, and both sets of ranges.
  • Boot: the exact effective command line, and its digest.
  • Disks: the length and block sizes of the distro and of the scratch.
  • Devices and isolation: each device's ID, kind, state unit, transport, address ranges, interrupt, features, and queues, and the interrupt delivery; the filesystem slot's version and tag; the fixed NIC, if any, with its profile, MAC addresses, addresses, prefixes, and gateways, and the identity of the network policy that OpenVMM enforces there; and the invariant settings.

A capture's own metadata stays out of the key, so the generations of one key differ in it: the capture time, the time contract's anchors, host and boot IDs, and lineage counter, and the artifacts' lengths. TemplateManifest keeps them.

The encoding (canonical.rs). A key and a class each have one canonical, versioned encoding, which the module documents value by value. Their identities, a KeyIdentity and a ClassIdentity, are SHA-256 digests of it after a domain of their own. TemplateKey::from_bytes reads a key back, and refuses any other encoding and bytes left after it. A KeyIdentity converts to and from the nvx::activation::Identity by which a clone's activation record names its template's key, without loss.

Fixtures and vectors. tests/fixtures/template/ holds two manifests that OpenVMM's own writer encoded, and the program that wrote them, so the tests check the reader against OpenVMM's encoding and not only against the tests' encoder. tests/vectors/template-key-v1.txt fixes version 1 of the encoding: keys, classes, keys of other versions, and invalid keys, which an independent encoder wrote from the documented encoding.

Assumptions to review

  • "Invariant isolation and resource settings" are the adapter's Invariants: named settings, at most 64, in the order of their names, which the class and the key hold as they are.
  • "The rate policy" is the time contract's tolerance of a restoring host's TSC rate, in parts per million.
  • A class has an identity of its own, a ClassIdentity, so that the generations of one class can be found before their keys are known.
  • TemplateKey::new leaves the identity of the network policy to OpenVMM, which derives it from the class's network arguments by an encoding of its own. A capture that enforced another policy than its class requested makes a key of its own, which no restore with the class's arguments serves: OpenVMM refuses a restore whose network, its policy included, is not the one that the snapshot records.
  • TemplateManifest::read refuses a link at the manifest itself. It names the manifest by its path, so the snapshot's directory and those above it must be ones that only the user, or an administrator, may change, as the sandbox store requires of its own.

Validation

At d74181e, rebased on dev's b57760b, on Windows and on Linux x86_64 (the head, 0a9fdce, on dev's ce8c7d6, differs from it only in dev's performance baselines):

  • cargo fmt --check; cargo clippy with --all-targets --all-features, --lib, and --lib --no-default-features, -D warnings; cargo test with all, default, and no features, among them tests/host_modules.rs, the layering of nvx::host; cargo doc with -D warnings, also of private items on Windows; cargo +1.89 check;
  • tests/openvmm_vectors.rs with --ignored, against the OpenVMM submodule at its pin, e68871c;
  • the runtime adapter's protocol-only and host-only configurations, with their MSRV checks, and, on Linux, its guest-only configuration;
  • the tests of the bundled artifacts of nvx, against a synthetic bundle;
  • the check that doc/usage.md documents every error code that nvx-rs/README.md lists, and the Markdown links;
  • on Linux, also the macOS check, the guest binaries' check, build, and tests, the template and sandbox-store tests three times, and the tests of the Specula CI wiring;
  • at each of its commits, on both hosts: cargo fmt --check, cargo clippy with --all-targets --all-features, --lib, and --lib --no-default-features, -D warnings, and tests/host_modules.rs, and, on Linux, the macOS check.

Every check passed on both hosts.

A final review by a separate model found three issues, which the head addresses: an expansion range now refuses its file offset, field 3, even when it is zero, as an unknown field; and the documentation now states that TemplateManifest::read refuses a link at the manifest itself, and that TemplateKey::new leaves the network policy's identity to OpenVMM, as above.

The restack on #874 resolved conflicts only in nvx-rs/README.md and tests/vectors/README.md, where #871 to #874 and dev changed the same lines. It keeps their text, such as the activation entries and the new listing of tests/, and adds this pull request's lines to it. It changed no code. A final review of the restack by a separate model found no issues.

A second restack, on #874's fa45e48, which drops the configuration identity of activation records, again resolved conflicts only in nvx-rs/README.md and tests/vectors/README.md, in this pull request's rows next to #874's activation entries: it keeps #874's new text and this pull request's rows. It changed no code, and the template key's vectors are unchanged, since nvx::activation::VERSION is still 1. A final review of it by a separate model found no issues.

The rebase onto dev, after #874 merged, resolved conflicts only in the crate tree of nvx-rs/README.md and the module documentation of nvx::host, keeping dev's text and adding this pull request's; it changed no code, tests/host_modules.rs places nvx::host::template in the top layer of nvx::host, without a cycle, the template key's vectors are unchanged, and a final review of it by a separate model found no issues.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The persistent compatibility encoding and hand-written manifest decoder need human confirmation of cross-platform behavior and long-term versioning assumptions.

No open issues 🎉

What changed in this PR

Adds metadata-only template compatibility keys to nvx-rs, establishing the first part of #839’s template-store foundation on top of #871’s activation records. Template capture, storage, and restore remain out of scope.

Changes:

  • Introduces capture classes, compatibility keys, canonical encodings, and domain-separated identities.
  • Adds strict manifest decoding, OpenVMM-generated fixtures, and compatibility conformance tests.
  • Centralizes restore contract versions and extends checks against the OpenVMM submodule.
File Description
nvx-rs/​tests/​vectors/​README.md Documents template-key vectors and contract checks.
nvx-rs/​tests/​openvmm_vectors.rs Checks restore contract versions against OpenVMM.
nvx-rs/​tests/​fixtures/​template/​README.md Documents manifest fixtures and regeneration.
nvx-rs/​tests/​fixtures/​template/​generate.rs Generates fixtures using OpenVMM’s writer.
nvx-rs/​src/​lib.rs Documents the new template and contract APIs.
nvx-rs/​src/​host/​template/​wire.rs Implements strict protobuf wire decoding.
nvx-rs/​src/​host/​template/​test_support.rs Supplies manifest and capture-class test builders.
nvx-rs/​src/​host/​template/​mod.rs Exposes template APIs and documents encoding rules.
nvx-rs/​src/​host/​template/​manifest/​tests.rs Tests manifest validation and safe file reading.
nvx-rs/​src/​host/​template/​key/​tests.rs Tests key derivation, identities, and conformance vectors.
nvx-rs/​src/​host/​template/​key.rs Defines capture classes and canonical compatibility keys.
nvx-rs/​src/​host/​template/​canonical.rs Implements bounded canonical encoding and decoding.
nvx-rs/​src/​host/​mod.rs Exports and documents the template module.
nvx-rs/​src/​guest/​time/​decoders.rs Uses the shared restore-packet version constant.
nvx-rs/​src/​contract.rs Defines snapshot and restore contract versions.
nvx-rs/​README.md Documents template metadata and validation.
doc/​ci.md Describes the expanded OpenVMM consistency checks.
.github/​workflows/​ci.yml Updates comments describing existing checks.
.gitattributes Preserves manifest fixtures as binary files.

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The persistent compatibility encoding and cross-platform manifest reader establish long-lived restore contracts that warrant final human review.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The handwritten manifest decoder and persistent compatibility encoding need domain review for parity with OpenVMM’s schema and restore semantics because the pinned submodule sources are unavailable.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are blocking the template compatibility metadata changes.

No open issues 🎉

Review effort: Balanced

@ppenna
Pedro Henrique Penna (ppenna) force-pushed the esaurez/838-pr3-configuration-identity-20261024 branch from fa45e48 to f341f09 Compare October 27, 2026 14:38
Base automatically changed from esaurez/838-pr3-configuration-identity-20261024 to dev October 27, 2026 15:28
Enrique Saurez and others added 3 commits October 27, 2026 09:15
A template's compatibility key must hold the version of every contract
through which its clones are restored. A runtime identity records the boot
and control versions, but none of those of OpenVMM's snapshot format, the
schema of the saved device state, the time ABI, and the restore packet.

Add them to nvx::contract with the values of the pinned OpenVMM, and derive
the guest's restore-packet decoder version from its constant. The ignored
test of tests/openvmm_vectors.rs, which the NVX CLI jobs run with the
submodule checked out, now also checks each version against the constant
in OpenVMM's source that it must match.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
A template is a capture of a microVM from which independent clones are
restored. Which clones it can serve, and on which hosts, depends on small
metadata: the runtime that captured it, the contracts through which its
clones are restored, and the machine that they restore. Stores, adapters,
and restores need one key that holds exactly that metadata, which they
compare, and whose canonical bytes and identity never vary between hosts or
builds.

Add nvx::host::template:

- TemplateManifest::read reads the manifest that OpenVMM wrote for a
  capture, at most 1 MiB of a regular file that is not a link or other
  reparse point, through a strict reader of the protobuf wire format that
  refuses unknown, mistyped, repeated, and truncated fields. It refuses a
  capture that is not a template's: a platform-tier capture for clones,
  with a read-only distro whose identity is unbound, a fresh scratch, and
  the dormant filesystem slot.
- CaptureClass holds the invariant inputs of a capture, known before it,
  and TemplateKey::new checks that the capture made the machine that its
  class requested before it derives the key from metadata alone. It never
  reads or digests an artifact.
- A key and a class each have one canonical, versioned encoding, and their
  identities are domain-separated SHA-256 digests of it. TemplateKey's
  from_bytes refuses any other encoding. Besides the versions of the
  snapshot contracts in nvx::contract, a key holds the version of the
  activation contract, nvx::activation::VERSION, and a key's identity
  converts without loss to and from the nvx::activation::Identity by which
  a clone's activation record names its template.

The fixtures are manifests that OpenVMM's own writer encoded, so the tests
check the reader against OpenVMM's encoding and not only against the test
encoder. The vectors of tests/vectors/template-key-v1.txt, which an
independent encoder wrote, fix version 1 of the encoding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
The row of nvx::host::template ended mid-sentence: the conformance vectors
of version 1 pin the key's encoding and identity.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/839-pr1-compatibility-key-20261024 branch from ff7ddec to 0a9fdce Compare October 27, 2026 16:18
Copilot AI balanced review requested due to automatic review settings October 27, 2026 16:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The handwritten manifest decoder and persistent compatibility encoding warrant final human review of their OpenVMM contracts, despite no confirmed blocking defects.

No open issues 🎉

Review effort: Balanced

@@ -0,0 +1,988 @@
//! Builders of the manifests and capture classes of templates, for their tests.
//!
//! [`Proto`] writes the protobuf wire format as OpenVMM's `mesh` does, independently of the reader

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not actually using protobuf, we should create an issue to streamline the way we are serializing data in NVX, we should have a single way instead of multiple ones scatter around the repos, potentially using a library, let's try to converge to a single canonical way of serializing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants