-
Notifications
You must be signed in to change notification settings - Fork 11
Expose sandbox nested-virt capability and pre-check create on the host #375
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -37,6 +37,33 @@ async fn sandbox_channel() -> Result<Channel> { | |||||||||||||||||||||
| }) | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| /// Fails fast when the daemon reports that sandboxes cannot run on this host | ||||||||||||||||||||||
| /// or System VM backend, instead of booting a microVM that would land in | ||||||||||||||||||||||
| /// `failed` with an opaque KVM error. This is a host-side check — no round-trip | ||||||||||||||||||||||
| /// into the guest. Transport errors (e.g. an older daemon without the RPC) fall | ||||||||||||||||||||||
| /// through so the create still proceeds, where the guest agent remains the | ||||||||||||||||||||||
| /// backstop. | ||||||||||||||||||||||
| async fn ensure_sandbox_supported(channel: &Channel) -> Result<()> { | ||||||||||||||||||||||
| use arcbox_grpc::SystemServiceClient; | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| let mut client = SystemServiceClient::new(channel.clone()); | ||||||||||||||||||||||
| match client | ||||||||||||||||||||||
| .get_sandbox_capability(tonic::Request::new(arcbox_protocol::v1::Empty {})) | ||||||||||||||||||||||
| .await | ||||||||||||||||||||||
| { | ||||||||||||||||||||||
| Ok(resp) => { | ||||||||||||||||||||||
| let cap = resp.into_inner(); | ||||||||||||||||||||||
| if !cap.supported { | ||||||||||||||||||||||
| anyhow::bail!("{}", cap.reason); | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| Ok(()) | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| // The capability RPC is unavailable (older daemon, not ready); let the | ||||||||||||||||||||||
| // create proceed rather than blocking on a missing pre-check. | ||||||||||||||||||||||
| Err(_) => Ok(()), | ||||||||||||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This arm treats every gRPC status as an unavailable capability RPC. A permission, data-loss, or internal server error therefore starts sandbox creation instead of reporting the failed pre-check; only compatibility and temporary-availability statuses should fall through.
Suggested change
|
||||||||||||||||||||||
| } | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| /// Attaches the default `x-machine` metadata header to a tonic request for | ||||||||||||||||||||||
| /// daemon-side routing to the guest VM agent. | ||||||||||||||||||||||
| fn attach_machine<T>(mut request: tonic::Request<T>) -> tonic::Request<T> { | ||||||||||||||||||||||
|
|
@@ -294,6 +321,10 @@ fn parse_labels(raw: &[String]) -> Result<HashMap<String, String>> { | |||||||||||||||||||||
|
|
||||||||||||||||||||||
| async fn execute_create(args: CreateArgs) -> Result<()> { | ||||||||||||||||||||||
| let channel = sandbox_channel().await?; | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| // Reject unsupported hosts/backends up front with an actionable message. | ||||||||||||||||||||||
| ensure_sandbox_supported(&channel).await?; | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| let mut client = SandboxServiceClient::new(channel); | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| let labels = parse_labels(&args.label)?; | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -145,6 +145,49 @@ pub struct SandboxPortExposure { | |
| pub guest_port: u16, | ||
| } | ||
|
|
||
| /// Whether this host and System VM backend can run sandboxes. | ||
| /// | ||
| /// Produced by [`Runtime::sandbox_capability`]. | ||
| #[derive(Debug, Clone)] | ||
| pub struct SandboxCapability { | ||
| /// Whether sandboxes are runnable on the current host and backend. | ||
| pub supported: bool, | ||
| /// Actionable reason when unsupported; empty when supported. | ||
| pub reason: String, | ||
| /// The System VM backend the capability was evaluated against. | ||
| pub backend: arcbox_vmm::VmBackend, | ||
| } | ||
|
|
||
| /// Pure decision behind [`Runtime::sandbox_capability`], split out for testing. | ||
| /// | ||
| /// Host nested-virt support is the hard gate (M3+ / macOS 15+ hardware), so it | ||
| /// is checked first: on hardware that lacks it, switching backends cannot help. | ||
| /// Only when the hardware is capable does the HV backend become the actionable | ||
| /// blocker (nested virtualization is unavailable under Hypervisor.framework). | ||
| fn evaluate_sandbox_capability( | ||
| backend: arcbox_vmm::VmBackend, | ||
| host_nested_virt: bool, | ||
| ) -> (bool, String) { | ||
| if !host_nested_virt { | ||
| return ( | ||
| false, | ||
| "sandbox requires nested virtualization: the VZ backend on Apple \ | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The reason strings here are macOS-specific ("the VZ backend on Apple Silicon M3 or newer with macOS 15+" and, below, " Technical details# macOS-centric reason strings on a cross-platform helper
## Affected sites
- `app/arcbox-core/src/runtime.rs:172-186` — both `reason` strings name Apple Silicon / macOS 15+ / `arcbox system backend vz`, yet the function is reached on Linux via `Runtime::sandbox_capability` → `arcbox_hypervisor::host_supports_nested_virt` (Linux KVM probe in `virt/arcbox-hypervisor/src/linux/mod.rs`).
## Required outcome
- On Linux, an unsupported result should read as a KVM-nesting message rather than an Apple-Silicon one — or the reason should be platform-conditional.
## Open questions for the human
- Is `arcbox sandbox create` a supported surface on Linux at all? If the System VM / sandbox architecture is macOS-only in practice, this is cosmetic and can be left as-is; if Linux is a real target, the strings should branch per platform. |
||
| Silicon M3 or newer with macOS 15+; this host does not support it" | ||
| .to_string(), | ||
| ); | ||
| } | ||
| if backend == arcbox_vmm::VmBackend::Hv { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
On a Linux x86_64 host with KVM nesting enabled and Useful? React with 👍 / 👎. |
||
| return ( | ||
| false, | ||
| "sandbox requires nested virtualization, which is unavailable under \ | ||
| the HV backend (current backend: HV); switch with \ | ||
| `arcbox system backend vz`" | ||
| .to_string(), | ||
| ); | ||
| } | ||
| (true, String::new()) | ||
| } | ||
|
|
||
| impl Runtime { | ||
| /// Creates a new runtime with the given configuration. | ||
| /// | ||
|
|
@@ -415,6 +458,25 @@ impl Runtime { | |
| self.vm_lifecycle.backend() | ||
| } | ||
|
|
||
| /// Reports whether this host and System VM backend can run sandboxes. | ||
| /// | ||
| /// Sandboxes are nested Firecracker microVMs; they need `/dev/kvm` inside | ||
| /// the System VM, which the host exposes only with nested virtualization | ||
| /// enabled (VZ backend on Apple Silicon M3+ with macOS 15+). Computing this | ||
| /// on the host lets `arcbox sandbox create` and clients fail fast without a | ||
| /// round-trip into the guest. | ||
| #[must_use] | ||
| pub fn sandbox_capability(&self) -> SandboxCapability { | ||
| let backend = self.system_vm_backend(); | ||
| let (supported, reason) = | ||
| evaluate_sandbox_capability(backend, arcbox_hypervisor::host_supports_nested_virt()); | ||
| SandboxCapability { | ||
| supported, | ||
| reason, | ||
| backend, | ||
| } | ||
| } | ||
|
|
||
| /// Switches the System VM's hypervisor backend (HV <-> VZ) and restarts the | ||
| /// VM so it takes effect. | ||
| /// | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -24,3 +24,32 @@ pub use hypervisor::KvmHypervisor; | |
| pub use memory::KvmMemory; | ||
| pub use vcpu::KvmVcpu; | ||
| pub use vm::{KvmVm, VirtioDeviceInfo}; | ||
|
|
||
| /// Whether the Linux host has nested virtualization enabled for KVM. | ||
| /// | ||
| /// Reads the `nested` module parameter of the Intel/AMD KVM drivers. x86-only; | ||
| /// other architectures always report `false`. | ||
| #[must_use] | ||
| pub(crate) fn host_supports_nested_virt() -> bool { | ||
| #[cfg(target_arch = "x86_64")] | ||
| { | ||
| // Intel VMX and AMD SVM expose nesting via a module parameter that | ||
| // reads "Y"/"1" when enabled. | ||
| for path in [ | ||
| "/sys/module/kvm_intel/parameters/nested", | ||
| "/sys/module/kvm_amd/parameters/nested", | ||
| ] { | ||
| if let Ok(content) = std::fs::read_to_string(path) { | ||
| let value = content.trim(); | ||
| if value == "Y" || value == "1" { | ||
| return true; | ||
| } | ||
| } | ||
| } | ||
| false | ||
| } | ||
| #[cfg(not(target_arch = "x86_64"))] | ||
| { | ||
| false | ||
| } | ||
|
Comment on lines
+51
to
+54
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
On Linux ARM64, this branch always reports Artifacts
|
||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This host-only query uses
shared_runtime, which is unavailable until full runtime initialization and System VM readiness. During daemon startup, clients receiveUNAVAILABLEeven afterearly_runtimecontains the configured backend, so they cannot discover the capability without first booting the VM.Context Used: AGENTS.md (source)