diff --git a/docs/backends/lxc/lxc-backend.md b/docs/backends/lxc/lxc-backend.md index d3ff17f04..e8254b0e8 100644 --- a/docs/backends/lxc/lxc-backend.md +++ b/docs/backends/lxc/lxc-backend.md @@ -24,7 +24,8 @@ Creates an LXC container to provide: - Linux kernel >= 2.6.32, or >= 3.12 to run unprivileged - LXC >= 5.0 installed (`liblxc-dev` for building, `lxc-utils` for runtime) -- Root privileges, or unprivileged LXC for a policy that asks for no network. +- Root privileges, or [unprivileged LXC](#running-without-root) for a policy + that asks for no network. Filtering egress or ingress installs iptables rules in the container's network namespace, which requires root. @@ -39,28 +40,29 @@ sudo apt install lxc lxc-utils liblxc-dev ```bash sudo dnf install lxc lxc-templates dnsmasq lxc-devel ``` -RHEL needs EPEL enabled first, and a few more steps besides — see -[Red Hat Enterprise Linux setup](#red-hat-enterprise-linux-setup) for the whole -process. +RHEL needs EPEL enabled first — see +[Red Hat Enterprise Linux setup](#red-hat-enterprise-linux-setup). **Arch Linux:** ```bash sudo pacman -S lxc ``` -### Container networking +### Running without root -A policy that permits any network needs `lxcbr0` to hand the container an IPv4 -lease, so the bridge has to be up before a run: `sudo systemctl start lxc-net`. +A config with no `network` section can run without sudo once the host is set up +for unprivileged LXC. Everything else needs root. -On a host running firewalld, which is the default on Fedora and RHEL, the bridge -also has to sit in a zone that permits the traffic. The default zone rejects -IPv4 DHCP while permitting router advertisement, so the container configures -itself an IPv6 address, never receives a lease, and the run fails: +1. Follow LXC's + [Create Unprivileged Containers as a User](https://linuxcontainers.org/lxc/getting-started/#create-unprivileged-containers-as-a-user). +2. Let the container's root user reach its files under your home directory: + `chmod o+x ~`. -```bash -sudo firewall-cmd --zone= --change-interface=lxcbr0 -``` + +### Container networking + +Refer to your distribution's manual to configure networking for LXC +containers. A config with a `network` section fails without it. ## Configuration @@ -294,65 +296,31 @@ streaming in a long-lived service. Add `lxc-devel` as well if you are building MXC against liblxc rather than running a released binary. -3. **Start the bridge.** - - ```bash - sudo systemctl enable --now lxc-net - ip -4 addr show lxcbr0 - ``` - -4. **Move the bridge into a firewalld zone that permits its traffic.** - firewalld is running by default on RHEL, and its default `public` zone - admits `ssh`, `cockpit` and `dhcpv6-client` but not IPv4 DHCP. Containers - therefore configure an IPv6 address from the router advertisement, never - receive a lease, and every run that asks for network fails: - - ```bash - sudo firewall-cmd --zone= --change-interface=lxcbr0 - ``` - - `` has to admit DHCP and DNS from the bridge, since dnsmasq answers - both on the bridge address. - - Add `--permanent` and reload to keep the assignment across reboots. +3. **Configure container networking.** See + [Container networking](#container-networking). ### Verifying the host ```bash sudo lxc-checkconfig -ip -4 addr show lxcbr0 -sudo firewall-cmd --get-zone-of-interface=lxcbr0 ``` -The zone query should answer the zone you assigned. - ## Limitations -- **Default-deny is not a containment boundary against the workload.** Both the - inbound and outbound chains live in the container's own network namespace, and - container init keeps `CAP_NET_ADMIN` there, so a process running as root inside - the container can flush or delete them. The command MXC runs is attached with - `CAP_NET_ADMIN` dropped from its bounding set whenever chains are installed, so - it cannot. Default-deny closes external reachability for a container that does - not deliberately tear it down, including services the workload itself starts. -- **Raw sockets bypass egress filtering.** `CAP_NET_RAW` is retained so that an - explicit `protocol: "icmp"` allow works. It also permits `AF_PACKET` sockets, - which write link-layer frames straight to the interface without traversing the - filter chain. -- **Policy is not in force while the container starts.** The chains are installed - after the container has started and its address has settled, so container init - and anything it starts run unfiltered in both directions for that interval. The - requested command is attached afterwards. A connection opened during that window - keeps working once the rules land, because the chains accept established flows. -- **A filtered container cannot renew a DHCP lease.** The chains permit loopback - and established flows, with no implicit carve-out for DHCP or DNS. A container that - outlives its lease loses its address; one that finishes within the lease period - is unaffected. -- **A container that needs a network waits for an IPv4 address.** A dual-stack - `lxcbr0` answers router solicitation seconds before its DHCP lease arrives, and - the bridge NATs IPv4 only, so an IPv6 address alone does not mean the container - can reach the destinations its policy names. A bridge that never provides an - IPv4 address fails the run rather than starting a workload that reaches nothing. +- **The workload can send only ICMP Echo.** A `protocol: "icmp"` rule lets every + ICMP type through the firewall, but the command MXC runs can originate only Echo + requests, the ones `ping` sends. +- **The network restrictions bind only the command MXC runs and its children.** + MXC takes away that command's ability to change the firewall or send raw + packets, but the container's init process (PID 1) keeps both. The command + runs as root inside the container, so it can ask init to start a new process, + for example with `systemd-run` on Ubuntu. That process is not a child of the + command, and it can change the firewall and send raw packets. +- **Network access stops after about an hour.** The stock `lxcbr0` hands out + DHCP leases that last one hour, and MXC's firewall blocks the DHCP request the + container sends to renew one. When the lease expires the container loses its + address and the workload can no longer reach the network. A run that finishes + within the hour is unaffected. - **IPv6 egress is not supported.** An `egress` rule naming an IPv6 destination installs and reports success, but no IPv6 traffic reaches that destination. Stock `lxcbr0` gives the container no IPv6 address, so this surfaces only on a diff --git a/src/mxc-sdk/src/backends/lxc/common/lxc_bindings.rs b/src/mxc-sdk/src/backends/lxc/common/lxc_bindings.rs index 93ff101a4..cb04ee555 100644 --- a/src/mxc-sdk/src/backends/lxc/common/lxc_bindings.rs +++ b/src/mxc-sdk/src/backends/lxc/common/lxc_bindings.rs @@ -135,21 +135,29 @@ pub enum ContainerFirewall { Absent, } +/// Drop Network capabilities for sandboxing purposes. #[cfg(target_os = "linux")] fn confine_network_capabilities(command: &mut std::process::Command) { use std::os::unix::process::CommandExt; - // `libc` does not export this; the value is from linux/capability.h. + // `libc` does not export these; the values are from linux/capability.h. const CAP_NET_ADMIN: libc::c_ulong = 12; + const CAP_NET_RAW: libc::c_ulong = 13; // SAFETY: `pre_exec` runs between fork and exec, where only - // async-signal-safe work is permitted. `prctl` is a bare syscall and this + // async-signal-safe work is permitted. `prctl` is a bare syscall, and this // closure allocates nothing and captures nothing. unsafe { command.pre_exec(|| { + // Drop CAP_NET_ADMIN and CAP_NET_RAW from the bounding set. if libc::prctl(libc::PR_CAPBSET_DROP, CAP_NET_ADMIN, 0, 0, 0) != 0 { return Err(std::io::Error::last_os_error()); } + + if libc::prctl(libc::PR_CAPBSET_DROP, CAP_NET_RAW, 0, 0, 0) != 0 { + return Err(std::io::Error::last_os_error()); + } + Ok(()) }); } @@ -1450,9 +1458,9 @@ mod tests { ); } - // The attach paths drop this capability only when chains are installed: the - // drop is privileged, so applying it to every run costs an unprivileged - // caller the whole execution. + // The attach paths confine the workload only when chains are installed: + // dropping a capability is privileged, so applying it to every run costs an + // unprivileged caller the whole execution. #[cfg(target_os = "linux")] #[test] fn confining_a_command_takes_a_privilege_an_unprivileged_caller_lacks() { diff --git a/src/mxc-sdk/src/backends/lxc/common/lxc_runner.rs b/src/mxc-sdk/src/backends/lxc/common/lxc_runner.rs index 9abf2c93b..ec3cefd4a 100644 --- a/src/mxc-sdk/src/backends/lxc/common/lxc_runner.rs +++ b/src/mxc-sdk/src/backends/lxc/common/lxc_runner.rs @@ -399,31 +399,6 @@ impl LxcScriptRunner { ))) } - fn enforce_netns_discovery( - &self, - netns_pid: Option, - installs_firewall: bool, - container_created: bool, - logger: &mut Logger, - mut release_container: R, - ) -> Result - where - R: FnMut(ContainerRelease) -> Result<(), String>, - { - if let Some(hook_point) = egress_hook_point(netns_pid, installs_firewall) { - return Ok(hook_point); - } - - let release = self.release_kind(container_created); - Self::report_release_failure(release, release_container(release), logger); - - Err(ScriptResponse::error( - "Failed to discover the container init PID; cannot enter the container \ - network namespace to enforce the requested firewall. Aborting rather \ - than running with enforcement silently disabled.", - )) - } - /// Everything that must succeed before a workload can launch; a failure /// releases whatever it had already acquired. fn prepare( @@ -582,16 +557,14 @@ impl LxcScriptRunner { // The init PID names the container's network namespace. let netns_pid = container.init_pid(); - let hook_point = self.enforce_netns_discovery( - netns_pid, - plan.installs_firewall(), - container_created, - logger, - |release| match release { - ContainerRelease::Destroy => container.destroy(), - ContainerRelease::Stop => container.stop(), - }, - )?; + let Some(hook_point) = egress_hook_point(netns_pid, plan.installs_firewall()) else { + self.release_after_failure(&container, container_created, logger); + return Err(ScriptResponse::error( + "Failed to discover the container init PID; cannot enter the container \ + network namespace to install its firewall. Aborting rather than \ + running without it.", + )); + }; if let Some(pid) = netns_pid { let _ = writeln!(logger, "Container init PID: {}", pid); @@ -616,12 +589,25 @@ impl LxcScriptRunner { } }; - let firewall = container_firewall( - fw_manager.rules_applied(), - ingress_manager + // figure out if a firewall needs to be installed. + let is_firewall_installed = if fw_manager.rules_applied() + || ingress_manager .as_ref() - .is_some_and(|mgr| mgr.rules_applied()), - ); + .is_some_and(|mgr| mgr.rules_applied()) + { + ContainerFirewall::Installed + } else { + ContainerFirewall::Absent + }; + + // Dropping CAP_NET_RAW also removed raw ICMP. Widen the container's + // ping_group_range so unprivileged ping still works. + if is_firewall_installed == ContainerFirewall::Installed { + if let Err(e) = fw_manager.allow_ping_sockets(logger) { + self.release_after_failure(&container, container_created, logger); + return Err(ScriptResponse::error(&e)); + } + } // A reused container can carry a pin left by an older run. let stale_pin_error = Self::clear_stale_hosts_pin(container_created, |command| { @@ -632,7 +618,7 @@ impl LxcScriptRunner { &[], true, Some(HOSTS_COMMAND_TIMEOUT), - firewall, + is_firewall_installed, ) .map_err(|e| e.to_string()) }); @@ -659,7 +645,7 @@ impl LxcScriptRunner { ingress_manager, container, container_created, - firewall, + firewall: is_firewall_installed, released: false, force_stopped: false, stop_failed: false, @@ -1529,6 +1515,8 @@ fn uuid_simple() -> String { format!("{:08x}", (t & 0xFFFF_FFFF) as u32) } +/// Where egress rules attach, or `None` when a firewall is needed and the +/// container's network namespace cannot be found. fn egress_hook_point(netns_pid: Option, installs_firewall: bool) -> Option { match netns_pid { Some(pid) => Some(EgressHookPoint::ContainerNetns(pid)), @@ -1553,15 +1541,6 @@ fn launch_failure_message(reason: &str, cleanup_failures: &[String]) -> String { ) } -/// Whether the workload must be kept away from chains in its own namespace. -fn container_firewall(egress_applied: bool, ingress_applied: bool) -> ContainerFirewall { - if egress_applied || ingress_applied { - ContainerFirewall::Installed - } else { - ContainerFirewall::Absent - } -} - #[cfg(test)] mod tests { use super::*; @@ -2090,70 +2069,18 @@ mod tests { } #[test] - fn a_known_netns_hooks_the_chain_inside_the_container() { - assert!(matches!( - egress_hook_point(Some(4242), true), - Some(EgressHookPoint::ContainerNetns(4242)) - )); + fn a_missing_init_pid_stops_a_run_that_needs_a_firewall() { + assert!(egress_hook_point(None, true).is_none()); } #[test] - fn an_unknown_netns_refuses_to_run_when_a_firewall_was_requested() { - let mut logger = Logger::new(Mode::Buffer); - - let response = runner_for_network_readiness_tests(true) - .enforce_netns_discovery(None, true, true, &mut logger, |_release| Ok(())) - .expect_err("a firewall was requested and the namespace is unknown"); - - assert_ne!(response.exit_code, 0); - assert!(response.standard_out.is_empty()); - } - - #[test] - fn a_known_netns_lets_the_run_proceed() { - let mut logger = Logger::new(Mode::Buffer); - - assert!(matches!( - runner_for_network_readiness_tests(true).enforce_netns_discovery( - Some(4242), - true, - true, - &mut logger, - |_release| Ok(()) - ), - Ok(EgressHookPoint::ContainerNetns(4242)) - )); - } - - #[test] - fn an_unknown_netns_is_allowed_when_no_firewall_was_requested() { + fn a_missing_init_pid_does_not_stop_a_run_with_no_firewall() { assert!(matches!( egress_hook_point(None, false), Some(EgressHookPoint::Unhooked) )); } - #[test] - fn either_installed_chain_confines_the_workload() { - for (egress, ingress) in [(true, false), (false, true), (true, true)] { - assert_eq!( - container_firewall(egress, ingress), - ContainerFirewall::Installed, - "egress={egress} ingress={ingress} leaves a chain the workload could flush" - ); - } - } - - #[test] - fn a_policy_that_installs_no_chain_leaves_the_capability_alone() { - // Dropping it needs CAP_SETPCAP, which an unprivileged caller lacks. - assert_eq!( - container_firewall(false, false), - ContainerFirewall::Absent, - "a run with no firewall must not be confined" - ); - } - /// `process.env` resolution, which schema 0.9 gave a default block. mod env { use super::*; diff --git a/src/mxc-sdk/src/backends/lxc/common/network_iptables.rs b/src/mxc-sdk/src/backends/lxc/common/network_iptables.rs index 983452633..55729fffe 100644 --- a/src/mxc-sdk/src/backends/lxc/common/network_iptables.rs +++ b/src/mxc-sdk/src/backends/lxc/common/network_iptables.rs @@ -1169,6 +1169,16 @@ impl NetworkIptablesManager { Ok(()) } + /// The CAP_NET_RAW drop removes raw ICMP. Opening net.ipv4.ping_group_range + /// to every gid lets the workload still ping through a datagram ICMP socket. + pub fn allow_ping_sockets(&self, logger: &mut Logger) -> Result { + self.run_firewall_command( + "sysctl", + &["-w", "net.ipv4.ping_group_range=0 2147483647"], + logger, + ) + } + // Clean up the iptables state recorded by another thread. // The ownership record keeps signal-time cleanup from flushing a later // run's chain under the same container name. @@ -1329,6 +1339,28 @@ mod tests { } } + #[test] + fn allowing_ping_opens_the_ping_group_range_in_the_container_netns() { + let fake = super::test_firewall::install(); + let manager = + NetworkIptablesManager::new("lxc-ping", EgressHookPoint::ContainerNetns(4242)); + let mut logger = Logger::new(Mode::Buffer); + + manager + .allow_ping_sockets(&mut logger) + .expect("the fake firewall accepts every command"); + + assert_eq!( + fake.issued(), + vec![vec![ + "sysctl".to_string(), + "-w".to_string(), + "net.ipv4.ping_group_range=0 2147483647".to_string(), + ]], + "ping restore must open the ping group range to every gid" + ); + } + #[test] fn an_unhooked_caller_is_not_refused_in_firewall_mode() { let _fake = super::test_firewall::install(); diff --git a/src/mxc-sdk/tests/support/raw_socket_probe.py b/src/mxc-sdk/tests/support/raw_socket_probe.py new file mode 100644 index 000000000..be09fd8ce --- /dev/null +++ b/src/mxc-sdk/tests/support/raw_socket_probe.py @@ -0,0 +1,11 @@ +import errno, socket + +for name, family, kind, protocol in ( + ("raw", socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP), + ("packet", socket.AF_PACKET, socket.SOCK_RAW, 0), +): + try: + socket.socket(family, kind, protocol).close() + print(name, "OPENED") + except OSError as error: + print(name, errno.errorcode[error.errno]) diff --git a/src/mxc-sdk/tests/wxc_e2e_tests_e2e_lxc_network_capability.rs b/src/mxc-sdk/tests/wxc_e2e_tests_e2e_lxc_network_capability.rs index 453a12534..dbdc8f560 100644 --- a/src/mxc-sdk/tests/wxc_e2e_tests_e2e_lxc_network_capability.rs +++ b/src/mxc-sdk/tests/wxc_e2e_tests_e2e_lxc_network_capability.rs @@ -9,6 +9,8 @@ use serde_json::json; use wxc_e2e_tests::{has_lxc_host, has_platform_exec, run_platform_config_value}; const CAP_NET_ADMIN: u64 = 1 << 12; +const CAP_NET_RAW: u64 = 1 << 13; +const NETWORK_CAPABILITIES: u64 = CAP_NET_ADMIN | CAP_NET_RAW; /// Whether the LXC capability prerequisites are present. fn ready() -> bool { @@ -26,12 +28,11 @@ fn capability_mask(status: &str, field: &str) -> u64 { } #[test] -fn workload_cannot_reconfigure_the_network() { +fn workload_holds_neither_network_capability() { if !ready() { return; } - // The drop only happens when chains exist, so this policy asks for a firewall. let config = json!({ "version": "0.9.0-alpha", "containerId": "lxc-network-capability", @@ -39,15 +40,12 @@ fn workload_cannot_reconfigure_the_network() { "process": { "commandLine": "sh -c \"cat /proc/self/status\"" }, "lifecycle": { "destroyOnExit": true }, "lxc": { "distribution": "alpine", "release": "3.23" }, + + // An allow rule puts the workload behind the firewall. "network": { "egress": { "default": "deny", - "allow": [ - { - "to": [{ "cidr": "140.82.112.0/20" }], - "ports": [{ "protocol": "tcp", "port": 443 }] - } - ] + "allow": [{ "to": [{ "cidr": "198.51.100.2/32" }] }] } } }); @@ -55,22 +53,115 @@ fn workload_cannot_reconfigure_the_network() { let result = run_platform_config_value("lxc network capability", &config, &[], None); let status = result.combined_output(); + let held = capability_mask(&status, "CapEff:") | capability_mask(&status, "CapPrm:"); + + assert_eq!( + held & NETWORK_CAPABILITIES, + 0, + "the workload still holds CAP_NET_ADMIN or CAP_NET_RAW\n{status}" + ); +} + +#[test] +fn workload_cannot_regain_a_network_capability() { + if !ready() { + return; + } + + let config = json!({ + "version": "0.9.0-alpha", + "containerId": "lxc-capability-regain", + "containment": "lxc", + "process": { "commandLine": "sh -c \"cat /proc/self/status\"" }, + "lifecycle": { "destroyOnExit": true }, + "lxc": { "distribution": "alpine", "release": "3.23" }, + "network": { + "egress": { + "default": "deny", + "allow": [{ "to": [{ "cidr": "198.51.100.2/32" }] }] + } + } + }); + + let result = run_platform_config_value("lxc capability regain", &config, &[], None); + let status = result.combined_output(); + + // An exec can only grant a capability that is in one of these sets. + // https://man7.org/linux/man-pages/man7/capabilities.7.html + let regainable = capability_mask(&status, "CapBnd:") + | capability_mask(&status, "CapInh:") + | capability_mask(&status, "CapAmb:"); + + assert_eq!( + regainable & NETWORK_CAPABILITIES, + 0, + "an exec could hand the workload CAP_NET_ADMIN or CAP_NET_RAW back\n{status}" + ); +} + +#[test] +fn workload_can_still_ping() { + if !ready() { + return; + } + + let config = json!({ + "version": "0.9.0-alpha", + "containerId": "lxc-ping-allowed", + "containment": "lxc", + "process": { "commandLine": "ping -c 1 127.0.0.1" }, + "lifecycle": { "destroyOnExit": true }, + "lxc": { "distribution": "ubuntu", "release": "noble" }, + "network": { + "egress": { + "default": "deny", + "allow": [{ "to": [{ "cidr": "198.51.100.2/32" }] }] + } + } + }); + + let result = run_platform_config_value("lxc ping allowed", &config, &[], None); + assert_eq!( result.code, Some(0), - "lxc-exec did not run the container\n--- stderr ---\n{}", - result.stderr + "the workload could not ping after the capability drop\n{}", + result.combined_output() ); +} - // The kernel writes these masks; the workload cannot forge them. Effective - // alone would not settle it: a process raises a permitted capability into - // its effective set whenever it likes, and only a bounding-set drop - // survives execve. - for field in ["CapEff:", "CapPrm:", "CapBnd:"] { - assert_eq!( - capability_mask(&status, field) & CAP_NET_ADMIN, - 0, - "{field} still carries CAP_NET_ADMIN; the workload can rewrite the firewall confining it\n{status}" - ); +#[test] +fn workload_cannot_open_a_raw_or_packet_socket() { + if !ready() { + return; } + + let probe = format!( + "python3 -c '{}'", + include_str!("support/raw_socket_probe.py") + ); + + // Use Ubuntu for python3. + let config = json!({ + "version": "0.9.0-alpha", + "containerId": "lxc-raw-socket-refusal", + "containment": "lxc", + "process": { "commandLine": probe }, + "lifecycle": { "destroyOnExit": true }, + "lxc": { "distribution": "ubuntu", "release": "noble" }, + "network": { + "egress": { + "default": "deny", + "allow": [{ "to": [{ "cidr": "198.51.100.2/32" }] }] + } + } + }); + + let result = run_platform_config_value("lxc raw socket refusal", &config, &[], None); + let output = result.combined_output(); + + assert!( + output.contains("raw EPERM") && output.contains("packet EPERM"), + "the workload opened a raw or packet socket the capability drop must refuse\n{output}" + ); } diff --git a/src/tools/lxc/src/linux_executor_arguments.rs b/src/tools/lxc/src/linux_executor_arguments.rs new file mode 100644 index 000000000..bbe5fcdc1 --- /dev/null +++ b/src/tools/lxc/src/linux_executor_arguments.rs @@ -0,0 +1,219 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use clap::Parser; +use mxc_sdk::mxc_common::config_parser::load_one_shot_request; +use mxc_sdk::mxc_common::logger::Logger; +use mxc_sdk::mxc_common::models::ExecutionRequest; +use std::process; + +#[derive(Parser)] +#[command(name = "lxc-exec", about = "Linux Container Executor")] +pub struct LinuxExecutorArguments { + /// Path to config JSON file (positional) + #[arg(value_name = "CONFIG_PATH")] + config_path: Option, + + /// Path to config JSON file + #[arg(long = "config")] + config: Option, + + /// Base64-encoded JSON config + #[arg(long = "config-base64")] + config_base64: Option, + + /// Enable debug/console output + #[arg(long)] + debug: bool, + + /// Delete container mode + #[arg(long)] + delete: bool, + + /// Container name (required with --delete) + #[arg(long = "containername")] + containername: Option, + + /// Enable experimental features + #[arg(long)] + experimental: bool, + + /// Report host backend availability as JSON and exit + #[arg(long = "available-backends")] + available_backends: bool, + + /// Parse and validate config then exit without executing + #[arg(long = "dry-run")] + dry_run: bool, + + /// Path to diagnostic log file (appends, creates if missing) + #[arg(long = "log-file")] + log_file: Option, + + /// Install the warmed Hyperlight snapshot and exit, for the default + /// runtime (`agent`) or the comma-separated runtimes given with `=`: + /// `--setup-hyperlight=python,node`. Names are `agent`, `python`, + /// `python-shell`, `node`, `bash` and `dotnet-jit`. For each, pulls the + /// published rootfs from GHCR unless the image home already holds it, + /// boots it once, and writes the snapshot into the default user data + /// dir (~/.local/share/mxc-hyperlight on Linux, + /// %LOCALAPPDATA%\mxc-hyperlight on Windows). $MXC_HYPERLIGHT_HOME + /// overrides the destination if set. Intended for tool install hooks + /// so first-run has zero warmup cost. + #[arg( + long = "setup-hyperlight", + value_name = "RUNTIMES", + num_args = 0..=1, + require_equals = true, + value_delimiter = ',' + )] + setup_hyperlight: Option>, + + /// Rebuild the snapshot even if one already exists. + #[arg(long, requires = "setup_hyperlight")] + force: bool, +} + +impl LinuxExecutorArguments { + pub fn get_log_path(&self) -> Option { + self.log_file.clone() + } + + pub fn should_run_in_debug(&self) -> bool { + self.debug + } + + pub fn should_display_available_backends(&self) -> bool { + self.available_backends + } + + pub fn parse_and_validate(&self) -> bool { + self.dry_run + } + + pub fn get_request(&self, logger: &mut Logger) -> ExecutionRequest { + let (config_data, is_base64) = if let Some(ref b64) = self.config_base64 { + (b64.clone(), true) + } else if let Some(ref path) = self.config { + (path.clone(), false) + } else if let Some(ref path) = self.config_path { + (path.clone(), false) + } else { + eprintln!( + "Error: No config provided. Use a positional path, --config, or --config-base64" + ); + process::exit(1); + }; + + let mut request = match load_one_shot_request(&config_data, logger, is_base64) { + Ok(r) => r, + Err(_) => { + eprintln!("Request error"); + crate::emit_warnings(logger); + eprint!("{}", logger.get_buffer()); + process::exit(1); + } + }; + + request.experimental_enabled = self.experimental; + request.dry_run = self.dry_run; + + request + } + + pub fn does_user_want_hyperlight_setup(&self) -> bool { + self.setup_hyperlight.is_some() + } + + pub fn setup_hyperlight(&self) -> Result { + if let Some(config_path) = &self.config_path { + return Err(format!( + "Error: --setup-hyperlight takes no config path; name runtimes with \ + --setup-hyperlight={config_path}" + )); + } + + #[cfg(not(all(feature = "hyperlight", target_arch = "x86_64")))] + { + Err( + "Error: --setup-hyperlight requires x86_64 (Hyperlight needs KVM or WHP)" + .to_string(), + ) + } + + #[cfg(all(feature = "hyperlight", target_arch = "x86_64"))] + { + // WHP is delay-loaded; check before setup boots a VM. + #[cfg(target_os = "windows")] + if !mxc_sdk::hyperlight_common::is_whp_available() { + return Err( + "Error: --setup-hyperlight requires Windows Hypervisor Platform (WHP). \ + Enable the HypervisorPlatform optional feature and reboot." + .to_string(), + ); + } + + //KVM is checked before anything is pulled. + #[cfg(target_os = "linux")] + if !mxc_sdk::hyperlight_common::is_kvm_available() { + return Err( + "Error: --setup-hyperlight requires KVM: /dev/kvm must be readable and \ + writable by this user." + .to_string(), + ); + } + + // Setup is an interactive install: the pull and the warm-up + // report progress as they go. + let runtimes = + mxc_sdk::hyperlight_common::parse_runtimes(&self.setup_hyperlight.clone().unwrap()) + .map_err(|message| format!("Error: {message}"))?; + + let mut logger = mxc_sdk::mxc_common::logger::Logger::new( + mxc_sdk::mxc_common::logger::Mode::Console, + ); + match mxc_sdk::hyperlight_common::setup(self.force, &runtimes, &mut logger) { + Ok(home) => Ok(format!("hyperlight setup: image home ready at {home:?}")), + Err(msg) => Err(format!("hyperlight setup failed: {msg}")), + } + } + } + + pub fn does_user_want_to_delete_a_container(&self) -> bool { + self.delete + } + + pub fn delete_container(&self, logger: &mut Logger) -> bool { + let name = match self.containername { + Some(ref n) => n, + None => { + eprintln!("Error: --containername is required with --delete"); + process::exit(1); + } + }; + + Self::delete_lxc_container(name, logger) + } + + fn delete_lxc_container(name: &str, logger: &mut Logger) -> bool { + use mxc_sdk::lxc_common::lxc_bindings::LxcContainer; + + let container = LxcContainer::new(name, None); + + if !container.is_defined() { + logger.log_line(&format!("Container '{}' does not exist.", name)); + return false; + } + + match container.destroy() { + Ok(()) => { + logger.log_line(&format!("Deleted LXC container: {}", name)); + true + } + Err(e) => { + logger.log_line(&format!("Failed to delete LXC container '{}': {}", name, e)); + false + } + } + } +} diff --git a/src/tools/lxc/src/main.rs b/src/tools/lxc/src/main.rs index 795538f1c..1b482ad86 100644 --- a/src/tools/lxc/src/main.rs +++ b/src/tools/lxc/src/main.rs @@ -1,294 +1,81 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. +mod linux_executor_arguments; + +use clap::Parser; use std::fmt::Write; -use std::process; use std::time::Instant; +use std::{eprint, process}; -use clap::Parser; -use mxc_sdk::mxc_common::config_parser::load_one_shot_request; use mxc_sdk::mxc_common::logger::{Logger, Mode}; use mxc_sdk::mxc_common::models::{ExecutionRequest, ScriptResponse}; -use mxc_sdk::mxc_common::script_runner::handle_dry_run_exit; +use mxc_sdk::mxc_common::script_runner; use mxc_sdk::mxc_common::telemetry; use mxc_sdk::lxc_common::signal_cleanup; -#[derive(Parser)] -#[command(name = "lxc-exec", about = "Linux Container Executor")] -struct Cli { - /// Path to config JSON file (positional) - #[arg(value_name = "CONFIG_PATH")] - config_path: Option, - - /// Path to config JSON file - #[arg(long = "config")] - config: Option, - - /// Base64-encoded JSON config - #[arg(long = "config-base64")] - config_base64: Option, - - /// Enable debug/console output - #[arg(long)] - debug: bool, - - /// Delete container mode - #[arg(long)] - delete: bool, - - /// Container name (required with --delete) - #[arg(long = "containername")] - containername: Option, - - /// Enable experimental features - #[arg(long)] - experimental: bool, - - /// Report host backend availability as JSON and exit - #[arg(long = "available-backends")] - available_backends: bool, - - /// Parse and validate config then exit without executing - #[arg(long = "dry-run")] - dry_run: bool, - - /// Path to diagnostic log file (appends, creates if missing) - #[arg(long = "log-file")] - log_file: Option, - - /// Install the warmed Hyperlight snapshot and exit, for the default - /// runtime (`agent`) or the comma-separated runtimes given with `=`: - /// `--setup-hyperlight=python,node`. Names are `agent`, `python`, - /// `python-shell`, `node`, `bash` and `dotnet-jit`. For each, pulls the - /// published rootfs from GHCR unless the image home already holds it, - /// boots it once, and writes the snapshot into the default user data - /// dir (~/.local/share/mxc-hyperlight on Linux, - /// %LOCALAPPDATA%\mxc-hyperlight on Windows). $MXC_HYPERLIGHT_HOME - /// overrides the destination if set. Intended for tool install hooks - /// so first-run has zero warmup cost. - #[arg( - long = "setup-hyperlight", - value_name = "RUNTIMES", - num_args = 0..=1, - require_equals = true, - value_delimiter = ',' - )] - setup_hyperlight: Option>, - - /// Rebuild the snapshot even if one already exists. Use after - /// replacing `initrd.cpio` so the warm state matches the new - /// bits. Requires --setup-hyperlight. - #[arg(long, requires = "setup_hyperlight")] - force: bool, -} - -fn log_request(request: &ExecutionRequest, logger: &mut Logger) { - let _ = writeln!(logger, "Script code length: {}", request.script_code.len()); - let _ = writeln!(logger, "Working directory: {}", request.working_directory); - let _ = writeln!(logger, "Script timeout: {}", request.script_timeout); - let _ = writeln!(logger, "Container name: {}", request.container_id); -} - -fn display_script_results(response: &ScriptResponse, logger: &mut Logger) { - let code = response.exit_code; - let _ = writeln!(logger, "Exit code: {} (0x{:08X})", code, code as u32); - if !response.error_message.is_empty() { - let _ = writeln!(logger, "Error: {}", response.error_message); - } -} - -/// Surface warnings the run recorded (e.g. Bubblewrap reporting an IPv6 allow -/// the sandbox namespace cannot reach). -/// -/// The logger only retains these rather than writing them itself, so that -/// `mxc_engine` embedders don't get unannounced writes to a terminal they own. -/// `lxc-exec` *does* own its terminal, so it opts in here — matching wxc-exec. -/// stderr, not stdout: stdout carries the workload's own output. -fn emit_warnings(logger: &Logger) { - for warning in logger.warnings() { - eprintln!("{warning}"); - } -} - -fn delete_lxc_container(name: &str, logger: &mut Logger) -> bool { - use mxc_sdk::lxc_common::lxc_bindings::LxcContainer; - - let container = LxcContainer::new(name, None); +fn main() { + let arguments = linux_executor_arguments::LinuxExecutorArguments::parse(); - if !container.is_defined() { - logger.log_line(&format!("Container '{}' does not exist.", name)); - return false; - } + // Get the logger + let mut logger = Logger::new(if arguments.should_run_in_debug() { + Mode::Console + } else { + Mode::Buffer + }); - match container.destroy() { - Ok(()) => { - logger.log_line(&format!("Deleted LXC container: {}", name)); - true - } - Err(e) => { - logger.log_line(&format!("Failed to delete LXC container '{}': {}", name, e)); - false + // Enable the file sink if needed + if let Some(ref log_path) = arguments.get_log_path() { + if let Err(e) = logger.enable_file_sink(std::path::Path::new(log_path)) { + eprintln!("Warning: could not open log file '{}': {}", log_path, e); } } -} - -fn main() { - // Install before spawning any other threads so the signal mask propagates. - // Failure here is fatal: install() either succeeds with the watchdog - // running, or restores the original signal mask and returns Err. We - // refuse to continue without it because containers leaked on SIGTERM/INT - // are exactly the failure mode this code exists to prevent. - if let Err(e) = signal_cleanup::install() { - eprintln!("Error: failed to install signal cleanup handler: {}", e); - process::exit(1); - } - - let cli = Cli::parse(); - // Detection-only fast path used by SDK `getPlatformSupport()`; runs before - // config handling so no JSON file is needed just to ask what the host can - // do, and mutates no host state. - if cli.available_backends { + // Go through the different operating modes + if arguments.should_display_available_backends() { match mxc_sdk::mxc_engine::to_json_pretty(&mxc_sdk::mxc_engine::available_backends()) { - Ok(json) => println!("{json}"), + Ok(json) => { + println!("{json}"); + process::exit(0); + } Err(e) => { eprintln!("Error: probe serialization failed: {e}"); process::exit(1); } } - return; - } - - // --setup-hyperlight: eagerly warm up the snapshot and exit. Runs - // before config parsing so the user doesn't need a JSON file on - // disk just to install. - if let Some(runtime_names) = &cli.setup_hyperlight { - // Setup exits before any config is read, so a positional here is - // most likely a runtime name given with a space instead of `=`. - if let Some(stray) = &cli.config_path { - eprintln!( - "Error: --setup-hyperlight takes no config path; name runtimes with \ - --setup-hyperlight={stray}" - ); - process::exit(1); - } - #[cfg(all(feature = "hyperlight", target_arch = "x86_64"))] - { - // WHP is delay-loaded; check before setup boots a VM. - #[cfg(target_os = "windows")] - if !mxc_sdk::hyperlight_common::is_whp_available() { - eprintln!( - "Error: --setup-hyperlight requires Windows Hypervisor Platform (WHP). \ - Enable the HypervisorPlatform optional feature and reboot." - ); - process::exit(1); + } else if arguments.does_user_want_hyperlight_setup() { + match arguments.setup_hyperlight() { + Ok(message) => { + eprintln!("{message}"); + process::exit(0); } - // KVM is checked before anything is pulled. - #[cfg(target_os = "linux")] - if !mxc_sdk::hyperlight_common::is_kvm_available() { - eprintln!( - "Error: --setup-hyperlight requires KVM: /dev/kvm must be readable and \ - writable by this user." - ); + Err(message) => { + eprintln!("{message}"); process::exit(1); } - - // Setup is an interactive install: the pull and the warm-up - // report progress as they go. - let mut logger = Logger::new(Mode::Console); - let runtimes = match mxc_sdk::hyperlight_common::parse_runtimes(runtime_names) { - Ok(runtimes) => runtimes, - Err(msg) => { - eprintln!("Error: {msg}"); - process::exit(1); - } - }; - match mxc_sdk::hyperlight_common::setup(cli.force, &runtimes, &mut logger) { - Ok(home) => { - eprintln!("hyperlight setup: image home ready at {:?}", home); - process::exit(0); - } - Err(msg) => { - eprintln!("hyperlight setup failed: {msg}"); - process::exit(1); - } - } - } - #[cfg(not(all(feature = "hyperlight", target_arch = "x86_64")))] - { - let _ = runtime_names; - eprintln!("Error: --setup-hyperlight requires x86_64 (Hyperlight needs KVM or WHP)"); - process::exit(1); } - } - - // Determine config input - let (config_data, is_base64) = if let Some(ref b64) = cli.config_base64 { - (b64.clone(), true) - } else if let Some(ref path) = cli.config { - (path.clone(), false) - } else if let Some(ref path) = cli.config_path { - (path.clone(), false) - } else if !cli.delete { - eprintln!("Error: No config provided. Use a positional path, --config, or --config-base64"); - process::exit(1); - } else { - (String::new(), false) - }; - - let mut logger = Logger::new(if cli.debug { - Mode::Console - } else { - Mode::Buffer - }); - - if let Some(ref log_path) = cli.log_file { - if let Err(e) = logger.enable_file_sink(std::path::Path::new(log_path)) { - eprintln!("Warning: could not open log file '{}': {}", log_path, e); - } - } - - // Delete mode - if cli.delete { - let name = match cli.containername { - Some(ref n) => n.as_str(), - None => { - eprintln!("Error: --containername is required with --delete"); - process::exit(1); - } - }; - let success = delete_lxc_container(name, &mut logger); + } else if arguments.does_user_want_to_delete_a_container() { + let success = arguments.delete_container(&mut logger); print!("{}", logger.get_buffer()); process::exit(if success { 0 } else { 1 }); } - // Load request - let mut request = match load_one_shot_request(&config_data, &mut logger, is_base64) { - Ok(r) => r, - Err(_) => { - eprint!("Request error\n{}", logger.get_buffer()); - process::exit(1); - } - }; - - request.experimental_enabled = cli.experimental; - request.dry_run = cli.dry_run; + // User wants to run a request. + // Make the request. + let request = arguments.get_request(&mut logger); - // ── Telemetry init ────────────────────────────────────────────── let telemetry_active = request .telemetry .as_ref() .map(|c| telemetry::init(c, &mut logger)) .unwrap_or(false); + let requested_sandbox_kind = request .telemetry .as_ref() .and_then(|config| config.requested_sandbox_kind); - // Install a crash-telemetry panic hook once telemetry is active, chaining - // the previously-installed hook so the default stderr backtrace still - // prints. The hook body is panic-free and emits no message text. if telemetry_active { telemetry::set_process_context_with_kind(&request.containment, requested_sandbox_kind); telemetry::install_panic_hook(); @@ -296,15 +83,15 @@ fn main() { log_request(&request, &mut logger); - // Dispatch by containment backend. Backend selection and runner - // construction — Bubblewrap (the Linux default for abstract intents), LXC - // (explicit `containment: "lxc"`, plus the catch-all for anything else such - // as `processcontainer`), and the experimental Hyperlight / MicroVM - // backends — live in `mxc_sdk::mxc_engine::run`, the single home for one-shot backend - // dispatch. It runs the selected backend to completion and returns the - // response; experimental backends that require `--experimental` (or that - // aren't compiled in) surface an error here. + // Setup watchdog for unexpected shutdowns + if let Err(e) = signal_cleanup::install() { + eprintln!("Error: failed to install signal cleanup handler: {e}"); + process::exit(1); + } + let run_start = Instant::now(); + + // Run the request. let response = match mxc_sdk::mxc_engine::run(&request, &mut logger) { Ok(response) => response, Err(e) => { @@ -320,19 +107,20 @@ fn main() { process::exit(1); } }; + let run_elapsed = run_start.elapsed(); let _ = writeln!(logger, "Runner completed in {}ms", run_elapsed.as_millis()); - // Emitted before the dry-run branch below, which exits the process. emit_warnings(&logger); - if cli.dry_run { - handle_dry_run_exit(&response, &mut logger); + // Handle dry_run request. + if arguments.parse_and_validate() { + script_runner::handle_dry_run_exit(&response, &mut logger); } + // Show output to the user. display_script_results(&response, &mut logger); - // ── Telemetry emit ────────────────────────────────────────────── telemetry::emit_completion_with_kind( telemetry_active, &request.containment, @@ -344,11 +132,28 @@ fn main() { print!("{}", response.standard_out); eprint!("{}", response.standard_err); - // Never exit non-zero on an infrastructure failure without a diagnostic: - // `display_script_results` only writes the error into the (buffered, - // non-debug-suppressed) logger, so surface it on stderr here for parity - // with wxc-exec (issue #564). - mxc_sdk::mxc_common::script_runner::emit_backend_error_envelope(&response); + script_runner::emit_backend_error_envelope(&response); process::exit(response.exit_code); } + +fn log_request(request: &ExecutionRequest, logger: &mut Logger) { + let _ = writeln!(logger, "Script code length: {}", request.script_code.len()); + let _ = writeln!(logger, "Working directory: {}", request.working_directory); + let _ = writeln!(logger, "Script timeout: {}", request.script_timeout); + let _ = writeln!(logger, "Container name: {}", request.container_id); +} + +fn emit_warnings(logger: &Logger) { + for warning in logger.warnings() { + eprintln!("{warning}"); + } +} + +fn display_script_results(response: &ScriptResponse, logger: &mut Logger) { + let code = response.exit_code; + let _ = writeln!(logger, "Exit code: {} (0x{:08X})", code, code as u32); + if !response.error_message.is_empty() { + let _ = writeln!(logger, "Error: {}", response.error_message); + } +} diff --git a/src/tools/lxc/tests/cli.rs b/src/tools/lxc/tests/cli.rs new file mode 100644 index 000000000..05c45571f --- /dev/null +++ b/src/tools/lxc/tests/cli.rs @@ -0,0 +1,29 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use std::fs; +use std::path::Path; +use std::process::Command; + +#[test] +fn writes_config_parse_error_to_log_file() { + let directory = Path::new(env!("CARGO_TARGET_TMPDIR")); + let config_path = directory.join("unparseable_config.json"); + let log_path = directory.join("unparseable_config.log"); + fs::write(&config_path, "{ not json").expect("write config"); + fs::write(&log_path, "").expect("clear log left by an earlier run"); + + let output = Command::new(env!("CARGO_BIN_EXE_lxc-exec")) + .arg("--log-file") + .arg(&log_path) + .arg(&config_path) + .output() + .expect("run lxc-exec"); + + let log = fs::read_to_string(&log_path).expect("read log file"); + assert!( + log.contains("Configuration parse error"), + "lxc-exec did not write the parse error to --log-file\nlog:\n{log}\nstderr:\n{}", + String::from_utf8_lossy(&output.stderr) + ); +} diff --git a/tests/configs/lxc_network_ga_egress_icmp_allowed.json b/tests/configs/lxc_network_ga_egress_icmp_allowed.json index a7712fc4e..f575b14ee 100644 --- a/tests/configs/lxc_network_ga_egress_icmp_allowed.json +++ b/tests/configs/lxc_network_ga_egress_icmp_allowed.json @@ -4,7 +4,7 @@ "containerId": "CLI-LXC-GA-Egress-Icmp-Allowed", "containment": "lxc", "process": { - "commandLine": "sh -c \"timeout 8 ping -c 1 -W 5 203.0.113.2 >/dev/null 2>&1 && echo MXC_NET_ALLOWED || echo MXC_NET_BLOCKED\"" + "commandLine": "sh -c \"command -v ping || echo MXC_PING_MISSING; timeout 8 ping -c 1 -W 5 203.0.113.2 2>&1 && echo MXC_NET_ALLOWED || echo MXC_NET_BLOCKED\"" }, "lifecycle": { "destroyOnExit": true diff --git a/tests/configs/lxc_rootless_no_network.json b/tests/configs/lxc_rootless_no_network.json new file mode 100644 index 000000000..b416c3f64 --- /dev/null +++ b/tests/configs/lxc_rootless_no_network.json @@ -0,0 +1,16 @@ +{ + "$schema": "../../schemas/stable/mxc-config.schema.1.0.0.json", + "version": "1.0.0", + "containerId": "CLI-LXC-Rootless-No-Network", + "containment": "lxc", + "process": { + "commandLine": "echo MXC_ROOTLESS_OK" + }, + "lifecycle": { + "destroyOnExit": true + }, + "lxc": { + "distribution": "alpine", + "release": "3.23" + } +} diff --git a/tests/scripts/README.md b/tests/scripts/README.md index 2ccffdb3e..e3383a7bf 100644 --- a/tests/scripts/README.md +++ b/tests/scripts/README.md @@ -94,6 +94,11 @@ one raises MXC-FATAL and stops the whole suite (child exit code 78). Individual `run_bwrap_*.sh`, `run_lxc_*.sh`, and `run_seatbelt_*.sh` scripts run focused backend suites. +`run_lxc_rootless_no_network_test.sh` checks that `lxc-exec` runs a config +with no network section without root. Neither `run_lxc_all_tests.sh` nor CI +runs it, because it adds a temporary user account to the machine. Run it +manually with `sudo bash tests/scripts/run_lxc_rootless_no_network_test.sh`. + ### Manual smoke tests Manual smokes are visual-inspection scripts for rendering and event-propagation diff --git a/tests/scripts/lib/io_uring_egress_probe.py b/tests/scripts/lib/io_uring_egress_probe.py new file mode 100644 index 000000000..32c979e7a --- /dev/null +++ b/tests/scripts/lib/io_uring_egress_probe.py @@ -0,0 +1,77 @@ +import ctypes, mmap, os, socket, struct, sys +PEER = sys.argv[1] +libc = ctypes.CDLL(None, use_errno=True) + +def checksum(data): + total = sum(struct.unpack("!%dH" % (len(data) // 2), data)) + total = (total >> 16) + (total & 0xFFFF) + return ~(total + (total >> 16)) & 0xFFFF + +def echo(ident): + return struct.pack("!BBHHH", 8, 0, checksum(struct.pack("!BBHHH", 8, 0, 0, ident, 1)), ident, 1) + +def replied(sock, ip_start): + sock.settimeout(3) + try: + while True: + data = sock.recv(2048) + icmp = ip_start + (data[ip_start] & 0x0F) * 4 + if socket.inet_ntoa(data[ip_start + 12:ip_start + 16]) == PEER and data[icmp] == 0: + return True + except OSError: + return False + +def io_uring_socket(family, kind, protocol): + params = ctypes.create_string_buffer(120) + ring = libc.syscall(425, 1, params) + if ring < 0: + raise SystemExit("MXC_IO_URING_SETUP_FAILED errno=%d" % ctypes.get_errno()) + _, sq_tail, _, _, _, _, sq_array = struct.unpack_from("7I", params, 40) + sq = mmap.mmap(ring, sq_array + 4, offset=0) + sqes = mmap.mmap(ring, 64, offset=0x10000000) + # IORING_OP_SOCKET takes the family in fd, the type in off, and the protocol in len. + sqes[0:64] = struct.pack("BBHiQQII", 45, 0, 0, family, kind, 0, protocol, 0) + bytes(32) + struct.pack_into("I", sq, sq_array, 0) + struct.pack_into("I", sq, sq_tail, 1) + # The kernel installs the new socket at the lowest free descriptor. + fd = os.open("/dev/null", os.O_RDONLY) + os.close(fd) + if libc.syscall(426, ring, 1, 1, 1, None, 0) < 0: + raise SystemExit("MXC_IO_URING_ENTER_FAILED errno=%d" % ctypes.get_errno()) + try: + return socket.socket(fileno=fd) + except OSError: + return None + +# A socket ping proves the peer answers and fills the neighbor entry the packet frame needs. +ping = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_ICMP) +ping.settimeout(3) +ping.sendto(echo(1), (PEER, 0)) +try: + ping.recv(64) +except OSError: + raise SystemExit("MXC_PEER_UNREACHABLE") + +escaped = False + +raw = io_uring_socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP) +if raw: + raw.sendto(echo(2), (PEER, 0)) + escaped |= replied(raw, 0) + +packet = io_uring_socket(socket.AF_PACKET, socket.SOCK_RAW, socket.htons(0x0800)) +if packet: + route = next(line.split() for line in open("/proc/net/route") if line.split()[1] == "00000000") + gateway = socket.inet_ntoa(struct.pack(""$IO_URING_CONFIG" <<'PY' +import json, sys +probe, peer, cidr = sys.argv[1:] +print(json.dumps({ + "version": "1.0.0", + "containerId": "CLI-LXC-GA-Egress-Io-Uring", + "containment": "lxc", + "process": {"commandLine": "python3 -c '" + open(probe).read() + "' " + peer}, + "lifecycle": {"destroyOnExit": True}, + "lxc": {"distribution": "ubuntu", "release": "noble"}, + "network": {"egress": {"default": "deny", "allow": [{"to": [{"cidr": cidr}], "ports": [{"protocol": "icmp"}]}]}}, +})) +PY + +run_case "io_uring case: peer allowed on protocol icmp, echo sent from raw and packet sockets opened through io_uring" "$IO_URING_CONFIG" +assert_blocked "an echo from a raw or packet socket opened through io_uring reached the peer. io_uring hands the workload the sockets the capability drop takes away." + run_case "port-range case: peer allowed on tcp/440-445, probe tcp/443" "$PORT_RANGE_INSIDE_CONFIG" assert_allowed "a port inside the allowed range 440-445 was unreachable. endPort is not reaching the chain, so a range selector blocks the traffic it was written to permit." @@ -415,5 +435,5 @@ assert_allowed "udp/$PEER_UDP_PORT was unreachable while protocol any allowed th run_case "protocol-any case: peer allowed on any port 8054, probe udp/$PEER_UDP_PORT" "$ANY_UDP_WRONG_PORT_CONFIG" assert_blocked "udp/$PEER_UDP_PORT succeeded while protocol any allowed only port 8054. The UDP half of the fan-out ignores the port selector." -echo "PASS: egress rules filtered by destination, by port, by port range, by protocol, by resolver, by deny rule, and by exclusion, and no exclusion answered for a destination a later rule denied." +echo "PASS: egress rules filtered by destination, by port, by port range, by protocol, by resolver, by deny rule, and by exclusion, no exclusion answered for a destination a later rule denied, and no socket opened through io_uring reached the peer." echo "LXC egress enforcement test complete." diff --git a/tests/scripts/run_lxc_rootless_no_network_test.sh b/tests/scripts/run_lxc_rootless_no_network_test.sh new file mode 100644 index 000000000..e260d942b --- /dev/null +++ b/tests/scripts/run_lxc_rootless_no_network_test.sh @@ -0,0 +1,65 @@ +#!/bin/bash +# LXC rootless test: a config with no network section runs without root +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_DIR="$(dirname "$(dirname "$SCRIPT_DIR")")" +LXC_EXEC="$REPO_DIR/src/target/release/lxc-exec" + +if [ ! -f "$LXC_EXEC" ]; then + LXC_EXEC="$REPO_DIR/src/target/debug/lxc-exec" +fi + +if [ ! -f "$LXC_EXEC" ]; then + echo "Error: lxc-exec not found. Run build.sh first." + exit 1 +fi + +if [ "$(id -u)" -ne 0 ]; then + echo "Error: run this under sudo, so it can create the unprivileged user that runs lxc-exec." + exit 1 +fi + +RUN_USER=mxc-rootless-e2e +useradd -m "$RUN_USER" +trap 'userdel -r "$RUN_USER" 2>/dev/null' EXIT + +RUN_HOME="/home/$RUN_USER" +SUBUID_START=$(grep "^$RUN_USER:" /etc/subuid | cut -d: -f2) +SUBGID_START=$(grep "^$RUN_USER:" /etc/subgid | cut -d: -f2) + +# The container's mapped root must be able to walk into the user's home to reach its rootfs. +chmod 755 "$RUN_HOME" +mkdir -p "$RUN_HOME/.config/lxc" +cat > "$RUN_HOME/.config/lxc/default.conf" <&1) || RC=$? +echo "$OUTPUT" + +if [ "$RC" -ne 0 ]; then + echo "FAIL: lxc-exec without root could not run a config that has no network section." + exit 1 +fi + +if ! grep -q "MXC_ROOTLESS_OK" <<< "$OUTPUT"; then + echo "FAIL: the container ran without root but the command printed nothing." + exit 1 +fi + +echo "LXC rootless no-network test complete." \ No newline at end of file