From 2d9f0417f8c23b2237482be0e36e6c0f04cb68d0 Mon Sep 17 00:00:00 2001 From: Jess Sullivan Date: Fri, 28 Aug 2026 02:55:35 -0400 Subject: [PATCH] TIN-539: NixOS VM tests for ser2net, NUT, and tailscale modules Adds pkgs.testers.runNixOSTest coverage for the three modules named in the ticket, using virtual hardware so no physical serial adapter, UPS, or Tailscale account is needed: - tests/vm/ser2net.nix: a socat PTY pair stands in for the serial device; asserts the generated /etc/ser2net/ser2net.yaml carries the module's own port/device/description, the firewall opens the configured port, and connecting over TCP reaches the configured serial device (proven via the connection banner, since ser2net sends it immediately on accept before any serial-side bytes flow). - tests/vm/nut.nix: NUT netserver mode against the dummy-ups driver (file-backed simulated readings, no real UPS). Verified the module internals against the live nixpkgs source (nixos/modules/services/monitoring/ups.nix, nixos-24.11) rather than assuming them: NUT_CONFPATH is hardcoded to /etc/nut, the relevant systemd units are upsd.service/upsdrv.service/upsmon.service (upsdrv is one oneshot registering every configured UPS, not one unit per UPS), and passwordFile is the only auth option -- there is no plaintext `password`. - tests/vm/tailscale.nix: tailscaled starts without an auth key (asserts an unauthenticated status rather than attempting a real join), firewall trusts tailscale0, and IPv4/IPv6 forwarding are enabled by useRoutingFeatures = "server" (read from /proc/sys/net/{ipv4,ipv6}/conf/all/forwarding, matching exactly what the nixpkgs module sets rather than a generic guess). - tests/vm/integration.nix: all three on one machine, asserting the configured ports are disjoint by construction and each service starts and stays reachable alongside the other two. None of these import modules/nut-server or hosts/common/tailscale.nix directly -- both wire config.sops.secrets.*.path, which needs live sops-nix secrets this hermetic VM does not have. Each test reconstructs the same power.ups / services.tailscale shape with a plaintext/writeText stand-in for the secret, which exercises the same module logic the production config does. Wired into flake.nix's checks output as checks.x86_64-linux.{vm-ser2net,vm-nut,vm-tailscale,vm-integration}, gated to x86_64-linux only (VM tests need KVM; there is no CI signal that aarch64 runners here have it, and a check that silently no-ops on arches that lack KVM would be exactly the gate-theater failure mode TIN-3457 warns about -- this is an explicit restriction, not an omission). Added a "NixOS VM tests" step to the existing .github/workflows/ci.yml `check` job's nix build sequence. Not verified locally, by design: this session's environment does not build or evaluate Nix (bazel/nix build is out of scope here). Every new file passed nix-instantiate --parse (syntax only) plus statix and deadnix (both directly available, no build needed) with zero findings. The one thing genuinely unverifiable without a live run: the `check` job runs on the self-hosted `tinyland-nix` ARC runner, not GitHub's own `ubuntu-latest` -- whether that runner pod has /dev/kvm passthrough is unproven from here. Flagged inline in ci.yml so a KVM-availability failure reads as an infra question for tinyland-nix's owner, not a defect in the tests. --- .github/workflows/ci.yml | 19 ++++++ flake.nix | 15 +++++ tests/vm/integration.nix | 139 +++++++++++++++++++++++++++++++++++++++ tests/vm/nut.nix | 119 +++++++++++++++++++++++++++++++++ tests/vm/ser2net.nix | 90 +++++++++++++++++++++++++ tests/vm/tailscale.nix | 66 +++++++++++++++++++ 6 files changed, 448 insertions(+) create mode 100644 tests/vm/integration.nix create mode 100644 tests/vm/nut.nix create mode 100644 tests/vm/ser2net.nix create mode 100644 tests/vm/tailscale.nix diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index cbb7dfb..68ac1e7 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -83,6 +83,25 @@ jobs: nix build .#checks.x86_64-linux.shellcheck-scripts --print-build-logs nix build .#checks.x86_64-linux.tesmart-ctl-build --print-build-logs + # TIN-539: NixOS VM tests (pkgs.testers.runNixOSTest) for ser2net, NUT, + # and tailscale module logic, no physical hardware required. These need + # /dev/kvm inside the runner. `tinyland-nix` is a self-hosted ARC + # runner, not GitHub's own `ubuntu-latest` (which has had KVM since + # 2023 per DeterminateSystems' installer setting + # `system-features = nixos-test kvm` automatically) -- this repo's CI + # was not previously proven to have KVM passthrough at all. If this + # step fails with a `/dev/kvm` or `qemu-kvm` permission/availability + # error rather than an actual test assertion failure, that is the + # runner's nested-virtualization posture, not a defect in the tests + # themselves; route it to whoever owns tinyland-nix's ARC pod spec + # rather than re-authoring the tests. + - name: NixOS VM tests (ser2net, NUT, tailscale) + run: | + nix build .#checks.x86_64-linux.vm-ser2net --print-build-logs + nix build .#checks.x86_64-linux.vm-nut --print-build-logs + nix build .#checks.x86_64-linux.vm-tailscale --print-build-logs + nix build .#checks.x86_64-linux.vm-integration --print-build-logs + build-packages: name: Build Packages runs-on: tinyland-nix diff --git a/flake.nix b/flake.nix index 4224a00..418438b 100644 --- a/flake.nix +++ b/flake.nix @@ -181,6 +181,21 @@ tesmart-ctl-build = self.packages.${system}.tesmart-ctl or (pkgs.runCommand "tesmart-ctl-skip" { } "echo 'skipped on ${system}'; touch $out"); + } + # NixOS VM tests (TIN-539) need KVM, which is only sanctioned + # here on x86_64-linux CI runners (GitHub-hosted `ubuntu-latest` + # has had KVM since 2023; module logic under test is + # arch-independent, so one arch's VM proves the Nix code path + # aarch64 hosts also run). Do not add these under + # aarch64-linux/darwin -- there is no KVM there in CI, and a + # `nix flake check` that silently skips VM tests on the arches + # that lack it is the gate-theater failure mode TIN-3457 warns + # about, so this is an explicit restriction, not an omission. + // pkgs.lib.optionalAttrs (system == "x86_64-linux") { + vm-ser2net = import ./tests/vm/ser2net.nix { inherit pkgs; }; + vm-nut = import ./tests/vm/nut.nix { inherit pkgs; }; + vm-tailscale = import ./tests/vm/tailscale.nix { inherit pkgs; }; + vm-integration = import ./tests/vm/integration.nix { inherit pkgs; }; }; in { diff --git a/tests/vm/integration.nix b/tests/vm/integration.nix new file mode 100644 index 0000000..098a2f1 --- /dev/null +++ b/tests/vm/integration.nix @@ -0,0 +1,139 @@ +# tests/vm/integration.nix — TIN-539 +# +# Combined VM test: ser2net + NUT (dummy-ups) + tailscale on one machine, +# validating service ordering and that none of their fixed ports collide +# (ser2net 3001, NUT upsd 3493, tailscale UDP port -- disjoint by +# construction, asserted here rather than merely assumed). +{ pkgs, ... }: + +let + ser2netModule = ../../modules/ser2net; + testDevice = "/tmp/lab-test-tty"; + testDevicePeer = "/tmp/lab-test-tty-peer"; + ser2netPort = 3001; + nutPort = 3493; + testPasswordFile = pkgs.writeText "nut-vm-test-password" "vm-test-only-not-a-real-secret"; +in +pkgs.testers.runNixOSTest { + name = "lab-services-integration"; + + nodes.machine = + { pkgs, config, ... }: + { + imports = [ ser2netModule ]; + + environment.systemPackages = [ + pkgs.socat + pkgs.nut + ]; + + services.lab-ser2net = { + enable = true; + connections.test-port = { + port = ser2netPort; + device = testDevice; + speed = "9600n81"; + description = "integration test serial connection"; + }; + }; + + power.ups = { + enable = true; + mode = "netserver"; + ups."dummy-ups" = { + description = "VM Test Dummy UPS"; + driver = "dummy-ups"; + port = "dummy-ups.dev"; + }; + upsd.listen = [ + { + address = "0.0.0.0"; + port = nutPort; + } + ]; + users.upsmon = { + passwordFile = testPasswordFile; + upsmon = "primary"; + }; + upsmon = { + enable = true; + monitor."dummy-ups@localhost" = { + powerValue = 1; + type = "primary"; + user = "upsmon"; + passwordFile = testPasswordFile; + }; + settings = { + MINSUPPLIES = 1; + SHUTDOWNCMD = "${pkgs.coreutils}/bin/true"; + POLLFREQ = 5; + POLLFREQALERT = 2; + FINALDELAY = 5; + }; + }; + openFirewall = true; + }; + + services.tailscale = { + enable = true; + useRoutingFeatures = "server"; + }; + + networking.firewall = { + trustedInterfaces = [ "tailscale0" ]; + allowedUDPPorts = [ config.services.tailscale.port ]; + checkReversePath = "loose"; + }; + }; + + testScript = '' + machine.start() + machine.wait_for_unit("multi-user.target") + + with subtest("configured TCP/UDP ports are disjoint"): + ser2net_port = ${builtins.toString ser2netPort} + nut_port = ${builtins.toString nutPort} + assert ser2net_port != nut_port, "ser2net and NUT ports collide by construction" + + with subtest("serial device pair exists for ser2net"): + machine.succeed( + "socat -d -d " + "pty,raw,echo=0,link=${testDevice} " + "pty,raw,echo=0,link=${testDevicePeer} " + ">/tmp/socat.log 2>&1 /etc/nut/dummy-ups.dev <<'DUMMYEOF'\n" + "battery.charge: 100\n" + "ups.status: OL\n" + "DUMMYEOF\n" + ) + machine.succeed("systemctl restart upsd.service") + machine.wait_for_unit("upsd.service") + machine.succeed("systemctl restart upsdrv.service") + machine.wait_for_unit("upsdrv.service") + machine.wait_for_open_port(nut_port) + + with subtest("tailscaled starts alongside the other two services"): + machine.wait_for_unit("tailscaled.service") + + with subtest("no port conflict: both services are independently reachable"): + machine.wait_for_open_port(ser2net_port) + machine.wait_for_open_port(nut_port) + status = machine.succeed("upsc dummy-ups@localhost ups.status").strip() + assert "OL" in status, f"expected OL status, got: {status!r}" + + with subtest("firewall trusts tailscale0 without blocking the other two services' ports"): + fw = machine.succeed("iptables -L INPUT -n") + assert str(ser2net_port) in fw + assert str(nut_port) in fw + ''; +} diff --git a/tests/vm/nut.nix b/tests/vm/nut.nix new file mode 100644 index 0000000..c63ecc6 --- /dev/null +++ b/tests/vm/nut.nix @@ -0,0 +1,119 @@ +# tests/vm/nut.nix — TIN-539 +# +# NixOS VM test for NUT (Network UPS Tools) in netserver mode, using the +# `dummy-ups` driver so no physical UPS is required. This deliberately does +# NOT import modules/nut-server directly: that module wires +# `config.sops.secrets.nut-password.path`, which needs live sops-nix +# secrets infrastructure this hermetic VM does not have. Instead it +# reconstructs the same `power.ups` shape (netserver mode, upsd listening, +# upsmon primary monitor) using a `pkgs.writeText` password file in place +# of a sops secret, which is the documented safe substitution for a VM +# test of module *logic*. +# +# Verified against the live module source +# (nixos/modules/services/monitoring/ups.nix, nixos-24.11) rather than +# assumed: +# * NUT_CONFPATH is hardcoded to /etc/nut -- not host/version-dependent. +# * the systemd units are upsd.service, upsdrv.service (a oneshot that +# runs `upsdrvctl start` for every configured UPS, NOT one unit per +# UPS) and upsmon.service. +# * `users.` and `upsmon.monitor.` only accept +# `passwordFile`, never a plaintext `password`. +{ pkgs, ... }: + +let + testPasswordFile = pkgs.writeText "nut-vm-test-password" "vm-test-only-not-a-real-secret"; +in +pkgs.testers.runNixOSTest { + name = "nut"; + + nodes.machine = + { pkgs, ... }: + { + environment.systemPackages = [ pkgs.nut ]; + + power.ups = { + enable = true; + mode = "netserver"; + + ups."dummy-ups" = { + description = "VM Test Dummy UPS"; + driver = "dummy-ups"; + port = "dummy-ups.dev"; + }; + + upsd.listen = [ + { + address = "0.0.0.0"; + port = 3493; + } + ]; + + users.upsmon = { + passwordFile = testPasswordFile; + upsmon = "primary"; + }; + + upsmon = { + enable = true; + monitor."dummy-ups@localhost" = { + powerValue = 1; + type = "primary"; + user = "upsmon"; + passwordFile = testPasswordFile; + }; + settings = { + MINSUPPLIES = 1; + SHUTDOWNCMD = "${pkgs.coreutils}/bin/true"; + POLLFREQ = 5; + POLLFREQALERT = 2; + FINALDELAY = 5; + }; + }; + + openFirewall = true; + }; + }; + + testScript = '' + machine.start() + machine.wait_for_unit("multi-user.target") + + with subtest("write the dummy-ups simulated reading file"): + # NUT_CONFPATH is fixed at /etc/nut by the module; dummy-ups reads + # its simulated readings from /. + machine.succeed( + "cat > /etc/nut/dummy-ups.dev <<'DUMMYEOF'\n" + "battery.charge: 100\n" + "battery.runtime: 3600\n" + "ups.status: OL\n" + "ups.mfr: Dummy\n" + "ups.model: VM Test UPS\n" + "DUMMYEOF\n" + ) + + with subtest("upsd starts and serves on the configured port"): + machine.succeed("systemctl restart upsd.service") + machine.wait_for_unit("upsd.service") + machine.wait_for_open_port(3493) + + with subtest("upsdrv registers the dummy driver against the reading file"): + # upsdrv is a oneshot (RemainAfterExit); restarting re-runs + # `upsdrvctl start` now that both upsd and the reading file exist. + machine.succeed("systemctl restart upsdrv.service") + machine.wait_for_unit("upsdrv.service") + + with subtest("upsc query returns the dummy reading"): + status = machine.succeed("upsc dummy-ups@localhost ups.status").strip() + assert "OL" in status, f"expected OL status, got: {status!r}" + charge = machine.succeed("upsc dummy-ups@localhost battery.charge").strip() + assert charge == "100", f"expected battery.charge 100, got: {charge!r}" + + with subtest("upsmon comes up and can reach upsd"): + machine.wait_for_unit("upsmon.service") + + with subtest("firewall opens the configured upsd port"): + fw = machine.succeed("iptables -L INPUT -n") + assert "3493" in fw, "configured NUT port is not in the firewall accept rules" + ''; +} diff --git a/tests/vm/ser2net.nix b/tests/vm/ser2net.nix new file mode 100644 index 0000000..2aa486a --- /dev/null +++ b/tests/vm/ser2net.nix @@ -0,0 +1,90 @@ +# tests/vm/ser2net.nix — TIN-539 +# +# NixOS VM test for modules/ser2net: validates the module's generated +# systemd unit, config, and firewall rule without any physical serial +# hardware. A `socat` PTY pair stands in for the serial device the module +# expects at `conn.device`; ser2net's own `Restart = "on-failure"` policy +# means start order between socat and ser2net does not matter. +# +# Run via `nix flake check` (CI only — this repo's local dev machine does +# not build/eval; see justfile/CI for the sanctioned entry point). +{ pkgs, ... }: + +let + ser2netModule = ../../modules/ser2net; + testDevice = "/tmp/lab-test-tty"; + testDevicePeer = "/tmp/lab-test-tty-peer"; + testPort = 3001; + testDescription = "VM test serial connection"; +in +pkgs.testers.runNixOSTest { + name = "ser2net"; + + nodes.machine = + { pkgs, ... }: + { + imports = [ ser2netModule ]; + + environment.systemPackages = [ pkgs.socat ]; + + services.lab-ser2net = { + enable = true; + connections.test-port = { + port = testPort; + device = testDevice; + speed = "9600n81"; + description = testDescription; + }; + }; + }; + + testScript = '' + machine.start() + machine.wait_for_unit("multi-user.target") + + with subtest("generated config carries the module's own values"): + config_text = machine.succeed("cat /etc/ser2net/ser2net.yaml") + assert "${builtins.toString testPort}" in config_text, "port missing from generated config" + assert "${testDevice}" in config_text, "device path missing from generated config" + assert "${testDescription}" in config_text, "banner description missing from generated config" + + with subtest("firewall opens the configured port"): + fw = machine.succeed("iptables -L INPUT -n") + assert "${builtins.toString testPort}" in fw, "configured port is not in the firewall accept rules" + + with subtest("serial device pair exists before ser2net needs it"): + # Trailing `&` backgrounds socat so `succeed()` (which waits for the + # shell it runs to exit) returns immediately instead of blocking on + # a long-running process; redirecting all three fds detaches it from + # the test driver's connection. + machine.succeed( + "socat -d -d " + "pty,raw,echo=0,link=${testDevice} " + "pty,raw,echo=0,link=${testDevicePeer} " + ">/tmp/socat.log 2>&1 connector -> serial-device chain is + # live, not just that something is listening on the port. + banner = machine.succeed( + "timeout 5 bash -c " + "'exec 3<>/dev/tcp/127.0.0.1/${builtins.toString testPort}; head -c 200 <&3'" + ) + assert "${testDescription}" in banner, f"banner missing expected description, got: {banner!r}" + ''; +} diff --git a/tests/vm/tailscale.nix b/tests/vm/tailscale.nix new file mode 100644 index 0000000..1673710 --- /dev/null +++ b/tests/vm/tailscale.nix @@ -0,0 +1,66 @@ +# tests/vm/tailscale.nix — TIN-539 +# +# NixOS VM test for the tailscale subnet-router posture used by +# hosts/common/tailscale.nix. Does not import that file directly: it wires +# `authKeyFile = config.sops.secrets.tailscale-auth-key.path`, which needs +# live sops-nix secrets this hermetic VM does not have. This test +# reconstructs the same `services.tailscale` + firewall shape without an +# auth key -- tailscaled starts and is reachable, it just reports +# "not logged in" (no network join is attempted or expected). +# +# Verified against the live module source +# (nixos/modules/services/networking/tailscale.nix, nixos-24.11): +# * the daemon's own shipped unit is `tailscaled.service` (this module +# only sets its Environment, via `systemd.packages = [ cfg.package ]`). +# * `useRoutingFeatures = "server"` sets +# `boot.kernel.sysctl."net.ipv4.conf.all.forwarding"` (and the ipv6 +# equivalent) to true -- that is the actual forwarding assertion this +# test makes, matching what the module claims rather than a general +# "IP forwarding enabled" guess. +{ pkgs, ... }: + +pkgs.testers.runNixOSTest { + name = "tailscale"; + + nodes.machine = + { config, ... }: + { + services.tailscale = { + enable = true; + useRoutingFeatures = "server"; + }; + + networking.firewall = { + trustedInterfaces = [ "tailscale0" ]; + allowedUDPPorts = [ config.services.tailscale.port ]; + checkReversePath = "loose"; + }; + }; + + testScript = '' + machine.start() + machine.wait_for_unit("multi-user.target") + + with subtest("tailscaled starts without an auth key"): + machine.wait_for_unit("tailscaled.service") + + with subtest("tailscale reports not logged in (no join was attempted)"): + status = machine.succeed("tailscale status 2>&1 || true") + assert "Logged out" in status or "NeedsLogin" in status or "not logged in" in status.lower(), ( + f"expected an unauthenticated status, got: {status!r}" + ) + + with subtest("firewall trusts the tailscale0 interface"): + trusted = machine.succeed("iptables -L INPUT -n -v") + # nixpkgs' firewall implementation adds an ACCEPT rule keyed on the + # trusted interface name; assert the interface name appears in the + # accept chain rather than parsing exact iptables formatting. + assert "tailscale0" in trusted, "tailscale0 is not a trusted firewall interface" + + with subtest("IPv4 and IPv6 forwarding are enabled by useRoutingFeatures=server"): + ipv4_forward = machine.succeed("cat /proc/sys/net/ipv4/conf/all/forwarding").strip() + ipv6_forward = machine.succeed("cat /proc/sys/net/ipv6/conf/all/forwarding").strip() + assert ipv4_forward == "1", f"expected IPv4 forwarding enabled, got: {ipv4_forward!r}" + assert ipv6_forward == "1", f"expected IPv6 forwarding enabled, got: {ipv6_forward!r}" + ''; +}