From c60153c6234d68b21a0e21d8cde0466499d06216 Mon Sep 17 00:00:00 2001 From: Jonathan Shin <85975340+j0nsh1n@users.noreply.github.com> Date: Thu, 19 Nov 2026 12:29:36 -0800 Subject: [PATCH] fix: the Linux Wine runner fails fast when Proton's wine hangs headless, and takes --wine With no display, GE-Proton 11-11's own wine (11.0) waits forever when a program starts Vulkan (two explorer.exe /desktop processes block on an RPC to rpcss, which never starts); a system wine 11.21 runs the same prefix. Only 11-11 was observed. The runner then sat in the tests' 300 s to 3600 s limits and left wine processes behind. The runner now lists the GPU adapters first with a small probe (dxgi_probe.cpp, --probe-timeout, default 90 s). If it hangs, the tests that would start a D3D12 program (interop, ags-layout, rr-leak with --runtime) fail at once with a message that names the cause and the override. A test that would skip (no --runtime, no dxc, no compiler) still skips without the probe or the test program (the prefix is still set up, as before). --wine / WG_WINE_BIN runs the tests with another wine executable, as an absolute path, with the wineserver beside its real install (else beside the link, else on PATH). The DLLs still come from the Proton install. The default is still Proton's own wine. The probe and the test programs started through run_wine (ags-layout, rr-leak) run under xvfb-run when there is no display and it is installed, as interop already does; the launcher's unit tests need no display and run with Wine directly, as before, so a broken xvfb-run cannot stop them. A timed-out probe or run_wine program stops the Wine client first, lets xvfb-run exit and clean up by itself (bounded: TERM, then KILL), kills what is left in the process group, then runs `wineserver -k` scoped to the prefix. (interop's own shell script keeps its own timeout, outside this clean-up.) A failing or stuck clean-up step, a probe build or start failure, and a probe or wrapper that exits with an error (xvfb-run's own setup error) are per-test results that carry the error and the log; the launcher step and the summary still run. Verified: clean/linux/test_run_wine_tests.py (skips on non-Linux, so the Windows CI shard still discovers it), 33 tests with fake wine, wineserver, compiler and an xvfb-run that stays alive around its child with an exit trap; each review finding has a test that fails on the code before it. Real Wine on the previous revision 4c65ddd (no display, no xvfb-run installed; this revision only puts the launcher's programs back on direct Wine, as on main, and skips the tests off Linux): GE-Proton 11-11's wine fails ags-layout in 30 s with the message and the launcher step still runs; a system wine 11.21 given as a relative symlink passes the probe, ags-layout (7 of 7) and interop (32 tests). The xvfb-run path is covered by the fake only. Line endings and git diff --check are clean. Signed-off-by: Jonathan Shin <85975340+j0nsh1n@users.noreply.github.com> --- clean/linux/README.md | 10 +- clean/linux/dxgi_probe.cpp | 22 ++ clean/linux/run_wine_tests.py | 183 ++++++++++- clean/linux/test_run_wine_tests.py | 482 +++++++++++++++++++++++++++++ clean/linux/wine_env.py | 46 ++- 5 files changed, 716 insertions(+), 27 deletions(-) create mode 100644 clean/linux/dxgi_probe.cpp create mode 100644 clean/linux/test_run_wine_tests.py diff --git a/clean/linux/README.md b/clean/linux/README.md index e7249ff1..639b0733 100644 --- a/clean/linux/README.md +++ b/clean/linux/README.md @@ -6,7 +6,7 @@ RADV, `d3dcompiler_47.dll` is Wine's own (vkd3d-shader), and the driver channel WrongGPU's D3D12 code on that stack. They don't make Linux a supported platform. ```sh -python3 clean/linux/run_wine_tests.py --proton [--vkd3d ] [--runtime ] +python3 clean/linux/run_wine_tests.py --proton [--wine ] [--vkd3d ] [--runtime ] ``` The tests use WrongGPU's own synthetic data only: no model converted from a DLSS file, no game files. @@ -29,6 +29,14 @@ programs only, never shipped (the release rule in `clean/fsr/build-mingw.sh`). - Linux with an AMD RDNA 4 GPU (RX 9070 or RX 9060 family) and Mesa's RADV driver. - A Proton install: `--proton`, or the environment variable `WG_PROTON`. For example, a GE-Proton folder in `~/.local/share/Steam/compatibilitytools.d/`, or Steam's `steamapps/common/Proton - Experimental`. +- Without a display (a console or SSH session), a Wine that can start Vulkan programs. Proton's own wine (seen with + GE-Proton 11-11, wine 11.0) can wait forever there, while a system Wine (11.21 tried) runs the same prefix. The runner + first lists the GPU adapters with a small probe (`dxgi_probe.cpp`, `--probe-timeout`, default 90 s); if that hangs, the tests that would + start a D3D12 program (`interop`, `ags-layout`, and `rr-leak` only with `--runtime`) fail at once with a message, and the + stuck Wine is stopped (the probe and test programs run under `xvfb-run` when it is installed). A probe or `xvfb-run` that + exits with an error fails them too, with its output. A test that would skip (`rr-leak` without `--runtime`, `ags-layout` + without `dxc`) skips as before, without the probe or the test program (the prefix is still set up). `--wine ` (or `WG_WINE_BIN`) runs the tests with another wine + executable while the DLLs still come from `--proton`. In a desktop session Proton's wine is used as before. - A MinGW-w64 C++ compiler: `MINGW_CXX`, otherwise `x86_64-w64-mingw32-g++-posix` or `x86_64-w64-mingw32-g++` on `PATH`. llvm-mingw works too. The tests add `-fms-extensions` for clang. - For `ags-layout`: the Linux `dxc` from the DXC release named in BUILDING.md, in `WG_DXC` or on `PATH`. diff --git a/clean/linux/dxgi_probe.cpp b/clean/linux/dxgi_probe.cpp new file mode 100644 index 00000000..fcc2ee01 --- /dev/null +++ b/clean/linux/dxgi_probe.cpp @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: MIT +// Lists the DXGI adapters and exits. run_wine_tests.py runs it first with a time limit: a Wine that hangs when a program +// starts Vulkan (the D3D12 tests' first step) hangs here too, in seconds instead of the tests' long limits. +#include +#include + +int main() { + IDXGIFactory1* f = nullptr; + if (FAILED(CreateDXGIFactory1(__uuidof(IDXGIFactory1), reinterpret_cast(&f)))) { + std::printf("probe: CreateDXGIFactory1 failed\n"); + return 1; + } + IDXGIAdapter1* a = nullptr; + unsigned n = 0; + while (f->EnumAdapters1(n, &a) == S_OK) { + a->Release(); + ++n; + } + f->Release(); + std::printf("probe: %u adapter(s)\n", n); + return 0; +} diff --git a/clean/linux/run_wine_tests.py b/clean/linux/run_wine_tests.py index 40a5101a..f7dbc693 100644 --- a/clean/linux/run_wine_tests.py +++ b/clean/linux/run_wine_tests.py @@ -2,7 +2,7 @@ """WrongGPU's D3D12 tests on Linux, under Wine with the D3D12 stack of a Proton install (vkd3d-proton, DXVK's dxgi, vkd3d-shader), the stack a game runs on in Steam. - python3 clean/linux/run_wine_tests.py --proton [--vkd3d ] [--runtime ] + python3 clean/linux/run_wine_tests.py --proton [--wine ] [--vkd3d ] [--runtime ] [--tests interop,rr-leak,ags-layout,launcher] [--work ] Tests (all run by default; each prints PASS, FAIL or SKIP with the reason; the exit status is 1 when one fails): @@ -26,11 +26,14 @@ launcher test also needs CMake, Ninja and the MinGW-w64 windres beside MINGW_CXX. """ import argparse +import contextlib import os import re import shutil +import signal import subprocess import sys +import tempfile from pathlib import Path import numpy as np @@ -61,17 +64,148 @@ def build_exe(cxx, sources, out, includes=(), libs=()): raise RuntimeError(f"building {out.name} failed:\n{r.stderr[-3000:]}") -def run_wine(env, args, cwd, timeout): - r = subprocess.run([env["WG_WINE"], *args], cwd=cwd, env=env, capture_output=True, text=True, errors="replace", - timeout=timeout) - return r.returncode, r.stdout.replace("\r", "") + r.stderr.replace("\r", "") +def skip_reason(name, a): + """Why a test would SKIP before it runs (None when it would run); the adapter probe is run only when some D3D12 + test is eligible, so a test that skips still skips without the probe or its test program (the prefix is set up + before, as on main).""" + if name in ("interop", "rr-leak", "ags-layout", "launcher") and not mingw_cxx(): + return "no MinGW-w64 C++ compiler (MINGW_CXX)" + if name == "rr-leak" and not a.runtime: + return "needs --runtime with the RR runtime linked in" + if name == "ags-layout" and not shutil.which(os.environ.get("WG_DXC", "dxc")): + return "no dxc (WG_DXC: the DXC release's Linux dxc)" + return None + + +def wine_cmd(env, display=True): + """The command that starts a Windows program: wine, under xvfb-run when `display` is wanted, there is no display + and xvfb-run is installed (what clean/runtime/interop/tests/run-tests.sh does for the interop test). The launcher + unit tests need no display and run with Wine directly, as before, so a broken xvfb-run cannot stop them.""" + cmd = [env["WG_WINE"]] + xvfb = None if env.get("DISPLAY") or not display else shutil.which("xvfb-run", path=env.get("PATH")) + return [xvfb, "-a", *cmd] if xvfb else cmd + + +KILL_TIMEOUT = 60 # seconds each step of stopping a stuck wine client (and `wineserver -k`) gets + + +class ProbeFailed(RuntimeError): + """The adapter probe (or the wrapper around it) exited with an error: not a hang, and not a completed probe.""" + + +def _group_members(pgid): + """Pids of the process group `pgid` (from /proc; a stuck wine's helpers stay in the group of the command).""" + members = [] + for d in Path("/proc").iterdir(): + if d.name.isdigit(): + try: + fields = (d / "stat").read_text().rsplit(")", 1)[1].split() + except (OSError, IndexError): + continue + if int(fields[2]) == pgid: + members.append(int(d.name)) + return members + + +def _signal(pid, sig): + with contextlib.suppress(ProcessLookupError, PermissionError): + os.kill(pid, sig) + + +def launch(env, args, cwd, timeout, notes, display=True): + """Run `wine args` (under xvfb-run when wine_cmd says so) with its output in a file in `cwd` and return + (exit status, output text), or (None, text) after `timeout` seconds, with the stuck client stopped in this order: + the Wine client gets SIGTERM, a wrapper such as xvfb-run is left to exit and clean up by itself (waited for, then + SIGTERM, then SIGKILL, each step bounded by KILL_TIMEOUT), anything left in the group is killed, and then + `wineserver -k` (env["WG_WINESERVER"], WINEPREFIX of env) stops the prefix's wine processes. A step that fails or + hangs is added to `notes`, not raised. Output goes to a file, not a pipe: the wine processes a hung run leaves + behind keep a pipe open and would block the read for good.""" + cmd = wine_cmd(env, display) + wrapped = len(cmd) > 1 + fd, name = tempfile.mkstemp(prefix=".wg-wine-", suffix=".log", dir=cwd) + try: + with os.fdopen(fd, "w") as out: + p = subprocess.Popen([*cmd, *args], cwd=cwd, env=env, stdout=out, stderr=subprocess.STDOUT, + start_new_session=True) + try: + rc = p.wait(timeout) + except subprocess.TimeoutExpired: + rc = None + if rc is None: + # the wine client first; a wrapper (xvfb-run) then ends by itself and removes what it made + for pid in ([m for m in _group_members(p.pid) if m != p.pid] if wrapped else [p.pid]): + _signal(pid, signal.SIGTERM) + for sig in ((signal.SIGTERM, signal.SIGKILL) if wrapped else (signal.SIGKILL,)): + try: + p.wait(KILL_TIMEOUT) + break + except subprocess.TimeoutExpired: + notes.append(f"{Path(cmd[0]).name} (pid {p.pid}) did not exit; sent {sig.name}") + _signal(p.pid, sig) + else: + with contextlib.suppress(subprocess.TimeoutExpired): + p.wait(KILL_TIMEOUT) + with contextlib.suppress(ProcessLookupError, PermissionError): + os.killpg(p.pid, signal.SIGKILL) # what the wrapper left behind, after its clean-up + server = env.get("WG_WINESERVER") + if not server: + notes.append("no wineserver known, the prefix's wine processes were not stopped") + else: + try: + r = subprocess.run([server, "-k"], env=env, capture_output=True, text=True, errors="replace", + timeout=KILL_TIMEOUT) + if r.returncode != 0: + notes.append(f"`{server} -k` exited {r.returncode}: {(r.stdout + r.stderr).strip()[-300:]}") + except subprocess.TimeoutExpired: + notes.append(f"`{server} -k` did not return within {KILL_TIMEOUT} s") + except OSError as e: + notes.append(f"`{server} -k` could not start: {e}") + text = Path(name).read_text(errors="replace").replace("\r", "") + finally: + with contextlib.suppress(OSError): + os.unlink(name) + return rc, text + + +def run_wine(env, args, cwd, timeout, display=True): + notes = [] + rc, text = launch(env, args, cwd, timeout, notes, display) + if rc is None: + raise RuntimeError(f"{Path(args[0]).name} timed out after {timeout} s and was stopped" + + ("; " + "; ".join(notes) if notes else "")) + return rc, text + + +PROBE_SRC = HERE / "dxgi_probe.cpp" +HANG_NEEDS = ("interop", "rr-leak", "ags-layout") # the tests that start a D3D12 / Vulkan program in the prefix + + +def probe_hang(env, exe, cwd, timeout, notes=None): + """True when the probe program (it lists the DXGI adapters, where a stuck Wine waits) does not finish in + `timeout` seconds (stopped as launch() describes; clean-up problems go to `notes`), False when it finishes with + status 0. A probe or wrapper (xvfb-run) that exits with an error raises ProbeFailed with its output.""" + notes = [] if notes is None else notes + rc, text = launch(env, [str(exe)], cwd, timeout, notes) + if rc is None: + return True + if rc != 0: + raise ProbeFailed(f"the probe exited {rc}: {text.strip()[-600:]}") + return False + + +def hang_message(env, timeout, notes=()): + msg = (f"{env['WG_WINE']} did not list the GPU adapters within {timeout} s (a Wine program that starts Vulkan or " + "D3D12 waits forever; observed with GE-Proton 11-11's own wine (11.0) when no display is available); " + "pass a system Wine with --wine (it runs the same prefix with the Proton install's DLLs), or run " + "in a desktop session") + return msg + ("; clean-up problems: " + "; ".join(notes) if notes else "") # ---- interop -------------------------------------------------------------------------------------------------------- def test_interop(a, env, work): + if skip_reason("interop", a): + return "SKIP", skip_reason("interop", a) cxx = mingw_cxx() - if not cxx: - return "SKIP", "no MinGW-w64 C++ compiler (MINGW_CXX)" out = work / "interop" e = dict(env, MINGW_CXX=cxx, WG_TEST_DIR=str(out)) # run-tests.sh makes its own prefix under WG_TEST_DIR unless WINEPREFIX is set: use the prepared one @@ -90,11 +224,9 @@ def test_interop(a, env, work): # ---- rr-leak -------------------------------------------------------------------------------------------------------- def test_rr_leak(a, env, work): - if not a.runtime: - return "SKIP", "needs --runtime with the RR runtime linked in" + if skip_reason("rr-leak", a): + return "SKIP", skip_reason("rr-leak", a) cxx = mingw_cxx() - if not cxx: - return "SKIP", "no MinGW-w64 C++ compiler (MINGW_CXX)" out = work / "rr-leak" stage, run = out / "stage", out / "run" for d in (stage, run): @@ -230,12 +362,10 @@ def ags_compare(got, exp): def test_ags_layout(a, env, work): + if skip_reason("ags-layout", a): + return "SKIP", skip_reason("ags-layout", a) cxx = mingw_cxx() - if not cxx: - return "SKIP", "no MinGW-w64 C++ compiler (MINGW_CXX)" dxc = shutil.which(os.environ.get("WG_DXC", "dxc")) - if not dxc: - return "SKIP", "no dxc (WG_DXC: the DXC release's Linux dxc)" out = work / "ags-layout" out.mkdir(parents=True, exist_ok=True) for entry in ("main32", "main64"): @@ -294,7 +424,7 @@ def test_launcher(a, env, work): raise RuntimeError("building the launcher's unit tests and the WrongGPU program failed:\n" + (r.stdout + r.stderr)[-3000:]) passed, failed = [], [] for t in LAUNCHER_TESTS: - rc, text = run_wine(env, [str(build / (t + ".exe"))], build, 600) + rc, text = run_wine(env, [str(build / (t + ".exe"))], build, 600, display=False) # the test's own summary line (Wine's messages are mixed into the output) last = [line.strip() for line in text.splitlines() if "checks passed" in line or "FAILED" in line][-1:] or ["(no summary line)"] log(f" {t}: exit {rc}, {last[0]}") @@ -310,6 +440,10 @@ def test_launcher(a, env, work): def main(argv=None): ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) ap.add_argument("--proton", default=os.environ.get("WG_PROTON"), help="Proton install (env WG_PROTON)") + ap.add_argument("--wine", default=os.environ.get("WG_WINE_BIN"), + help="wine executable to run the tests with instead of the Proton install's own (env WG_WINE_BIN)") + ap.add_argument("--probe-timeout", type=int, default=90, + help="seconds the adapter probe may take before the D3D12 tests fail as 'Wine hangs' (default 90)") ap.add_argument("--vkd3d", help="folder with a self-built vkd3d-proton d3d12.dll + d3d12core.dll") ap.add_argument("--runtime", help="wronggpu_runtime.dll with the RR runtime (test rr-leak)") ap.add_argument("--tests", default=",".join(TESTS), help="comma list of: " + ", ".join(TESTS)) @@ -325,10 +459,25 @@ def main(argv=None): if " " in str(work): ap.error("--work must not contain spaces (the RR test host's job file is split on spaces)") work.mkdir(parents=True, exist_ok=True) - env = wine_env.prepare(a.proton, work / "prefix", a.vkd3d, log=log) + env = wine_env.prepare(a.proton, work / "prefix", a.vkd3d, log=log, wine=a.wine) + hung = None + if any(n in HANG_NEEDS and not skip_reason(n, a) for n in names): + try: + cxx = mingw_cxx() + probe = work / "dxgi_probe.exe" + build_exe(cxx, [str(PROBE_SRC)], probe, libs=["-ldxgi"]) + notes = [] + if probe_hang(env, probe, work, a.probe_timeout, notes): + hung = hang_message(env, a.probe_timeout, notes) + except Exception as e: # the probe's own failure fails the tests that need it, not the whole run + hung = f"the adapter probe could not run: {type(e).__name__}: {e}" results = {} for name in names: log(f"== {name}") + if hung and name in HANG_NEEDS and not skip_reason(name, a): + results[name] = ("FAIL", hung) + log(f"FAIL {name}: {hung}") + continue try: results[name] = TESTS[name](a, env, work) except Exception as e: # a build or tool failure is this test's failure, not the run's diff --git a/clean/linux/test_run_wine_tests.py b/clean/linux/test_run_wine_tests.py new file mode 100644 index 00000000..44b8d7d6 --- /dev/null +++ b/clean/linux/test_run_wine_tests.py @@ -0,0 +1,482 @@ +# SPDX-License-Identifier: MIT +"""CPU-only tests of run_wine_tests.py's hang probe and wine_env's --wine override, with fake wine, wineserver, compiler +and xvfb-run scripts (no Wine, no GPU, no MinGW): python3 -I clean/linux/test_run_wine_tests.py""" +import contextlib +import io +import os +import re +import signal +import stat +import subprocess +import sys +import tempfile +import time +import unittest +from pathlib import Path +from unittest import mock + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +import run_wine_tests as rwt # noqa: E402 +import wine_env # noqa: E402 + +FAKE_WINE = """#!/bin/sh +echo "wine $* DISPLAY=${DISPLAY-unset} WINEPREFIX=${WINEPREFIX-}" >> "$FAKE_LOG" +if [ "$1" = wineboot ]; then + mkdir -p "$WINEPREFIX/drive_c/windows/system32"; : > "$WINEPREFIX/system.reg"; exit 0 +fi +case "${FAKE_MODE-}" in + hang) sleep 300 & echo $! >> "$FAKE_PIDS"; echo $$ >> "$FAKE_PIDS"; exec sleep 300 ;; + needs_display) [ -n "${DISPLAY-}" ] || { echo $$ >> "$FAKE_PIDS"; exec sleep 300; } ;; + exit1) echo "fake wine: probe failed" >&2; exit 1 ;; + summary) echo "wine ran: $1"; echo "5 checks passed"; exit 0 ;; +esac +exit 0 +""" +FAKE_WINESERVER = """#!/bin/sh +echo "wineserver $* WINEPREFIX=${WINEPREFIX-}" >> "$FAKE_LOG" +if [ "$1" = -k ]; then # only the clean-up call misbehaves, so a prefix can be set up first + case "${FAKE_WS-}" in + fail) echo "wineserver: cannot kill" >&2; exit 3 ;; + slow) sleep 30 ;; + esac +fi +exit 0 +""" +FAKE_XVFB = """#!/bin/sh +echo "xvfb-run $*" >> "$FAKE_LOG" +if [ -n "${FAKE_XVFB_FAIL-}" ]; then echo "xvfb-run: error: xauth command not found" >&2; exit 3; fi +[ "$1" = -a ] && shift +: > "$FAKE_XAUTH" +trap 'echo xvfb-cleanup >> "$FAKE_LOG"; rm -f "$FAKE_XAUTH"' EXIT +[ -n "${FAKE_XVFB_STUBBORN-}" ] && trap '' TERM +DISPLAY=:99 "$@" & +child=$! +echo $$ >> "$FAKE_PIDS" +wait $child +rc=$? +[ -n "${FAKE_XVFB_STUBBORN-}" ] && while :; do sleep 1; done +exit $rc +""" +FAKE_CXX = """#!/bin/sh +echo "cxx $*" >> "$FAKE_LOG" +if [ "$1" = --version ]; then echo fake-cxx; exit 0; fi +[ -n "${FAKE_CXX_FAIL-}" ] && { echo "fake compiler: cannot find -ldxgi" >&2; exit 1; } +while [ $# -gt 0 ]; do [ "$1" = -o ] && { : > "$2"; break; }; shift; done +exit 0 +""" + + +def script(path, body): + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(body) + path.chmod(path.stat().st_mode | stat.S_IEXEC) + return path + + +LINUX_ONLY = "needs Linux: the fake wine, wineserver and xvfb-run are shell scripts, and the clean-up uses /proc and process groups" + + +class Fixture(unittest.TestCase): + @classmethod + def setUpClass(cls): # at run time, not import time: a Windows shard must still discover (and skip) the module + if not sys.platform.startswith("linux"): + raise unittest.SkipTest(LINUX_ONLY) + + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.dir = Path(self.tmp.name).resolve() + self.log = self.dir / "fake.log" + self.pids = self.dir / "fake.pids" + self.log.write_text("") + self.addCleanup(self.stop_pids) + self.xauth = self.dir / "fake.xauth" + base = {"FAKE_LOG": str(self.log), "FAKE_PIDS": str(self.pids), "FAKE_XAUTH": str(self.xauth)} + p = mock.patch.dict(os.environ, base) + p.start() + self.addCleanup(p.stop) + # xvfb-run is found in the fixture's own bin folder only, never in a host install + os.environ["PATH"] = os.pathsep.join(d for d in os.environ["PATH"].split(os.pathsep) + if not os.access(os.path.join(d, "xvfb-run"), os.X_OK)) + for k in ("DISPLAY", "WAYLAND_DISPLAY", "FAKE_MODE", "FAKE_WS", "FAKE_CXX_FAIL", "FAKE_XVFB_FAIL", "FAKE_XVFB_STUBBORN", "MINGW_CXX", "WG_WINE_BIN", "WG_DXC"): + os.environ.pop(k, None) + # a Proton install with the DLLs the runner copies, and a fake wine + wineserver in it + self.proton = self.dir / "proton" + for rel in ("vkd3d-proton/x86_64-windows/d3d12.dll", "vkd3d-proton/x86_64-windows/d3d12core.dll", + "dxvk/x86_64-windows/dxgi.dll", "vkd3d/x86_64-windows/libvkd3d-utils-1.dll", + "vkd3d/x86_64-windows/libvkd3d-1.dll", "vkd3d/x86_64-windows/libvkd3d-shader-1.dll"): + f = self.proton / "files" / "lib" / "wine" / rel + f.parent.mkdir(parents=True, exist_ok=True) + f.write_bytes(b"x") + self.pwine = script(self.proton / "files" / "bin" / "wine", FAKE_WINE) + script(self.proton / "files" / "bin" / "wineserver", FAKE_WINESERVER) + self.prefix = self.dir / "prefix" + + def stop_pids(self): + if self.pids.exists(): + for line in self.pids.read_text().split(): + with contextlib.suppress(ProcessLookupError): + os.kill(int(line), signal.SIGTERM) + + def alive_pids(self): + alive = [] + for line in (self.pids.read_text().split() if self.pids.exists() else []): + try: + os.kill(int(line), 0) + alive.append(int(line)) + except ProcessLookupError: + pass + return alive + + def use_xvfb(self): + script(self.dir / "bin" / "xvfb-run", FAKE_XVFB) + os.environ["PATH"] = f"{self.dir / 'bin'}{os.pathsep}{os.environ['PATH']}" + + def calls(self, prefix=""): + return [l for l in self.log.read_text().splitlines() if l.startswith(prefix)] + + def env_for(self, wine=None, **extra): + env = wine_env.prepare(self.proton, self.prefix, log=lambda m: None, wine=wine) + env.update(extra) + return env + + def run_main(self, *args, stubs=None): + out = io.StringIO() + saved = dict(rwt.TESTS) + try: + for k, v in (stubs or {}).items(): + rwt.TESTS[k] = v + with contextlib.redirect_stdout(out): + rc = rwt.main(["--proton", str(self.proton), "--work", str(self.dir / "work"), "--probe-timeout", "2", *args]) + finally: + rwt.TESTS.clear() + rwt.TESTS.update(saved) + return rc, out.getvalue() + + +class ProbeHang(Fixture): + def test_finished_probe_is_not_a_hang(self): + self.assertFalse(rwt.probe_hang(self.env_for(), "x.exe", self.dir, 5)) + + def test_stuck_probe_is_a_hang_even_when_a_child_keeps_the_output_open(self): + os.environ["FAKE_MODE"] = "hang" + self.assertTrue(rwt.probe_hang(self.env_for(), "x.exe", self.dir, 2)) + + # finding 1: the probe gets interop's display setup (xvfb-run when DISPLAY is unset and it is installed) + def test_probe_runs_under_xvfb_run_when_there_is_no_display(self): + script(self.dir / "bin" / "xvfb-run", FAKE_XVFB) + os.environ["PATH"] = f"{self.dir / 'bin'}{os.pathsep}{os.environ['PATH']}" + os.environ["FAKE_MODE"] = "needs_display" + self.assertFalse(rwt.probe_hang(self.env_for(), "x.exe", self.dir, 3)) + self.assertTrue(self.calls("xvfb-run")) + + def test_test_programs_run_under_xvfb_run_too(self): + script(self.dir / "bin" / "xvfb-run", FAKE_XVFB) + os.environ["PATH"] = f"{self.dir / 'bin'}{os.pathsep}{os.environ['PATH']}" + os.environ["FAKE_MODE"] = "needs_display" + rc, _ = rwt.run_wine(self.env_for(), ["x.exe"], self.dir, 5) + self.assertEqual(rc, 0) + + # finding 3: the wineserver that belongs to the wine, used for cleanup, scoped to the prepared prefix + def test_timeout_cleanup_uses_the_resolved_wineserver_and_the_prefix(self): + os.environ["FAKE_MODE"] = "hang" + env = self.env_for() + self.assertTrue(rwt.probe_hang(env, "x.exe", self.dir, 2)) + self.assertEqual(self.calls("wineserver -k"), [f"wineserver -k WINEPREFIX={self.prefix}"]) + + def test_cleanup_uses_wg_wineserver_not_a_sibling_guess(self): + # a wine whose wineserver is not beside it (a symlink in a user bin directory) + os.environ["FAKE_MODE"] = "hang" + ws = script(self.dir / "elsewhere" / "wineserver", FAKE_WINESERVER) + env = self.env_for() + env["WG_WINE"] = str(script(self.dir / "userbin" / "wine", FAKE_WINE)) + env["WG_WINESERVER"] = str(ws) + self.assertTrue(rwt.probe_hang(env, "x.exe", self.dir, 2)) + self.assertEqual(len(self.calls("wineserver -k")), 1) + + # finding 4: a failing or stuck cleanup is reported, never raised + def test_cleanup_failure_is_reported_not_discarded(self): + os.environ["FAKE_MODE"] = "hang" + os.environ["FAKE_WS"] = "fail" + env = self.env_for() + notes = [] + self.assertTrue(rwt.probe_hang(env, "x.exe", self.dir, 2, notes=notes)) + self.assertIn("cannot kill", " ".join(notes)) + + def test_cleanup_timeout_is_reported_not_raised(self): + os.environ["FAKE_MODE"] = "hang" + os.environ["FAKE_WS"] = "slow" + env = self.env_for() + notes = [] + with mock.patch.object(rwt, "KILL_TIMEOUT", 1): + self.assertTrue(rwt.probe_hang(env, "x.exe", self.dir, 2, notes=notes)) + self.assertTrue(notes) + + # round 2, item 1: a timeout under a wrapper that stays alive around its child (xvfb-run) + def test_probe_timeout_under_xvfb_run_lets_the_wrapper_clean_up_and_stops_the_client(self): + self.use_xvfb() + os.environ["FAKE_MODE"] = "hang" + env = self.env_for() + self.assertTrue(rwt.probe_hang(env, "x.exe", self.dir, 2)) + self.assertEqual(self.alive_pids(), []) + self.assertFalse(self.xauth.exists(), "the wrapper's own clean-up (X authority file) did not run") + self.assertEqual(self.calls("xvfb-cleanup"), ["xvfb-cleanup"]) + self.assertEqual(self.calls("wineserver -k"), [f"wineserver -k WINEPREFIX={self.prefix}"]) + + def test_test_program_timeout_under_xvfb_run_stops_the_client_and_cleans_up(self): + self.use_xvfb() + os.environ["FAKE_MODE"] = "hang" + env = self.env_for() + with self.assertRaises(Exception) as c: + rwt.run_wine(env, ["x.exe"], self.dir, 2) + self.assertIn("timed out", str(c.exception)) + self.assertEqual(self.alive_pids(), []) + self.assertFalse(self.xauth.exists()) + self.assertEqual(self.calls("wineserver -k"), [f"wineserver -k WINEPREFIX={self.prefix}"]) + + def test_test_program_timeout_without_xvfb_stops_the_client(self): + os.environ["FAKE_MODE"] = "hang" + env = self.env_for() + with self.assertRaises(Exception): + rwt.run_wine(env, ["x.exe"], self.dir, 2) + self.assertEqual(self.alive_pids(), []) + self.assertEqual(len(self.calls("wineserver -k")), 1) + + def test_a_wrapper_that_ignores_term_is_escalated_within_a_bound(self): + self.use_xvfb() + os.environ["FAKE_MODE"] = "hang" + os.environ["FAKE_XVFB_STUBBORN"] = "1" + env = self.env_for() + notes = [] + t = time.monotonic() + with mock.patch.object(rwt, "KILL_TIMEOUT", 1): + self.assertTrue(rwt.probe_hang(env, "x.exe", self.dir, 2, notes=notes)) + self.assertLess(time.monotonic() - t, 20) + self.assertEqual(self.alive_pids(), []) + self.assertEqual(len(self.calls("wineserver -k")), 1) + + # item 3: a non-zero probe or wrapper exit is a failure with its log, not a finished probe + def test_wrapper_setup_error_is_a_probe_failure_with_its_log(self): + self.use_xvfb() + os.environ["FAKE_XVFB_FAIL"] = "1" + with self.assertRaises(Exception) as c: + rwt.probe_hang(self.env_for(), "x.exe", self.dir, 5) + self.assertIn("xauth command not found", str(c.exception)) + + def test_probe_that_exits_non_zero_is_a_probe_failure(self): + os.environ["FAKE_MODE"] = "exit1" + with self.assertRaises(Exception) as c: + rwt.probe_hang(self.env_for(), "x.exe", self.dir, 5) + self.assertIn("probe failed", str(c.exception)) + + # finding 6 + def test_message_names_the_cause_the_override_and_only_the_observed_case(self): + msg = rwt.hang_message({"WG_WINE": "/p/files/bin/wine"}, 90) + self.assertIn("--wine", msg) + self.assertIn("GE-Proton 11-11", msg) + self.assertNotIn("11-12", msg) + self.assertIn("90 s", msg) + + def test_readme_names_only_the_observed_case(self): + text = (Path(__file__).resolve().parent / "README.md").read_text() + self.assertNotIn("11-12", text) + + def test_readme_says_which_tests_fail_and_that_ineligible_ones_skip(self): + text = " ".join((Path(__file__).resolve().parent / "README.md").read_text().replace("`", "").split()) + self.assertIn("A test that would skip (rr-leak without --runtime, ags-layout without dxc) skips as before", text) + + +class WineOverride(Fixture): + def test_missing_wine_is_refused(self): + with self.assertRaises(SystemExit) as c: + wine_env.prepare(self.proton, self.prefix, wine="/nonexistent/wine") + self.assertIn("--wine", str(c.exception)) + + # finding 2 + def test_relative_wine_becomes_absolute(self): + script(self.dir / "rel" / "bin" / "wine", FAKE_WINE) + script(self.dir / "rel" / "bin" / "wineserver", FAKE_WINESERVER) + cwd = os.getcwd() + os.chdir(self.dir / "rel") + self.addCleanup(os.chdir, cwd) + env = self.env_for(wine="./bin/wine") + self.assertTrue(os.path.isabs(env["WG_WINE"]), env["WG_WINE"]) + os.chdir(self.dir) # a child process runs from another folder (the --work folder) + self.assertFalse(rwt.probe_hang(env, "x.exe", self.dir, 5)) + + def test_wine_from_path_name_becomes_absolute(self): + script(self.dir / "pbin" / "wine", FAKE_WINE) + script(self.dir / "pbin" / "wineserver", FAKE_WINESERVER) + os.environ["PATH"] = f"{self.dir / 'pbin'}{os.pathsep}{os.environ['PATH']}" + self.assertTrue(os.path.isabs(self.env_for(wine="wine")["WG_WINE"])) + + # finding 3 + def test_wine_with_its_wineserver_elsewhere_prepares_a_fresh_prefix(self): + real = script(self.dir / "real" / "bin" / "wine", FAKE_WINE) + (self.dir / "userbin").mkdir() + (self.dir / "userbin" / "wine").symlink_to(real) + script(self.dir / "wsdir" / "wineserver", FAKE_WINESERVER) + os.environ["PATH"] = f"{self.dir / 'wsdir'}{os.pathsep}{os.environ['PATH']}" + env = self.env_for(wine=str(self.dir / "userbin" / "wine")) + self.assertEqual(Path(env["WG_WINESERVER"]), self.dir / "wsdir" / "wineserver") + self.assertEqual(self.calls("wineserver -w"), [f"wineserver -w WINEPREFIX={self.prefix}"]) + + # round 2, item 2: a symlinked wine uses the wineserver beside its target, not an unrelated one beside the link + def test_symlinked_wine_prefers_the_wineserver_beside_its_target(self): + real = script(self.dir / "real" / "bin" / "wine", FAKE_WINE) + script(self.dir / "real" / "bin" / "wineserver", FAKE_WINESERVER) + (self.dir / "userbin").mkdir() + (self.dir / "userbin" / "wine").symlink_to(real) + script(self.dir / "userbin" / "wineserver", FAKE_WINESERVER.replace("echo ", "echo WRONG ", 1)) + env = self.env_for(wine=str(self.dir / "userbin" / "wine")) + self.assertEqual(Path(env["WG_WINESERVER"]), self.dir / "real" / "bin" / "wineserver") + self.assertFalse([l for l in self.calls() if "WRONG" in l]) + + def test_wine_without_any_wineserver_is_refused_with_a_message(self): + script(self.dir / "lonely" / "wine", FAKE_WINE) + os.environ["PATH"] = os.pathsep.join(p for p in os.environ["PATH"].split(os.pathsep) if not (Path(p) / "wineserver").exists()) + with self.assertRaises(SystemExit) as c: + self.env_for(wine=str(self.dir / "lonely" / "wine")) + self.assertIn("wineserver", str(c.exception)) + + def test_new_prefix_log_names_the_wine_that_made_it(self): + script(self.dir / "sys" / "wine", FAKE_WINE) + script(self.dir / "sys" / "wineserver", FAKE_WINESERVER) + msgs = [] + wine_env.prepare(self.proton, self.prefix, log=msgs.append, wine=str(self.dir / "sys" / "wine")) + self.assertIn(str(self.dir / "sys" / "wine"), msgs[0]) + + +class MainFlow(Fixture): + def setUp(self): + super().setUp() + self.cxx = script(self.dir / "tools" / "x86_64-w64-mingw32-g++", FAKE_CXX) + os.environ["MINGW_CXX"] = str(self.cxx) + self.called = [] + + def stub(name, result): + def f(a, env, work): + self.called.append(name) + return result + return f + self.stub = stub + + # finding 4 + def test_probe_build_failure_is_a_per_test_result_and_launcher_still_runs(self): + os.environ["FAKE_CXX_FAIL"] = "1" + rc, out = self.run_main("--tests", "interop,launcher", stubs={"launcher": self.stub("launcher", ("PASS", "ok"))}) + self.assertEqual(rc, 1) + self.assertIn("FAIL interop", out) + self.assertIn("-ldxgi", out) + self.assertIn("PASS launcher", out) + self.assertIn("== summary", out) + self.assertEqual(self.called, ["launcher"]) + + def test_probe_start_failure_is_a_per_test_result(self): + script(self.pwine, FAKE_WINE.replace("exit 0\n", "exit 0\n", 1)) + with mock.patch.object(rwt, "probe_hang", side_effect=OSError("cannot start wine")): + rc, out = self.run_main("--tests", "interop,launcher", stubs={"launcher": self.stub("launcher", ("PASS", "ok"))}) + self.assertIn("cannot start wine", out) + self.assertIn("PASS launcher", out) + + def test_hang_with_cleanup_error_puts_the_error_in_the_message(self): + os.environ["FAKE_MODE"] = "hang" + os.environ["FAKE_WS"] = "fail" + rc, out = self.run_main("--tests", "interop,launcher", stubs={"launcher": self.stub("launcher", ("PASS", "ok"))}) + self.assertIn("FAIL interop", out) + self.assertIn("cannot kill", out) + self.assertIn("PASS launcher", out) + + # finding 5 + def test_rr_leak_without_runtime_still_skips_without_probe_or_test_program(self): + os.environ["FAKE_MODE"] = "hang" + rc, out = self.run_main("--tests", "rr-leak") + self.assertIn("SKIP rr-leak", out) + self.assertEqual(rc, 0) + # the prefix is still set up (wineboot, as on main); no adapter probe and no test program start + self.assertEqual([l for l in self.calls("wine ") if "wineboot" not in l], []) + self.assertFalse([l for l in self.calls("cxx") if "dxgi_probe" in l]) + + def test_ags_layout_without_dxc_still_skips_without_probe_or_test_program(self): + os.environ["FAKE_MODE"] = "hang" + os.environ["WG_DXC"] = str(self.dir / "no-such-dxc") + rc, out = self.run_main("--tests", "ags-layout") + self.assertIn("SKIP ags-layout", out) + self.assertEqual(rc, 0) + self.assertEqual([l for l in self.calls("wine ") if "wineboot" not in l], []) # wineboot: prefix setup only + + def test_wrapper_setup_error_fails_the_tests_that_need_the_probe_with_its_log(self): + self.use_xvfb() + os.environ["FAKE_XVFB_FAIL"] = "1" + rc, out = self.run_main("--tests", "interop,launcher", stubs={"launcher": self.stub("launcher", ("PASS", "ok"))}) + self.assertEqual(rc, 1) + self.assertIn("FAIL interop", out) + self.assertIn("xauth command not found", out) + self.assertIn("PASS launcher", out) + + def test_eligible_test_still_fails_when_wine_hangs(self): + os.environ["FAKE_MODE"] = "hang" + rc, out = self.run_main("--tests", "interop", stubs={"interop": self.stub("interop", ("PASS", "ok"))}) + self.assertEqual(rc, 1) + self.assertIn("did not list the GPU adapters", out) + self.assertEqual(self.called, []) + + +class NonLinux(unittest.TestCase): + """What a Windows CI shard sees: the module must be discoverable and skip, not fail.""" + + def test_fixture_tests_skip_when_the_platform_is_not_linux(self): + code = ("import sys, unittest\nsys.path.insert(0, sys.argv[1])\n" + "import test_run_wine_tests as t\nsys.platform = 'win32'\n" + "suite = unittest.TestSuite(unittest.defaultTestLoader.loadTestsFromTestCase(c) for c in vars(t).values()\n" + " if isinstance(c, type) and issubclass(c, t.Fixture) and c is not t.Fixture)\n" + "r = unittest.TextTestRunner(verbosity=0).run(suite)\n" + "print('SKIPPED', len(r.skipped), 'RAN', r.testsRun, 'FAILED', len(r.failures) + len(r.errors))\n" + "print('REASON', sorted({x[1] for x in r.skipped})[0] if r.skipped else None)\n") + r = subprocess.run([sys.executable, "-I", "-c", code, str(Path(__file__).resolve().parent)], + capture_output=True, text=True, timeout=120) + self.assertEqual(r.returncode, 0, r.stderr) + m = re.search(r"SKIPPED (\d+) RAN (\d+) FAILED (\d+)", r.stdout) + self.assertTrue(m, r.stdout + r.stderr) + skipped, ran, failed = map(int, m.groups()) + self.assertEqual(failed, 0) + self.assertGreater(skipped, 0) + self.assertEqual(ran, 0) # no fixture test body ran + self.assertIn("needs Linux", r.stdout) + + +class LauncherWithoutDisplay(Fixture): + """test_launcher runs its programs directly with Wine, as before xvfb-run was added: a broken xvfb-run + (here: xauth missing) must not stop them.""" + + def test_launcher_programs_run_directly_when_xvfb_run_fails(self): + self.use_xvfb() + os.environ["FAKE_XVFB_FAIL"] = "1" + os.environ["FAKE_MODE"] = "summary" + tools = self.dir / "tools" + script(tools / "x86_64-w64-mingw32-g++", FAKE_CXX) + script(tools / "x86_64-w64-mingw32-windres", "#!/bin/sh\nexit 0\n") + for t in ("cmake", "ninja"): + script(tools / t, "#!/bin/sh\nexit 0\n") + os.environ["MINGW_CXX"] = str(tools / "x86_64-w64-mingw32-g++") + os.environ["PATH"] = f"{tools}{os.pathsep}{os.environ['PATH']}" + work = self.dir / "work" + (work / "launcher-build").mkdir(parents=True) + real_run = subprocess.run + + def fake_run(argv, *a, **k): # the installer's --prepare-only and the cmake steps: nothing to build here + if argv[0] == sys.executable or Path(argv[0]).name in ("cmake", "ninja"): + return subprocess.CompletedProcess(argv, 0, "", "") + return real_run(argv, *a, **k) + + env = self.env_for() + a = mock.Mock(runtime=None) + with mock.patch.object(rwt.subprocess, "run", fake_run): + status, why = rwt.test_launcher(a, env, work) + self.assertEqual(status, "PASS", why) + self.assertFalse(self.calls("xvfb-run"), "the launcher programs went through xvfb-run") + self.assertEqual(len([l for l in self.calls("wine ") if ".exe" in l or "launcher-build" in l]), 4) + + +if __name__ == "__main__": + unittest.main() diff --git a/clean/linux/wine_env.py b/clean/linux/wine_env.py index 9869f695..26f22cf9 100644 --- a/clean/linux/wine_env.py +++ b/clean/linux/wine_env.py @@ -48,15 +48,42 @@ def proton_dlls(proton_dir): return found -def prepare(proton_dir, prefix_dir, vkd3d_dir=None, log=print): - """Create (once) the prefix with Proton's wine, copy the DLLs into its system32 (every call, so a changed vkd3d_dir - takes effect) and return the environment: WINEPREFIX, WINEDLLOVERRIDES, WINEDEBUG, PATH with Proton's bin first, - and WG_WINE (Proton's wine executable).""" +def resolve_wine(wine): + """Absolute path of a wine executable given as a path or a name on PATH (children run from other folders).""" + found = shutil.which(str(wine)) + if not found: + raise SystemExit(f"--wine {wine}: not an executable") + return Path(os.path.abspath(found)) + + +def find_wineserver(wine): + """The wineserver that goes with `wine`: beside what it links to (the real install), else beside the link, else the + one on PATH. A system Wine can be a link in a user bin folder whose neighbour is some other wineserver.""" + for folder in (Path(os.path.realpath(wine)).parent, wine.parent): + cand = folder / "wineserver" + if cand.is_file() and os.access(cand, os.X_OK): + return cand + found = shutil.which("wineserver") + if found: + return Path(os.path.abspath(found)) + raise SystemExit(f"no wineserver found beside {wine} or on PATH; a wine needs its wineserver") + + +def prepare(proton_dir, prefix_dir, vkd3d_dir=None, log=print, wine=None): + """Create (once) the prefix with the selected wine, copy the Proton install's DLLs into its system32 (every call, so + a changed vkd3d_dir takes effect) and return the environment: WINEPREFIX, WINEDLLOVERRIDES, WINEDEBUG, PATH with the + selected wine's folder first, WG_WINE (its absolute path) and WG_WINESERVER (its wineserver, also given to wine as + WINESERVER; the runner's clean-up of a hung run uses it). + + wine: the wine executable (a system Wine, say) to create and run the prefix with. Default: Proton's own + (/files/bin/wine). Proton's own wine can hang headless (see run_wine_tests.py's probe).""" proton_dir = Path(proton_dir).resolve() prefix_dir = Path(prefix_dir).resolve() - wine = proton_dir / "files" / "bin" / "wine" + own = wine is None + wine = proton_dir / "files" / "bin" / "wine" if own else resolve_wine(wine) if not wine.is_file(): - raise SystemExit(f"{proton_dir}: no files/bin/wine; is this a Proton install?") + raise SystemExit(f"{wine}: no wine here; is {proton_dir} a Proton install?") + wineserver = find_wineserver(wine) dlls = proton_dlls(proton_dir) if vkd3d_dir: for name in ("d3d12.dll", "d3d12core.dll"): @@ -66,13 +93,14 @@ def prepare(proton_dir, prefix_dir, vkd3d_dir=None, log=print): dlls[name] = p env = dict(os.environ) env.update(WINEPREFIX=str(prefix_dir), WINEDLLOVERRIDES=DLL_OVERRIDES, WINEDEBUG="-all", - PATH=f"{wine.parent}{os.pathsep}{env.get('PATH', '')}", WG_WINE=str(wine)) + PATH=f"{wine.parent}{os.pathsep}{env.get('PATH', '')}", WG_WINE=str(wine), + WG_WINESERVER=str(wineserver), WINESERVER=str(wineserver)) if not (prefix_dir / "system.reg").is_file(): - log(f"creating Wine prefix {prefix_dir} with {proton_dir.name}") + log(f"creating Wine prefix {prefix_dir} with {proton_dir.name if own else wine}") prefix_dir.mkdir(parents=True, exist_ok=True) subprocess.run([str(wine), "wineboot", "-i"], env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True) - subprocess.run([str(wine.parent / "wineserver"), "-w"], env=env, check=True) + subprocess.run([str(wineserver), "-w"], env=env, check=True) system32 = prefix_dir / "drive_c" / "windows" / "system32" for name, src in dlls.items(): shutil.copyfile(src, system32 / name)