Skip to content

tests: skip the cuda-test recipe checks when make is absent - #1569

Merged
JustVugg merged 1 commit into
JustVugg:devfrom
SurefireStudios:fix/cuda-test-makefile-skip-without-make
Sep 17, 2026
Merged

JustVugg merged 1 commit into
JustVugg:devfrom
SurefireStudios:fix/cuda-test-makefile-skip-without-make

Conversation

@SurefireStudios

@SurefireStudios SurefireStudios commented Sep 16, 2026

Copy link
Copy Markdown

Summary

CudaTestMakefileTest shells out to make to dry-run the cuda-test recipe but never checks that make is there, so on a host without it the six recipe tests raise FileNotFoundError β€” an absent prerequisite dressed up as a regression.

Five test files reach for make; four guard it and this is the one that does not, so this applies test_makefile_platform.py's existing idiom: MAKE = shutil.which("make"), @unittest.skipUnless(MAKE, ...), and invoke by the resolved path.

test_setup_openmp_probe_does_not_require_tmp only reads setup.sh, so it moves to its own class rather than being skipped along with the rest.

Validation

  • make -C c check β€” via this repo's own check and CI workflows run unmodified on a fork: 28 jobs green, Linux/Windows UCRT64/macOS, sanitizers, musl, ARM and every tiny-oracle gate.
  • CUDA changes were tested with make -C c cuda-test β€” n/a, no CUDA code changed.
  • Performance claims include hardware, commands, and repeatable measurements β€” n/a, no performance claim.
  • Performance claims include a validated experiment manifest with raw evidence β€” n/a.

python -m unittest discover -s tests -p 'test_*.py' on Windows 11 / Python 3.12 with no make on PATH: before FAILED (errors=6), 819 ran / 132 skipped; after OK, 813 ran / 138 skipped.

The check that matters is the opposite one β€” a guard that silently disabled six build-contract assertions on CI would be worse than the errors it replaces β€” so I confirmed unmodified dev (b5968eb) and this branch both report 934 ran, 119 skipped in the Linux Python tests job.

Compatibility

  • The default CPU build remains dependency-free β€” test-only change; no engine or build code touched.
  • No model files, generated binaries, or benchmark artifacts are included.

A scan turned up two other bare-name invocations and I left both alone: test_mw_ur3_raw_adapter.py's make call is inside an already-@unittest.skipped test, and test_supervisor_scope.py's bash sits behind sys.platform.startswith("linux") and shutil.which("flock").

CudaTestMakefileTest shells out to make to dry-run the cuda-test recipe,
but unlike its siblings it never checks that make is there. Without it the
six recipe tests raise FileNotFoundError from CreateProcess/exec and the
file reports six errors -- an absent prerequisite dressed up as a
regression.

test_makefile_platform.py already has the shape: resolve make once with
shutil.which, invoke it by that path, and skipUnless on the result.
test_makefile_cuda_scope.py and test_backend_loader.py guard the same way.
This file is the only one of the five that reaches for make and does not.

test_setup_openmp_probe_does_not_require_tmp only reads setup.sh, so it
moves to its own class instead of being skipped along with the rest --
a class-level guard would have cost that coverage.

Running make through the resolved absolute path rather than the bare name
also matches the sibling, and avoids relying on PATHEXT resolution.

python -m unittest discover -s tests, on a host with no make:
  before   FAILED (errors=6)   819 ran, 132 skipped
  after    OK                  813 ran, 138 skipped
With make present every test still runs; nothing is skipped on CI.
Copilot AI lite review requested due to automatic review settings September 16, 2026 06:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@SurefireStudios

Copy link
Copy Markdown
Author

Fork CI results, as promised in the description. Both of this repo's workflows, unmodified, on SurefireStudios/colibri.

check β€” all green

success  Linux                                     success  Windows UCRT64
success  macOS (colibri + V4 platform gate)        success  Linux (generated GLM-5.3 oracle contract)
success  Linux (generated tiny K3 vendor oracle)   success  Linux (generated tiny V4 target oracle)

CI β€” all 22 jobs green, including Engine (Linux, CPU), All engines (linux/macos/windows), Sanitizers (ASan + UBSan), musl libc (Alpine), ARM (engines + NEON kernel exactness + tiny oracle), every tiny-oracle gate, Python tests and Python tests (Windows focused).

The check that actually matters. A guard that quietly disabled six build-contract assertions on CI would be worse than the errors it replaces, so I ran the baseline too β€” the same CI workflow on unmodified dev (b5968eb), same runner image, and compared the Linux Python tests job:

Ran skipped
baseline, unmodified dev 934 119
this branch 934 119

Identical. Where make exists, all six tests still execute β€” the skip only engages on a host that cannot run them at all.

@JustVugg
JustVugg merged commit dfa6182 into JustVugg:dev Sep 17, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants