Skip to content

tests: link $(VK_OBJ) into the rest of the engine-including rules - #1763

Open
Kenneth-Javier wants to merge 4 commits into
JustVugg:devfrom
Kenneth-Javier:build/vk-obj-engine-tests
Open

Kenneth-Javier wants to merge 4 commits into
JustVugg:devfrom
Kenneth-Javier:build/vk-obj-engine-tests

Conversation

@Kenneth-Javier

@Kenneth-Javier Kenneth-Javier commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #1728. Under VK=1, 13 rules on current dev (4e28e39) still compile a source with COLI_VULKAN hooks without linking backend_vulkan.o, so they stop at link on the coli_vk_* symbols (26 distinct on Linux).

Fix. Each rule lists $(VK_OBJ) as a prerequisite and links it right after $< (after backend_xdna.c in the three XDNA rules), the #1728 pattern. $(VK_OBJ) is empty unless VK=1, so default builds are unchanged.

Guard. c/tests/test_makefile_vk_obj.py keeps this from drifting again; it is what found the nine. Whether a link needs the backend is a preprocessor question, not a textual one: code can pick CUDA over Vulkan with #elif, a hook can be an #ifdef that only reads an environment variable, and a test can define the coli_vk_* functions itself to fake the device (all three occur in #1338). So the test reads no #if lines. It asks the tools:

  • make -Bn VK=1 gives every expanded link line (recipe continuations folded), including rules that write a differently named file;
  • each .c on a line is preprocessed with that line's own flags (link-only and -MMD/-MF arguments dropped);
  • the line needs $(VK_OBJ) when this tree's surviving code, outside backend_vulkan.h, calls a function backend_vulkan.h declares and does not define it; the rule must then link it and list it as a prerequisite;
  • the engines whose own source calls the backend anchor a second test, so a broken dry run or preprocess cannot pass silently.

Nothing is hand-listed: the rules come from the Makefile, the API from backend_vulkan.h. A line the host cannot preprocess (bench_idot without ARCH=native on Windows) is left to the platforms that can build it, and .build-config is restored afterwards so the dry run does not make the next real build relink. It needs make (GNU Make 3.81 is enough: $(EXE) comes from make -pn, not --eval) and the Makefile's $(CC), and takes about 20 s. A first, text-only version (the first commit) was exact on dev but flagged 25 rules with #1338 merged where the linker fails one; the second commit replaces it, and the third makes it run under the macOS runner's make 3.81 and read multi-line recipes, so all 204 link lines are covered.

Against the linker (make -k test-c VK=1 plus the on-demand targets):

Validation

Windows 11, MSYS2 UCRT64 gcc 16.1.0, make 4.4.1, vulkan-loader 1.4.357.0, shaderc 2026.3. On dev, no VK=1 link succeeds on Windows at all (-lvulkan under -static, fixed separately in #1762), so these runs have that fix applied:

  • make -k test-c VK=1: exit 0; all 170 TEST_BINS build and pass (4 link failures without this change).
  • The nine on-demand targets link and import vulkan-1.dll (bench_idot with ARCH=native, which its #error requires on any build).
  • make check (default build, this change alone on dev, at the second commit): exit 0; 170 C test binaries, 1571 Python tests OK (192 skipped), the new test included.

Windows CUDA_DLL=1 and HIP_DLL=1 hosts (no SDK needed for the host build), make -k test-c plus the nine on-demand targets, VK arms on top of the link fix:

  • Without VK=1 the link lines of dev and this change are identical (176 of 176, both flavours).
  • With VK=1, on dev 12 of the 13 rules have coli_vk_* among their undefined symbols (300 in total); with this change none do, and the failing set is exactly the one without VK=1: 52 targets that do not link backend_loader.o (undefined coli_cuda_* / ink_cuda_*), plus bench_idot without ARCH=native. Pre-existing, the same shape as the Linux CUDA=1 gap below.

WSL Ubuntu 24.04, gcc 13.3.0, make 4.3, libvulkan-dev 1.3.275.0, glslc 2023.8:

  • make -k test-c VK=1: exit 0; all 170 TEST_BINS build and pass. test_xdna_qt_state now contains the 31 coli_vk_* definitions.
  • The eight portable on-demand targets link. xdna_physical_probe is a Windows tool; on Linux only GetCurrentProcessId stays undefined.
  • The new test fails on dev's Makefile with the 13 rules and passes with this change (about 20 s).

Same WSL box, CUDA 13.4.92 + cuBLAS 13.8.0.4, CUDA_ARCH=portable, build and link only (no NVIDIA GPU), make -k test-c plus the nine on-demand targets, dev against this change:

  • CUDA=1: both fail on the same 72 of 175 targets, and the link lines of all 172 linking targets are identical token for token. With VK=1 off, the change does nothing.
  • CUDA=1 VK=1: on dev, 12 of the 13 rules have coli_vk_* among their undefined symbols (300 in total; the 13th, test_xdna_failure, is Windows-only). With this change none do, and the failing set is exactly the CUDA=1 one.

Those 72 are pre-existing and not about Vulkan, so this PR leaves them alone: 49 stop on undefined coli_cuda_* (engine-including rules without $(CUDA_OBJ), the same shape as this PR, and all 13 rules here are among them); 19 kimi/olmoe/qwen38/dsv41 tests hit cannot find -lcublas because NOCUDA_LDFLAGS filters out -L$(CUDA_HOME)/lib64 but keeps -lcublasLt -lcublas; and 4 V4 targets stop at backend_cuda_dsv4.o, which needs cuda_profiler_api.h, absent from this toolkit install.

CI on this PR, macOS runner: the new test runs and passes under Apple clang and the runner's make, which is GNU Make 3.81. The second commit's version errored there on make --eval (3.82+), which is what the third commit fixes. The suite went from 1479 run with that one error to 1481 run, OK, with the same 156 skipped, so the two new tests ran rather than skipped. All 29 checks are green.

Not verified: a VK=1 build on macOS / MoltenVK, where #1728 was measured, and METAL=1; anything on an NVIDIA GPU; running the benches or the XDNA probe (built only).

  • make -C c check
  • CUDA changes were tested with make -C c cuda-test (not applicable)
  • Performance claims include hardware, commands, and repeatable measurements (no performance claim)
  • Performance claims include a validated experiment manifest with raw evidence (no performance claim)

Compatibility

  • The default CPU build remains dependency-free ($(VK_OBJ) is empty unless VK=1)
  • No model files, generated binaries, or benchmark artifacts are included

Overlap with #1758. #1758 rewrites these same prerequisite lines, so whichever lands second gets a mechanical conflict: 10 hunks, prerequisite lines only. The resolution is to keep #1758's line and append $(VK_OBJ) (in the XDNA rules, after tests/backend_xdna_lane.o). I tried it on a throwaway merge: this test and #1758's test_makefile_deps both pass, and the new test runs unchanged on #1758's Makefile. I'll resolve it on whichever side lands second. With -MMD -MP in CFLAGS the new test writes no .d file.

Overlap with #1338. It merges cleanly with this PR, and together the new test reports tests/test_qwen36_slot_int8: that rule compiles qwen36_tier.c under VK=1 without $(VK_OBJ) and does fail to link. I'll note it on #1338; whichever lands second will need that one line.

Under VK=1, CFLAGS carries -DCOLI_VULKAN, so a translation unit that
includes colibri.c calls coli_vk_* and does not link without
backend_vulkan.o. JustVugg#1728 fixed the 43 rules that `make -k test-c VK=1`
reported at the time. Thirteen more still failed on dev 4e28e39. On
Linux twelve stop on the same 26 undefined coli_vk_* symbols; the
thirteenth, test_xdna_failure, is Windows-only and fails the same way
there:

- four gates that arrived in pull requests merged after JustVugg#1728:
  test_xdna_qt_state and test_xdna_failure (JustVugg#1261), test_logprob_status
  and test_ablate_mode (JustVugg#1356);
- nine on-demand rules that test-c never builds, so no such run could
  report them: xdna_physical_probe, test_e8x4g64_loader, and the benches
  bench_topp, bench_dsa_select, bench_router_select, bench_idot,
  bench_i4p_gidot, bench_gemv_stream and bench_mla_simd.

Each now lists $(VK_OBJ) as a prerequisite and links it after $< (after
backend_xdna.c for the three XDNA rules), the JustVugg#1728 pattern. $(VK_OBJ)
is empty unless VK=1, so default builds are unchanged.

tests/test_makefile_vk_obj.py keeps it that way without a hand-kept
list. It derives the hooked sources from their preprocessor conditionals
on COLI_VULKAN (colibri.c, glm53.c, kimi_k3.c, and telemetry.h through
colibri.c), follows quoted #includes from every .c a $(CC) link recipe
compiles ($<, $^, literal files, and variables such as QWEN36_TIER_SRC),
and requires $(VK_OBJ) both as a prerequisite and on the link line. -c
compiles are not links, and flag variables whose definition filters
-DCOLI_VULKAN out (SEGMENT_CPU_CFLAGS) are recognised from that
definition. A second test fails if the scan stops finding the engines'
own rules. It needs no compiler and no make. Against the previous
Makefile it fails with exactly the thirteen rules above; with this
change the 60 rules it says need $(VK_OBJ) are exactly the 60 that link
it.

Measured, Windows 11, gcc 16.1.0 (UCRT64), make 4.4.1, with the
Windows Vulkan link fix (build/windows-vulkan-link) applied:
- make -k test-c VK=1: exit 0; all 170 TEST_BINS build and pass, no
  undefined reference (four failed to link before).
- The nine on-demand targets link and import vulkan-1.dll; bench_idot
  with ARCH=native, which its own #error requires on any build.
- make check (default build, this change alone on dev): exit 0; 170 C
  test binaries with no failure, 1571 Python tests OK (192 skipped).

Measured, WSL Ubuntu 24.04, gcc 13.3.0, make 4.3, libvulkan-dev
1.3.275.0, glslc 2023.8:
- The new test fails on the previous Makefile with exactly the thirteen
  rules, and passes with this change.
- make -k test-c VK=1: exit 0; all 170 TEST_BINS build and pass, no
  undefined reference; test_xdna_qt_state now defines 31 coli_vk_*
  symbols.
- The eight portable on-demand targets link. xdna_physical_probe, a
  Windows tool, is left with GetCurrentProcessId undefined and no
  coli_vk_* reference.

Not verified: macOS (MoltenVK), where JustVugg#1728 was measured; the
MinGW-hosted CUDA and METAL=1 variants of these rules; running the
benches or the XDNA probe, which were only built.
The first version of test_makefile_vk_obj.py read preprocessor
conditionals as text: a rule needed $(VK_OBJ) when a .c it compiled
reached a file with `#if ... COLI_VULKAN`. That is exact on dev, but
not in general. With JustVugg#1338 (the qwen36 Vulkan expert tier) merged on
top, it flagged 25 rules where the linker fails one:

- the qwen36 tier tests pass -DCOLI_CUDA, and qwen36_tier.c picks CUDA
  over Vulkan with #elif;
- qwen36.c's Vulkan hook only reads COLI_VULKAN from the environment;
- test_qwen36_tier_vk_fake defines the coli_vk_* functions itself.

Skipping lines that pass -DCOLI_CUDA is not a fix either: kimi_k3.c
keeps independent #ifdef COLI_VULKAN blocks, and test_kimi_cuda_expert
does need backend_vulkan.o.

The test now asks the tools. `make -Bn VK=1` gives each expanded link
line, and every .c on it is preprocessed with that line's own flags
(link-only and -MMD/-MF arguments dropped). A line needs $(VK_OBJ) when
the surviving code of this tree, outside backend_vulkan.h, calls a
function backend_vulkan.h declares and does not define it. String
literals are ignored. A line this host cannot preprocess is left to the
platforms that can build it. .build-config is restored afterwards, so
the dry run does not make the next real build relink. The engines whose
own source calls the backend anchor the non-vacuity check.

Measured against the linker (make -k test-c VK=1 plus the on-demand
targets):
- dev 4e28e39: flags exactly the thirteen rules the previous commit
  fixed on Linux; on Windows twelve, since bench_idot cannot even
  preprocess there without ARCH=native (its #error).
- dev + the previous commit: passes. On Linux the 60 lines it says call
  the backend are exactly the 60 that link backend_vulkan.o.
- dev + JustVugg#1338 + this branch: flags only test_qwen36_slot_int8, which is
  also the only link the linker fails there.
- dev + JustVugg#1758 + this branch, conflicts resolved: passes together with
  JustVugg#1758's test_makefile_deps; with -MMD -MP in CFLAGS no .d file is
  written.

Runtime: 15 s on Windows (gcc 16.1.0, 32 threads), 20 s on WSL Ubuntu
24.04 (gcc 13.3.0). The first version took 1.5 s.

Not verified: macOS / Apple clang, where the linemarker format is the
same but the test was not run.
The VK_OBJ test errored in setUpClass on the macOS runner: it read
$(CC) and $(EXE) through `make --eval`, and /usr/bin/make on macOS is
GNU Make 3.81, which predates --eval (3.82). Its stdout was empty; the
rest of the suite passed (1479 run, 156 skipped, this one error).

$(EXE) now comes from make's own database (`make -pn`), which 3.81
prints the same way, and the compiler is simply the first word of each
link line.

Two holes in reading `make -n` go with it:
- recipe continuations are printed as written, so the five link
  commands that span lines (test_deepseek_v4 and the four
  segment/edge adapter tests) were never checked. They are folded
  first now.
- a rule that writes a differently named file (fuzz-rans,
  bench-omp-grain, the dsv4 CUDA tests, glm53-metal-check) was skipped,
  because its output was not a rule name. Every printed command that
  compiles a .c into an output without -c is checked now, and such a
  rule is reported by its output name if it ever needs $(VK_OBJ).

The scan covers 204 link lines on dev plus this branch (198 before).
None of the added ones calls the backend, so the results against the
linker are unchanged:
- Linux (WSL Ubuntu 24.04, make 4.3): dev fails with the thirteen
  rules; this branch passes, and the 60 lines that call the backend are
  the 60 that link backend_vulkan.o; with JustVugg#1338 merged only
  test_qwen36_slot_int8 is reported. About 22 s.
- Windows (make 4.4.1): the same, with twelve on dev, since bench_idot
  cannot preprocess without ARCH=native. About 20 s.
- dev + JustVugg#1758 + this branch, conflicts resolved: passes together with
  JustVugg#1758's test_makefile_deps and writes no .d file.

Not verified locally: GNU Make 3.81 itself. The macOS runner is the
check.
JustVugg#1775 added tok_unicode_deepseek.h to the prerequisites of every rule
that includes colibri.c, twelve of which are rules this branch adds
$(VK_OBJ) to. Resolved by taking dev's prerequisite lines and appending
$(VK_OBJ) again. The recipes are this branch's.

Against dev, c/Makefile differs in exactly the 26 lines of this branch
(13 rules, prerequisite and link line each), and each differs from
dev's line only by one inserted $(VK_OBJ).

Checked on the merged tree: test_makefile_vk_obj, test_makefile_deps
and test_makefile_platform pass on Windows (8 tests); the VK_OBJ test
sees 205 link lines, dev's new test_tok_deepseek included.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant