Skip to content

build(vk): link the Vulkan loader on Windows under -static - #1762

Open
Kenneth-Javier wants to merge 1 commit into
JustVugg:devfrom
Kenneth-Javier:build/windows-vulkan-link
Open

Kenneth-Javier wants to merge 1 commit into
JustVugg:devfrom
Kenneth-Javier:build/windows-vulkan-link

Conversation

@Kenneth-Javier

Copy link
Copy Markdown
Contributor

Summary

make colibri.exe VK=1 fails at link time under MSYS2 UCRT64 on current dev (4e28e39):

ld.exe: cannot find -lvulkan: No such file or directory
ld.exe: have you installed the static version of the vulkan library ?

Windows has no libvulkan. The loader is vulkan-1.dll, installed by the GPU driver, and MSYS2's mingw-w64-ucrt-x86_64-vulkan-loader ships only its import library, libvulkan-1.dll.a. The Windows LDFLAGS carry -static, and under it ld does not consider .dll.a import libraries for -l at all, so -lvulkan-1 fails the same way (and -Wl,-Bdynamic -lvulkan finds nothing: the library is named vulkan-1). CI builds VK=1 only on Linux (the vulkan job), which is why nothing reported it.

Fix. On Windows, VK=1 appends -Wl,-Bdynamic -lvulkan-1 -Wl,-Bstatic: dynamic lookup for the loader alone, then back to static, so libgomp, winpthreads and libgcc stay linked in as before. Linux keeps -lvulkan. backend_vulkan.o and the SPIR-V rules are unchanged. Resolving the import library's path with $(CC) -print-file-name=libvulkan-1.dll.a also links, but it puts an absolute toolchain path into LDFLAGS and .build-config and costs a parse-time shell call.

Test. tests/test_makefile_platform.py pins the tail of the VK=1 link line from a dry run, for x86_64-w64-mingw32 and x86_64-unknown-linux-gnu, so it runs on any host with make. Against the previous Makefile the mingw case fails ('-lpsapi', '-lvulkan') and the Linux case passes.

Docs. docs/vulkan.md only showed Linux. It gains a short Windows section: the three MSYS2 packages (vulkan-headers, vulkan-loader, and shaderc for glslc), the one added import, the fact that on Windows the shaders are found from the current directory or COLI_VK_SHADERS (the next-to-the-binary lookup in vk_resolve_spv is __linux__ only), and the CI's config.json init probe as a PowerShell snippet, for checking a driver before downloading a model.

Validation

Windows 11, MSYS2 UCRT64 gcc 16.1.0, make 4.4.1, vulkan-loader 1.4.357.0, shaderc 2026.3, Radeon 8060S:

  • make colibri.exe VK=1 links, with 0 compiler warnings. Its import table differs from the default build's by exactly one entry, vulkan-1.dll (13 vs 12).
  • The docs snippet, run from c\ with PATH set to System32 and Windows only, prints [VK] ready: AMD Radeon(TM) 8060S Graphics and expert tier active, in pwsh 7.6 and in Windows PowerShell 5.1. From another directory it reports cannot open shaders/qmatmul.spv; with COLI_VK_SHADERS set it initialises again.
  • Without VK=1 nothing changes: the default colibri.exe command and all 256 test-c commands from make -Bn are identical to dev's, so the default binary and the oracle are unaffected.
  • make check (default build): exit 0; 170 C test binaries with no failure, 1570 Python tests OK (192 skipped).
  • make -k test-c VK=1: every link finds the loader, and 166 of 170 test binaries build and pass. The other four (test_xdna_qt_state, test_xdna_failure, test_logprob_status, test_ablate_mode) include colibri.c without linking $(VK_OBJ) and stop on undefined coli_vk_*. That gap is separate and exists on Linux too; I'll send it as a follow-up to tests: link backend_vulkan.o into the engine-including tests under VK=1 #1728. With that follow-up on top, all 170 build and pass.
  • With CUDA_DLL=1 or HIP_DLL=1 added: all 172 VK=1 link lines carry -lvulkan-1 and none fails to find it. The 52 targets that fail also fail without VK=1: they do not link backend_loader.o.

WSL Ubuntu 24.04, gcc 13.3.0, make 4.3, libvulkan-dev 1.3.275.0, glslc 2023.8, Mesa 25.2.8:

  • The VK=1 compile-and-link command from make -Bn is byte-identical to the previous Makefile's, so the Linux build and its diagnostics are unchanged.
  • make colibri VK=1 links, ldd shows libvulkan.so.1, and the ci.yml Lavapipe probe prints [VK] ready: llvmpipe and expert tier active.
  • The platform tests pass (4/4).

Not verified: the MINGW64 and CLANG64 environments, the LunarG SDK's vulkan-1.lib, Cygwin, cross-compiling with a mingw gcc from Linux, any GPU other than the Radeon 8060S and Lavapipe, and a Windows VK=1 run on real weights (the probe stops after the backend initialises). This PR adds no Windows VK=1 CI job.

  • make -C c check
  • CUDA changes were tested with make -C c cuda-test (not applicable)
  • Performance claims include hardware, commands, and repeatable measurements (no performance claim)
  • Performance claims include a validated experiment manifest with raw evidence (no performance claim)

Compatibility

  • The default CPU build remains dependency-free (only VK=1 links change, and only on Windows)
  • No model files, generated binaries, or benchmark artifacts are included

Checked locally: this merges cleanly with dev, with #1758, and with #1338, which also edits docs/vulkan.md.

`make colibri.exe VK=1` failed at link time under MSYS2 UCRT64 with
"cannot find -lvulkan". Windows has no libvulkan: the loader is
vulkan-1.dll, installed by the GPU driver, and MSYS2's vulkan-loader
package ships only its import library, libvulkan-1.dll.a. The Windows
LDFLAGS carry -static, and under it ld does not consider .dll.a import
libraries for -l at all, so -lvulkan-1 failed the same way.

On Windows, VK=1 now appends -Wl,-Bdynamic -lvulkan-1 -Wl,-Bstatic:
dynamic lookup for the loader alone, then back to static, so libgomp,
winpthreads and libgcc stay linked in as before. Linux keeps -lvulkan.
backend_vulkan.o and the SPIR-V rules are unchanged.

Resolving the import library's path with $(CC) -print-file-name also
links, but it puts an absolute toolchain path into LDFLAGS and
.build-config and costs a parse-time shell call; the -Bdynamic window
needs neither.

tests/test_makefile_platform.py pins the tail of the VK=1 link line
from a dry run for x86_64-w64-mingw32 and x86_64-unknown-linux-gnu, so
it runs on any host with make. Against the previous Makefile its mingw
case fails ('-lpsapi', '-lvulkan') and its Linux case passes.

docs/vulkan.md gains a Windows section: the three MSYS2 packages, the
one added import, the CWD-relative shader lookup (vk_resolve_spv's
next-to-the-binary lookup is __linux__ only), and the CI's config.json
init probe as a PowerShell snippet.

Measured on Windows 11, gcc 16.1.0 (UCRT64), make 4.4.1,
vulkan-loader 1.4.357.0, shaderc 2026.3, Radeon 8060S:
- make colibri.exe VK=1 links. Its import table differs from the
  default build's by exactly one entry, vulkan-1.dll (13 vs 12).
- The docs snippet, run from c\ with PATH set to System32 and Windows
  only, prints "[VK] ready: AMD Radeon(TM) 8060S Graphics" and "expert
  tier active" in pwsh 7.6 and in Windows PowerShell 5.1. From another
  directory it reports "cannot open shaders/qmatmul.spv"; with
  COLI_VK_SHADERS set it initialises again.
- make check (default build): exit 0; 170 C test binaries with no
  failure, 1570 Python tests OK (192 skipped).
- make -k test-c VK=1: every link resolved vulkan-1; 166 of 170 test
  binaries build and pass. The other four (test_xdna_qt_state,
  test_xdna_failure, test_logprob_status, test_ablate_mode) include
  colibri.c under -DCOLI_VULKAN without linking $(VK_OBJ) and stop on
  undefined coli_vk_* symbols. test_xdna_qt_state fails the same way
  with the previous Makefile on Linux; that gap is left to a separate
  change.

Measured on WSL Ubuntu 24.04, gcc 13.3.0, make 4.3, libvulkan-dev
1.3.275.0, glslc 2023.8, Mesa 25.2.8: the VK=1 link line from make -Bn
is byte-identical to the previous Makefile's; make colibri VK=1 links,
ldd shows libvulkan.so.1, and the ci.yml Lavapipe probe prints
"[VK] ready: llvmpipe" and "expert tier active". The platform tests
pass (4/4).

Not verified: the MINGW64 and CLANG64 environments, the LunarG SDK's
vulkan-1.lib, Cygwin, cross-compiling with a mingw gcc from Linux, and
any GPU other than the Radeon 8060S and Lavapipe.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant