fix(qwen36): the pre-init device-count probe must load the backend DLL (#1577) - #1579
Conversation
coli_cuda_available_device_count() is asked by qwen36_tier.c BEFORE coli_cuda_init -- the count decides which device indices are handed to init -- so gating the wrapper on g_cuda.available answered 0 on every Windows host and the tier silently fell back to the CPU path unless COLI_GPUS or COLI_GPU was set explicitly (JustVugg#1577). Load the DLL on demand, exactly as coli_cuda_attention_project_ragged already does, and keep the fallback for a DLL predating the export. The "no lock needed" note above coli_cuda_load() now names all three callers and records why the new one is still inside the single-threaded start-up window: the tier asks while it is still choosing the devices to hand to init. Tests: the generated stub backend answers the two device-count exports with distinct non-zero values (2 and 5) and can be built without the optional one, so LoaderPreInitAvailabilityTest can tell "the DLL answered" from "nothing answered" -- the difference the g_cuda.available gate hid. Four cases (backend present / absent / older DLL / HIP control); reverting the one-line change fails the two that need the DLL. Fixes JustVugg#1577
|
Real-hardware verification of the fix on a native Windows host (no WSL), following the Before the change the first line read Host: Windows 11, RTX 5060 Ti (sm_120), CUDA 12.9, VS 2022 MSVC, MinGW w64devkit 2.10.0, Unrelated to this PR: on a 32 GB host this container still dies at slot allocation |
|
Reviewed together with #1542 and #1580, full comparison in #1577 (comment). Short version: the load fix and the precedent you cite are right, and your test is the best of the three. The one thing to change is the fallback you kept: if(!g_cuda.available_device_count) return g_cuda.device_count(); /* a DLL from before the export */
Take that behaviour into this branch and I will merge it. |
Adapt DLL module-state probes and the HIP refusal control from ZhiyangK's PR JustVugg#1579. Preserve initialized-context semantics, verify missing-export diagnostics and explicit initialization, and distinguish a valid zero-device response. Clarify the three loader callers and their startup ordering.
Extend the discovery fixture with separate visible-device and initialized-context state. Cover CUDA/HIP discovery, explicit device forwarding, old DLL diagnostics, failure cleanup, runtime ownership, and repeated calls without changing the original direct-init tests. Exercise real qt_init and qt_init_fp8 selection paths with a stateful backend and add both suites to the mandatory Windows CI job. Adapt module-state and HIP control ideas from ZhiyangK's JustVugg#1579. Validation: 105 focused tests passed without skips; make check passed (Python: 913 passed, 100 skipped); seven regression mutations detected. On RTX 4070 Ti SUPER, nine CUDA test programs and sixteen real DLL/tier scenarios passed. Multi-GPU, AMD HIP hardware, and full-model inference remain unverified. Co-authored-by: ZhiyangK <ZhiyangK@users.noreply.github.com>
Summary
coli_cuda_available_device_count()is called byqwen36_tier.cbeforecoli_cuda_init()—the count decides which device indices are handed to init — but the wrapper gated on
g_cuda.available, a flag only init sets. On Windows that answered0on every host, so the tierfell back to the CPU path unless
COLI_GPUS/COLI_GPUwas set by hand (#1577).This is the fix #1577 proposed, plus the regression test that issue said it did not have.
Changes
c/backend_loader.ccoli_cuda_available_device_count()now callscoli_cuda_load()instead of readingg_cuda.available, exactly ascoli_cuda_attention_project_ragged()already does. Theoptional-export fallback to
device_count()is unchanged, so a DLL predating [Bug]: Windows CUDA_DLL build of qwen36 is broken in four independent ways (Makefile gating, missing symbol, missing marker string, cuda_binary() checks the wrong engine) #1533's exportstill loads and still answers.
coli_cuda_load()'s HIP branch named exactly twocallers; it now names all three and records why the new one stays inside the single-threaded
start-up window — the tier asks the count while it is still choosing the devices to hand to init,
before it creates any thread of its own.
c/tests/test_backend_loader.py(2 and 5) and can now be built without the optional export.
0is both a legitimate device countand what the
g_cuda.availablegate returned without loading anything, so a test that cannotseparate those two cases cannot see this bug.
probe, then still runs init.
LoaderPreInitAvailabilityTest, four cases:COLI_GPUS→ the count comes from the DLL (2), the harnessobserves the DLL being loaded by the probe (
0 → 2 → 1), and init still succeeds after;0devices, nothing loaded, no crash, init still reached;oldvariantfixture) → falls back todevice_count()(5);COLI_HIP_RUNTIME_DIRbefore init and leaves thebackend unmapped.
Test plan
Reverting the one-line change in
coli_cuda_available_device_count()makes cases 1 and 3 fail(
0instead of2/5) — which is what these tests exist for.The build path from #1533 is exercised too:
— the tier is compiled in by the Makefile alone (no variable overrides) and links against the
patched loader with no missing symbol.
Environment: Windows 11, MinGW w64devkit 2.10.0, Python 3.11. The fixture skips honestly where
gcc/objdumpare absent, and loads no real vendor runtime.Fixes #1577