fix(coli): --gpu on DeepSeek V4 refused an install with only the DeepGEMM DLL - #1586
Merged
Merged
Conversation
…GEMM DLL dsv4_cuda_available checked only coli_cuda_dsv4.dll next to the engine on Windows. backend_loader_dsv4.c loads coli_cuda_dsv4_dg.dll first and coli_cuda_dsv4.dll second, and doctor's cuda_linkage accepts either. So with only the DeepGEMM build installed, `coli doctor` reported the GPU engine as available while `coli chat/serve/run --gpu` and `--vram` exited with "--gpu needs the CUDA build". Accept either name, as the loader and doctor do. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
On Windows, a DeepSeek V4 install with
coli_cuda_dsv4_dg.dllnext todeepseek_v4.exebut nocoli_cuda_dsv4.dllgets two contradicting answers. That is the DeepGEMM builddocs/deepseek-v4.mdlists for RTX 50-series:--vramrefuses the same way. The engine itself would have loaded that DLL.Root cause
dsv4_cuda_availableinc/colitests onlycoli_cuda_dsv4.dllon win32. Two other places accept either DLL:backend_loader_dsv4.cdsv4_cuda_load, triescoli_cuda_dsv4_dg.dllfirst andcoli_cuda_dsv4.dllsecond. Either one starts the tier.doctor.cuda_linkageaccepts either name for a[DSV4 CUDA]engine.Fix
On win32,
dsv4_cuda_availableaccepts either of the two names the loader tries, the same pair doctor accepts. Linux and macOS are unchanged, and so is the build hint.Tests
In
c/tests/test_env_defaults.py(Dsv4CudaDetectTest):test_win32_deepgemm_dll_alone_detected: onlycoli_cuda_dsv4_dg.dllnext to the engine is detected.test_win32_unrelated_dll_rejected: a lonecoli_cuda.dll(the GLM backend) is still not accepted._detecttakes the DLL name as a parameter. Existing cases keep the old default.Fail-before on unfixed
dev:Verification
tests.test_cli_output,test_cuda_binary_engine,test_env_defaults,test_stop_scope,test_v4_cli(every module that exercises the launcher's GPU gating): 92 tests OK (1 skipped), in the cp949 locale and withPYTHONUTF8=1.deepseek_v4.exefrommake -C c deepseek-v4(gcc 16.2), the tiny checkpoint fromtools/make_deepseek_v4_tiny.py, and an emptycoli_cuda_dsv4_dg.dllnext to the engine:coli run --gpu 0 --ngen 4 --model v4_tiny "x"exits 1 with the refusal above.🤖 Generated with Claude Code