Skip to content

docs: document which CUDA extension build a run loads - #316

Merged
lvyufeng merged 1 commit into
masterfrom
docs/v41-cuda-extension-builds
Sep 21, 2026
Merged

lvyufeng merged 1 commit into
masterfrom
docs/v41-cuda-extension-builds

Conversation

@lvyufeng

Copy link
Copy Markdown
Owner

Summary

Adds docs/guides/cuda_extension_builds.md, indexed from docs/guides/index.md and pinned in mkdocs.yml's nav.

cuda_kernel is resolved at runtime by cpython tag out of build/extensions/ and then the repository root, and every failure mode in that resolution is silent: load_cuda_kernel() returns None rather than raising, --expert-device falls back to the host experts with one line on progress, and a missing op surfaces only when a call site reaches it. A full run of the wrong configuration therefore looks like a run.

What the page records

  • The resolution order, read from src/kernels/cuda_loader.py:13-26: two directories, and inside each the bare cuda_kernel.so followed by the names in the running interpreter's EXTENSION_SUFFIXES. The mtime glob at :25 is the only fallback and the only path that crosses cpython tags. In this tree the root holds cpython-310 (2026-06-15) and cpython-311 (2026-09-20) builds of the same name while build/extensions/ holds only the 3.11 one, so a base-environment run falls through to the June build without a word.
  • The two wrong-build failures and why they need different checks: the wrong tag is a missing op, a stale tree .so is the right op answering for the previous revision. Three copy-pasteable checks: the resolved path, md5 against the sources, and a symbol probe of the op the run needs — the last being the only one that sees a stale build.
  • That src/csrc/ is not the compilation unit. setup.py:205-215 lists eight sources for cuda_kernel; three files in the directory are outside that list. dot_microbench.cpp is deliberate (main() at :172 would collide with the extension's init, and scripts/run_best_scheduler.sh:94 already keeps it as a research artefact with the reason). minimax_gqa_kernel.cu and fused_decode_gqa_attention.cu are not compiled and not referenced from the build either: neither is bound in cuda_kernel.cpp, and the current 3.11 extension exports 51 bindings, none of them gqa_decode_qk_gemv, gqa_decode_attn_v_gemv or fused_decode_gqa_attention.
  • The test consequence. importorskip guards the extension load, not an individual op, so the four tests that call those names fail with AttributeError rather than skipping — a failure there is a missing source or a missing registration, not a missing GPU. All three files arrived in 51ab5ab (Replace environment variables with API-based configuration #108), which added the sources and the tests without touching setup.py.

Testing

mkdocs build --strict passes (7.12 s). No code changed. The four named tests were identified by binding list (dir() on the loaded extension) rather than executed: the cards are occupied by a 262144-token prefill probe.

🤖 Generated with Claude Code

`cuda_kernel` is resolved at runtime by cpython tag out of two directories,
and every failure mode in that resolution is silent: the loader returns None
rather than raising, the device path falls back to the host experts with one
line on `progress`, and a missing op surfaces only when a call site reaches
it. A full run of the wrong configuration therefore looks like a run.

The new guide records the resolution order from `cuda_loader.py`, the two
wrong-build failures it produces (the 3.10 build that predates the fp4 MoE
ops, and a tree `.so` left behind by a merge) and the three checks that
separate them -- resolved path, md5 against the sources, and a symbol probe
of the op the run needs, which is the only one that sees a stale build.

It also lists the three files in `src/csrc/` that `setup.py` does not
compile. `dot_microbench.cpp` is deliberately out and already documented as a
research artefact in `scripts/run_best_scheduler.sh:94`; `minimax_gqa_kernel.cu`
and `fused_decode_gqa_attention.cu` are not compiled and not referenced from
the build either, so the four tests that call them -- three in
`tests/test_minimax_gqa_kernel.py`, one in `tests/test_fused_decode_gqa_real.py`
-- fail with an AttributeError on a binding that no extension exports. All
three arrived in `51ab5ab` (#108), which added the sources and the tests
without touching `setup.py`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@lvyufeng
lvyufeng merged commit a972f12 into master Sep 21, 2026
@lvyufeng
lvyufeng deleted the docs/v41-cuda-extension-builds branch September 21, 2026 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant