docs: document which CUDA extension build a run loads - #316
Merged
Merged
Conversation
`cuda_kernel` is resolved at runtime by cpython tag out of two directories, and every failure mode in that resolution is silent: the loader returns None rather than raising, the device path falls back to the host experts with one line on `progress`, and a missing op surfaces only when a call site reaches it. A full run of the wrong configuration therefore looks like a run. The new guide records the resolution order from `cuda_loader.py`, the two wrong-build failures it produces (the 3.10 build that predates the fp4 MoE ops, and a tree `.so` left behind by a merge) and the three checks that separate them -- resolved path, md5 against the sources, and a symbol probe of the op the run needs, which is the only one that sees a stale build. It also lists the three files in `src/csrc/` that `setup.py` does not compile. `dot_microbench.cpp` is deliberately out and already documented as a research artefact in `scripts/run_best_scheduler.sh:94`; `minimax_gqa_kernel.cu` and `fused_decode_gqa_attention.cu` are not compiled and not referenced from the build either, so the four tests that call them -- three in `tests/test_minimax_gqa_kernel.py`, one in `tests/test_fused_decode_gqa_real.py` -- fail with an AttributeError on a binding that no extension exports. All three arrived in `51ab5ab` (#108), which added the sources and the tests without touching `setup.py`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
docs/guides/cuda_extension_builds.md, indexed fromdocs/guides/index.mdand pinned inmkdocs.yml's nav.cuda_kernelis resolved at runtime by cpython tag out ofbuild/extensions/and then the repository root, and every failure mode in that resolution is silent:load_cuda_kernel()returnsNonerather than raising,--expert-devicefalls back to the host experts with one line onprogress, and a missing op surfaces only when a call site reaches it. A full run of the wrong configuration therefore looks like a run.What the page records
src/kernels/cuda_loader.py:13-26: two directories, and inside each the barecuda_kernel.sofollowed by the names in the running interpreter'sEXTENSION_SUFFIXES. The mtime glob at:25is the only fallback and the only path that crosses cpython tags. In this tree the root holdscpython-310(2026-06-15) andcpython-311(2026-09-20) builds of the same name whilebuild/extensions/holds only the 3.11 one, so a base-environment run falls through to the June build without a word..sois the right op answering for the previous revision. Three copy-pasteable checks: the resolved path, md5 against the sources, and a symbol probe of the op the run needs — the last being the only one that sees a stale build.src/csrc/is not the compilation unit.setup.py:205-215lists eight sources forcuda_kernel; three files in the directory are outside that list.dot_microbench.cppis deliberate (main()at:172would collide with the extension's init, andscripts/run_best_scheduler.sh:94already keeps it as a research artefact with the reason).minimax_gqa_kernel.cuandfused_decode_gqa_attention.cuare not compiled and not referenced from the build either: neither is bound incuda_kernel.cpp, and the current 3.11 extension exports 51 bindings, none of themgqa_decode_qk_gemv,gqa_decode_attn_v_gemvorfused_decode_gqa_attention.importorskipguards the extension load, not an individual op, so the four tests that call those names fail withAttributeErrorrather than skipping — a failure there is a missing source or a missing registration, not a missing GPU. All three files arrived in51ab5ab(Replace environment variables with API-based configuration #108), which added the sources and the tests without touchingsetup.py.Testing
mkdocs build --strictpasses (7.12 s). No code changed. The four named tests were identified by binding list (dir()on the loaded extension) rather than executed: the cards are occupied by a 262144-token prefill probe.🤖 Generated with Claude Code