glm5next: avoid soft_max gridDim.y overflow in the indexer - #214
Open
AIalliAI wants to merge 2 commits into
Open
glm5next: avoid soft_max gridDim.y overflow in the indexer#214AIalliAI wants to merge 2 commits into
AIalliAI wants to merge 2 commits into
Conversation
Assisted-by: Cursor
Assisted-by: Cursor
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
This was referenced Sep 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Reshape k-pool indexer gate logits before
ggml_soft_maxson_new_maxdoes not map togridDim.y, which is capped at 65535 on CUDA. During k-pool rebuild at long context (n_kv >= 262144,kpool = 4),n_new_maxreaches 65538 and the launch aborts withSOFT_MAX failed.Fold
n_new_maxandn_streamintone1viaggml_reshape_2d, then reshape back to 4D for downstreammul/sum_rows. Same pattern asqwen4exp.cpp.Stacked on ggml-org#27754.
Additional information
Related: ggml-org#27773, timkhronos#11, ggml-org#27901
Requirements