Skip to content

glm5next: avoid soft_max gridDim.y overflow in the indexer - #214

Open
AIalliAI wants to merge 2 commits into
unslothai:glm5next/upstreamfrom
AIalliAI:fix/glm5next-kpool-softmax-grid-y
Open

glm5next: avoid soft_max gridDim.y overflow in the indexer#214
AIalliAI wants to merge 2 commits into
unslothai:glm5next/upstreamfrom
AIalliAI:fix/glm5next-kpool-softmax-grid-y

Conversation

@AIalliAI

@AIalliAI AIalliAI commented Sep 11, 2026

Copy link
Copy Markdown

Overview

Reshape k-pool indexer gate logits before ggml_soft_max so n_new_max does not map to gridDim.y, which is capped at 65535 on CUDA. During k-pool rebuild at long context (n_kv >= 262144, kpool = 4), n_new_max reaches 65538 and the launch aborts with SOFT_MAX failed.

Fold n_new_max and n_stream into ne1 via ggml_reshape_2d, then reshape back to 4D for downstream mul / sum_rows. Same pattern as qwen4exp.cpp.

Stacked on ggml-org#27754.

Additional information

Related: ggml-org#27773, timkhronos#11, ggml-org#27901

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES - assisted implementation and review; I understand the change and can maintain it.

@AIalliAI
AIalliAI requested a review from CISC as a code owner September 11, 2026 02:23
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant