Commit 79de1f5
committed
model : fix k-pool indexer softmax gridDim.y overflow
Reshape the k-pool gate logits before softmax so n_new does not map to
gridDim.y, which is capped at 65535 on CUDA. Fixes prompt processing
abort at n_kv >= 262144 reported in ggml-org#27773.
Assisted-by: Cursor1 parent de25343 commit 79de1f5
1 file changed
Lines changed: 2 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
617 | 617 | | |
618 | 618 | | |
619 | 619 | | |
| 620 | + | |
620 | 621 | | |
| 622 | + | |
621 | 623 | | |
622 | 624 | | |
623 | 625 | | |
| |||
0 commit comments