add GLM-5.3-Flash (GLM5-Next) support

[timkhronos]

Overview

Add support for GLM -5.3-flash a 320B hybrid model, supporting both text and vision.

Additional information

Architecture

GLM 5.3 flash mixes 34 KDA linear layers with 11 DSA laters, with mHC and Deepseek style Moe. Most of the parts are already in llama.cpp so I reused whatever I could:

• KDA layers reuse the Kimi-K3 implementation
• Attention layers are nope only MLA
• For mHC I reused the Deepseek V4 implementation. I moved the build_hc helpers from the DSV4 graph into graph_context so both models can share them.
• Moe and swiglu clamping follow DSV4.

What I implemented new:

• Here, the DSA indexer scores pools of 4 consecutive token, and always keeps the incomplete tail. I implemented this on top of the existing DSA cache. The indexer cache stores key | gate per token and pooling happens in the graph. No new backend ops have been added.
• llama_memory_hybrid_dsa: recurrent state + DSA cache, cloned from hybrid ISWA. Rebased onto llama_memory_hybrid_idx instead of the earlier ISWA clone.
• Vision Tower: the encoder is the same family as glmv4 with per head qk-norm, clamped Swiglu and no post conv norm. It reuses glm4v projector with a swiglu_limit key and an optional image token budget. Added as glm5v as GLM 5.3 Flash requires a different pre processing method than what glm4v uses.
• Small, precision sensitive tensors (indexer, mHC mixers, KDA gates, MLA low rank paths, roughly 1GB total) are kept unquantized.

Tests

• Logits match transformers on a small random model across full prefill, small ubatches and single token decode while sparse selection is active, covering both scatter and gather.
• Vision embeddings match to ~1e-5.
• The converted model generates coherently and correctly, and vision is working as expected.

Limitations

• MTP is in a separate pr (Add MTP for GLM-5.3-Flash #27917), but NextN tensors are already preserved here.
• Multiple sequences require --kv-unified

Quantized GGUFs converted with this PR are available here.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES, AI was used in an assistive capacity, and helped figure out and solve several conversion issues, and helped validate the correctness of the implementation.

[danielhanchen]

Hey @timkhronos great work on the PR! A few requests if possible:

1. You're using glm5-next for general.architecture, whilst model: add GLM-5-Next (GLM-5.3-Flash) #27754 and model : add GLM-5.3-Flash (glm5next) #27752 uses glm5next - Qwen3-Next for eg does qwen3next - I'm unsure what the convention is @ngxson but adopting glm5next might be more generalized? If glm5-next is accepted, a simple first shard rewrite for our uploads should suffice.

2. The bigger issue is blk.N.indexer.kpool_ape / kpool_gate vs blk.N.indexer_compressor_ape / _gate. deepseek4 mainline already uses blk.N.indexer_compressor_ape and blk.N.indexer_compressor_gate but your PR changes it - the ones we uploaded uses deepseek4's convention. If this PR is accepted, we have to provide a script to rewrite all tensor names or folks have to re-download. If not, can you add aliases so the ones we published works - thanks in advance. Seems like it's more complex than I expected.

Tagging @ngxson for visibility as well.

I re-checked and if (1) + (2) is applied, the quants we uploaded work fine (+ the small shard-1 rewrite) and KLD / PPL are correct under this PR.

Seems like a simple alias isn't possible actually :( It breaks the quants made with this PR

[danielhanchen]

Hmmm https://github.com/timkhronos/llama.cpp/pull/9/changes would alias the tensors but it looks a bit problematic hmmm

[Sciguy429]

Throwing up some performance numbers here from the lower end of consumer hardware (128GB DDR5 + 24GB VRAM (4090)).

avar6 has some freshly converted imatrix quants from this PR up as of now if anyone else wants to give them a go: https://huggingface.co/avar6/GLM-5.3-Flash-BF16-gguf

For the IQ3_S, I am getting roughly 300t/s prefill at 256K context and 2048 b/ub size. Generation speed starts off at around 9t/s and drops down considerably by mid window (~128K) to around 6t/s. This seems to track with the 'pooled indexer keys' issue. The model is fully coherent and seems to be working fine. I don't have PPL/KL numbers at the moment as I still need to generate a logit dump.

I have noticed an interesting memory quirk, which I haven't seen before. This is the only model I have ever seen have inconsistent checkpoint sizes. As the context fills the checkpoints grow alarmingly fast in size. At ~90K they are already up to nearly 1.6GB. I don't know if this is an expected behavior for this model arch, or if this is a something which needs to be looked into.

Also, something of note for you @danielhanchen which I found last night while looking over the three PRs for this arch. The vision towers between this PR and yours differ as well. This PR reuses the name clip.vision.projector_type = "glm4v" while you built a new one clip.vision.projector_type = "glm5next". Likely not much of an issue given how easy it is to regenerate mmproj files, but it will need to be delt with as well.

[danielhanchen]

Yes I'll re-do the vision! This is fine!

@timkhronos I confirmed timkhronos#9 works fine and does not break your GGUFs. We will however need to do a cheap shard-1 update so that should be fine

[danielhanchen]

@timkhronos I saw you changed the tensor naming - but my solution I provided was to allow everyone's quants to work - now your own ones you uploaded don't work haha.

We still need to provide the shard rewrite for the naming (glm5-next) which we're fine with, but now the DeepSeek convention means you yourself have to reupload all shards or do a tensor rename inplace with a script - was this your intention?

[timkhronos]

@danielhanchen Hey!

I ended up going with the the indexer_compressor naming scheme, as it is closer to what's already there, and I was meaning to ask Avar to reconvert anyways, as his ggufs were made when we were missing quantization protection for some crucial tensors so they are not ideal.

Your vision projectors will need reconverting though most likely, and your main model ggufs might be missing the index_share_for_mtp_iteration key as well.

[danielhanchen]

@timkhronos Hey! I made some shard rewrites to https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF/tree/main/Shard_Rewrite for in preparation!

[nicholasshirley]

Vision works on 5c4bd50 with UD-Q4_K_XL, -fa on, ctx 524288, projector on CPU. 1500x500 logo -> 1003 prompt tokens, 2875x1500 diagram -> 5586 tokens, labels read correctly.

Note for anyone hitting "Failed to load CLIP model": the unsloth Shard_Rewrite mmproj predates the swiglu_clamp rename; avar6's mmproj has the current keys.

[LifesLight]

I input an 8MP image with --image-max-tokens 2048, still came out using 8000 tokens (mmproj max). editing image_max_pixels in the mmproj GGUF directly did cap it though, so the arg doesn't seem to be having any effect here.

[JPFlare]

I'm running into an issue involving a soft_max crash on commit 8134115, I had the model do some of its own investigation and it came up with the following, apologies if it's already fixed in the last few commits, or if bug reports investigated by AI are unwelcome, I'm not knowledgeable enough to chase it down myself:

Bug: llama.cpp server aborts ("SOFT_MAX failed: invalid argument") during prompt processing at n_kv ≈ 262,144 tokens (2^18). 100% reproducible, context-size independent.

Cause: The k-pool indexer pooling softmax (glm5-next.cpp:620) builds a [4, 128, n_pool] tensor. The CUDA softmax backend maps ne[2] to gridDim.y, and CUDA caps gridDim.y at 65535. During prompt processing ALL pools are re-pooled each graph (n_pool = n_kv / kpool = n_kv / 4), so at n_kv = 65536 × 4 = 262,144 the launch exceeds the limit and CUDA rejects it with cudaErrorInvalidValue. Not a softmax math bug, not a CUDA version issue.

Fix: 3-line reshape so the big dimension lands in gridDim.x (limit 2^31): ggml_reshape_2d(logits, kpool, 128*n_pool) before the softmax, reshape back after.

The symptom is a crash at 262k context consistently across all sessions, my current configuration is 3x NVIDIA GeForce RTX 5090 + 2x NVIDIA RTX PRO 6000 Blackwell Workstation, and I was originally chasing it down as a blackwell bug since a few results came up for softmax crashes on blackwell, but my setup doesn't match any of the conditions.

Brief log excerpt:

5.16.078.713 I slot print_timing: id 0 | prompt processing, n_tokens = 260621, progress = 0.99, t = 258.59 s / 1007.87 tokens per second
5.17.772.207 I slot print_timing: id 0 | prompt processing, n_tokens = 261529, progress = 0.99, t = 260.28 s / 1004.80 tokens per second
Desktop/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:108: CUDA error
5.18.215.470 E ggml_cuda_compute_forward: SOFT_MAX failed
5.18.215.476 E CUDA error: invalid argument
5.18.215.477 E current device: 0, in function ggml_cuda_compute_forward at .../ggml/src/ggml-cuda/ggml-cuda.cu:2417
5.18.215.478 E err
Aborted (core dumped)

Backtrace (from the same crash, according to and written by AI): ggml_cuda_error → ggml_backend_cuda_graph_compute → ggml_backend_sched_graph_compute_async → llama_context::graph_compute → llama_context::process_ubatch → llama_context::decode → server_context_impl::update_slots.

Launch flags: --ctx-size 420000 --n-gpu-layers 999 --tensor-split 65,21,21,21,70 --flash-attn on --parallel 1 --ubatch-size 2048 --batch-size 2048

Worth noting in case it affects prefill that this configuration gives an error/warning at launch that it couldn't use pipeline parallel, and that it's falling back. Performance seems to be about the same regardless on my setup, so I've been ignoring it in favor of a higher context window for both Deepseek V4 flash 0731 and this model.

Tested across different batch/ubatch sizes, no effect on crashes.

[AIalliAI]

Cross-ref: k-pool softmax gridDim.y fix at timkhronos#11 (for the SOFT_MAX crash at n_kv >= 262144).

Also see timkhronos#9 (tensor naming), #27752, #27754, #27917.

CC @timkhronos @ggerganov @CISC @ngxson