Repository navigation
Eval bug: Vulkan Backend no-kv-offload on Qwen3-Coder-Next and Qwen3.6-35B-A3B produces gibberish output #23321
Description
Name and Version
bash ./llama-server -v
0.00.043.048 I common_params_print_info: build 9205 (dd7cad7) with GNU 13.3.0 for Linux x86_64
0.00.043.050 I log_info: verbosity = 2147483647 (adjust with the -lv N CLI arg)
0.00.043.077 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
Operating systems
Linux
GGML backends
BLAS, Vulkan
Hardware
bash hostnamectl
Operating System: Ubuntu 24.04.3 LTS
Kernel: Linux 6.14.0-33-generic
Architecture: x86-64
Hardware Vendor: GMKtec
Hardware Model: M5 PLUS
Firmware Version: M5 PLUS 1.03
Device Name AMD Radeon Graphics
PCI (domain:bus:dev.func) 0000:03:00.0
DeviceID:RevID 0x15E7.0xC1
OpenGL Driver Version Mesa 26.1.0 - kisak-mesa PPA
gfx_target_version gfx90c
Models
Qwen36-35B-A3B and Qwen3-Coder-Next
Problem description & steps to reproduce
When no-kv-offload is enabled on these model. The model starts producing gibberish output.
First Bad Commit
Do not know.
Relevant log output
Model: Qwen-A3B-35B
Output:
User: Provide a quick sort in python.
Model:
觉得 :,った??
(快速1, (不会/)“, "一直2 No跟 line觉得/;一（ .ت q ak ,曾,觉得
",一sった跟 y.ens不会, 开始 -,觉得. **还是
,只,q已经 . ,一,一1..., ,跟** ' , "/一 , ,对这个/ 一直 ",ä,った **-,“, ,曾, , ( ", public,不会 \ ,ت,觉得,觉得 ,3, ;, ,った . ,还是,った,...,**,跟, ,觉得 , ,ä 跟,一直, ,心里 iu,1,String **觉得, (/ ,った iel !, media,< 一 ak, ,越来越 还是个,觉得, *.跟,,觉得,った " ',觉得,;,った,说完:و,った **还是 那双,った,った,觉得,,一直 不会,总是/った,iu, public,觉得 ** media,った,<.一 觉得 还是个, /*,跟 对社会,觉得 *,,!, facing, 心里,った,,yer,说完 那双,\(, ,还是,越来越 ** ,xx,iu/iel ingly/ public,不会.った,觉得 و,; 一直 ö **<,还是个,觉得,* 甚至,< 跟,った,った /*,** !,/*, '觉得 那双,... 跟, < 心里, , *,< 觉得 还是 った,说完,った,った/iel \( ; 不会,觉得.跟 觉得,越来越,一直,< **ak *,跟 \( public,<, /*,觉得,还是个 **String,总是 (甚至,对社会,った,yer,! った **iu,< *那双 觉得 心里/跟 / った,...:说完 (,< iel ::,说完,不会,说完 **,,还是.*,; った,<,< った,说着,一直 ak,还是个, /* 觉得,那句 った 越来越, * った,ingly,跟 **ö,说完,那双,觉得//*,心里,iu <,跟,! yer 不会 还是,った った,跟 *跟 <: ( ,った.一直 甚至, < <,iel,**, *,<,[,<,った,觉得,::/总是, /*,<,った,iu <,跟,越来越 "不会,说完,yer,对你的 还是个 还得,;,った,不会,还是,/*,那双,*,ak,<,った **った,った.iel った:跟 った,觉得,[, * **った ,::,った,总是,心里 =/った,った **跟,说着 <,った 甚至,った, /* (った,跟.<,* ****,った,跟 还是,yer.说完,iel 'った.った < ak,った,った,还是个 跟,[, ,觉得 觉得 /跟 ;, =,ingly,< った/心里 说完,对你的 < iu,った * **った,一,< * /* < った 说完,不会,<,/*,那双.::,った "对这个,总是 <,<,还是,跟 った,还得, =,った,った ,った,< '那句 还是个,对你的,<,越来越 *った,</<,ingly った,觉得 った,った 说完 心里 った,说着 った.,那双 /*,近期,* **对这个 ;.开始-< 跟 **不会,った,总是, =,った った った,那句, / 对社会,った,还是 还得 った,越来越:った,了,ich 觉得 **::,nce ingly った,った,[/说完,那双,说完,/.对你的 った,<,った **近期,iance '觉得,说着,った った 不会,った,说完,った った,心里 った,那句,说完, /*.跟,<, :总是,觉得,还是,对社会 越来越,*,一;った,< nce 对你的/inge,那双 った < った/ye,说完 leih,った,った **:: **还是个 觉得 った 还得 对这个 った 了,った,跟 ingly,说完,った,说完 (心里 /*,还是,近期,一,说完,iance 对社会/ ,不会 总是 * 那双,<,说完 **那句 '
Logs
0.00.227.156 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.227.185 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.227.188 I srv main: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.227.516 I srv init: running without SSL
0.00.227.543 I srv init: using 15 threads for HTTP server
0.00.231.248 I srv load_models: Loaded 0 cached model presets
0.00.234.775 I srv load_models: Loaded 18 custom model presets from /home/tipu/Applications/llamacpp/llamacpp-config.ini
0.00.241.007 I srv operator(): Available models (18) (*: custom preset)
0.00.241.010 I srv operator(): * DotsOCR
0.00.241.010 I srv operator(): * Gemma-4-26B-A4B
0.00.241.011 I srv operator(): * Gemma-4-26B-A4B-NM
0.00.241.011 I srv operator(): * LightOnOCR-2
0.00.241.011 I srv operator(): * LightOnOCR-2-soup
0.00.241.012 I srv operator(): * NVIDIA-Nemotron-Cascade
0.00.241.013 I srv operator(): * Nanonets-OCR (aliases: Nanonets-OCR)
0.00.241.013 I srv operator(): * Qwen-A3B-35B-Instruct
0.00.241.014 I srv operator(): * Qwen-A3B-35B-Instruct-Agent
0.00.241.014 I srv operator(): * Qwen-A3B-35B-Instruct-Agent-NM
0.00.241.014 I srv operator(): * Qwen-A3B-35B-Instruct-Agent-NM-exp
0.00.241.015 I srv operator(): * Qwen-A3B-35B-Thinking
0.00.241.015 I srv operator(): * Qwen-A3B-35B-Thinking-Agent
0.00.241.015 I srv operator(): * Qwen-A3B-35B-Thinking-Agent-NM
0.00.241.015 I srv operator(): * Qwen-A3B-35B-Thinking-NM
0.00.241.016 I srv operator(): * Qwen3-Coder-Next
0.00.241.016 I srv operator(): * Qwen3-Coder-Next-Agent
0.00.241.016 I srv operator(): * default
0.00.241.403 W srv main: -----------------
0.00.241.404 W srv main: CORS proxy is enabled, do not expose server to untrusted environments
0.00.241.405 W srv main: This feature is EXPERIMENTAL and may be removed or changed in future versions
0.00.241.405 W srv main: -----------------
0.00.241.415 W srv main: -----------------
0.00.241.415 W srv main: Built-in tools are enabled, do not expose server to untrusted environments
0.00.241.415 W srv main: This feature is EXPERIMENTAL and may be changed in the future
0.00.241.415 W srv main: -----------------
0.00.241.419 I srv main: starting router server, no model will be loaded in this process
0.00.241.420 I srv start: binding port with default address family
0.00.242.587 I srv main: router server is listening on http://127.0.0.1:8080
0.00.242.588 W srv main: NOTE: router mode is experimental
0.00.242.589 W srv main: it is not recommended to use this mode in untrusted environments
0.10.086.187 I srv load: spawning server instance with name=Qwen-A3B-35B-Instruct on port 38739
0.10.086.221 I srv load: spawning server instance with args:
0.10.086.222 I srv load: /home/tipu/Applications/llamacpp/llama-server
0.10.086.222 I srv load: --cache-idle-slots
0.10.086.223 I srv load: --host
0.10.086.223 I srv load: 127.0.0.1
0.10.086.223 I srv load: --jinja
0.10.086.223 I srv load: --keep
0.10.086.224 I srv load: -1
0.10.086.224 I srv load: --min-p
0.10.086.224 I srv load: 0.0
0.10.086.224 I srv load: --mlock
0.10.086.225 I srv load: --no-mmap
0.10.086.225 I srv load: --offline
0.10.086.225 I srv load: --port
0.10.086.226 I srv load: 38739
0.10.086.230 I srv load: --presence-penalty
0.10.086.230 I srv load: 1.5
0.10.086.231 I srv load: --prio
0.10.086.231 I srv load: 2
0.10.086.231 I srv load: --prio-batch
0.10.086.231 I srv load: 2
0.10.086.232 I srv load: --repack
0.10.086.232 I srv load: --repeat-penalty
0.10.086.232 I srv load: 1.0
0.10.086.232 I srv load: --sampling-seq
0.10.086.232 I srv load: ekpmt
0.10.086.233 I srv load: --spec-draft-n-max
0.10.086.233 I srv load: 48
0.10.086.233 I srv load: --spec-type
0.10.086.234 I srv load: ngram-map-k
0.10.086.234 I srv load: --temperature
0.10.086.234 I srv load: 1.0
0.10.086.234 I srv load: --tools
0.10.086.235 I srv load: all
0.10.086.235 I srv load: --top-k
0.10.086.235 I srv load: 20
0.10.086.235 I srv load: --top-p
0.10.086.236 I srv load: 0.95
0.10.086.236 I srv load: --webui-config-file
0.10.086.236 I srv load: /home/tipu/Applications/llamacpp/front-end-config.json
0.10.086.236 I srv load: --webui-mcp-proxy
0.10.086.237 I srv load: --alias
0.10.086.237 I srv load: Qwen-A3B-35B-Instruct
0.10.086.237 I srv load: --batch-size
0.10.086.237 I srv load: 2048
0.10.086.237 I srv load: --ctx-size
0.10.086.239 I srv load: 65536
0.10.086.239 I srv load: --cache-ram
0.10.086.240 I srv load: -1
0.10.086.240 I srv load: --flash-attn
0.10.086.240 I srv load: on
0.10.086.240 I srv load: --fit
0.10.086.241 I srv load: off
0.10.086.241 I srv load: --no-kv-offload
0.10.086.241 I srv load: --kv-unified
0.10.086.241 I srv load: --log-verbosity
0.10.086.242 I srv load: 4
0.10.086.242 I srv load: --model
0.10.086.242 I srv load: /home/tipu/AI/models/unsloth/Qwen36-35-A3B/Qwen36-35B-A3B-Q8-mtp.gguf
0.10.086.243 I srv load: --mmproj
0.10.086.243 I srv load: /home/tipu/AI/models/unsloth/Qwen36-35-A3B/Qwen36-35B-A3B-mmproj-F16.gguf
0.10.086.243 I srv load: --n-predict
0.10.086.243 I srv load: -1
0.10.086.244 I srv load: --n-gpu-layers
0.10.086.244 I srv load: 99
0.10.086.244 I srv load: --parallel
0.10.086.244 I srv load: 1
0.10.086.245 I srv load: --reasoning
0.10.086.245 I srv load: off
0.10.086.245 I srv load: --ubatch-size
0.10.086.245 I srv load: 1088
[38739] 0.00.038.743 W migrate_old_cache_to_hf_cache: skipping migration in offline mode (will run when online)
[38739] 0.00.039.017 I common_params_print_info: build 9205 (dd7cad719) with GNU 13.3.0 for Linux x86_64
[38739] 0.00.039.018 I log_info: verbosity = 4 (adjust with the `-lv N` CLI arg)
[38739] 0.00.039.019 I device_info:
[38739] 0.00.039.156 I - Vulkan0 : AMD Radeon Graphics (RADV RENOIR) (60416 MiB, 59336 MiB free)
[38739] 0.00.039.158 I - BLAS : BLIS (0 MiB, 0 MiB free)
[38739] 0.00.039.169 I - CPU : AMD Ryzen 7 5825U with Radeon Graphics (47998 MiB, 47998 MiB free)
[38739] 0.00.039.196 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
[38739] 0.00.039.214 I srv init: running without SSL
[38739] 0.00.039.243 I srv init: using 15 threads for HTTP server
[38739] 0.00.039.388 W srv main: -----------------
[38739] 0.00.039.390 W srv main: CORS proxy is enabled, do not expose server to untrusted environments
[38739] 0.00.039.391 W srv main: This feature is EXPERIMENTAL and may be removed or changed in future versions
[38739] 0.00.039.391 W srv main: -----------------
[38739] 0.00.039.399 W srv main: -----------------
[38739] 0.00.039.400 W srv main: Built-in tools are enabled, do not expose server to untrusted environments
[38739] 0.00.039.400 W srv main: This feature is EXPERIMENTAL and may be changed in the future
[38739] 0.00.039.400 W srv main: -----------------
[38739] 0.00.039.406 I srv start: binding port with default address family
[38739] 0.00.040.570 I srv main: loading model
[38739] 0.00.040.573 I srv load_model: loading model '/home/tipu/AI/models/unsloth/Qwen36-35-A3B/Qwen36-35B-A3B-Q8-mtp.gguf'
[38739] 0.00.096.388 I llama_model_loader: loaded meta data with 55 key-value pairs and 753 tensors from /home/tipu/AI/models/unsloth/Qwen36-35-A3B/Qwen36-35B-A3B-Q8-mtp.gguf (version GGUF V3 (latest))
[38739] 0.00.096.424 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
[38739] 0.00.096.429 I llama_model_loader: - kv 0: general.architecture str = qwen35moe
[38739] 0.00.096.431 I llama_model_loader: - kv 1: general.type str = model
[38739] 0.00.096.432 I llama_model_loader: - kv 2: general.sampling.top_k i32 = 20
[38739] 0.00.096.437 I llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000
[38739] 0.00.096.438 I llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000
[38739] 0.00.096.439 I llama_model_loader: - kv 5: general.name str = Qwen3.6-35B-A3B
[38739] 0.00.096.439 I llama_model_loader: - kv 6: general.basename str = Qwen3.6-35B-A3B
[38739] 0.00.096.440 I llama_model_loader: - kv 7: general.quantized_by str = Unsloth
[38739] 0.00.096.440 I llama_model_loader: - kv 8: general.size_label str = 35B-A3B
[38739] 0.00.096.441 I llama_model_loader: - kv 9: general.license str = apache-2.0
[38739] 0.00.096.443 I llama_model_loader: - kv 10: general.license.link str = https://huggingface.co/Qwen/Qwen3.6-3...
[38739] 0.00.096.443 I llama_model_loader: - kv 11: general.repo_url str = https://huggingface.co/unsloth
[38739] 0.00.096.444 I llama_model_loader: - kv 12: general.base_model.count u32 = 1
[38739] 0.00.096.445 I llama_model_loader: - kv 13: general.base_model.0.name str = Qwen3.6 35B A3B
[38739] 0.00.096.445 I llama_model_loader: - kv 14: general.base_model.0.organization str = Qwen
[38739] 0.00.096.447 I llama_model_loader: - kv 15: general.base_model.0.repo_url str = https://huggingface.co/Qwen/Qwen3.6-3...
[38739] 0.00.096.456 I llama_model_loader: - kv 16: general.tags arr[str,3] = ["qwen3_5_moe", "qwen", "image-text-t...
[38739] 0.00.096.458 I llama_model_loader: - kv 17: qwen35moe.block_count u32 = 41
[38739] 0.00.096.459 I llama_model_loader: - kv 18: qwen35moe.context_length u32 = 262144
[38739] 0.00.096.459 I llama_model_loader: - kv 19: qwen35moe.embedding_length u32 = 2048
[38739] 0.00.096.460 I llama_model_loader: - kv 20: qwen35moe.attention.head_count u32 = 16
[38739] 0.00.096.460 I llama_model_loader: - kv 21: qwen35moe.attention.head_count_kv u32 = 2
[38739] 0.00.096.462 I llama_model_loader: - kv 22: qwen35moe.rope.dimension_sections arr[i32,4] = [11, 11, 10, 0]
[38739] 0.00.096.464 I llama_model_loader: - kv 23: qwen35moe.rope.freq_base f32 = 10000000.000000
[38739] 0.00.096.465 I llama_model_loader: - kv 24: qwen35moe.attention.layer_norm_rms_epsilon f32 = 0.000001
[38739] 0.00.096.465 I llama_model_loader: - kv 25: qwen35moe.expert_count u32 = 256
[38739] 0.00.096.466 I llama_model_loader: - kv 26: qwen35moe.expert_used_count u32 = 8
[38739] 0.00.096.466 I llama_model_loader: - kv 27: qwen35moe.attention.key_length u32 = 256
[38739] 0.00.096.467 I llama_model_loader: - kv 28: qwen35moe.attention.value_length u32 = 256
[38739] 0.00.096.467 I llama_model_loader: - kv 29: qwen35moe.expert_feed_forward_length u32 = 512
[38739] 0.00.096.468 I llama_model_loader: - kv 30: qwen35moe.expert_shared_feed_forward_length u32 = 512
[38739] 0.00.096.469 I llama_model_loader: - kv 31: qwen35moe.ssm.conv_kernel u32 = 4
[38739] 0.00.096.469 I llama_model_loader: - kv 32: qwen35moe.ssm.state_size u32 = 128
[38739] 0.00.096.474 I llama_model_loader: - kv 33: qwen35moe.ssm.group_count u32 = 16
[38739] 0.00.096.474 I llama_model_loader: - kv 34: qwen35moe.ssm.time_step_rank u32 = 32
[38739] 0.00.096.475 I llama_model_loader: - kv 35: qwen35moe.ssm.inner_size u32 = 4096
[38739] 0.00.096.475 I llama_model_loader: - kv 36: qwen35moe.full_attention_interval u32 = 4
[38739] 0.00.096.476 I llama_model_loader: - kv 37: qwen35moe.rope.dimension_count u32 = 64
[38739] 0.00.096.476 I llama_model_loader: - kv 38: qwen35moe.nextn_predict_layers u32 = 1
[38739] 0.00.096.477 I llama_model_loader: - kv 39: tokenizer.ggml.model str = gpt2
[38739] 0.00.096.477 I llama_model_loader: - kv 40: tokenizer.ggml.pre str = qwen35
[38739] 0.00.121.280 I llama_model_loader: - kv 41: tokenizer.ggml.tokens arr[str,248320] = ["!", "\"", "#", "$", "%", "&", "'", ...
[38739] 0.00.128.326 I llama_model_loader: - kv 42: tokenizer.ggml.token_type arr[i32,248320] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
[38739] 0.00.151.957 I llama_model_loader: - kv 43: tokenizer.ggml.merges arr[str,247587] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
[38739] 0.00.151.964 I llama_model_loader: - kv 44: tokenizer.ggml.eos_token_id u32 = 248046
[38739] 0.00.151.965 I llama_model_loader: - kv 45: tokenizer.ggml.padding_token_id u32 = 248055
[38739] 0.00.151.966 I llama_model_loader: - kv 46: tokenizer.ggml.bos_token_id u32 = 248044
[38739] 0.00.151.967 I llama_model_loader: - kv 47: tokenizer.ggml.add_bos_token bool = false
[38739] 0.00.151.971 I llama_model_loader: - kv 48: tokenizer.chat_template str = {%- set image_count = namespace(value...
[38739] 0.00.151.972 I llama_model_loader: - kv 49: general.quantization_version u32 = 2
[38739] 0.00.151.972 I llama_model_loader: - kv 50: general.file_type u32 = 7
[38739] 0.00.151.974 I llama_model_loader: - kv 51: quantize.imatrix.file str = Qwen3.6-35B-A3B-GGUF/imatrix_unsloth....
[38739] 0.00.151.976 I llama_model_loader: - kv 52: quantize.imatrix.dataset str = unsloth_calibration_Qwen3.6-35B-A3B.txt
[38739] 0.00.151.976 I llama_model_loader: - kv 53: quantize.imatrix.entries_count u32 = 510
[38739] 0.00.151.977 I llama_model_loader: - kv 54: quantize.imatrix.chunks_count u32 = 77
[38739] 0.00.151.977 I llama_model_loader: - type f32: 308 tensors
[38739] 0.00.151.979 I llama_model_loader: - type q8_0: 443 tensors
[38739] 0.00.151.979 I llama_model_loader: - type bf16: 2 tensors
[38739] 0.00.151.980 I print_info: file format = GGUF V3 (latest)
[38739] 0.00.151.981 I print_info: file type = Q8_0
[38739] 0.00.151.985 I print_info: file size = 35.19 GiB (8.51 BPW)
[38739] 0.00.152.210 I llama_prepare_model_devices: using device Vulkan0 (AMD Radeon Graphics (RADV RENOIR)) (0000:03:00.0) - 59345 MiB free
[38739] 0.00.381.454 I load: 0 unused tokens
[38739] 0.00.436.909 I load: printing all EOG tokens:
[38739] 0.00.436.913 I load: - 248044 ('<|endoftext|>')
[38739] 0.00.436.914 I load: - 248046 ('<|im_end|>')
[38739] 0.00.436.915 I load: - 248063 ('<|fim_pad|>')
[38739] 0.00.436.916 I load: - 248064 ('<|repo_name|>')
[38739] 0.00.436.916 I load: - 248065 ('<|file_sep|>')
[38739] 0.00.437.365 I load: special tokens cache size = 33
[38739] 0.00.514.373 I load: token to piece cache size = 1.7581 MB
[38739] 0.00.514.388 I print_info: arch = qwen35moe
[38739] 0.00.514.388 I print_info: vocab_only = 0
[38739] 0.00.514.389 I print_info: no_alloc = 0
[38739] 0.00.514.389 I print_info: n_ctx_train = 262144
[38739] 0.00.514.389 I print_info: n_embd = 2048
[38739] 0.00.514.390 I print_info: n_embd_inp = 2048
[38739] 0.00.514.390 I print_info: n_layer = 41
[38739] 0.00.514.399 I print_info: n_head = 16
[38739] 0.00.514.400 I print_info: n_head_kv = 2
[38739] 0.00.514.401 I print_info: n_rot = 64
[38739] 0.00.514.401 I print_info: n_swa = 0
[38739] 0.00.514.401 I print_info: is_swa_any = 0
[38739] 0.00.514.402 I print_info: n_embd_head_k = 256
[38739] 0.00.514.402 I print_info: n_embd_head_v = 256
[38739] 0.00.514.403 I print_info: n_gqa = 8
[38739] 0.00.514.405 I print_info: n_embd_k_gqa = 512
[38739] 0.00.514.406 I print_info: n_embd_v_gqa = 512
[38739] 0.00.514.407 I print_info: f_norm_eps = 0.0e+00
[38739] 0.00.514.408 I print_info: f_norm_rms_eps = 1.0e-06
[38739] 0.00.514.409 I print_info: f_clamp_kqv = 0.0e+00
[38739] 0.00.514.409 I print_info: f_max_alibi_bias = 0.0e+00
[38739] 0.00.514.409 I print_info: f_logit_scale = 0.0e+00
[38739] 0.00.514.411 I print_info: f_attn_scale = 0.0e+00
[38739] 0.00.514.412 I print_info: f_attn_value_scale = 0.0000
[38739] 0.00.514.413 I print_info: n_ff = 0
[38739] 0.00.514.414 I print_info: n_expert = 256
[38739] 0.00.514.414 I print_info: n_expert_used = 8
[38739] 0.00.514.414 I print_info: n_expert_groups = 0
[38739] 0.00.514.414 I print_info: n_group_used = 0
[38739] 0.00.514.415 I print_info: causal attn = 1
[38739] 0.00.514.415 I print_info: pooling type = -1
[38739] 0.00.514.415 I print_info: rope type = 40
[38739] 0.00.514.415 I print_info: rope scaling = linear
[38739] 0.00.514.417 I print_info: freq_base_train = 10000000.0
[38739] 0.00.514.417 I print_info: freq_scale_train = 1
[38739] 0.00.514.418 I print_info: n_ctx_orig_yarn = 262144
[38739] 0.00.514.418 I print_info: rope_yarn_log_mul = 0.0000
[38739] 0.00.514.419 I print_info: rope_finetuned = unknown
[38739] 0.00.514.419 I print_info: mrope sections = [11, 11, 10, 0]
[38739] 0.00.514.420 I print_info: ssm_d_conv = 4
[38739] 0.00.514.420 I print_info: ssm_d_inner = 4096
[38739] 0.00.514.420 I print_info: ssm_d_state = 128
[38739] 0.00.514.421 I print_info: ssm_dt_rank = 32
[38739] 0.00.514.421 I print_info: ssm_n_group = 16
[38739] 0.00.514.421 I print_info: ssm_dt_b_c_rms = 0
[38739] 0.00.514.422 I print_info: model type = 35B.A3B
[38739] 0.00.514.423 I print_info: model params = 35.51 B
[38739] 0.00.514.423 I print_info: general.name = Qwen3.6-35B-A3B
[38739] 0.00.514.424 I print_info: vocab type = BPE
[38739] 0.00.514.425 I print_info: n_vocab = 248320
[38739] 0.00.514.425 I print_info: n_merges = 247587
[38739] 0.00.514.425 I print_info: BOS token = 248044 '<|endoftext|>'
[38739] 0.00.514.426 I print_info: EOS token = 248046 '<|im_end|>'
[38739] 0.00.514.426 I print_info: EOT token = 248046 '<|im_end|>'
[38739] 0.00.514.427 I print_info: PAD token = 248055 '<|vision_pad|>'
[38739] 0.00.514.427 I print_info: LF token = 198 'Ċ'
[38739] 0.00.514.427 I print_info: FIM PRE token = 248060 '<|fim_prefix|>'
[38739] 0.00.514.428 I print_info: FIM SUF token = 248062 '<|fim_suffix|>'
[38739] 0.00.514.428 I print_info: FIM MID token = 248061 '<|fim_middle|>'
[38739] 0.00.514.428 I print_info: FIM PAD token = 248063 '<|fim_pad|>'
[38739] 0.00.514.429 I print_info: FIM REP token = 248064 '<|repo_name|>'
[38739] 0.00.514.429 I print_info: FIM SEP token = 248065 '<|file_sep|>'
[38739] 0.00.514.429 I print_info: EOG token = 248044 '<|endoftext|>'
[38739] 0.00.514.430 I print_info: EOG token = 248046 '<|im_end|>'
[38739] 0.00.514.430 I print_info: EOG token = 248063 '<|fim_pad|>'
[38739] 0.00.514.431 I print_info: EOG token = 248064 '<|repo_name|>'
[38739] 0.00.514.431 I print_info: EOG token = 248065 '<|file_sep|>'
[38739] 0.00.514.431 I print_info: max token length = 256
[38739] 0.00.514.432 I load_tensors: loading model tensors, this can take a while... (mmap = false, direct_io = false)
[38739] 0.02.343.144 I load_tensors: offloading output layer to GPU
[38739] 0.02.343.150 I load_tensors: offloading 40 repeating layers to GPU
[38739] 0.02.343.150 I load_tensors: offloaded 42/42 layers to GPU
[38739] 0.02.343.158 I load_tensors: Vulkan0 model buffer size = 35524.14 MiB
[38739] 0.02.343.159 I load_tensors: Vulkan_Host model buffer size = 515.31 MiB
[38739] ...................................................................................................
[38739] 0.19.860.329 I common_init_result: added <|endoftext|> logit bias = -inf
[38739] 0.19.860.334 I common_init_result: added <|im_end|> logit bias = -inf
[38739] 0.19.860.335 I common_init_result: added <|fim_pad|> logit bias = -inf
[38739] 0.19.860.335 I common_init_result: added <|repo_name|> logit bias = -inf
[38739] 0.19.860.336 I common_init_result: added <|file_sep|> logit bias = -inf
[38739] 0.19.860.410 I llama_context: constructing llama_context
[38739] 0.19.860.415 I llama_context: n_seq_max = 1
[38739] 0.19.860.416 I llama_context: n_ctx = 65536
[38739] 0.19.860.416 I llama_context: n_ctx_seq = 65536
[38739] 0.19.860.417 I llama_context: n_batch = 2048
[38739] 0.19.860.417 I llama_context: n_ubatch = 1088
[38739] 0.19.860.418 I llama_context: causal_attn = 1
[38739] 0.19.860.418 I llama_context: flash_attn = enabled
[38739] 0.19.860.419 I llama_context: kv_unified = true
[38739] 0.19.860.423 I llama_context: freq_base = 10000000.0
[38739] 0.19.860.424 I llama_context: freq_scale = 1
[38739] 0.19.860.424 I llama_context: n_rs_seq = 0
[38739] 0.19.860.425 W llama_context: n_ctx_seq (65536) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
[38739] 0.19.860.706 I llama_context: Vulkan_Host output buffer size = 0.95 MiB
[38739] 0.19.861.130 I llama_kv_cache: CPU KV buffer size = 1280.00 MiB
[38739] 0.20.464.195 I llama_kv_cache: size = 1280.00 MiB ( 65536 cells, 10 layers, 1/1 seqs), K (f16): 640.00 MiB, V (f16): 640.00 MiB
[38739] 0.20.464.201 I llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
[38739] 0.20.464.201 I llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
[38739] 0.20.493.603 I llama_memory_recurrent: CPU RS buffer size = 62.81 MiB
[38739] 0.20.493.615 I llama_memory_recurrent: size = 62.81 MiB ( 1 cells, 41 layers, 1 seqs 0 rs_seq), R (f32): 2.81 MiB, S (f32): 60.00 MiB
[38739] 0.20.493.625 I sched_reserve: reserving ...
[38739] 0.20.510.393 I sched_reserve: resolving fused Gated Delta Net support:
[38739] 0.20.512.619 I sched_reserve: fused Gated Delta Net (autoregressive) enabled
[38739] 0.20.514.330 I sched_reserve: fused Gated Delta Net (chunked) enabled
[38739] 0.20.722.631 I sched_reserve: Vulkan0 compute buffer size = 1652.69 MiB
[38739] 0.20.722.636 I sched_reserve: Vulkan_Host compute buffer size = 314.65 MiB
[38739] 0.20.722.636 I sched_reserve: graph nodes = 3758
[38739] 0.20.722.637 I sched_reserve: graph splits = 202
[38739] 0.20.722.638 I sched_reserve: reserve took 229.01 ms, sched copies = 1
[38739] 0.20.722.752 W common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
[38739] 0.22.432.596 I clip_model_loader: model name: Qwen3.6-35B-A3B
[38739] 0.22.432.600 I clip_model_loader: description:
[38739] 0.22.432.600 I clip_model_loader: GGUF version: 3
[38739] 0.22.432.601 I clip_model_loader: alignment: 32
[38739] 0.22.432.601 I clip_model_loader: n_tensors: 334
[38739] 0.22.432.602 I clip_model_loader: n_kv: 35
[38739] 0.22.432.602 I
[38739] 0.22.432.603 I clip_model_loader: has vision encoder
[38739] 0.22.433.097 I clip_ctx: CLIP using Vulkan0 backend
[38739] 0.22.433.445 W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
[38739] 0.22.433.447 W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
[38739] 0.22.433.447 W load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842
[38739]
[38739] 0.22.433.448 I load_hparams: projector: qwen3vl_merger
[38739] 0.22.433.448 I load_hparams: n_embd: 1152
[38739] 0.22.433.449 I load_hparams: n_head: 16
[38739] 0.22.433.449 I load_hparams: n_ff: 4304
[38739] 0.22.433.449 I load_hparams: n_layer: 27
[38739] 0.22.433.449 I load_hparams: ffn_op: gelu
[38739] 0.22.433.450 I load_hparams: projection_dim: 2048
[38739] 0.22.433.450 I
[38739] --- vision hparams ---
[38739] 0.22.433.450 I load_hparams: image_size: 768
[38739] 0.22.433.450 I load_hparams: patch_size: 16
[38739] 0.22.433.451 I load_hparams: has_llava_proj: 0
[38739] 0.22.433.451 I load_hparams: minicpmv_version: 0
[38739] 0.22.433.451 I load_hparams: n_merge: 2
[38739] 0.22.433.452 I load_hparams: n_wa_pattern: 0
[38739] 0.22.433.452 I load_hparams: image_min_pixels: 8192
[38739] 0.22.433.452 I load_hparams: image_max_pixels: 4194304
[38739] 0.22.433.452 I
[38739] 0.22.433.457 I load_hparams: model size: 857.60 MiB
[38739] 0.22.433.458 I load_hparams: metadata size: 0.12 MiB
[38739] 0.23.172.134 I warmup: warmup with image size = 1472 x 1472
[38739] 0.23.199.738 I alloc_compute_meta: Vulkan0 compute buffer size = 248.10 MiB
[38739] 0.23.199.744 I alloc_compute_meta: CPU compute buffer size = 24.93 MiB
[38739] 0.23.199.745 I alloc_compute_meta: graph splits = 1, nodes = 823
[38739] 0.23.199.861 I warmup: flash attention is enabled
[38739] 0.23.199.886 I srv load_model: loaded multimodal model, '/home/tipu/AI/models/unsloth/Qwen36-35-A3B/Qwen36-35B-A3B-mmproj-F16.gguf'
[38739] 0.23.199.893 I srv load_model: initializing slots, n_slots = 1
[38739] 0.24.820.396 I common_context_can_seq_rm: the context does not support partial sequence removal
[38739] 0.24.913.686 W srv load_model: speculative decoding will use checkpoints
[38739] 0.24.913.704 I common_speculative_init: adding speculative implementation 'ngram-map-k'
[38739] 0.24.914.430 I srv load_model: speculative decoding context initialized
[38739] 0.24.914.433 I slot load_model: id 0 | task -1 | new slot, n_ctx = 65536
[38739] 0.24.914.499 I srv load_model: prompt cache is enabled, size limit: no limit
[38739] 0.24.914.505 I srv load_model: use `--cache-ram 0` to disable the prompt cache
[38739] 0.24.914.507 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
[38739] 0.24.914.527 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
[38739] 0.24.928.582 I init: chat template, example_format: '<|im_start|>system
[38739] You are a helpful assistant<|im_end|>
[38739] <|im_start|>user
[38739] Hello<|im_end|>
[38739] <|im_start|>assistant
[38739] Hi there<|im_end|>
[38739] <|im_start|>user
[38739] How are you?<|im_end|>
[38739] <|im_start|>assistant
[38739] <think>
[38739]
[38739] </think>
[38739]
[38739] '
[38739] 0.24.939.208 I srv init: init: chat template, thinking = 0
[38739] 0.24.939.226 I srv main: model loaded
[38739] 0.24.939.228 I srv main: server is listening on http://127.0.0.1:38739
[38739] cmd_child_to_router:ready
[38739] cmd_child_to_router:info:{"id":"Qwen-A3B-35B-Instruct","aliases":["Qwen-A3B-35B-Instruct"],"tags":[],"object":"model","created":1779170911,"owned_by":"llamacpp","meta":{"vocab_type":2,"n_vocab":248320,"n_ctx":65536,"n_ctx_train":262144,"n_embd":2048,"n_params":35505251456,"size":37790106112}}
[38739] 0.24.942.270 I srv operator(): child server monitoring thread started, waiting for EOF on stdin...
[38739] 0.24.942.376 I srv update_slots: all slots are idle
0.35.265.071 I srv proxy_reques: proxying request to model Qwen-A3B-35B-Instruct on port 38739
0.38.791.455 I srv proxy_reques: proxying request to model Qwen-A3B-35B-Instruct on port 38739
[38739] 0.28.720.324 I srv params_from_: Chat format: peg-native
[38739] 0.28.721.440 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
[38739] 0.28.721.442 I srv get_availabl: updating prompt cache
[38739] 0.28.721.446 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
[38739] 0.28.721.448 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 0.000 MiB, 65536 tokens, 65536 est)
[38739] 0.28.721.449 I srv get_availabl: prompt cache update took 0.01 ms
[38739] 0.28.721.700 I reasoning-budget: activated, budget=2147483647 tokens
[38739] 0.28.721.703 I reasoning-budget: deactivated (natural end)
[38739] 0.28.721.715 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> top-k -> top-p -> ?min-p -> ?temp-ext -> dist
[38739] 0.28.721.722 I slot launch_slot_: id 0 | task -1 | sampler params:
[38739] repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 1.500
[38739] dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 65536
[38739] top_k = 20, top_p = 0.950, min_p = 0.000, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 1.000
[38739] mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
[38739] 0.28.721.727 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0
[38739] 0.28.721.732 I slot update_slots: id 0 | task 0 | new prompt, n_ctx_slot = 65536, n_keep = -1, task.n_tokens = 1378
[38739] 0.28.721.740 I slot update_slots: id 0 | task 0 | cached n_tokens = 0, memory_seq_rm [0, end)
[38739] 0.28.721.991 I srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
[38739] 0.32.456.795 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 286, progress = 0.21, t = 3.74 s / 76.57 tokens per second
[38739] 0.32.456.931 I slot update_slots: id 0 | task 0 | cached n_tokens = 286, memory_seq_rm [286, end)
[38739] 0.32.583.195 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 285, pos_max = 285, n_tokens = 286, size = 62.813 MiB)
[38739] 0.40.778.063 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 1374, progress = 1.00, t = 12.06 s / 113.97 tokens per second
[38739] 0.40.778.078 I slot update_slots: id 0 | task 0 | cached n_tokens = 1374, memory_seq_rm [1374, end)
[38739] 0.40.779.093 I slot init_sampler: id 0 | task 0 | init sampler, took 0.74 ms, tokens: text = 1378, total = 1378
[38739] 0.40.978.267 I slot create_check: id 0 | task 0 | created context checkpoint 2 of 32 (pos_min = 1373, pos_max = 1373, n_tokens = 1374, size = 62.813 MiB)
[38739] 0.54.427.190 I slot print_timing: id 0 | task 0 | n_decoded = 100, tg = 7.57 t/s
[38739] 0.57.551.625 I slot print_timing: id 0 | task 0 | n_decoded = 123, tg = 7.53 t/s
[38739] 1.00.643.062 I slot print_timing: id 0 | task 0 | n_decoded = 146, tg = 7.51 t/s
[38739] 1.03.667.754 I slot print_timing: id 0 | task 0 | n_decoded = 169, tg = 7.53 t/s
[38739] 1.06.745.697 I slot print_timing: id 0 | task 0 | n_decoded = 192, tg = 7.52 t/s
[38739] 1.09.886.027 I slot print_timing: id 0 | task 0 | n_decoded = 215, tg = 7.50 t/s
[38739] 1.13.000.502 I slot print_timing: id 0 | task 0 | n_decoded = 238, tg = 7.49 t/s
[38739] 1.16.083.349 I slot print_timing: id 0 | task 0 | n_decoded = 261, tg = 7.48 t/s
[38739] 1.19.136.767 I slot print_timing: id 0 | task 0 | n_decoded = 284, tg = 7.49 t/s
[38739] 1.22.152.363 I slot print_timing: id 0 | task 0 | n_decoded = 306, tg = 7.47 t/s
[38739] 1.25.185.392 I slot print_timing: id 0 | task 0 | n_decoded = 329, tg = 7.48 t/s
[38739] 1.28.212.620 I slot print_timing: id 0 | task 0 | n_decoded = 352, tg = 7.49 t/s
[38739] 1.31.323.410 I slot print_timing: id 0 | task 0 | n_decoded = 375, tg = 7.48 t/s
[38739] 1.34.399.622 I slot print_timing: id 0 | task 0 | n_decoded = 398, tg = 7.48 t/s
[38739] 1.37.411.423 I slot print_timing: id 0 | task 0 | n_decoded = 420, tg = 7.47 t/s
[38739] 1.40.519.655 I slot print_timing: id 0 | task 0 | n_decoded = 443, tg = 7.47 t/s
[38739] 1.43.563.939 I slot print_timing: id 0 | task 0 | n_decoded = 466, tg = 7.47 t/s
[38739] 1.46.693.526 I slot print_timing: id 0 | task 0 | n_decoded = 488, tg = 7.45 t/s
[38739] 1.49.707.157 I slot print_timing: id 0 | task 0 | n_decoded = 510, tg = 7.45 t/s
[38739] 1.52.847.886 I slot print_timing: id 0 | task 0 | n_decoded = 533, tg = 7.44 t/s
[38739] 1.55.977.719 I slot print_timing: id 0 | task 0 | n_decoded = 556, tg = 7.44 t/s
[38739] 1.59.077.691 I slot print_timing: id 0 | task 0 | n_decoded = 579, tg = 7.44 t/s
[38739] 2.02.133.394 I slot print_timing: id 0 | task 0 | n_decoded = 602, tg = 7.44 t/s
[38739] 2.05.150.958 I slot print_timing: id 0 | task 0 | n_decoded = 625, tg = 7.45 t/s
[38739] 2.08.273.972 I slot print_timing: id 0 | task 0 | n_decoded = 646, tg = 7.42 t/s
[38739] 2.11.275.047 I slot print_timing: id 0 | task 0 | n_decoded = 668, tg = 7.42 t/s
[38739] 2.14.319.124 I slot print_timing: id 0 | task 0 | n_decoded = 690, tg = 7.41 t/s
[38739] 2.17.407.816 I slot print_timing: id 0 | task 0 | n_decoded = 713, tg = 7.41 t/s
[38739] 2.20.440.079 I slot print_timing: id 0 | task 0 | n_decoded = 735, tg = 7.41 t/s
[38739] 2.23.475.601 I slot print_timing: id 0 | task 0 | n_decoded = 758, tg = 7.41 t/s
[38739] 2.26.514.300 I slot print_timing: id 0 | task 0 | n_decoded = 781, tg = 7.42 t/s
[38739] 2.29.565.307 I slot print_timing: id 0 | task 0 | n_decoded = 804, tg = 7.42 t/s
[38739] 2.32.618.941 I slot print_timing: id 0 | task 0 | n_decoded = 827, tg = 7.42 t/s
[38739] 2.35.663.018 I slot print_timing: id 0 | task 0 | n_decoded = 850, tg = 7.43 t/s
[38739] 2.38.718.541 I slot print_timing: id 0 | task 0 | n_decoded = 873, tg = 7.43 t/s
[38739] 2.41.765.386 I slot print_timing: id 0 | task 0 | n_decoded = 896, tg = 7.43 t/s
[38739] 2.44.828.572 I slot print_timing: id 0 | task 0 | n_decoded = 919, tg = 7.43 t/s
[38739] 2.47.869.895 I slot print_timing: id 0 | task 0 | n_decoded = 941, tg = 7.43 t/s
[38739] 2.50.993.343 I slot print_timing: id 0 | task 0 | n_decoded = 964, tg = 7.43 t/s
[38739] 2.54.018.872 I slot print_timing: id 0 | task 0 | n_decoded = 986, tg = 7.42 t/s
[38739] 2.57.106.478 I slot print_timing: id 0 | task 0 | n_decoded = 1009, tg = 7.42 t/s
[38739] 3.00.219.577 I slot print_timing: id 0 | task 0 | n_decoded = 1032, tg = 7.42 t/s
[38739] 3.03.299.909 I slot print_timing: id 0 | task 0 | n_decoded = 1055, tg = 7.43 t/s
[38739] 3.06.394.699 I slot print_timing: id 0 | task 0 | n_decoded = 1077, tg = 7.42 t/s
[38739] 3.09.395.339 I slot print_timing: id 0 | task 0 | n_decoded = 1099, tg = 7.42 t/s
^C[38739] 3.10.055.129 I srv operator(): operator(): cleaning up before exit...
3.20.145.785 E srv operator(): http client error: Failed to read connection
[38739] 3.10.055.381 W srv stop: cancel task, id_task = 0
3.20.146.128 I srv operator(): operator(): cleaning up before exit...
3.20.146.137 I srv unload_all: stopping model instance name=Qwen-A3B-35B-Instruct
3.20.146.192 I srv operator(): stopping model instance name=Qwen-A3B-35B-Instruct
[38739] 3.10.055.989 I srv operator(): exit command received, exiting...
[38739] 3.10.056.208 I common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
[38739] 3.10.056.210 I common_memory_breakdown_print: | - Vulkan0 (Graphics (RADV RENOIR)) | 60416 = 20399 + (37176 = 35524 + 0 + 1652) + 2839 |
[38739] 3.10.056.210 I common_memory_breakdown_print: | - Host | 2172 = 515 + 1342 + 314 |
3.20.962.556 I srv operator(): instance name=Qwen-A3B-35B-Instruct exited with status 0
Activity
engrtipusultan commentedon May 19, 2026
Model: Qwen3-Coder-Next
Output:
User: Provide a quick sort in python.
Model:
the
ark user-T
.
br
es [
.>
upst>onymaminationy indexed
atedamin t
thearam.co name definition utirectoryification
aplroduction
.ット
Logs
[-]Eval bug: Vulkan Backend `no-kv-offload` on Qwen3-Coder-Next and Qwen3.6-35B-A3B[/-][+]Eval bug: Vulkan Backend `no-kv-offload` on Qwen3-Coder-Next and Qwen3.6-35B-A3B produces gibberish output[/+]engrtipusultan commentedon May 29, 2026
@0cc4m sorry to bump you, is it working on your hardware.
LVLUP-Studio commentedon Jun 4, 2026
I’m seeing a very similar issue on a dual Intel Arc Pro B60 setup using the llama.cpp
server-intelDocker image with SYCL.The important detail is that
Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.ggufandQwen3-Coder-30B-A3B-Instruct-Q6_K.ggufboth work cleanly on the same stack, across both GPUs.But
Qwen3.6-35B-A3B-UD-Q4_K_M.ggufandQwen3-Coder-Next-Q4_K_M.ggufproduce gibberish / random multilingual output even on simple prompts likeHello?.I tested with:
The server loads the model, but generation is corrupt. Logs for Qwen3.6 show
thinking = 1and repeated checkpoint invalidation due to hybrid/recurrent memory.This feels specific to Qwen3.6 / Coder-Next architecture or UD quant handling on GPU backends, not a general Intel Arc/SYCL setup problem, because Qwen3-Coder-30B Q4/Q6 work correctly.
github-actions commentedon Jul 20, 2026
This issue was closed because it has been inactive for 14 days since being marked as stale.
0cc4m commentedon Jul 23, 2026
Please check if #26040 fixes it for you.
engrtipusultan commentedon Jul 24, 2026
Hi, thank you for looking into it. On llama-server version: 10108 (0a50d99) with load-mode = none. This problem did not exist. Also I merged your PR with 10108 (0a50d99) that is also working fine for both models mentioned in opening post.
When I initially shared the problem I was using mmap = 0 and mlock = 1. With new introduction of load mode when using load-mode = mlock I cannot load qwen3 next. So I checked that with load-mode = none only.
marcobarreto007 commentedon Aug 16, 2026
Corroborating report: Gated DeltaNet (Qwen3.5-9B) on NVIDIA Vulkan — device lost + silent state corruption at long context
Setup: Windows 11, Quadro M4000 8GB, driver 582.78, llama.cpp b10448 (also reproduced on b10452), model Huihui-Qwen3.5-9B-abliterated Q6_K, ngl 99, temperature 0.
A/B matrix (controlled, same prompt):
-fa 1(default), short context (~500 tok)-fa 0, short context-fa 0, long context ~6.5-7.5k tok, KV on GPU (default)vk::Queue::submit: ErrorDeviceLost→ server crash (exit 9)-fa 0, long context,--no-kv-offload(KV in RAM)Related: #20462 (device lost bisected to PR #20334), #20610 / #20662 (subgroup rewrite gibberish), #20097 (fa on → gibberish; fa off works).
Full logs available on request. No Pascal-era GPU + GDN reports existed before this one, so sharing in case it helps narrow the subgroup/scheduler race.
Adds testcase addressing ggml-org#23321
Adds testcase addressing ggml-org#23321