Repository navigation

WIP: Strix Halo Vulkan stack (FA KV-quant, mmid, dense GEMM, delta-net prefill, DSv4 sparse attention) - #13

Closed

Nathanw1014 wants to merge 115 commits into

masterfrom

staging/vulkan-stack-20260830

Closed

WIP: Strix Halo Vulkan stack (FA KV-quant, mmid, dense GEMM, delta-net prefill, DSv4 sparse attention)#13
Nathanw1014 wants to merge 115 commits into
masterfrom
staging/vulkan-stack-20260830

Commits on Aug 27, 2026
Commits on Aug 28, 2026
- tests : run test-save-load-state across all architectures (#27755)
  ggerganov
  authored
  4e97ac8
- metal : add fa-vec tunings for M4 Pro (#27824)
  infinitewarp
  authored
  6d6b697
- metal : add fa-vec tunings for M3 Max, M5 and M5 Pro (#27863)
  ggerganov
  authored
  8963a9b
- sycl: bind the f16 KV cache in place for the oneDNN SDPA path (#27468)
  Titaniumtown
  authored
  be87620
- sycl: use TILE for quantized KV decode on BMG (#26689)
  johnkarlhill
  authored
  d077b4c
- convert: prevent ndarray conversion in LazyChunkedTensor (#27869)
  ngxson
  authored
  b19cbe9
- OpenVINO: Update OV to 2026.3.1, whisper.cpp support, Qwen3.5 on NPU, and new ops (#27843)
  
  authored
  511f9c1
- metal : add fa-vec tunings for M4 (#27875)
  Strongtut
  authored
  f5e85d4
- context : disable non-fused GDN and LID ops (#27877)
  ggerganov
  authored
  8663224
- Vulkan: add hoisting support for row IDs and expert count in shaders (#26686)
  
  ravel7524
  and
  jeffbolznv
  authored
  90c26fc
- ggml : fix conv_transpose_2d for multiple batches (#26132)
  
  tekinertekin
  and
  ggerganov
  authored
  a43c398
- vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize (#27812)
  
  Eric-A-Stalee
  and
  jeffbolznv
  authored
  b387ddf
- model: qwen4exp: reduce number of graph splits (#27880)
  ngxson
  authored
  6fe7498
- bench: add --tensor-read-lazy (#27881)
  ngxson
  authored
  50f068f
- snapdragon: python SDK setup (Windows) (#27903)
  
  kurquhar
  and
  Kristopher Urquhart
  authored
  d7bd3bf
Commits on Aug 29, 2026
Commits on Aug 30, 2026
- ggml: allow passing alloc dependencies in graph_optimize (#27301)
  am17an
  authored
  57291f2
- memory : copy Hadamard matrix to k_rot tensor only if it has buffer assigned to prevent crashes during context shift of unquantized K cache (#27967)
  
  authored
  bdf3955
- metal : fix null-pipeline crash for F16 src1 mul_mat/mul_mat_id (#25648)
  QuintinShaw
  authored
  d882575
- sycl: split long rows in TOP_K instead of one work-group per row (#27847)
  Titaniumtown
  authored
  370cb12
- hexagon: support for device discovery and create sessions on demand (#27785)
  
  authored
  9e54e68
- rpc : fix pre-rdma macOS versions (#27815)
  ryan5rdx
  authored
  2bf0415
- vendor : update cpp-httplib to 0.54.0 (#27919)
  cabelo
  authored
  dc7aecf
- metal : Add fa-vec tuning for M3 Pro (#27963)
  addianto
  authored
  b8b743c
- metal: add fa-vec tunings for M3 Ultra (#27999)
  ngladitz
  authored
  093adb2
- rpc: fix apple rdma error spew on teardown (#27908)
  ryan5rdx
  authored
  742347b
- ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (#27960)
  ggerganov
  authored
  73f56d1
- common: rename --tensor-read-lazy to --lazy-mode, add -lzm shorthand (#27969)
  ggerganov
  authored
  bebc935
- experiment: dequant-once FA scratch for all KV quant types (q4_0/q4_1/q5_0/q5_1/iq4_nl)
  Nathanw1014
  committed
  816c7bd
- vulkan : gate FA dequant-once scratch on device-local capacity
  Nathanw1014
  committed
  0aac213
- vulkan: contiguize strided f16 KV for FA prefill (GGML_VK_FA_KV_CONTIG, env-gated)
  
  Nathanw1014
  and
  claude
  committed
  c8230e7
- tests: dense-permuted K/V option + Strix FA prefill perf/probe cases
  
  Nathanw1014
  and
  claude
  committed
  40aaa49
- vulkan: enable the f16 KV contiguize pass by default (GGML_VK_FA_KV_CONTIG=0 opts out)
  
  Nathanw1014
  and
  claude
  committed
  e7a0e58
- vulkan: single source of truth for native FA K/V types + non-native hard-gate
  
  Nathanw1014
  and
  claude
  committed
  bf54e45
- vulkan: hoist the coopmat1 FA P-fragment load out of the hsv_tile loop
  Nathanw1014
  committed
  66e7348
- tests: dense-permuted iq4_nl FA cases (dequant-once route coverage)
  
  Nathanw1014
  and
  claude
  committed
  6a6896c
- vulkan: store coopmat1 FA Psh query-major so the GEMM2 A load vectorizes
  Nathanw1014
  committed
  306eecf
- vulkan: bill the FA K/V contiguize pass on its own perf-logger line
  Nathanw1014
  committed
  48407ef
- vulkan: pin a 32-wide subgroup for coopmat1 FA where narrowing is free
  Nathanw1014
  committed
  d7124e9
- vulkan: scale the FA MMQ dot product in fp32 before narrowing
  
  Nathanw1014
  and
  claude
  committed
  c4444f9
- vulkan: enable the coopmat1 FA wave32 narrowing rule by default
  
  Nathanw1014
  and
  claude
  committed
  a94369d
- vulkan: mul_mat_id per-expert-n tile selection (Stage 2a, env-gated)
  
  Nathanw1014
  and
  claude
  committed
  41d17f1
- vulkan: mul_mat_id small-tile shape probes (env-gated)
  
  Nathanw1014
  and
  claude
  committed
  7f0c243
- vulkan: mul_mat_id taller medium tile probe (GGML_VK_MMID_M128, env-gated)
  
  Nathanw1014
  and
  claude
  committed
  afedbab
- vulkan: mmid wave32 probe (GGML_VK_MMID_WAVE32, env-gated)
  
  Nathanw1014
  and
  claude
  committed
  7722eb4
- vulkan: mul_mat_id f16-B probe (GGML_VK_MMID_F16B, env-gated)
  
  Nathanw1014
  and
  claude
  committed
  0986850
- vulkan: guard mmid f16-B path on pipeline existence (Q2_0 fallback)
  Nathanw1014
  committed
  633ab28
- tests: cover MMQ tile boundaries in MUL_MAT and MUL_MAT_ID
  
  Nathanw1014
  and
  claude
  committed
  bc235d8
- vulkan: create coopmat2 mul_mat_id pipelines with the real param count
  
  Nathanw1014
  and
  claude
  committed
  3df1fd5
- vulkan: enable the Strix mmid tile gates by default
  
  Nathanw1014
  and
  claude
  committed
  266166f
- vulkan : optional f16 B operand for quantized MUL_MAT on coopmat1
  Nathanw1014
  committed
  fcf2f38
- vulkan : add auto mode to GGML_VK_DENSE_F16B
  Nathanw1014
  committed
  74b3efa
- vulkan: run the quantised dense coopmat pipelines at wave32
  Nathanw1014
  committed
  bc3638c
- vulkan: tune the coopmat matmul LDS pad per path on RADV
  Nathanw1014
  committed
  beea09e
- vulkan: gate the RADV coopmat pad 2 on driver >= 25.3
  Nathanw1014
  committed
  73608d2
- vulkan: four env-gated Strix Halo prefill fixes for delta-net MoE
  
  Nathanw1014
  and
  claude
  committed
  b9e43c5
- vulkan: reject ne[3] > 1 in the mul_mat_id scale epilogue
  
  Nathanw1014
  and
  claude
  committed
  7a1eb6d
- vulkan : default the transposed-concat path on
  Nathanw1014
  committed
  0d542b8
- vulkan: flush pending compute ctx before perf logger timestamps
  Nathanw1014
  committed
  ce93d29
- vulkan: bound command buffers by memory traffic, not just flops
  Nathanw1014
  committed
  8a8c218
- ggml: cut backend splits on the input constant, not the grown capacity
  
  Nathanw1014
  and
  claude
  committed
  9125bb2
- vulkan : take the copy path for bulk UMA reads from uncached mappings
  Nathanw1014
  committed
  e4615cc
- vulkan: DeepSeek V4 lightning indexer kernels + indexed sparse FA
  
  gaetan-puleo
  authored and
  Nathanw1014
  committed
  69e3052
- vulkan: harden the sparse-FA shader and document the top-k API
  
  Nathanw1014
  and
  claude
  committed
  aa04dcc
- tests: sparse top-k FA parity + perf coverage (V4 CSA shape)
  
  Nathanw1014
  and
  claude
  committed
  5046bf6
- vulkan: gather-to-compact sparse decode FA for DeepSeek V4 top-k selection
  
  Nathanw1014
  and
  claude
  committed
  8a15795
- vulkan: fused DeepSeek V4 hyper-connection ops (HC pre / comb / post)
  
  Nathanw1014
  and
  claude
  committed
  a27636c
- llama: keep DeepSeek lightning-indexer key cache f16 under quantized -ctk
  Nathanw1014
  committed
  631fd48
- llama: contiguize grouped o-proj input for small multi-token batches (DSv4)
  Nathanw1014
  committed
  0e71642
- vulkan: accelerate DeepSeek V4 sparse prefill FA
  
  Mushoz
  authored and
  Nathanw1014
  committed
  aca6680
- vulkan: split sparse prefill attention
  
  Mushoz
  authored and
  Nathanw1014
  committed
  10c5457
- vulkan: tile sparse prefill scratch
  
  Mushoz
  authored and
  Nathanw1014
  committed
  aa5085f
- vulkan: reuse sparse FA probability fragments
  
  Mushoz
  authored and
  Nathanw1014
  committed
  8a4a20b
- vulkan: cache sparse FA masks per key block
  
  Mushoz
  authored and
  Nathanw1014
  committed
  13bedfe
- vulkan: fix DeepSeek V4 sparse split attention with multiple sequences
  
  Nathanw1014
  and
  claude
  committed
  2c79376
- vulkan: harden the sparse FA split path and drop its debug scaffolding
  
  Nathanw1014
  and
  claude
  committed
  9c950c5
- test-backend-ops: cover sparse top-k FA with more than one sequence
  
  Nathanw1014
  and
  claude
  committed
  09f6a7d
- vulkan: let sparse FA query tiling and multiple sequences coexist
  
  Nathanw1014
  and
  claude
  committed
  932245a
- vulkan: parallelize DSV4 Lightning Indexer prefill
  
  Mushoz
  authored and
  Nathanw1014
  committed
  ff7ac3d
- vulkan: extend DeepSeek V4 gather-to-compact to small batches
  
  Nathanw1014
  and
  claude
  committed
  93f18c3
- vulkan: record the resource limits behind the two DSV4 prefill kernels
  
  Nathanw1014
  and
  claude
  committed
  4c35864
- vulkan: deduplicated union for DeepSeek V4 small-batch decode
  
  Nathanw1014
  and
  claude
  committed
  7532a73
- vulkan: gate the DeepSeek V4 small-batch union on the measured union size
  
  Nathanw1014
  and
  claude
  committed
  0ff9e37
- vulkan: measure the V4 union on real draft tokens, and correct the fixture calibration
  
  Nathanw1014
  and
  claude
  committed
  ce9fef7
- vulkan: default the DeepSeek V4 small-batch union on
  
  Nathanw1014
  and
  claude
  committed
  7dfe8c8
- vulkan: let the DeepSeek V4 small-batch gather serve quantised K/V
  
  Nathanw1014
  and
  claude
  committed
  8851c37
- vulkan: dequantise q8_0 K/V inside the DeepSeek V4 small-batch gather
  
  Nathanw1014
  and
  claude
  committed
  6b7c998
- vulkan: decode q4_0 in the V4 gather too, with the real element mapping
  
  Nathanw1014
  and
  claude
  committed
  2cd621b
- vulkan : restore the unrolled row copy in the DSV4 gather shaders
  Nathanw1014
  committed
  7d45323
- vulkan: decode quantised K/V inside the DeepSeek V4 per-token gather
  
  Nathanw1014
  and
  claude
  committed
  14f7c55
- vulkan: dequantise the cache for the DeepSeek V4 sparse prefill
  
  Nathanw1014
  and
  claude
  committed
  a2b7675
- vulkan: use small Lightning Indexer CM for batches 4-15
  
  pepuscz
  authored and
  Nathanw1014
  committed
  ca50763
- vulkan: hoist the Lightning Indexer K fragments out of the head loop
  
  Nathanw1014
  and
  claude
  committed
  1ca0f9a
- vulkan: route the whole small-batch Lightning Indexer window to the decode CM shader
  
  Nathanw1014
  and
  claude
  committed
  06b89c3
- vulkan: hoist the Lightning Indexer K fragments out of the CM head loop
  
  Nathanw1014
  and
  claude
  committed
  849aca6
- vulkan: support arbitrary Lightning Indexer head counts via specialization constant
  
  Nathanw1014
  and
  claude
  committed
  e85148e
- vulkan: N_HEAD spec constant for the scalar-64 lightning indexer
  Nathanw1014
  committed
  46b7ed7
- Merge branch 'vulkan/dsv4' into stage/strix-stacked
  Nathanw1014
  committed
  b19d076
- Merge branch 'vulkan/fa-coopmat1-micro' into stage/strix-stacked
  Nathanw1014
  committed
  d24b48c
- Merge branch 'vulkan/mmid-stack' into stage/strix-stacked
  Nathanw1014
  committed
  3f32113
- Merge branch 'vulkan/dense-gemm' into stage/strix-stacked
  Nathanw1014
  committed
  4366d29
- Merge branch 'vulkan/deltanet-prefill' into stage/strix-stacked
  Nathanw1014
  committed
  f34e16a
- Merge branch 'vulkan/strix-misc' into stage/strix-stacked
  Nathanw1014
  committed
  a6e38dd

Commits

Commits on Aug 27, 2026

Commits on Aug 28, 2026

Commits on Aug 29, 2026

Commits on Aug 30, 2026