Repository navigation

sync : ggml - #4034

Merged

ggerganov merged 104 commits into

masterfrom

sync-ggml-26-09-04

Sep 4, 2026

Merged

sync : ggml#4034
ggerganov merged 104 commits into
masterfrom
sync-ggml-26-09-04

Commits on Sep 4, 2026
- metal : null-check buffer alloc to fix OOM crash (llama/25371)
  
  ykhrustalev
  and
  ggerganov
  committed
  be12d39
- rpc: support apple RDMA as an RPC transport (llama/26421)
  
  authored and
  ggerganov
  committed
  e820c28
- kleidiai: Rework KleidiAI Build System/Integration (llama/26077)
  
  JonathanC-ARM
  authored and
  ggerganov
  committed
  482956e
- ggml-meta: propagate buffer usage and call init on the new tensors (llama/27586)
  
  max-krasnyansky
  authored and
  ggerganov
  committed
  8df657a
- ggml-metal: add chunked SSD MMA for Mamba-2 prefill optimization (llama/26647)
  
  authored and
  ggerganov
  committed
  8c0adb0
- cuda: unblock mmq for MoE on sm_60 (llama/26264)
  
  dfriehs
  authored and
  ggerganov
  committed
  9d8e6b9
- rpc : implement event and async backend APIs (llama/18626)
  
  rgerganov
  authored and
  ggerganov
  committed
  82f5f85
- Implemented vulkan cross_entropy_loss and cross_entropy_loss_back (llama/27216)
  
  PranavUttarkar
  authored and
  ggerganov
  committed
  0a02697
- vulkan: warptiles currently assume warp sizes <= 64, clamp to work around larger warps (llama/27726)
  
  0cc4m
  authored and
  ggerganov
  committed
  5271734
- hexagon: support for multi-NPU devices (IQ9, IQ10) and fully asynchronous backend (llama/26501)
  
  max-krasnyansky
  authored and
  ggerganov
  committed
  3fea10d
- metal : fix memory leaks due to missing autoreleasepools (llama/27758)
  
  nikwen
  authored and
  ggerganov
  committed
  a5db1d6
- Feature: Added LIGHTNING_INDEXER support for Deepseek V4 ops on Vulkan Backend (llama/27453)
  
  shenron0101
  authored and
  ggerganov
  committed
  55ab1e5
- opencl: add bin kernels `kernel_gemm_moe_q4_0_q8_1_dp4a_bin`, `kernel_gemm_moe_mxfp4_q8_1_dp4a_bin` (llama/27768)
  
  shawngu-quic
  authored and
  ggerganov
  committed
  8529971
- hex-unary: fix RMS_NORM_MUL weight-offset bugs for grouped/broadcast norms (llama/27798)
  
  aparmp-quic
  authored and
  ggerganov
  committed
  a0614d9
- ggml-hexagon: add HTP unary ops for ABS and LOG (llama/27786)
  
  cqderek
  authored and
  ggerganov
  committed
  b6571e4
- metal : add fa-vec tunings for M4 Pro (llama/27824)
  
  infinitewarp
  authored and
  ggerganov
  committed
  ff38b98
- metal : add fa-vec tunings for M3 Max, M5 and M5 Pro (llama/27863)
  ggerganov
  committed
  530e3f4
- sycl: bind the f16 KV cache in place for the oneDNN SDPA path (llama/27468)
  
  Titaniumtown
  authored and
  ggerganov
  committed
  97d0da2
- sycl: use TILE for quantized KV decode on BMG (llama/26689)
  
  johnkarlhill
  authored and
  ggerganov
  committed
  fa4d244
- OpenVINO: Update OV to 2026.3.1, whisper.cpp support, Qwen3.5 on NPU, and new ops (llama/27843)
  
  authored and
  ggerganov
  committed
  7f78e1b
- metal : add fa-vec tunings for M4 (llama/27875)
  
  Strongtut
  authored and
  ggerganov
  committed
  0a15087
- Vulkan: add hoisting support for row IDs and expert count in shaders (llama/26686)
  
  authored and
  ggerganov
  committed
  ba99c09
- ggml : fix conv_transpose_2d for multiple batches (llama/26132)
  
  tekinertekin
  and
  ggerganov
  committed
  caea96f
- vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize (llama/27812)
  
  authored and
  ggerganov
  committed
  4c38040
- vulkan: Change mul_mat_id to pad K rather than N (llama/27925)
  
  jeffbolznv
  authored and
  ggerganov
  committed
  d501a0a
- metal : add fa-vec tunings for M1 Max (llama/27932)
  
  jhen0409
  authored and
  ggerganov
  committed
  590fe18
- vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs (llama/27526)
  
  jeffbolznv
  authored and
  ggerganov
  committed
  e644752
- sycl: make --fit respect --fit-target better (llama/27629)
  
  nicois
  authored and
  ggerganov
  committed
  325c8d1
- metal : add remaining fa-vec tunings for M4 Pro (llama/27915)
  
  nikwen
  authored and
  ggerganov
  committed
  308fa4f
- metal : assert shared memory padding (llama/27951)
  ggerganov
  committed
  285f1ff
- opencl: use a better matmul path on two Adreno GPU generations (llama/27640)
  
  authored and
  ggerganov
  committed
  2a11026
- metal : add fa-vec tunings for M2 (llama/27940)
  
  ring2003
  authored and
  ggerganov
  committed
  b33bbc5
- ggml: allow passing alloc dependencies in graph_optimize (llama/27301)
  
  am17an
  authored and
  ggerganov
  committed
  c969c68
- metal : fix null-pipeline crash for F16 src1 mul_mat/mul_mat_id (llama/25648)
  
  QuintinShaw
  authored and
  ggerganov
  committed
  c68f205
- sycl: split long rows in TOP_K instead of one work-group per row (llama/27847)
  
  Titaniumtown
  authored and
  ggerganov
  committed
  3d4e0e9
- hexagon: support for device discovery and create sessions on demand (llama/27785)
  
  authored and
  ggerganov
  committed
  3ad8b9b
- rpc : fix pre-rdma macOS versions (llama/27815)
  
  ryan5rdx
  authored and
  ggerganov
  committed
  5e49459
- metal : Add fa-vec tuning for M3 Pro (llama/27963)
  
  addianto
  authored and
  ggerganov
  committed
  b66593e
- metal: add fa-vec tunings for M3 Ultra (llama/27999)
  
  ngladitz
  authored and
  ggerganov
  committed
  1e0f382
- rpc: fix apple rdma error spew on teardown (llama/27908)
  
  ryan5rdx
  authored and
  ggerganov
  committed
  43acf3d
- ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (llama/27960)
  ggerganov
  committed
  4b2243a
- hip : optimize Q2_0 dot-product path for gfx1201 (llama/26753)
  
  LunalFresh
  authored and
  ggerganov
  committed
  e5c9e3e
- hip: tune rdna 3 mmq config (llama/26284)
  
  itterative
  authored and
  ggerganov
  committed
  35d9e22
- CUDA: use the fast mm_ids_helper path for any n_expert_used (llama/27978)
  
  ServeurpersoCom
  authored and
  ggerganov
  committed
  e900a73
- ggml: add SWIGLU_CLAMP (llama/27930)
  
  am17an
  authored and
  ggerganov
  committed
  e9583f0
- ggml : fix ggml_backend_buft_get_alloc_size() guard (llama/28038)
  ggerganov
  committed
  749683d
- rpc: avoid serializing buffers from other servers (llama/26500)
  
  hmirin
  and
  ggerganov
  committed
  4089fa6
- metal : add remaining Q4_1/Q5_0/Q5_1 fa-vec tunings for M2 (llama/28017)
  
  ring2003
  authored and
  ggerganov
  committed
  e5c96ca
- hexagon: fix CPY fence bug (llama/28033)
  
  yshsharke
  authored and
  ggerganov
  committed
  01ebd22
- vulkan: top_k radix select for k >= 1024 for Qwen 3.8 Flash Next (llama/28032)
  
  0cc4m
  authored and
  ggerganov
  committed
  db00b01
- ggml : add MUL_MAT to the list of ops that may need additional memory (for WebGPU) (llama/28071)
  
  authored and
  ggerganov
  committed
  96dddd8
- vulkan: tune mat-vec rows for batched inference on Strix Halo (llama/27909)
  
  SimonTeixidor
  authored and
  ggerganov
  committed
  6ce7b89
- sycl : Enhance to get the free memory of Intel GPU (llama/27968)
  
  arthw
  authored and
  ggerganov
  committed
  76a51e8
- CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were restricted to 1 token (llama/27621)
  
  ynankani
  authored and
  ggerganov
  committed
  b0f4bc0
- metal : add fa-vec tunings for M1 (llama/28078)
  
  nikwen
  authored and
  ggerganov
  committed
  7614a4c
- ROCm: add radix TOP_K for long rows (llama/27466)
  
  jadenmach2
  authored and
  ggerganov
  committed
  c1be45b
- webgpu : avoid crash when offset is not multiple of 4 in WebGPU ggml_backend_tensor_get() implementation (llama/28045)
  
  authored and
  ggerganov
  committed
  c648b9a
- opencl: tune the quant paths for Intel Xe-LP GPUs to improve its TG and PP performance (llama/26438)
  
  wanghqc
  authored and
  ggerganov
  committed
  088c603
- metal : add top-k radix implementation (llama/28073)
  ggerganov
  committed
  c6934d0
- AVX2: Speed up large batch size prompt processing of IQ models (llama/27402)
  
  bartowski1182
  and
  ggerganov
  committed
  2f608ab
- metal : add concat support for quantized types (llama/28116)
  ggerganov
  committed
  dbc40ef
- CUDA: XOR swizzle flash attn K,V smem fp16 tiles (llama/25635)
  
  ynankani
  authored and
  ggerganov
  committed
  f22bb2e
- metal : add fa-vec tunings for M1 Ultra (llama/28088)
  
  ozgursoy
  authored and
  ggerganov
  committed
  8e54c65
- metal: enable Metal 4.0 tensor API on M5+/A19+ (llama/27461)
  
  JamesFranc
  authored and
  ggerganov
  committed
  5032008
- sycl : support limit max alloc memory within 2GB for host-pinned memory (llama/27559)
  
  arthw
  authored and
  ggerganov
  committed
  4f3a2a4
- metal : add fa-vec tuning for M2 Max (llama/28015)
  ggerganov
  committed
  870db2a
- metal : fix more leaks due to missing autoreleasepools (llama/27883)
  
  authored and
  ggerganov
  committed
  a245a8f
- metal : add fa-vec tunings for A18 Pro (MacBook Neo) (llama/28152)
  
  jhen0409
  authored and
  ggerganov
  committed
  8cca1a3
- metal : add fa-vec tuning for M2 Pro (llama/28122)
  
  lstolcman
  authored and
  ggerganov
  committed
  5f07f85
- sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 1280 (llama/28016)
  
  philip-jingxin
  authored and
  ggerganov
  committed
  408faaa
- Revert "sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 12…" (#28184)
  
  Titaniumtown
  authored and
  ggerganov
  committed
  f162a19
- cuda: fuse MoE weighted expert reduction (llama/25952)
  
  anujj
  authored and
  ggerganov
  committed
  2c48678
- metal : add metallib build support for xcframework (llama/28163)
  
  jhen0409
  authored and
  ggerganov
  committed
  fcc2fee
- hexagon: add missing FARF logs for cpy/get_rows/set_rows/gdn ops (llama/28217)
  
  trivikram-reddy1
  authored and
  ggerganov
  committed
  c94921f
- opencl: fix out‐of‐bound reads in the Adreno image kernels (#27632)
  
  wanghqc
  authored and
  ggerganov
  committed
  35133c9
- ggml-cpu : conditionally add SpacemiT IME kernel sources (llama/27961)
  
  alanhc
  authored and
  ggerganov
  committed
  d57ae98
- vulkan : only request VK_KHR_shader_bfloat16 extension if supported (llama/28155)
  
  madsmtm
  authored and
  ggerganov
  committed
  a9e5861
- vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec (llama/27449)
  
  authored and
  ggerganov
  committed
  1c7d35e
- hexagon: MUL_MAT and MUL_MAT_ID fusion and fixes (llama/28202)
  
  max-krasnyansky
  authored and
  ggerganov
  committed
  dc70853
- ggml: avoid KleidiAI buffer type init on dispatch (llama/27891)
  
  authored and
  ggerganov
  committed
  c2b4007
- CUDA + ggml: add sparse-fa for DSV4/GLM (llama/27970)
  
  am17an
  authored and
  ggerganov
  committed
  519df61
- ggml-cuda : remove unused vars (llama/28235)
  
  angt
  authored and
  ggerganov
  committed
  4d343d7
- metal : fix memory query under low-memory conditions (llama/27701)
  
  committed
  3a1c7d6
- metal : add fa-vec tunings for M3 (llama/28236)
  
  init-22
  authored and
  ggerganov
  committed
  1bdda1e
- ggml-hexagon: add F16 support for unary ops (llama/28228)
  
  cqderek
  authored and
  ggerganov
  committed
  37f0f44
- finetune: fix no KV cache (llama/27199)
  
  ngxson
  authored and
  ggerganov
  committed
  e560569
- sycl: reduce redundant work in Q4_K multi-column MMVQ (llama/27062)
  
  authored and
  ggerganov
  committed
  a704770
- sycl : enhance the api to support peer-to-peer copy (llama/27550)
  
  arthw
  authored and
  ggerganov
  committed
  f24a386
- vulkan: fix FA dequant path engagement (llama/28190)
  
  Nathanw1014
  authored and
  ggerganov
  committed
  47d348a
- CUDA: Allow concurrent streams per split for multi-GPU (llama/28198)
  
  authored and
  ggerganov
  committed
  25350b5
- metal : fix glu dispatch with ne00 = 1 (llama/28306)
  ggerganov
  committed
  d55d345
- metal : add sparse FA (llama/28098)
  ggerganov
  committed
  4dd48dd
- tune MMVQ to MMQ crossover for SM87 (llama/28285)
  
  kbenkhaled
  authored and
  ggerganov
  committed
  0a4a95c
- opencl: quant lm_head / decode GEMV and medium-batch GEMM optimizations (speculative decoding/MTP) (llama/26477)
  
  wanghqc
  authored and
  ggerganov
  committed
  d784add
- SYCL: Refactor GGML_SYCL_ENABLE_MKL_FA to global var (llama/26863)
  
  johnkarlhill
  authored and
  ggerganov
  committed
  36f170e
- sycl: fuse rms_norm+mul+add and add+add residual chains (llama/27610)
  
  newjordan
  authored and
  ggerganov
  committed
  d1e0e64
- ggml-cpu(s390x) : fix q5_1 uninitialized v_acc (llama/28332)
  
  taronaeo
  authored and
  ggerganov
  committed
  e1bbe40
- ggml : remove GGML_CUDA_PEER_MAX_BATCH_SIZE (llama/28177)
  
  angt
  authored and
  ggerganov
  committed
  f32e6fa
- ggml : don't crash when backend search path can't be read (llama/28271)
  
  angt
  authored and
  ggerganov
  committed
  1b37bea
- ggml : rename and make private ggml_op_alloc_size_may_expand() (ggml/0)
  ggerganov
  committed
  e2389eb
- ggml : replace compile definitions with version.h.in (llama/28364)
  
  danbev
  authored and
  ggerganov
  committed
  140e57a
- metal : add remaining fa-vec tunings for M3 Max (llama/28373)
  
  nikwen
  authored and
  ggerganov
  committed
  11d4eec
- ggml : bump version to 0.23.0 (ggml/1618)
  ggerganov
  committed
  a937f4e
- sync : ggml
  ggerganov
  committed
  52a939a

Commits

Commits on Sep 4, 2026