Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
401 commits
Select commit Hold shift + click to select a range
b5f6357
hip: select RDNA3.5 mul_mat_id tile width by rows-per-expert range
gaetan-puleo Sep 3, 2026
bdeb626
CUDA/HIP: use WMMA flash attention for D=256 prefill on RDNA3.5
gaetan-puleo Sep 3, 2026
e952926
llama: synchronize before mutating graph state
gaetan-puleo Sep 3, 2026
b6765cd
tests: avoid ambiguous top-k MoE scores
gaetan-puleo Sep 3, 2026
5f7cbd9
hip: optimize RDNA3.5 MoE inference paths
gaetan-puleo Sep 4, 2026
68ad200
hip: address PR review correctness issues
gaetan-puleo Sep 4, 2026
67e93a6
hip: tune Ling MoE inference on RDNA3.5
gaetan-puleo Sep 4, 2026
99cf804
hip: fix RDNA3.5 fused quantization edge cases
gaetan-puleo Sep 5, 2026
42f1e32
qwen4exp: --ngram-on-disk, forge ngram on ssd (#3)
dzannotti Aug 27, 2026
c752bb3
qwen4exp: MTP draft head for speculative decoding (#9)
LaurentZuijdwijk Aug 30, 2026
9943b67
qwen4exp: fix --ngram-on-disk, which crashes on every model (#8)
dzannotti Aug 30, 2026
935a676
server : add a `hidden` preset option to omit a model from GET /model…
LaurentZuijdwijk Aug 31, 2026
a356cdf
speculative-prefill : port upstream PR 27692
Sep 2, 2026
25f0abb
vulkan: driver-gate the coopmat LDS stride pad (#14)
LaurentZuijdwijk Sep 3, 2026
b82b3a3
Halo/vulkan topk radix (#17)
LaurentZuijdwijk Sep 3, 2026
a8475b5
vulkan: replace the EngramHalo radix top-k with upstream's radix select
Nathanw1014 Sep 4, 2026
3311813
vulkan: top_k radix select for k >= 1024 for Qwen 3.8 Flash Next (#28…
0cc4m Aug 31, 2026
88c1cdd
vulkan: deterministic slot assignment in top_k radix select
Nathanw1014 Sep 3, 2026
3d6971c
models: use flash-linear-attention's l2norm for gated delta net q/k
danielhanchen Aug 31, 2026
592a5ca
kv-cache: zero freed cells so masked-out rows never carry stale K/V
Nathanw1014 Sep 3, 2026
0399d7a
hip: fix ROCm 7.2 Flash Attention Werror build
gaetan-puleo Sep 5, 2026
cc4cbc2
hip: avoid compiling RDNA3.5 Q8 FA on CDNA
gaetan-puleo Sep 5, 2026
7620f12
readme : add Halo Box fork section (#19)
LaurentZuijdwijk Sep 4, 2026
6c22fb7
Update README to clarify features and fix formatting
LaurentZuijdwijk Sep 5, 2026
4422659
Halo/upstream sync 2026 09 05 (#20)
LaurentZuijdwijk Sep 6, 2026
e271ea3
readme : merge the halo-box/llama.cpp README into this fork's
LaurentZuijdwijk Sep 6, 2026
de9ed96
readme : align the Halo Box section with strix-llama.cpp (#21)
LaurentZuijdwijk Sep 6, 2026
5a83ffa
readme : adopt the strix-llama.cpp README for this fork (#22)
LaurentZuijdwijk Sep 6, 2026
a50bc8e
Halo/scheduler uma ring 27311 (#23)
gaetan-puleo Sep 6, 2026
908b07f
ROCmFPx: port the AMD-native FPx tensor types (CPU + Vulkan)
Aug 20, 2026
88b0aad
vulkan: integer-dot (MMQ/MMVQ) paths for the ROCmFPx types
Aug 20, 2026
f7fd302
vulkan: fix ROCmFPx mat-vec cost at batch 3-8
Aug 21, 2026
6e4ecf1
tests : set quantization error budgets for the ROCmFPx 2/3-bit types
Aug 31, 2026
7776dcf
rocmfpx : credit charlie12345/ROCmFPX as the origin of the format
Aug 31, 2026
1e792a6
llama : give Q2_0_ROCMFPX the ftype id ciru-ai/ROCmFPX uses (122)
LaurentZuijdwijk Sep 6, 2026
9066331
tests : run the ROCmFPx codec self-tests under ctest
LaurentZuijdwijk Sep 6, 2026
f095c74
gguf-py : dequantize every ROCmFPx type in Python, and test it against C
LaurentZuijdwijk Sep 6, 2026
8186503
llama : keep Q2_0_ROCMFPX ftype 119, matching the ROCmFPX origin and …
LaurentZuijdwijk Sep 6, 2026
be025b5
rocmfpx : drop the unused HIP helpers and describe what is actually here
LaurentZuijdwijk Sep 6, 2026
f97931f
common : do not offer ROCmFPx types as KV cache types
Nathanw1014 Sep 7, 2026
dd9e25f
readme : ROCmFPx types are weight formats, not KV-cache types
LaurentZuijdwijk Sep 7, 2026
1baadb1
vulkan : only chunk batched mat-vec for the quant types that were mea…
LaurentZuijdwijk Sep 6, 2026
bd8cebf
speculative : give MTP targets their recurrent rollback slots back
LaurentZuijdwijk Sep 6, 2026
f8a3a0c
experiment: dequant-once FA scratch for all KV quant types (q4_0/q4_1…
Nathanw1014 Jul 13, 2026
a4c513a
vulkan : gate FA dequant-once scratch on device-local capacity
Nathanw1014 Jul 14, 2026
f29c972
vulkan: contiguize strided f16 KV for FA prefill (GGML_VK_FA_KV_CONTI…
Nathanw1014 Jul 28, 2026
0e0dc44
tests: dense-permuted K/V option + Strix FA prefill perf/probe cases
Nathanw1014 Jul 28, 2026
3012884
vulkan: enable the f16 KV contiguize pass by default (GGML_VK_FA_KV_C…
Nathanw1014 Jul 28, 2026
e1abb25
vulkan: single source of truth for native FA K/V types + non-native h…
Nathanw1014 Jul 28, 2026
e174c39
tests: dense-permuted iq4_nl FA cases (dequant-once route coverage)
Nathanw1014 Jul 28, 2026
f6162b4
vulkan: bill the FA K/V contiguize pass on its own perf-logger line
Nathanw1014 Jul 31, 2026
144b2b4
vulkan: scale the FA MMQ dot product in fp32 before narrowing
Nathanw1014 Aug 9, 2026
bf6d0f0
vulkan: DeepSeek V4 lightning indexer kernels + indexed sparse FA
gaetan-puleo Aug 1, 2026
1c78755
vulkan: harden the sparse-FA shader and document the top-k API
Nathanw1014 Aug 1, 2026
cf1f991
tests: sparse top-k FA parity + perf coverage (V4 CSA shape)
Nathanw1014 Aug 1, 2026
8064cbb
vulkan: gather-to-compact sparse decode FA for DeepSeek V4 top-k sele…
Nathanw1014 Aug 1, 2026
8e3ea99
vulkan: fused DeepSeek V4 hyper-connection ops (HC pre / comb / post)
Nathanw1014 Aug 2, 2026
f9c0524
llama: keep DeepSeek lightning-indexer key cache f16 under quantized …
Nathanw1014 Aug 2, 2026
ab5662f
llama: contiguize grouped o-proj input for small multi-token batches …
Nathanw1014 Aug 3, 2026
8956377
vulkan: accelerate DeepSeek V4 sparse prefill FA
Mushoz Aug 12, 2026
86a169c
vulkan: split sparse prefill attention
Mushoz Aug 13, 2026
c4016b5
vulkan: tile sparse prefill scratch
Mushoz Aug 13, 2026
848d1f5
vulkan: reuse sparse FA probability fragments
Mushoz Aug 13, 2026
73af4b5
vulkan: cache sparse FA masks per key block
Mushoz Aug 13, 2026
f3364d6
vulkan: fix DeepSeek V4 sparse split attention with multiple sequences
Nathanw1014 Aug 13, 2026
befade1
vulkan: harden the sparse FA split path and drop its debug scaffolding
Nathanw1014 Aug 13, 2026
f501412
test-backend-ops: cover sparse top-k FA with more than one sequence
Nathanw1014 Aug 13, 2026
4205c5d
vulkan: let sparse FA query tiling and multiple sequences coexist
Nathanw1014 Aug 13, 2026
44aae54
vulkan: parallelize DSV4 Lightning Indexer prefill
Mushoz Aug 13, 2026
2799feb
vulkan: extend DeepSeek V4 gather-to-compact to small batches
Nathanw1014 Aug 13, 2026
a132077
vulkan: record the resource limits behind the two DSV4 prefill kernels
Nathanw1014 Aug 13, 2026
6b3e942
vulkan: deduplicated union for DeepSeek V4 small-batch decode
Nathanw1014 Aug 14, 2026
a7bf25c
vulkan: gate the DeepSeek V4 small-batch union on the measured union …
Nathanw1014 Aug 14, 2026
8323c4b
vulkan: measure the V4 union on real draft tokens, and correct the fi…
Nathanw1014 Aug 14, 2026
c24f7ed
vulkan: default the DeepSeek V4 small-batch union on
Nathanw1014 Aug 14, 2026
389b2c2
vulkan: let the DeepSeek V4 small-batch gather serve quantised K/V
Nathanw1014 Aug 14, 2026
00e44ca
vulkan: dequantise q8_0 K/V inside the DeepSeek V4 small-batch gather
Nathanw1014 Aug 14, 2026
009a142
vulkan: decode q4_0 in the V4 gather too, with the real element mapping
Nathanw1014 Aug 14, 2026
018480a
vulkan : restore the unrolled row copy in the DSV4 gather shaders
Nathanw1014 Aug 17, 2026
ec34789
vulkan: decode quantised K/V inside the DeepSeek V4 per-token gather
Nathanw1014 Aug 20, 2026
31628b3
vulkan: dequantise the cache for the DeepSeek V4 sparse prefill
Nathanw1014 Aug 20, 2026
45aed28
vulkan: use small Lightning Indexer CM for batches 4-15
pepuscz Aug 24, 2026
d9909df
vulkan: hoist the Lightning Indexer K fragments out of the head loop
Nathanw1014 Aug 26, 2026
88f0164
vulkan: route the whole small-batch Lightning Indexer window to the d…
Nathanw1014 Aug 26, 2026
c306728
vulkan: hoist the Lightning Indexer K fragments out of the CM head loop
Nathanw1014 Aug 26, 2026
1ba05d6
vulkan: support arbitrary Lightning Indexer head counts via specializ…
Nathanw1014 Aug 26, 2026
cace4db
vulkan: N_HEAD spec constant for the scalar-64 lightning indexer
Nathanw1014 Aug 30, 2026
0c59b86
vulkan: hoist the coopmat1 FA P-fragment load out of the hsv_tile loop
Nathanw1014 Jul 30, 2026
52a2150
vulkan: store coopmat1 FA Psh query-major so the GEMM2 A load vectorizes
Nathanw1014 Jul 30, 2026
dae3e07
vulkan: pin a 32-wide subgroup for coopmat1 FA where narrowing is free
Nathanw1014 Jul 30, 2026
7a36860
vulkan: enable the coopmat1 FA wave32 narrowing rule by default
Nathanw1014 Aug 30, 2026
bbc3f92
vulkan: mul_mat_id taller medium tile probe (GGML_VK_MMID_M128, env-g…
Nathanw1014 Jul 14, 2026
63a8acb
vulkan: mmid wave32 probe (GGML_VK_MMID_WAVE32, env-gated)
Nathanw1014 Jul 14, 2026
5364231
vulkan: mul_mat_id f16-B probe (GGML_VK_MMID_F16B, env-gated)
Nathanw1014 Jul 14, 2026
a63bd69
tests: cover MMQ tile boundaries in MUL_MAT and MUL_MAT_ID
Nathanw1014 Aug 6, 2026
74e9648
vulkan: enable the Strix mmid tile gates by default
Nathanw1014 Aug 30, 2026
6576c21
vulkan: run the quantised dense coopmat pipelines at wave32
Nathanw1014 Aug 18, 2026
96d20a8
vulkan: four env-gated Strix Halo prefill fixes for delta-net MoE
Nathanw1014 Aug 6, 2026
c19b728
vulkan: reject ne[3] > 1 in the mul_mat_id scale epilogue
Nathanw1014 Aug 6, 2026
414c62b
vulkan : default the transposed-concat path on
Nathanw1014 Aug 17, 2026
fcb822d
vulkan: flush pending compute ctx before perf logger timestamps
Nathanw1014 Aug 2, 2026
14c7341
vulkan: bound command buffers by memory traffic, not just flops
Nathanw1014 Aug 2, 2026
eb07b58
ggml: cut backend splits on the input constant, not the grown capacity
Nathanw1014 Aug 9, 2026
b8c5c8c
vulkan : optional f16 B operand for quantized MUL_MAT on coopmat1
Nathanw1014 Aug 16, 2026
8b795ec
vulkan : add auto mode to GGML_VK_DENSE_F16B
Nathanw1014 Aug 16, 2026
372b90b
vulkan: derive the mul_mat_id tiles from the wave64 dense tiles
Nathanw1014 Sep 8, 2026
faaa1ba
vulkan: keep the coopmat1 FA wave32 pin to multi-row dispatches
Nathanw1014 Sep 8, 2026
2df4282
server : do not re-verify replayed draft tokens after a checkpoint re…
Nathanw1014 Aug 22, 2026
325813e
docs : align issue templates with the fork's AI policy
Sep 8, 2026
0cd2a17
context : size device checkpoint views in quantized elements
arc-uri-el Sep 8, 2026
0be2118
tests : keep the test-rocmfpx assertions alive in Release builds
LaurentZuijdwijk Sep 7, 2026
5a9a3b2
tests : link rocmfpx from ggml-base instead of compiling a second copy
LaurentZuijdwijk Sep 7, 2026
be4e7b1
rocmfpx : let the ROCmFP2 encoder pick the zero scale
LaurentZuijdwijk Sep 7, 2026
a8aeb2b
rocmfpx : bound FP6 scale pruning by 32, the magnitude it can reach
LaurentZuijdwijk Sep 7, 2026
7fa9c7d
rocmfpx : clamp before the integer conversion in the FP6/FP8 encoders
LaurentZuijdwijk Sep 7, 2026
44e37b0
HIP : keep FA head size 192 off the WMMA kernel (#55)
SimonTeixidor Sep 15, 2026
fd65bf0
pwilkin/strix-halo official merge (#63)
pwilkin Sep 16, 2026
8f349bb
HIP : route Q6_K through MMQ up to ne11 = 1024 on RDNA 3.5 (#38)
Nyovelt Sep 17, 2026
33f41ac
chat : enforce string enums in Qwen XML tool arguments (#26)
arc-uri-el Sep 15, 2026
71591a7
sync upstream llama.cpp commits 2026 09 15 (#28)
dzannotti Sep 15, 2026
27abf49
--ngram-on-disk and lazymode merge (#29)
dzannotti Sep 15, 2026
63db99c
docs: add upstream sync procedure (#31)
dzannotti Sep 15, 2026
906439e
docs: record why upstream Vulkan sparse FA was deferred (#65)
dzannotti Sep 17, 2026
d3cdfe7
cuda: enable the BF16 WMMA matmul path only for qwen4exp
dzannotti Sep 17, 2026
0b8c2ce
qsa: maskless multi-sequence block selection
gaetan-puleo Sep 24, 2026
1822301
llama : fix qsa_n_kv_window crash when n_stream > 1
gaetan-puleo Sep 24, 2026
23d9a50
qsa: unify scalar visibility gate for single and multi sequence
gaetan-puleo Sep 24, 2026
db0b0b7
qsa: fix compact graph reuse, shared-block visibility, and 2-D scan cost
gaetan-puleo Sep 24, 2026
525b08e
llama : count the cache's streams in qsa_n_kv_window (#73)
pwilkin Sep 25, 2026
19e6b13
qsa : keep the masked path when a block splits a sequence over sequen…
pwilkin Sep 25, 2026
0ed0816
qsa : fill only the compact rows the queries read, harden qsa_metadata
pwilkin Sep 25, 2026
c79b20e
ggml-cpu : attend only the selected keys of flash_attn_ext
pwilkin Sep 25, 2026
575de20
qwen4exp : keep the one-sequence compact visibility a broadcast
pwilkin Sep 25, 2026
6181136
hip : isolate MMB scratch by backend context
Mushoz Sep 24, 2026
405ec97
cuda : gate mmb per backend context instead of per process
pwilkin Sep 25, 2026
505f3f3
tests : add --mmb on|off to test-backend-ops
pwilkin Sep 26, 2026
eb67776
hip: port gufo's Flash-Next prefill/decode ideas to qwen4exp on gfx1151
pwilkin Sep 24, 2026
fc605d4
qsa : take 1..512 queries in the per-query decode kernel
michal-zurkowski Sep 23, 2026
e0da608
qwen4exp : use complete-block selection for every ubatch size
michal-zurkowski Sep 23, 2026
0a2fc71
qwen4exp : incremental pooled-key update for ubatches up to 127 tokens
michal-zurkowski Sep 23, 2026
94d0b10
cuda : zero quantized padding without a synchronous memset
pwilkin Sep 26, 2026
c731477
fix: guard the qwen4exp PLE prefetch on the model architecture
dzannotti Sep 17, 2026
f23fd94
spec : MTP draft vocabulary subset (--spec-draft-mtp-vocab N)
SimonTeixidor Sep 13, 2026
9c6776f
spec : make the MTP draft vocabulary subset per context
SimonTeixidor Sep 15, 2026
eda61b3
tests : per-context MTP draft vocabulary regression test
SimonTeixidor Sep 15, 2026
36975f0
qwen4exp: own PLE prefetch inputs in the detached thread
gaetan-puleo Sep 27, 2026
e990df0
hip: keep topk-moe fusion when outputs alias router logits
gaetan-puleo Sep 27, 2026
edcb38c
hip: select top-k score ties by increasing column index
gaetan-puleo Sep 27, 2026
0337646
tests : check the tested backend's picks in test_top_k_stable_ties
LaurentZuijdwijk Sep 27, 2026
3bd1f78
hip: stabilize 128–512 token QSA incremental key updates
gaetan-puleo Sep 27, 2026
6bf8433
hip: keep HC residual BF16 marks stable across scheduler split cuts
gaetan-puleo Sep 27, 2026
fb9a58f
ggml-cuda : shape-aware graph cache key
SimonTeixidor Sep 27, 2026
fddfaf5
convert : drop unused imports in qwen4exp
dzannotti Sep 29, 2026
9f07ca2
ci : nightly ROCm release builds in the lemonade-sdk/llamacpp-rocm la…
dzannotti Sep 29, 2026
3b10095
ci : release every merge to master, Linux zips by default
dzannotti Sep 29, 2026
d877049
ci : add a Linux Vulkan zip to each release
dzannotti Sep 29, 2026
b56e40a
hip: optimize Qwen3.8-Flash-Next prefill on Strix Halo (#91)
gaetan-puleo Sep 29, 2026
d038d59
llama: flatten the GDN output projection input to 2-D
SimonTeixidor Sep 9, 2026
7caea13
qwen35, qwen35moe: run gated_delta_net in place on the recurrent stat…
SimonTeixidor Sep 2, 2026
de77b50
server : preserve context checkpoints across slot save/restore
Tough-Respawn Aug 15, 2026
9dd846a
ggml : backport persistent view initialization after buffer-size spli…
arc-uri-el Sep 29, 2026
c33b97b
cuda: ROCmFPx load/dequant/dispatch paths for HIP builds
baraxnaxgaming-commits Sep 10, 2026
b7d4b4c
cuda: MMQ tile path for Q4_0_ROCMFP4_FAST (dp4a + WMMA/MFMA layouts)
baraxnaxgaming-commits Sep 8, 2026
d40479f
cuda: gate Q4_0_ROCMFP4_FAST MMQ to validated architectures
baraxnaxgaming-commits Sep 28, 2026
cf0136b
cuda: return 0 for invalid ROCmFP4 scale bytes in the HIP decoder
baraxnaxgaming-commits Sep 30, 2026
1d6d2b7
cuda : fix read past the end of the top-k MoE group in the alloc-depe…
pwilkin Sep 30, 2026
c9093cc
hip: gate the MMB matmul path by quant type, not architecture
gaetan-puleo Sep 30, 2026
c243e80
qwen4exp : use maskless block selection only where the selected-key k…
rcfa Sep 30, 2026
e231404
vulkan : selected-key flash attention for QSA block selection
rcfa Sep 30, 2026
a1a5adc
vulkan : WMMA selected-key attention for QSA block selection
Nathanw1014 Oct 1, 2026
d932f1f
server : add --cache-disk for a persistent prompt cache
Aug 29, 2026
4f6d2bf
server : rename --cache-disk to --cache-dir
Sep 14, 2026
f87c55c
hip : default build target to gfx1151
gaetan-puleo Oct 2, 2026
8a50dd6
hip: lower the MMB gate to 32 rows for qwen4exp contexts
iTechMedic Sep 30, 2026
154890c
tests : MMB small-batch gate and indexer score cases at the qwen4exp …
iTechMedic Sep 30, 2026
dec2a3b
hip: comment and help-text cleanup for the MMB small-batch gate
iTechMedic Oct 1, 2026
b271822
hip: clarify the MMB opt-in, F32 gate and --mmb off wording
iTechMedic Oct 1, 2026
3334ebc
server : /dashboard live stats page and /dashboard/stats endpoint
LaurentZuijdwijk Oct 2, 2026
faef769
server : dashboard reports lazy_mode (strix) instead of ple_on_disk
LaurentZuijdwijk Oct 2, 2026
b354315
server : link the chat UI and the /dashboard page to each other
LaurentZuijdwijk Oct 2, 2026
da60a66
hip : drop the F32 HC norm output only when the gate mix takes the ga…
iTechMedic Oct 1, 2026
06ea455
tests : MMB_QUANT_HC with the down projection and mixed HC weight types
iTechMedic Oct 1, 2026
311fac9
hip : reword the HC gate-mix comments
iTechMedic Oct 1, 2026
f9b4ffa
hip : drop the F32 HC norm output only when the fused gate mix takes …
iTechMedic Oct 2, 2026
5976cc7
tests : MMB_QUANT_HC when the fused gate mix refuses: shapes, gate we…
iTechMedic Oct 2, 2026
3f81f00
tests : use _putenv_s for QSA_FAST_MAX on Windows
gaetan-puleo Oct 3, 2026
6341557
vulkan : top-k n-ary search never selected +inf
Nathanw1014 Sep 30, 2026
0edd908
vulkan : keep masked keys from leaking NaN/inf through flash attention
Nathanw1014 Sep 30, 2026
0441ffa
ci : disable the Fusion metal job on pull requests
gaetan-puleo Oct 3, 2026
206e049
vulkan: swap operands for small-M mul_mat with a transposed result
Nathanw1014 Oct 1, 2026
1b35211
vulkan : keep the small-M swapped mat-vec to F32 dst and f32/f16 src1
Nathanw1014 Oct 3, 2026
09c167c
vulkan: chunked gated delta net prefill on RDNA3
Nathanw1014 Sep 29, 2026
56c91ba
vulkan : declare the GDN scan accumulators separately for older glslang
Nathanw1014 Oct 1, 2026
9c4761e
vulkan : gate the chunked GDN scan to RADV, scope its test tolerance …
Nathanw1014 Oct 3, 2026
2b52285
ci : restrict the ROCm build and release to gfx1151
gaetan-puleo Oct 3, 2026
903d48c
hip : let routed experts take the measured mmb weight types, dense GE…
michal-zurkowski Oct 3, 2026
0e4f7fc
hip : mmb per-slice weight decoders for the K-quant and low-bit i-qua…
pwilkin Oct 6, 2026
86b24be
hip : reshape the routed small tile so every wave issues WMMA
pwilkin Oct 6, 2026
bd9d471
hip : keep fused gate_up experts on MMQ
pwilkin Oct 6, 2026
cb90d85
hip : admit the measured low-bit expert types to the routed mmb gate
pwilkin Oct 6, 2026
f28f076
cuda : route small-row wide-batch matmuls to the matvec kernel on AMD
pwilkin Oct 6, 2026
8d90a14
CUDA: enable the fused hc_mix_reduce at decode (qwen4exp)
Sep 28, 2026
684912b
cuda: IQ3_S vec-dot for the HIP MMVQ path
gaetan-puleo Sep 29, 2026
99779a4
CUDA: enable the GDN decode-chain fusion for qwen4exp (rms_norm+scale…
Sep 29, 2026
bcbcc38
CUDA: apply the qwen4exp GDN output gate in the decode norm kernel
gaetan-puleo Sep 29, 2026
4f0f33a
cuda : fuse GDN decode with rollback snapshots
pwilkin Oct 6, 2026
4c2a220
hip : keep the F32 HC gate output when no BF16 xn copy is expected
iTechMedic Oct 3, 2026
329b089
tests : MMB_QUANT_HC for the layer-0 mix without combine-norm
iTechMedic Oct 3, 2026
a3861fb
ggml-cuda : round growing pool requests up to a power of two
michal-zurkowski Sep 20, 2026
4d87594
scripts : log instead of print in gather_required_libs.py
dzannotti Sep 29, 2026
218432c
ci : add on-demand gfx1151 bench and ops lane for the DevLab Dispatch…
pwilkin Oct 7, 2026
2d331a0
ci : keep the bench steps running when an ops case fails, and report …
pwilkin Oct 7, 2026
87069fa
ci : record the first execution in the workflow header
pwilkin Oct 7, 2026
8c140d9
llama : load per-layer embedding tables from a separate GGUF (--ple)
michal-zurkowski Oct 7, 2026
5a75c3c
llama-bench : add --ple
michal-zurkowski Oct 7, 2026
3bdeda8
gguf-py : add gguf_extract_ple.py to write --ple files
michal-zurkowski Oct 7, 2026
8f3cbe9
gguf-py : write the PLE tables without faking an ndarray for GGUFWriter
michal-zurkowski Oct 7, 2026
a8b2a14
Revert "Merge pull request #81 from storm1er/persistent-prompt-cache"
dzannotti Oct 8, 2026
4f0fec1
ci : run test-backend-ops perf with HIP graphs disabled
pwilkin Oct 8, 2026
6d2a188
spec : add --spec-n-rs-seq to cap the rollback snapshot slots
Sep 3, 2026
8a88a7d
common : LLAMA_ARG_SPEC_DRAFT_MTP_VOCAB env var for --spec-draft-mtp-…
LaurentZuijdwijk Oct 1, 2026
5862894
updated readme.md to remove halo-box fork
dzannotti Oct 8, 2026
5be500f
vulkan : multi-row coopmat1 flash attention for prefill, with a V^T s…
Nathanw1014 Oct 8, 2026
f7d5792
vulkan: run the GDN conv straight from x and the state (CONCAT + SSM_…
Nathanw1014 Oct 1, 2026
80a5d62
vulkan: short-row RMS norm kernel + RMS_NORM_MUL_MUL / RMS_NORM_SCALE…
Nathanw1014 Oct 1, 2026
f049608
ggml-alloc : grow the chunk list on demand instead of an unbounded 16…
Nathanw1014 Oct 9, 2026
14767cb
vulkan: store int8 MMQ f32 results as vec4 on RDNA3 wave32
Nathanw1014 Oct 1, 2026
d3879b9
vulkan: apply the MoE router weights in the int8 MMQ MUL_MAT_ID epilogue
Nathanw1014 Oct 1, 2026
4a45b44
server : add --spec-draft-max-slots to stop drafting under concurrency
dzannotti Oct 7, 2026
f995c70
qsa : track a sequence's prefix while other sequences share the unifi…
rcfa Sep 30, 2026
5773190
qsa : enable the incremental QSA cache on Vulkan
rcfa Sep 30, 2026
eaac057
vulkan : grow the context-sized scratch in 64 MiB steps
rcfa Sep 30, 2026
f3cc942
vulkan : round the context-sized scratch only from 64 MiB, within the…
rcfa Oct 7, 2026
4d59fa8
vulkan : round the top-k scratch as well
rcfa Oct 7, 2026
e213269
tests : replace the correctness corpora with longer real text and data
gaetan-puleo Oct 13, 2026
4ba88b3
docs : the fork merges upstream every 3 days and never rebases master
gaetan-puleo Oct 13, 2026
157ae54
skills : add upstream-sync
gaetan-puleo Oct 13, 2026
92405ad
skills : add upstream-sync-verify
gaetan-puleo Oct 13, 2026
a222939
skills : make the repo skills discoverable by any harness
gaetan-puleo Oct 13, 2026
5188ebf
ci : post the upstream-sync plan to a pinned issue every 3 days
gaetan-puleo Oct 13, 2026
2186c52
sync: reconcile the rebased fork with its pre-rebase state on upstrea…
gaetan-puleo Oct 14, 2026
080ff5b
sync: fix the build of the reconciled tree
gaetan-puleo Oct 14, 2026
300cced
sync: fix the k-pool reserve context, the MMQ precision defaults and …
gaetan-puleo Oct 14, 2026
8ffb24c
sync: fix the HIP build (MMQ precision template args, moved/renamed u…
gaetan-puleo Oct 14, 2026
a3bbcbe
sync: routed-compact MMQ kernel takes the src1 precision
gaetan-puleo Oct 14, 2026
cc672b6
sync: the fork syncs by rebasing its commits onto upstream
gaetan-puleo Oct 14, 2026
640440b
sync: the RDNA3.5 fused-quantize MMVQ leaves the shared-expert fusion…
gaetan-puleo Oct 14, 2026
13b16cf
skills : validate.sh runs the KLD without mmap
gaetan-puleo Oct 14, 2026
7a964d1
skills : validate.sh shortens the long KLD of a model that fills the GPU
gaetan-puleo Oct 14, 2026
e0187a3
sync : join the rebased history onto the old master
gaetan-puleo Oct 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .devops/cann.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app

Expand Down
1 change: 1 addition & 0 deletions .devops/cpu.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app

Expand Down
1 change: 1 addition & 0 deletions .devops/cuda.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app

Expand Down
1 change: 1 addition & 0 deletions .devops/intel.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,7 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/lib/ /app
COPY --from=build /app/full/llama /app/full/llama-server /app
Expand Down
1 change: 1 addition & 0 deletions .devops/musa.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app

Expand Down
13 changes: 7 additions & 6 deletions .devops/openvino.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
ARG OPENVINO_VERSION_MAJOR=2026.4
ARG OPENVINO_VERSION_FULL=2026.4.0.22959.99c81491cc3
ARG OPENVINO_VERSION_MAJOR=2026.4.1
ARG OPENVINO_VERSION_FULL=2026.4.1.22982.07f9c262b05
ARG UBUNTU_VERSION=24.04

# Intel GPU driver versions. https://github.com/intel/compute-runtime/releases
ARG IGC_VERSION=v2.40.13
ARG IGC_VERSION_FULL=2_2.40.13+22418
ARG COMPUTE_RUNTIME_VERSION=26.31.39395.13
ARG COMPUTE_RUNTIME_VERSION_FULL=26.31.39395.13-0
ARG IGC_VERSION=v2.41.5
ARG IGC_VERSION_FULL=2_2.41.5+22716
ARG COMPUTE_RUNTIME_VERSION=26.35.39758.10
ARG COMPUTE_RUNTIME_VERSION_FULL=26.35.39758.10-0
ARG IGDGMM_VERSION=22.10.0

# Intel NPU driver versions. https://github.com/intel/linux-npu-driver/releases
Expand Down Expand Up @@ -227,6 +227,7 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app/

Expand Down
1 change: 1 addition & 0 deletions .devops/rocm.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app

Expand Down
1 change: 1 addition & 0 deletions .devops/s390x.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ ENTRYPOINT [ "/llama.cpp/bin/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

WORKDIR /llama.cpp/bin

Expand Down
1 change: 1 addition & 0 deletions .devops/vulkan.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app

Expand Down
1 change: 1 addition & 0 deletions .devops/zendnn.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app

Expand Down
3 changes: 3 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,9 @@ insert_final_newline = unset
trim_trailing_whitespace = unset
insert_final_newline = unset

[vendor/**.patch]
trim_trailing_whitespace = unset

[tools/ui/**]
indent_style = unset
indent_size = unset
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/ai-issues.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ on:
issues:
types: [opened]

cache-mode: none
permissions:
contents: read

jobs:
find-related:
if: github.event.action == 'opened'
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/build-3rd-party.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ on:
'**/*.cpp'
]

cache-mode: none
permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
Expand Down
16 changes: 16 additions & 0 deletions .github/workflows/build-and-test-snapdragon.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ on:
- 'scripts/snapdragon/**'
- 'CMakePresets.json'

cache-mode: none
permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
Expand All @@ -37,6 +41,10 @@ jobs:
run:
shell: bash

permissions:
actions: write
contents: read

steps:
- name: Clone
uses: actions/checkout@v6
Expand Down Expand Up @@ -67,6 +75,10 @@ jobs:
run:
shell: bash

permissions:
actions: write
contents: read

steps:
- name: Clone
uses: actions/checkout@v6
Expand Down Expand Up @@ -99,6 +111,10 @@ jobs:
matrix:
device: [SM8750, SM8850, QCS9075M]

permissions:
actions: read
contents: read

steps:
- name: Checkout
uses: actions/checkout@v6
Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/build-android.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@ on:
- '.github/workflows/build-android.yml'
- 'examples/llama.android/**'

cache-mode: none
permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
Expand Down Expand Up @@ -67,6 +71,10 @@ jobs:
run:
shell: bash

permissions:
actions: write
contents: read

steps:
- name: Clone
uses: actions/checkout@v6
Expand Down
19 changes: 16 additions & 3 deletions .github/workflows/build-apple.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ on:
'ggml/src/ggml-rpc/**'
]

cache-mode: none
permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
Expand All @@ -51,7 +55,7 @@ jobs:
- name: ccache
uses: ggml-org/ccache-action@v1.2.24
with:
key: apple-arm64
restore: false
save: false

- name: ccache-buckets-restore
Expand Down Expand Up @@ -100,8 +104,9 @@ jobs:
id: cmake_test
run: |
cd build
# Metal Paravirtual devices are difficult to support -> disable
# ref: https://github.com/ggml-org/llama.cpp/pull/19802#issuecomment-4013704023
ctest -L main -E "test-llama-archs|test-save-load-state" --verbose --timeout 900
ctest -L main -E "test-llama-archs|test-llama-context" --verbose --timeout 900

macos-latest-x64:
runs-on: macos-15-intel
Expand All @@ -114,7 +119,7 @@ jobs:
- name: ccache
uses: ggml-org/ccache-action@v1.2.24
with:
key: apple-x64
restore: false
save: false

- name: ccache-buckets-restore
Expand Down Expand Up @@ -162,6 +167,10 @@ jobs:
macos-latest-ios-xcode:
runs-on: macos-latest

permissions:
actions: write
contents: read

steps:
- name: Checkout code
uses: actions/checkout@v6
Expand Down Expand Up @@ -259,6 +268,10 @@ jobs:
runs-on: macos-latest
needs: macos-latest-ios-xcode

permissions:
actions: read
contents: read

strategy:
matrix:
destination: ['generic/platform=macOS', 'generic/platform=iOS', 'generic/platform=tvOS']
Expand Down
16 changes: 10 additions & 6 deletions .github/workflows/build-cache.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ on:
schedule:
- cron: '0 * * * *'

cache-mode: write
permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
Expand Down Expand Up @@ -40,9 +44,9 @@ jobs:
runs-on: ubuntu-24.04

env:
# Sync versions in build-openvino.yml, build-self-hosted.yml, release.yml, build-cache.yml, .devops/openvino.Dockerfile
OPENVINO_VERSION_MAJOR: "2026.4"
OPENVINO_VERSION_FULL: "2026.4.0.22959.99c81491cc3"
# Sync versions in build-openvino.yml, ci-self-hosted-openvino.yml, release.yml, release-publish.yml, build-cache.yml, .devops/openvino.Dockerfile, docs/backend/OPENVINO.md
OPENVINO_VERSION_MAJOR: "2026.4.1"
OPENVINO_VERSION_FULL: "2026.4.1.22982.07f9c262b05"

steps:
- name: Clone
Expand All @@ -68,9 +72,9 @@ jobs:
runs-on: windows-2022

env:
# Sync versions in build.yml, build-self-hosted.yml, release.yml, build-cache.yml, .devops/openvino.Dockerfile
OPENVINO_VERSION_MAJOR: "2026.4"
OPENVINO_VERSION_FULL: "2026.4.0.22959.99c81491cc3"
# Sync versions in build-openvino.yml, ci-self-hosted-openvino.yml, release.yml, release-publish.yml, build-cache.yml, .devops/openvino.Dockerfile, docs/backend/OPENVINO.md
OPENVINO_VERSION_MAJOR: "2026.4.1"
OPENVINO_VERSION_FULL: "2026.4.1.22982.07f9c262b05"

steps:
- name: Clone
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/build-cann.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ on:
'ggml/src/ggml-cann/**'
]

cache-mode: none
permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/build-cmake-pkg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ on:
workflow_dispatch:
workflow_call:

cache-mode: none
permissions:
contents: read

jobs:
linux:
runs-on: [self-hosted, Linux, CPU]
Expand Down
11 changes: 10 additions & 1 deletion .github/workflows/build-cpu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ on:
'**/*.cpp'
]

cache-mode: none
permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
Expand Down Expand Up @@ -66,7 +70,7 @@ jobs:
- name: ccache
uses: ggml-org/ccache-action@v1.2.24
with:
key: cpu-${{ matrix.os }}
restore: false
save: false

- name: Build Dependencies
Expand Down Expand Up @@ -143,6 +147,11 @@ jobs:
name: windows / ${{ matrix.build }}
runs-on: windows-2025

cache-mode: write
permissions:
actions: write
contents: read

env:
OPENBLAS_VERSION: 0.3.23
SDE_VERSION: 9.33.0-2024-01-07
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/build-cross.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ on:
schedule:
- cron: '0 0 * * 0'

cache-mode: none
permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
Expand Down
Loading
Loading