Merge halo-box/llama.cpp master into strix (2026-09-15)

[dzannotti]

Brings strix-llama.cpp up to date with halo-box/llama.cpp master (was 77 behind /
164 ahead), and brings in strix master's #63 (pwilkin/strix-halo), which landed after
this branch was cut. This is a real merge commit — do not squash it. Squashing an upstream
sync is what left halo-box stuck at "72 commits behind" and silently dropped two
upstream commits; see SYNC.md in halo-box.

Conflict resolutions (5 files)

CONTRIBUTING.md — kept strix's policy. Agents may author code, commit messages
and PR descriptions, and open PRs, per AGENTS.md. The incoming side was upstream's
stricter policy (mandatory disclosure, ~1h review per 200-400 LOC, no AI-written
posts); explicitly rejected.

tests/test-backend-ops.cpp — union, both sides kept. strix's Strix Halo probes
(MALL-spill, L2-residence, KV-cache-layout) and upstream's sparse-FA cases are
independent list entries. Verified the merged test_flash_attn_ext constructor
accepts both call signatures: it combines strix's permute/kv_view with upstream's
v_is_view_of_k/n_kv_max, and VARS_TO_STR17 exists to match the 17 vars.

src/models/qwen4exp.cpp — resolved per hunk, not wholesale:

• Kept strix — the rms_norm+mul fusion (gamma reshaped to [n_embd, hc] and
  force-expanded so RMS_NORM and MUL stay graph-adjacent for the backend fuser).
  Traced to 6130b7262 hip: optimize RDNA3.5 MoE inference paths. Upstream's
  incoming 41abbfd59 does the same thing less carefully as a one-liner.
• Took halo-box — the PLE-disk-entangled hunks, per halo-box Merge: from halobox 2026 09 06 uma ring #29.

ggml/src/ggml-vulkan/ — reverted wholesale to strix (5 files). See below.

Vulkan: upstream sparse FA deliberately NOT integrated

Upstream fc82583e6 vulkan: support sparse Flash Attention (ggml-org#28105) cannot
be merged mechanically with strix's Vulkan stack:

• Both claim the same specialization-constant bit:
  strix DYNAMIC_KV = (Flags & 16) vs upstream USE_SPARSE = (Flags & 16).
• They declare incompatible flash-attn push-constant layouts
  (strix: m_row_len, gqa_ratio, split_k_num, output_k_num, partial_output;
  upstream: sparse_base). A mismatched layout produces silent garbage, not a crash.
• They are rival implementations of overlapping functionality — strix already has
  its own gather/compaction family (flash_attn_gather{,_dq,_union,_union_dq}.comp)
  and a DeepSeek-V4 sparse split path keyed on bit 31 of gqa_ratio.

Taking either side alone breaks the build: upstream's sparse code merged cleanly
into flash_attn.comp, flash_attn_cm1.comp and flash_attn_cm2.comp (strix had
not touched those regions), so keeping strix's flash_attn_base.glsl leaves those
three files calling a now-undefined fa_kv_index() / USE_SPARSE / data_sparse.

Reverting the whole directory to strix restores a self-consistent, known-good state.
Orphaned upstream shader flash_attn_sparse_compact.comp removed (the shader
generator uses an explicit list, so nothing referenced it).

Also deferred by this: f1e44dcc1 vulkan: workaround NV queuesubmit driver bug.

Follow-up PR needed to integrate sparse FA properly: renumber one feature's flag
bit, reconcile the push-constant structs and their C++ writer, and decide whether
DYNAMIC_KV and USE_SPARSE coexist or one supersedes. That work needs a GPU.

Update 2026-09-17: #63 merged in (82aed6017)

pwilkin's #63 landed on master after this PR was opened and conflicted in 4 files, all
around the qwen4exp PLE n-gram table. #63 builds on the PLE disk reader this sync had
deleted:

• --lazy-mode on-direct (LLAMA_LAZY_MODE_DIRECT) reads PLE rows through the reader
  with page-cached pread()s
• llama_ple_disk::prefetch() / page_cached(), and qwen4exp_ple_prefetch() called
  from llama_context for batches of 4096+ tokens
• the host_gather path and mmap madvise prefetch, kept and extended
• tests/test-ple-disk.cpp

Taking this branch's deletion would have silently dropped all of it. Resolution:

• src/llama-ple-disk.{cpp,h}: kept as in pwilkin/strix-halo official merge #63, re-added to src/CMakeLists.txt.
• src/models/qwen4exp.cpp: pwilkin/strix-halo official merge #63's file with two changes. The reader is gated on
  LAZY_MODE_DIRECT alone, using its defaults (64 readers, 256 MiB cache, no O_DIRECT,
  which is what pwilkin/strix-halo official merge #63 already chose for on-direct). Upstream's [n_embd, hc]
TENSOR_ALLOW_RESHAPE gamma shapes are re-applied, including pwilkin/strix-halo official merge #63's new
  nextn.hc_head_norm. The graph reshapes the gamma either way, so RMS_NORM+MUL fusion
  is unaffected.
• src/models/models.h: ple_disk member restored.
• src/llama-model.cpp: init_mappings populates unless on-direct.

halo-box #29's user-facing cleanup stays. --ngram-on-disk is an alias for
--lazy-mode on, and --ngram-io-threads/-cache/-direct-io and the ple_*
llama_model_params fields are removed. Net effect versus pre-sync strix: only
O_DIRECT reads and the tuning knobs are gone. The prefetch and host_gather paths
the original description listed as collateral loss are back.

Audit: every other file #63 touched is byte-identical to master. Where the merge differs
from master, the difference is exactly this branch's own delta from the merge base.

Also merged: halo-box #31 (SYNC.md, docs only). Now 0 behind halo-box, 0 behind strix
master.

Verification

Built and tested on a Strix Halo (Ryzen AI MAX+ 395 / Radeon 8060S, gfx1151) in
containers: rocm/dev-ubuntu-24.04:7.2.1-complete for HIP, ubuntu:26.04 for Vulkan.

	result
HIP build (GGML_HIP=ON, GPU_TARGETS=gfx1151, tests on)	✅ 602/602 targets
Vulkan + CPU build (GGML_VULKAN=ON, tests on)	✅
test-backend-ops -b ROCm0 (full)	✅ 29695/29695
test-backend-ops -b Vulkan0 FLASH_ATTN_EXT / MUL_MAT_ID / RMS_NORM / GET_ROWS	✅ 5339 / 7498 / 51 / 590
test-llama-archs (ROCm)	✅ no failures; qwen4exp OK (9.97e-14) on GPU
test-ple-disk	✅ 960 gathers byte-identical to CPU dequant, all configs
test-qsa-prefix	✅
test-arg-parser	✅

PP/TG: not measured yet. An A/B against master 0636c9aee ran, but the box was
serving inference at the same time, so the numbers are discarded.

Not verified: no qwen4exp GGUF was available, so the on-direct / prefetch /
host_gather paths are covered only by test-ple-disk and test-llama-archs, not by
a real model run. CPU-only test-backend-ops was not run. The PP regression reported
on #63 (HIP 700→400, Qwen3.8-27B IQ4_XS halved) predates this PR, and I didn't
reproduce it. Tracked in #65's doc.

CI: the red checks on this PR are jobs that never got a runner and timed out after
exactly 24h with no logs. They aren't build failures.

Original description: PLE on disk (partly superseded by the update above)

Deliberately dropped: PLE on disk

halo-box #29 replaced the bespoke --ngram-on-disk with an alias to upstream
--lazy-mode, removing the custom Qwen4Exp PLE disk reader, O_DIRECT handling, row
cache, worker-thread controls, C API params and build wiring. strix still carried the
full implementation; this merge follows halo-box.

Note this was not visible as a conflict. strix had not modified those files since
the merge-base, so git applied halo-box's deletions silently. Only qwen4exp.cpp
conflicted — 1 of 12 affected files. What went:

src/llama-ple-disk.cpp, src/llama-ple-disk.h   (deleted)
src/CMakeLists.txt, src/llama-model.cpp        (wiring reverted)
common/common.{cpp,h}, include/llama.h         (C API params reverted)
src/models/models.h, README.md, docs/          (reverted)

Collateral loss, both from 6130b7262 and both referencing ple_disk, so neither
survives the feature's removal as written:

• madvise(MADV_WILLNEED) batch prefetch for random mmap PLE rows (each row is
  otherwise a serial page fault, ~0.2 ms from NVMe).
• the host_gather path, which avoided a CPU get_rows node splitting the graph
  into GPU→CPU→GPU with an extra launch and device sync per ubatch.

The host_gather optimization has a non-disk half (host-resident buffers) that could
be preserved independently. Not attempted here — it would be new hybrid code, and it
needs benchmarking to prove it is worth it.