Skip to content

Qualify v0.5.0 Android CPU dispatch and source policy - #93

Merged
leehack merged 8 commits into
mainfrom
fix/android-isa-v050
Sep 24, 2026
Merged

leehack merged 8 commits into
mainfrom
fix/android-isa-v050

Conversation

@leehack

@leehack leehack commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

#91 is merged; this branch now targets main and contains main up to it.

Fixes the build-android (arm64-v8a, arm64, opencl, …) failure in the scheduled v0.5.0 release run 35976011426. The job compiled and packaged successfully, then the Android CPU ISA source gate correctly stopped it:

Android CPU ISA audit failed: Unaudited ggml/src + kai source fingerprint pair
('68d5bc369749e78545f50dd5107368ec7ee1874619794795cd142b0043c747f6',
 '64189fc613c1c4c3aaeeb6bb12b38d85dd6728cafd2261a5a88f1b77b10fe59c');
review dispatch before updating policy

This is the gate working as designed on an upstream advance, the same situation as #78 for v0.4.1. The arm64 Vulkan job in that run died earlier, in shader generation (#92), and would have hit this same gate.

Following AGENTS.md and docs/platform_backend_strategy.md, the fingerprint was not refreshed just to silence the check. The dispatch was reviewed, and the full evidence is in docs/v050_android_isa_qualification.md.

Source review (v0.4.1 b29c606e28 → v0.5.0 7fe450e193)

  • 203 files under ggml/src changed: 8 CPU, 5 shared, 190 accelerator.
  • Byte-unchanged: ggml-cpu/kleidiai/, arch/arm/cpu-feats.cpp, ggml-cpu/CMakeLists.txt, ggml/src/CMakeLists.txt, ggml/cmake. KleidiAI is still v1.24.0, and the kai digest is unchanged.
  • The only ARM change is 8034c1d1f (Q1_0 repack kernels). The NEON bodies are guarded at compile time by __ARM_FEATURE_DOTPROD / __ARM_FEATURE_MATMUL_INT8 and fall back to generic C. Selection uses ggml_cpu_has_neon/dotprod/matmul_int8, which on ARM are compile-time constants of each isolated variant, so a variant can only pick kernels its own flags allow. The kernels contain no SVE or SME code.
  • Other CPU and shared changes add no intrinsics, assembly, or new #if guards: F16 FWHT input, hc ops, a SpacemiT fix, scheduler reserve-failure handling, the meta backend, ggml_permute, IQ1_M reference quantization, and GGUF alignment.
  • The local tree_digest of git archive v0.5.0 ggml/src equals the digest CI rejected.

Change

  • tools/validate_android_cpu_isa.py: adds only the pair (v0.5.0 ggml, unchanged kai). The 18-function allowlist and the containment logic are unchanged.
  • tests/test_android_cpu_isa.py: updates the exact-set assertion. The unknown, cross-combined, and mutated-source rejection tests are untouched and still pass.
  • validate_wrapper.yml: adds an exact v0.5.0 row to windows-arm64-kleidiai (the candidate Windows gate AGENTS.md requires for ARM64 upgrades; the release run never got past configure there). It also replaces that lane's dead v0.4.1 selector branch. And it adds LLAMADART_V050_QUALIFICATION_SHA (qualification only) and a v0.5.0 row in android-arm64-isa and kleidiai-dispatch-emulated. Those rows cover the release ARMv8.2 artifact audit, and the compiled Kleidi selector plus Q4/Q8 compute under QEMU cortex-a53 and max,sve=off,sme=off, each with and without GGML_KLEIDIAI_SME=1.

Validation

  • Local, exact SHA, NDK 28.2.13676358, release helper android_armv8.2_2 variant:
    • before: the production validator fails with the same pair as CI.
    • after: PASS: 260601 instructions; 2187 SVE/SME instructions contained in 18 exact audited functions. v0.4.1 had the same 2,187 scalable instructions in the same 18 functions.
    • ggml_gemm_q1_0_4x8_q8_0 in this non-I8MM variant is one tail-branch instruction into the generic kernel, with 0 smmla/ummla/usmmla.
  • python3 -m unittest discover -s tests: 157 tests OK. actionlint validate_wrapper.yml is clean.
  • Hosted: validate_wrapper v0.5.0 rows (links will be added once green).

Physical Android and SME hardware execution: N/A, not performed. Emulator results are CPU-compatibility evidence, not device qualification. The production pin is unchanged, and the policy change needs explicit maintainer approval.

Independent review

A fresh reviewer, with no involvement in this change, re-derived everything independently:

  • recomputed the v0.5.0 ggml/src digest from git archive (control: the v0.4.1 digest matches its existing entry);
  • recounted the 203-file delta and confirmed kleidiai/, cpu-feats.cpp and the build files are byte-unchanged;
  • rebuilt the android_armv8.0_1 and android_armv8.2_2 variants with NDK 28.2, reproduced PASS: 260601 instructions; 2187 … in 18 exact audited functions, and confirmed that the hosted v0.5.0 job prints the same line.

The reviewer would approve the pair on this evidence. Its should-fix items were a wrong "runtime detection" wording (corrected: selection is compile-time per variant, which is stronger) and the stacked head (disclosed above). It also flagged that no gate executes the new Q1_0 DOTPROD kernels numerically. That gap is now stated in the doc: this is ISA-safety evidence, not numerical qualification of Q1_0. Nits were addressed as well: ggml.h sits outside the fingerprinted tree, the artifact SHA-256 depends on the build path (dropped in favour of the reproducible counts), and I8MM/DOTPROD are not covered by the SVE/SME validator (checked by hand for this release).

GitHub is migrating the windows-11-arm label to the Windows 11 Arm64 with
Visual Studio 2026 image between 2026-09-21 and 2026-09-30
(actions/runner-images#14602). That image has no VS 2022 instance, so the
windows-arm64-full preset's hardcoded "Visual Studio 17 2022" generator
fails at configure. Native release run 35976011426 lost both arm64 lanes
(blas, vulkan) this way; the same lanes passed on the VS 2022 image in run
35850331498 on the same commit.

Switch the preset to "Visual Studio 18 2026" and pin the arm64 jobs to
windows-11-vs2026-arm so the image cannot flip mid-rollout. ClangCL, the
ARM64 and x64 MSVC tools, and CMake 4.4 are all present on that image.
The Android arm64 OpenCL job of native release run 35976011426 built
llama.cpp v0.5.0 (7fe450e19305) and then correctly rejected its new ggml/src
fingerprint at the CPU ISA source gate.

Review of the v0.4.1..v0.5.0 ggml/src delta: KleidiAI stays at v1.24.0 with
an unchanged kai digest, and kleidiai/, ARM cpu-feats and the CPU/ggml build
files are byte-unchanged. The only ARM change is 8034c1d1f's Q1_0 repack
kernels. They follow the existing pattern: compile-time DOTPROD/I8MM guards
fall back to generic C, runtime selection uses ggml_cpu_has_*, and there is no
SVE/SME code.

The release ARMv8.2_2 variant built from the exact SHA with NDK 28.2 passes the
unchanged containment policy: 260,601 instructions, with the same 2,187
scalable instructions as v0.4.1, all in the existing 18 functions. The Q1_0
I8MM GEMM compiles to its generic fallback in that non-I8MM variant.

Add only the audited pair, and exact v0.5.0 rows to the Android ISA and
emulated non-SVE Kleidi dispatch/compute lanes. The production pin is
unchanged. Evidence: docs/v050_android_isa_qualification.md.
@leehack
leehack changed the base branch from main to fix/windows-arm64-vs2026 September 24, 2026 12:17
AGENTS.md asks for the candidate Windows gate on ARM64 upstream upgrades.
The v0.5.0 release run never got past configure on Windows ARM64, and the
existing rows build only v0.4.1 and the post-v0.4.0 candidate. The lane's
selector also named a v0.4.1 row the matrix no longer has.
@leehack
leehack changed the base branch from fix/windows-arm64-vs2026 to main September 24, 2026 12:47
@leehack
leehack merged commit 6f2cb11 into main Sep 24, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant