Qualify v0.5.0 Android CPU dispatch and source policy - #93
Merged
Merged
Conversation
GitHub is migrating the windows-11-arm label to the Windows 11 Arm64 with Visual Studio 2026 image between 2026-09-21 and 2026-09-30 (actions/runner-images#14602). That image has no VS 2022 instance, so the windows-arm64-full preset's hardcoded "Visual Studio 17 2022" generator fails at configure. Native release run 35976011426 lost both arm64 lanes (blas, vulkan) this way; the same lanes passed on the VS 2022 image in run 35850331498 on the same commit. Switch the preset to "Visual Studio 18 2026" and pin the arm64 jobs to windows-11-vs2026-arm so the image cannot flip mid-rollout. ClangCL, the ARM64 and x64 MSVC tools, and CMake 4.4 are all present on that image.
The Android arm64 OpenCL job of native release run 35976011426 built llama.cpp v0.5.0 (7fe450e19305) and then correctly rejected its new ggml/src fingerprint at the CPU ISA source gate. Review of the v0.4.1..v0.5.0 ggml/src delta: KleidiAI stays at v1.24.0 with an unchanged kai digest, and kleidiai/, ARM cpu-feats and the CPU/ggml build files are byte-unchanged. The only ARM change is 8034c1d1f's Q1_0 repack kernels. They follow the existing pattern: compile-time DOTPROD/I8MM guards fall back to generic C, runtime selection uses ggml_cpu_has_*, and there is no SVE/SME code. The release ARMv8.2_2 variant built from the exact SHA with NDK 28.2 passes the unchanged containment policy: 260,601 instructions, with the same 2,187 scalable instructions as v0.4.1, all in the existing 18 functions. The Q1_0 I8MM GEMM compiles to its generic fallback in that non-I8MM variant. Add only the audited pair, and exact v0.5.0 rows to the Android ISA and emulated non-SVE Kleidi dispatch/compute lanes. The production pin is unchanged. Evidence: docs/v050_android_isa_qualification.md.
AGENTS.md asks for the candidate Windows gate on ARM64 upstream upgrades. The v0.5.0 release run never got past configure on Windows ARM64, and the existing rows build only v0.4.1 and the post-v0.4.0 candidate. The lane's selector also named a v0.4.1 row the matrix no longer has.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the
build-android (arm64-v8a, arm64, opencl, …)failure in the scheduled v0.5.0 release run 35976011426. The job compiled and packaged successfully, then the Android CPU ISA source gate correctly stopped it:This is the gate working as designed on an upstream advance, the same situation as #78 for v0.4.1. The arm64 Vulkan job in that run died earlier, in shader generation (#92), and would have hit this same gate.
Following
AGENTS.mdanddocs/platform_backend_strategy.md, the fingerprint was not refreshed just to silence the check. The dispatch was reviewed, and the full evidence is indocs/v050_android_isa_qualification.md.Source review (v0.4.1
b29c606e28→ v0.5.07fe450e193)ggml/srcchanged: 8 CPU, 5 shared, 190 accelerator.ggml-cpu/kleidiai/,arch/arm/cpu-feats.cpp,ggml-cpu/CMakeLists.txt,ggml/src/CMakeLists.txt,ggml/cmake. KleidiAI is still v1.24.0, and thekaidigest is unchanged.__ARM_FEATURE_DOTPROD/__ARM_FEATURE_MATMUL_INT8and fall back to generic C. Selection usesggml_cpu_has_neon/dotprod/matmul_int8, which on ARM are compile-time constants of each isolated variant, so a variant can only pick kernels its own flags allow. The kernels contain no SVE or SME code.#ifguards: F16 FWHT input, hc ops, a SpacemiT fix, scheduler reserve-failure handling, the meta backend,ggml_permute, IQ1_M reference quantization, and GGUF alignment.tree_digestofgit archive v0.5.0 ggml/srcequals the digest CI rejected.Change
tools/validate_android_cpu_isa.py: adds only the pair (v0.5.0 ggml, unchanged kai). The 18-function allowlist and the containment logic are unchanged.tests/test_android_cpu_isa.py: updates the exact-set assertion. The unknown, cross-combined, and mutated-source rejection tests are untouched and still pass.validate_wrapper.yml: adds an exactv0.5.0row towindows-arm64-kleidiai(the candidate Windows gate AGENTS.md requires for ARM64 upgrades; the release run never got past configure there). It also replaces that lane's deadv0.4.1selector branch. And it addsLLAMADART_V050_QUALIFICATION_SHA(qualification only) and av0.5.0row inandroid-arm64-isaandkleidiai-dispatch-emulated. Those rows cover the release ARMv8.2 artifact audit, and the compiled Kleidi selector plus Q4/Q8 compute under QEMUcortex-a53andmax,sve=off,sme=off, each with and withoutGGML_KLEIDIAI_SME=1.Validation
28.2.13676358, release helperandroid_armv8.2_2variant:PASS: 260601 instructions; 2187 SVE/SME instructions contained in 18 exact audited functions. v0.4.1 had the same 2,187 scalable instructions in the same 18 functions.ggml_gemm_q1_0_4x8_q8_0in this non-I8MM variant is one tail-branch instruction into the generic kernel, with 0smmla/ummla/usmmla.python3 -m unittest discover -s tests: 157 tests OK.actionlint validate_wrapper.ymlis clean.validate_wrapperv0.5.0 rows (links will be added once green).Physical Android and SME hardware execution: N/A, not performed. Emulator results are CPU-compatibility evidence, not device qualification. The production pin is unchanged, and the policy change needs explicit maintainer approval.
Independent review
A fresh reviewer, with no involvement in this change, re-derived everything independently:
ggml/srcdigest fromgit archive(control: the v0.4.1 digest matches its existing entry);kleidiai/,cpu-feats.cppand the build files are byte-unchanged;android_armv8.0_1andandroid_armv8.2_2variants with NDK 28.2, reproducedPASS: 260601 instructions; 2187 … in 18 exact audited functions, and confirmed that the hosted v0.5.0 job prints the same line.The reviewer would approve the pair on this evidence. Its should-fix items were a wrong "runtime detection" wording (corrected: selection is compile-time per variant, which is stronger) and the stacked head (disclosed above). It also flagged that no gate executes the new Q1_0 DOTPROD kernels numerically. That gap is now stated in the doc: this is ISA-safety evidence, not numerical qualification of Q1_0. Nits were addressed as well:
ggml.hsits outside the fingerprinted tree, the artifact SHA-256 depends on the build path (dropped in favour of the reproducible counts), and I8MM/DOTPROD are not covered by the SVE/SME validator (checked by hand for this release).