tools: FP4 expert matmul microbench (SIMD vs scalar arm, ref #1696)

[mfethe1]

What

Adds c/tools/bench_fp4_matmul.c + bench_fp4_matmul.sh: a self-contained microbench for coli_fp4_matmul_batch_rows16_order that builds the COLI_V4_UNIT_NATIVE_QUANT unit twice — default flags (SIMD arm) vs EXTRA_CFLAGS=-mno-avx2 (scalar arm) — verifies the two arms agree bit-exactly on seeded data, then times both at the real V4-Flash expert shapes.

Why (#1696)

To size what a NEON arm is worth on arm64, I measured the A/B on x86-64 (Zen 2 Ryzen 7 3700X, gcc 15.2, repo flags). Numbers in the issue comment:

shape (S×I×O)	AVX2 arm	scalar arm	ratio
128×4096×2048 (w1/w3)	~141 GFLOP/s	~1.9 GFLOP/s	75×
128×2048×4096 (w2)	~149 GFLOP/s	~3.8 GFLOP/s	41×
32×4096×2048	~156 GFLOP/s	~6.8 GFLOP/s	23×

Outputs bit-exact identical between arms (0/262144 floats differ), consistent with the design note at deepseek_v4.c:~16450 (per-row accumulation order preserved across arms).

The arm64 story: this kernel gates on __AVX2__ only, so Apple Silicon runs the scalar arm — which on an M-series (gcc 15.2, -march=armv8.2-a+dotprod) measures ~10 GFLOP/s vs ~2 on Zen 2 x86-64. The scalar arm is ~1.5–2.5% of SIMD throughput; a NEON arm does not need to match AVX2's absolute numbers to be a large win. The AVX2 arm's structure (pshufb nibble decode → 8×8 transpose → column-batched multiply/add) maps closely onto NEON (vqtbl1q lookup decode, vtrn/vzip transpose, vfma/mul+add lanes).

Harness details

• Mirrors the repo build: unit compiled with repo-equivalent flags (-O2 -fopenmp -march=…, COLI_V4_UNIT_NATIVE_QUANT), GPU-tier entry points stubbed (referenced but never taken on the CPU-only path).
• Deterministic seeded data (xorshift); e8m0 scale bytes constrained to 120–127 so products can never hit inf/NaN — full-range random scale bytes poison bit-comparison (NaN ≠ NaN).
• Bit-exact A/B: first build writes a float32 reference file, second build compares (MISMATCH → nonzero exit).
• Guards: rejects S > 128 (kernel's sums[128] batch cap; the caller chunks prefill to ≤128 rows) and non-multiple-of-32 I.
• Portable: POSIX + Windows (access/_access), arm64 (builds scalar-baseline mode until a NEON arm exists, per [Performance]: DeepSeek V4 on Apple Silicon (CPU path): prefill runs at decode speed, 1.3–1.75 prompt tok/s — 577-token prompt = 6 min TTFT #1696). Prefers real gcc over Apple clang (clang lacks -fopenmp).
• Verified: 0-warning build (-Wall -Wextra), run end-to-end on arm64 macOS and x86-64 Windows at S×I×O = 128×4096×2048.

How to run

cd c && bash tools/bench_fp4_matmul.sh            # defaults: 128 4096 2048 7
bash tools/bench_fp4_matmul.sh 128 2048 4096 7    # w2 shape

No model files, torch, or GPU needed — CPU-only, ~30 s.

[JustVugg]

The bench landed with #1730, which carries the same tools/bench_fp4_matmul.{c,sh} plus the NEON comparison on arm64 that now runs in the ARM CI job. Closing as included; thank you, it is what made the NEON arm checkable.