[ezoushen]

Refs #528.

What changes

Splash loads ornith-ai/Ornith-1.5-122B-MLX-4bit (Qwen3.5-122B-A10B) with incoai/Ornith-1.5-122B-DFlash2. On Apple M5 Max (40-core GPU, 128 GB), the attention replacement reduces cold 64K prefill 83.9→52.0 s (1.61x) and append 4K at 128K history 13.3→7.05 s (1.89x).

• Qwen3_6MoeLayout retains the 35B default and selects the 122B geometry by exact hidden size and layer count. Other configs retain the 35B fallback and refusal behavior. The 122B GGUF is refused until tested. install/families.py pairs the target with its draft.
• The draft config records its trained 16-row block; the engine drafts 8 rows and checks the trained block size. GDN adds the 64-value-head specializations.
• The 122B attention path replaces the kv2_g16 kernels with virtual groups for both INT8 and BF16 KV. Projection writes the grouped query layout directly, and the gate consumes it directly. Physical K/V storage and workspace capacity stay unchanged. The obsolete second softmax pass is removed.
• README model table and DEVELOPMENT draft documentation.

Attention performance

Upstream main cannot load this target, so the comparison is against this PR's previous head 2d6c8a3d (g16 kernels).

The old g16 tile fused 128 rows and ran at about half the per-FLOP rate of the 64-row g8 tile. Four virtual groups of eight query heads run over the two physical KV heads on the existing 8-head tile. Each physical KV head supplies two query groups, so query group g reads physical KV head g / 2.

Synthetic attention	Previous g16, ms	Virtual groups, ms	Speedup
Prefill 2048 rows, 64K history	227.334	105.235	2.16x
Prefill 2048 rows, 128K history	455.105	216.619	2.10x
Verify one 8-row lane, 64K history	1.358	0.651	2.09x
Verify one 8-row lane, 128K history	2.300	1.189	1.94x

INT8 KV, one layer, medians of 7 alternating runs after at least 1.5 s GPU warmup per arm, clock-qualified on Apple M5 Max (40-core GPU, 128 GB), Xcode 26.6. Every comparison is bit-equal (maximum absolute difference 0); 128/512-row prefill tails also improve. These recorded measurements were taken at 759e9dc5; this port preserves the measured attention instructions and host graph.

End-to-end wall times with the official Ornith 1.5 122B MLX 4-bit target and paired DFlash2 draft, comparing the prior g16 build (e2e239a3) with the measured virtual-group build (759e9dc5):

End-to-end request	Previous g16, s	Virtual groups, s	Speedup
Cold prefill, 64K tokens	83.879	52.006	1.61x
Append 4K at 128K history	13.317	7.047	1.89x

One clock-qualified AB pair on the same M5 Max, medians of 3 cold requests and 2 append requests per arm. Greedy 300-token outputs at 32K/64K matched byte-for-byte with all-row verification.

Existing models are unaffected

Compiled kernel identity against a fresh build of merge base 1482f6b: 1,816 existing kernels strictly AIR-identical; 26 additions (13 attention and 13 GDN), zero changed or removed.

Host route tests assert the existing split/reduce, QKV and gate pipeline names for every other supported attention geometry. The virtual-group route is selected only for dense 32-query/2-KV-head attention at dimension 256.

Tests

All attention cases passed, including 56 prefill and 22 verify cases across INT8/BF16 KV, CPU route selection and the full Q8 flash-attention matrix; the final cleanup production library is byte-identical to the full-check build. Timing-sensitive failures also fail or flake on main at 1482f6b on this machine and do not touch attention: backend watchdog 0/10 passes on both builds, Handoff 5/10 here versus 4/10 on main, and three Python timeouts that pass in isolation and 60/60 on main. The kernel-input fixture was fixed in 82368b00 and re-checked with positive and negative controls.

Earlier real-model validation at 2d6c8a3d

These measurements predate the attention replacement. Apple M5 Max 128 GB, macOS 27.0; official MLX 4-bit target at 310ebeb, draft at a5ed739, --language-only --verify-rows all --max-memory 80G. Loaded in 16 s, 256K context, no restarts, 5/5 smoke prompts. Greedy, thinking off.

	Splash	mlx-lm 0.32.2, no draft
Decode counting / code / prose, tok/s	232 / 157 / 81	57-66
Cold prefill 0.8K / 6K / 25K tokens, tok/s	1986 / 1739 / 925	1022 / 905 / 926
4 concurrent streams, aggregate tok/s	208

Medians of three trials, taken in a second pass when the machine was calm (load average 4-8, swap usage unchanged); the three 25K cold-prefill TTFTs were 21, 27 and 36 s.

Quality against mlx-lm on the same checkpoint: 10/10 arithmetic word problems on both; HumanEval first 40 executed 36/40 here (fails 10, 17, 32, 38) against 37/40 (fails 10, 32, 38); problem 17's solution misses the empty-string case. The sets are small, and the mlx-lm figures were measured earlier on the same machine.

Not covered

• Vision: the official MLX repo's index names model-vision.safetensors but does not contain it, so this is text only.
• GGUF, thinking mode and long-context quality are untested. make test-real and model-dependent benchmark targets were not run at 82368b00.
• Acceptance cost of drafting 8 of the draft's 16 rows is unmeasured (cumulative draft acceptance in the run above: 0.61 of drafted tokens).

[jianc99]

Thanks @ezoushen! Ornith-1.5-122B support landed through #539 and #560, which also serve the unsloth GGUF; your 122B tests were carried over with co-author credit. Closing as superseded.