[zhijian-liu]

Summary

Adds the Qwen3.5-122B-A10B family, so unsloth/Ornith-1.5-122B-GGUF:UD-Q4_K_XL serves with its DFlash2 draft, incoai/Ornith-1.5-122B-DFlash2. Three commits:

1. Read a split GGUF whose first part holds the metadata alone. llama.cpp's gguf-split writes that part with no tensors and no padding after the header, and the parser refused it as truncated. unsloth's Ornith GGUFs are split this way.
2. Serve Qwen3.5-122B-A10B and its fine-tune Ornith-1.5-122B.
   • It has the Qwen3.6-35B-A3B's architecture and layers at a larger size: hidden 3072, 48 layers, 32 query heads over 2 KV heads, 64 GDN value heads, experts of 1024. The draft was trained for a block of 16; a draft's block_size may now be 8 or more, and a cycle drafts the first 8 rows.
   • Families and kernel shapes are data, so another size of an architecture is one value, plus kernel instances and one table row for a shape no family had:
     • A layout type is an architecture, whose code its families share. A family is a value of it, named on the shared base layout, and every family is one entry of model::kFamilies, which refusals, family lookup and the tests walk. The 122B is kQwen3_5_122BLayout, a Qwen3_6MoeLayout with no code of its own.
     • A config's family is the one of its model type whose seven sizes it holds. A config of none is refused as it differs from the family it matches furthest.
     • GDN and attention kernels come from one table of compiled shapes each (kCompiledGdn, kCompiledAttention), with names built from a base and the shape's suffix. That replaces the per-shape enums, if-chains and ~10 literal name lists. New rows: GDN for 64 value heads (_vh64) and attention for 16 query heads per KV head (_kv2_g16).
     • The kernel tests take every family's GDN and attention shapes from kFamilies (dev/tests/engine/TestFamilies.hpp), so a new family's kernels are tested with no test edits.
   • One rule for wide GQA groups, SPLASH_ATTENTION_TILE_PARTS: a dense tile runs in the fewest parts that divide its 8 rows with at most 64 fused rows each. Existing tiles stay one part and are unchanged.
3. Read the draft's own mask embedding when its release ships one.
   • The mask token's row in Qwen3.5-122B-A10B's embedding table, which Ornith inherits, is of order 1e-6. That is below the smallest fp16 block scale of Q8_0 and the K-quants, so a GGUF's row dequantizes to zeros and draft acceptance halves.
   • A DFlash2 release may now ship mask_embedding.safetensors, one BF16 tensor of the hidden size. Installation fetches it, and one dispatch after the draft-input gather writes it to each lane's mask rows.
   • Drafts without the file load and run as before. incoai/Ornith-1.5-122B-DFlash2 ships it as of hub commit 9baefc1.

Performance

M5 Max (128 GB), unsloth/Ornith-1.5-122B-GGUF:UD-Q4_K_XL, all weights resident, before and after the mask embedding:

	Before	After
Backend benchmark, 1 / 2 / 3 / 4 lanes (tok/s aggregate)	71 / 129 / 149 / 187	150 / 277 / 356 / 443
Backend benchmark, tokens per cycle	3.05	6.40
HTTP coding, 1 stream (median tok/s)	40	105
HTTP coding, 4 concurrent (tok/s aggregate)	83	180

Prefill doesn't use the draft and is unchanged: a 2,048-token chunk takes 0.78 s.

Attention parts for the group of 16, from interleaved attention-sweep --shapes 122b rounds:

• Two 64-row parts ran 20–40% faster than one 128-row tile of 512 threads (32K prefill 67 vs 95 ms; 128K verify 1.21 vs 1.70 ms).
• Splitting further was slower at long histories.
• Outputs are bit-identical across part counts.

Verification

• make check-source check-native-build, make test-engine-cpu and make check-python (1,319 tests, ruff) pass.
• Metal tests under shader validation pass on every family's shapes: GDN prefill and decode, paged-attention plans and Q8 flash attention (all four geometries), plus draft preparation, model loading and runtime resources.
• The runtime oracle passes (fp32 and bf16 state) with the GGUF and its experts streamed.
• Text, tool calls and vision work over HTTP.
• make check-native-build also passes at the middle commit on its own.

Not covered:

• The MLX 4-bit build (ornith-ai/Ornith-1.5-122B-MLX-4bit) needs --language-only, because the installer refuses its separate vision shard. So the README gives no MLX example for this family.
• No llama.cpp parity check.

🤖 Generated with Claude Code