refactor(gemma4): run Gemma 4 layers as declarative Steps; calibration tools on the step program (carries #795)

[fivetide]

Targets beta. Rebased onto beta e268a0798; carries the #795 Qwen3.5 declarative port (its slices are the first 14 commits). Tracking: #666 G6.

Design and per-slice ledgers: docs/design/gemma4-declarative-program.md, docs/design/qwen35-declarative-program.md.

1. Gemma 4 declarative refactor

Ports #795's execution model (typed Step lists, executors and route choice in hipfire_dispatch, arch crate = declarations + bindings) to hipfire-arch-gemma4. Every Gemma 4 layer is

[SandwichAttention, SandwichMlp | ParallelMoeMlp, PerLayerInput?, Scale?]

declared in program.rs; the executors live in hipfire-dispatch/src/pipeline/sandwich.rs (model-neutral sandwich-norm ops). One program serves decode (rows == 1), batched prefill and EAGLE verify (rows > 1).

Slice
12B dense and E2B/E4B (PLE, KV sharing, double-wide FFN): decode, batched prefill, EAGLE verify; ~1.6k lines of hand layer code out of forward.rs
26B-A4B MoE (ParallelMoeMlp), lowered decode on the shared program, arch-22 EAGLE/MTP draft head as one step list (query-only attention over the target cache); lower_variant / Gemma4Bindings super-op path and HIPFIRE_FORWARD_LOWERED hand arms deleted
Calibration tools (calib_sweep, eval_hipfire, prefill_parity_gemma4) on the program; dispatch-owned calibration taps; old batched prefill deleted

Behaviour changes:

• 26B-A4B: the lowered stack loads MQ4G256V2 (qt 44) and runs MQ4G256V2 and MQ6G256 routed experts on the indexed kernels, which is what the current quantizer emits. The old local gemma4-26b-a4b.mq4 produced garbage on beta because its HFQ4-G128 expert down_proj (K = 704) was packed across rows by the quantizer before b4846285e; such files, and expert formats without an indexed kernel pair, now refuse at load. A fresh requantization of google/gemma-4-26B-A4B-it runs coherently (serve battery clean, Redline --pm4 shadow exact with 1202 launches, hipGraph decode equal to direct decode, prefill_parity_gemma4 same continuation).
• Removed developer switches: HIPFIRE_GEMMA4_FUSED_{FFN,QK,QK_ROPE,POSTNORM,ATTN_NORM,PROJ} (fused routes always on; greedy EAGLE HIPFIRE_GEMMA4_EAGLE=1 still turns off the two non-bitwise fusions), HIPFIRE_BATCHED_PREFILL / HIPFIRE_WMMA_PREFILL, HIPFIRE_GEMMA4_{BASELINE_ATTN,ATTN_VERIFY,GEMM_VERIFY}, HIPFIRE_MOE_{BYPASS,BUCKETED}.
• Fix: eval_hipfire's Gemma arm sat inside the llama branch, so Gemma models scored 0 tokens.

2. Rebase onto beta

Beta changed code this branch moves into new dispatch files, so those edits produced no textual conflict. They are carried into the moved copies, each in the commit that moves the code:

• pipeline/batched.rs (shared SwiGLU FFN): packed-MQ4 gate/up and down routes, the gfx1151 A4 gate/up epilogue (FfnGateOutput::Iu4A4, HIPFIRE_V2B_A4_EPI), s4_residual_fast on supports_dflash_f16_residual_fusions().
• pipeline/batched_attention.rs: fa2_gfx11_ctx_admitted for the wide Q8 attend; gfx1151 in the multirow assert.
• families/fused_qkv.rs: the MQ6G256/HFQ6G256 arms of fused_qkv(za)_key_for.
• mtp_head.rs: i_gpu_start: 0 in the MTP MoE config (build break on beta otherwise); qwen35_tensor_name_candidates stays pub.
• gemma4/lowered.rs: beta's gemma4_decode_flash_partials_len, lowered hipGraph capture (lowered_graph_binding, now capturing the program body) and Q8 full tier are kept.

Fixes on top:

• hybrid::swiglu_down_residual (Qwen3.5 dense decode) had dropped the generic arm of llama::weight_gemv_swiglu_residual: MFP4/MQ8/MQ4G128/ParoQ4G128 w_down refused with UnsupportedVariant on the first token. Restored, together with beta's memory.offload_exec=cpu arm.
• railgun-cert recording inventory follows the moved predicates; the sandwich calibration check is renamed calibrating (the scanner read capturing as graph capture).
• Ledger/doc bookkeeping: CHANGELOG entries move into v0.4.1 (check-changelog), dispatch-bypass rows qwen35 184 → 90 and gemma4 17 → 2 (bypass_slack 0), env-vars inventory and crate maps regenerated, changed files rustfmt'd. The beta-side formatting churn in the loader is dropped.

3. Evidence (gfx1151, against beta)

Beta daemon 94070f1ce9ce722fcbf4dcf7cc87da58 (e268a0798) vs this branch ea7b9af9f33b242a81a400486f169f02.

Gemma 4:

• Greedy daemon, 5 prompts per row, token stream and per-step top-16 logits bitwise: 12B AR, 12B prefill batch 64, 12B HIPFIRE_GEMMA4_EAGLE=1 (strict), E2B, E4B, E2B/E4B prefill batch 64.
• EAGLE (gemma-4-12B-it-assistant q8f16; opt-in on beta via HIPFIRE_GEMMA4_EAGLE=1): text identical, per-prompt tau 3.368/3.459/2.667/3.447/3.514 on both.
• 26B-A4B q8-experts: 3/5 identical, 2 diverge late (chars 295, 322), coherent (shared fused qk-norm+RoPE / post-norm routes).
• serve_harness.py --thinking off --sampling greedy --compare-transcript: 12B battery + chain, E4B battery, 26B battery transcript_byte_identical=true.
• 26B Redline --pm4 --skip-prefill: shadow exact on both; 1083 launches (beta 1263), tape hash f2675ac01238613c (beta 9f57d0ca48889bda); decode 38.2 vs 38.1 tok/s.
• prefill_parity_gemma4 (725 tokens): batched vs per-token same argmax, identical 24-token continuation; 12B 33.8 s → 4.2 s, 26B 19.2 s → 8.2 s.
• calib_sweep 12B: coverage 328/328, Hessian consistency 0, q/k/(v) and gate/up identity PASS.
• eval_hipfire vs the BF16 reference (ea5e297378c7ff692013af63d8411323), gemma4-12b.mq4, Q8 KV: per-token 0.733489, prefill 0.824203; both .kldseq byte-identical to the pre-rebase branch. The absolute level is the -it checkpoint on raw WT2 and the MQ4 recipe, not the engine (HF transformers cross-check in the previous revision of this description: hipfire F32 oracle vs HF F32 KL < 1e-4).

Qwen3.5 family (--compare-transcript, all transcript_byte_identical=true): qwen35-4b.mq4 battery, chain, 2k-token prompt; qwen3.8-27b.mq4-xts 2k-token prompt (MQ4V2 + AWQ, A4 epilogue route); ornith-1.5-35b-a3b.mq4r AR and MTP; qwen3.6-35b-a3b.mq4r MTP; qwen3.5-9b.mq4 DFlash and a 40k-token prompt (wide FA2 prefill past 32K). MFP4 Qwen3.5-4B greedy decode byte-identical to beta (the pre-rebase head failed it). qwen3.5-9b.mq4 with 20 resident layers and offload_exec=cpu byte-identical to beta, every host-mapped step on the CPU. qwen35-4b.mq4 Redline --pm4 shadow exact (426 dispatches).

Not measurable on this box: the packed-MQ4 FFN (gfx1100 opt-in), s4_residual_fast (gfx1100/gfx1201), MQ6/HFQ6 Qwen3.5 checkpoints (key mapping covered by the forward_slots unit test), gfx1201 graph capture of the lowered program.

Known gaps

• Two Gemma 4 weight stacks remain (gemma4.rs eager, lowered.rs MoE) with duplicate config/loaders; both bind the same program.
• The MoE expert-format refusal runs at carrier load, after the daemon unloads the prior model; admission does not probe expert dtypes yet.
• CI checks that also fail on beta e268a0798: leanup-ratchets daemon_lines (5392 > 5377), cargo test --lib in peacemaker-ir (two codec tests) and hipfire-arch-qwen4 (state_parity GPU test), cargo clippy in hipfire-xdna. railgun-cert/src/recording.rs is not rustfmt-clean on beta; formatting it would rewrite ~1.8k lines, so this branch leaves it as is.