perf(gfx1100): speed up long-context DFlash verify 13.4% with FA2 split-KV and retain HipGraph/Redline PM4

[HUSRCF]

Summary

Make the exact-gfx1100 FA2 split-KV S8 backend the default long-context Q8 DFlash verifier inside its measured envelope. The pinned product A/B improved from 42.6 to 48.3 tok/s (+13.38%) at unchanged tau and cycle count. The same fixed-grid route is retained through HipGraph and Redline/PM4 with live-context admission, compiler-free kernels, stable scratch, typed replay pointer effects, and fail-closed fallback.

This is the gfx1100 companion to the split-KV schedule evaluated in #760. It compares against—but does not claim authorship of—the existing R4/R8 multi-row verifier from #741. The implementation keeps hipfire's in-tree packed-Q8 KV path; it does not vendor or load CK source or a CK .so.

The branch is rebased onto beta@175b71fb3. The two semantic commits are 024f0b15c (kernel/retained route) and 52f2ef1d3 (default policy, review follow-up, and current-beta API adaptation). The performance A/B below was collected on the same split kernel/route before the rebase at beta@d5305333d; the final current-beta four-arm Redline rerun is reported separately below.

Default and admission policy

The route defaults on only on exact gfx1100. Either of these restores the established verifier path:

• kernel.gfx1100_fa2_split_verify=false / HIPFIRE_GFX1100_FA2_SPLIT_VERIFY=0
• parent switch kernel.verify_attn=false / HIPFIRE_VERIFY_ATTN=0

Admission is exact gfx1100, plain Q8 KV, dense Qwen H24/KV4/D256, sequential non-tree batches 4..32, and live logical context >4096. HIPFIRE_FA_PERTOKEN_MIN_CTX=0 disables the route; positive overrides below 4096 are clamped to the measured crossover. Preallocated max_seq does not influence admission.

HipGraph/retained capture additionally requires the complete precompiled split module, fixed-address Q16 scratch, and enough flash_partials capacity for the S8 layout. The real PM4 loader checks the target's actual workspace at fixed B=16. Unsupported shapes or not-ready resources fail closed to the established batched path; they cannot be recorded under a split identity.

Maintainer-review follow-up

• Removed the blanket rustfmt churn. kernel_registry.rs is a small semantic delta rather than a multi-thousand-line reflow, and the other affected files preserve current-beta formatting.
• Moved the pure split-verifier model/route admission into hipfire-dispatch; the Qwen arch crate only adapts GPU/config state into the dispatch predicate. This reduces the port surface for the declarative-Steps work in refactor(qwen35): run Qwen3.5 layers as declarative Steps #795.
• Honored the stable VerifyAttn parent opt-out and added an exact-gfx1100 route-specific stable config/escape hatch.
• Enforced dense/Q8/H24-KV4-D256/B4..32/live-context admission at both model and launcher boundaries.
• Split ChainVerifySplit from ordinary ChainVerify, so capture-only F16/LDS helpers are not widened below the crossover or on fallback routes.
• Kept retained recording fail-closed while preserving existing gfx1151/gfx1201 HipGraph behavior.
• Added preconvert, partial, merge, and direct symbols to one compiler-free gfx1100 module.
• Included the caller-owned S8 partial workspace in eager, HipGraph, Redline shadow, and product PM4 admission.
• Adapted the Gate 0 oracle to the current load_dflash_state workspace-capacity contract, so workspace admission is checked consistently in the example and product loader.

Which surface(s) does this touch?

• kernel — kernels/, crates/rdna-compute
• load — carries target scratch capacity into retained-route admission
• serve — crates/hipfire-generate, speculative verify and Redline plumbing
• arch crate(s): hipfire-arch-qwen35
• crates/hipfire-quantize / quant formats
• control plane — stable config key and environment alias
• docs / CI / scripts only (no hardware route)
• policy files

Performance

Optimization A/B: established R4/R8 vs FA2 split-KV

W7900, exact gfx1100, HIP 7.15, Q8 VMM KV, graph off, six fresh processes in order off,on,on,off,off,on, one unrecorded warmup per process:

verifier attention route	samples (tok/s)	median	tau / cycles
established R4/R8	43.5, 42.6, 42.5	42.6	1.97 / 67
FA2 split-KV S8	48.5, 48.3, 47.8	48.3 (+13.38%)	1.97 / 67

The unchanged tau/cycles isolate the improvement to verifier execution rather than draft acceptance.

Batch-16 kernel screen (10 warmups, 30 measurements):

logical context	R4/R8	split S8	speedup
8,192	368.80 us	177.36 us	2.079x
20,676	828.45 us	410.72 us	2.017x
32,768	1,369.01 us	648.21 us	2.112x

S1 is bit-identical to direct FA2. S8 relative L2 is about 2.46e-4 to 2.58e-4 with cosine about 0.999999970. Direct/partial/merge compile at 254/255/18 VGPR with zero spill and zero private scratch.

HipGraph capture neutrality: split route off vs on

Both arms below already use the new split-KV route; this is not the optimization A/B above. It checks only whether retaining that route in HipGraph changes output or speed.

split-KV execution	decode	result
graph off	4.8 tok/s	tau 2.06, 200 generated tokens
graph on	4.9 tok/s	tau 2.06, 200 generated tokens

The B16 graph captured 706 launch blobs. Transcripts were byte-identical, MD5 7b8dc5b28daef60f803fe2a466c888b2, with identical request MD5 8a54e9aa236f678362f89864bf000125. Both cold serve_harness.py runs ended inside the model's hidden thought channel and were flagged RUNAWAY,EMPTY, so these rates are not a performance or answer-quality claim. This run establishes only graph-off/on route parity; the 42.6→48.3 fresh-process native bench above is the performance A/B.

Final current-beta Redline/PM4 manual screen

On the final current-beta daemon, the four-arm manual oracle covered HipAuto, capture-safe direct HIP, recorded HIP, and PM4 at 12 consecutive B16 windows (positions 8176..8352):

• PASS; all four arms agreed byte-for-byte on tokens, argmax, ring head/written state, GDN frame, active KV hashes and guards, PBS guard, hidden staging/ring, final hidden, logits, GDN QKV/alpha/beta, recurrent state/scales/conv/error-feedback after forward and rollback.
• Capture contained 711 typed launches and 18 AQL contracts, with zero untyped launches or slot failures. Preparation covered 711 dispatches on one queue/phase.
• 17 successful replays; there were zero contract, prepare, replay, or poison failures.
• Five interleaved timing windows: HipGraph verify body 45.168 ms median / 45.277 ms p95; PM4 42.714 ms median / 43.006 ms p95, or -5.43% median latency and -5.02% p95 latency.

This is explicitly a manual capture/shadow correctness and timing screen, not a complete product timed-arm route proof or certification. Full immutable earlier route record: docs/perf-checkpoints/2026-10-05-gfx1100-fa2-splitkv-verifier-graph-pm4.md.

Test plan

• cargo build --release -p hipfire-cli -p hipfire-daemon
• cargo test -q -p hipfire-dispatch --lib — 311 passed, 1 ignored
• cargo test -q -p hipfire-arch-qwen35 --lib — 255 passed, 23 ignored
• cargo test -q -p rdna-compute --lib — 525 passed, 17 ignored
• cargo check -p hipfire-arch-qwen35 --example cb_vmm_state_oracle
• Current-beta W7900 four-arm manual Redline harness — exact parity across 12 B16 positions and no timing regression
• Crate maps for hipfire-arch-qwen35, hipfire-config, hipfire-dispatch, hipfire-generate, hipfire-loader, and rdna-compute
• env-doc scan, lifecycle check, and git diff --check
• ./scripts/no-gpu-ci.sh — Rust checks/tests and examples proceed after the Gate 0 API fix; the Python tail is 553 passed / 21 failed because scripts/hw-gate/select.py shadows Python's stdlib select while spawning review.py. This reproduces outside the socket-restricted sandbox and is unrelated to this PR's path.
• scripts/ci-rustfmt-changed.sh — advisory only; current beta is not rustfmt-clean in these full files, and applying it would recreate the unrelated multi-thousand-line reflow this revision removes.
• ./scripts/speed-gate.sh — this route is not a locked speed-gate case; the claim uses the fresh-process A/B above.

No gfx12 hardware rerun is included: this admission is exact gfx1100, and gfx1201 keeps its separate existing split-KV implementation. Static negative admission tests protect the cross-architecture boundary.

Final validation used target SHA-256 9f91556f7e0431a077d03756a7102d0154108757289e6e5fe9a2d204c0c9eeb7, pinned qwen3.8 draft SHA-256 d0a74a232a0e2166d889f823e91e0fbf778d21dd9668d7de055cdecb065401bc, CLI SHA-256 9d09edb8a60ee84756a8a6257077e03716c6736713ab740a0c06fc0e76ef42ea, and daemon SHA-256 3725b575f31fb5185fe2ab602586fee7c1463f42b92e3a4e2391f7fbadb0503a.

The local global config pointed at an older qwen3.6 draft, so the final hardware rerun explicitly set HIPFIRE_DFLASH_DRAFT to the pinned qwen3.8 artifact above; the daemon log confirms that exact file/digest was loaded.

Architecture-trait change?

No Architecture trait change.

[fivetide]

why opt in? sounds like no reason not to be default..

[HUSRCF]

@fivetide Thanks — I agree that default-on is probably the right final policy within the admitted envelope. I initially kept it opt-in because the validation scope was narrow, but a follow-up audit found a few things I would like to fix before flipping the default:

• HIPFIRE_VERIFY_ATTN=0 does not currently suppress the new split route, conflicting with the documented master opt-out.
• HipGraph admission is broader than the launcher’s exact H24/KV4/HD256 envelope, so a not-ready or unsupported capture can fall into an insufficiently validated multi-row path instead of a guaranteed batched fallback.
• The new partial/merge symbols are not yet present in the release kernel registry, leaving compiler-free cold installs uncovered.
• The same flag also enables capture-time F16 projection/LDS-stage paths below the 4K attention threshold, which is broader than its documented meaning.
• The PR says dense-only, but the eager/HipGraph admission does not currently enforce num_experts == 0.

Also, the measured kernel matrix starts at 8K, so I would like to pin the 4K crossover with a small boundary A/B before making it the product default.

I’ll rebase onto current beta now that the gfx1201 split-KV work has landed, tighten these guards, add the compiler-free and opt-out tests, and then make the exact admitted gfx1100 route default-on with HIPFIRE_GFX1100_FA2_SPLIT_VERIFY=0 retained as the escape hatch.

[HUSRCF]

@fivetide Thanks — I have now made the route default-on inside the exact measured gfx1100 envelope and rebased the PR onto current beta@212347998.

The follow-up is commit 4f07ac0bd. It now:

• honors both kernel.verify_attn=false / HIPFIRE_VERIFY_ATTN=0 and the route-specific kernel.gfx1100_fa2_split_verify=false / HIPFIRE_GFX1100_FA2_SPLIT_VERIFY=0 escape hatch;
• requires exact gfx1100, plain Q8 KV, dense H24/KV4/D256, sequential non-tree B4..32, and live context above the resolved 4K crossover;
• separates ChainVerifySplit from ordinary ChainVerify, so the capture-only F16/LDS helpers are not enabled below the crossover or on fallback routes;
• packages preconvert/partial/merge/direct in the compiler-free gfx1100 module;
• keeps capture fail-closed without changing the existing gfx1151/gfx1201 HipGraph paths; and
• checks the actual S8 flash_partials capacity in eager, HipGraph, Redline shadow, and product PM4 B16 admission, so a small workspace cannot record a batched fallback under a split identity.

The rebased head passes all-target compile checks for rdna-compute, hipfire-arch-qwen35, hipfire-generate, and hipfire-loader, plus the targeted default/opt-out, threshold, registry, capture fallback, and undersized-workspace tests; crate maps, env/lifecycle docs, changed-file rustfmt, and diff checks also pass. I did not rerun gfx12 hardware because this admission is exact gfx1100 and gfx1201 keeps its separate existing split-KV backend; a static negative admission test covers that boundary.

I also updated the PR body to distinguish the prior pinned performance evidence from the current-beta admission/packaging follow-up.

[fivetide]

Review notes (gfx1151 / Strix Halo, ROCm 7.2)

Follow-up on 4f07ac0bd: all five issues you listed are addressed, and default-on inside the exact gfx1100 Q8 H24/KV4/D256 >4K envelope (with both opt-outs) is what I asked for. Thanks.

Remaining:

• 🟠 medium — 4f07ac0bd rustfmt-reflows all of crates/rdna-compute/src/kernel_registry.rs (~3.9k changed lines, mostly unrelated add!/include_str! rows), plus feature_flags.rs and attention.rs. That reflow is why the PR conflicts with beta in kernel_registry.rs, and it hides the ~10 real registry rows. Please rebase with only the semantic hunks; the rustfmt CI job is advisory.
• 🟡 low — the prefill.rs / dflash_spec.rs / speculative.rs edits will conflict with refactor(qwen35): run Qwen3.5 layers as declarative Steps #795 (qwen35 → declarative Steps). Whichever lands second has to port; the ChainVerifySplit admission is easy to carry over if it stays a dispatch-level predicate.

I'll run a gfx1151 DFlash no-regression check (byte-identical output vs beta) once it's rebased.

_{Agent-assisted review; head 4f07ac0. Evidence: static read of the delta since my 2026-10-05 comment.}

[HUSRCF]

@fivetide Thanks — the branch is now cleanly rebased onto current beta@2b91b1fdd and force-pushed at b8eb3ab28.

Both remaining review items are addressed:

• The blanket rustfmt churn is gone. kernel_registry.rs is now only a 23-line semantic delta (22 additions / 1 deletion), and I preserved current-beta formatting elsewhere rather than rerunning full-file rustfmt.
• The pure gfx1100 split-verifier admission now lives in hipfire-dispatch; the Qwen arch crate only adapts GPU/config state into that predicate, reducing the future refactor(qwen35): run Qwen3.5 layers as declarative Steps #795 declarative-Steps port surface.

I also reran the required hardware gate on the final rebased HEAD, not the older checkpoint. On a W7900 / exact gfx1100, the Redline harness exercised HipAuto, capture-safe direct HIP, recorded HIP, and PM4 across 12 consecutive B16 positions (8176..8352). Tokens, argmax, logits, hidden staging/ring, final hidden, active KV hashes, GDN intermediates, and recurrent state after forward/rollback were byte-identical across all arms. Five interleaved timing windows measured HipGraph at 44.550 ms median / 44.874 ms p95 and PM4 at 41.908 ms / 42.323 ms, so there is no replay-path performance regression (-5.93% median latency).

The current-beta unit/build/map/env/lifecycle/diff/format-bomb checks listed in the updated PR body pass. no-gpu-ci.sh has one unrelated current-beta fixture failure (ServeRuntime missing vmm_batch), for which #841 carries the fix; I did not fold that unrelated change into this PR.

The PR body now records the new base/head, semantic cleanup, final hardware evidence, and artifact/binary hashes. The rebased branch is ready for the gfx1151 byte-parity no-regression check you offered.

[HUSRCF]

@fivetide I rebased the branch again because beta advanced, and the PR is now cleanly based on 175b71fb3 with final head 52f2ef1d3 (024f0b15c + 52f2ef1d3). The rebase kept the two semantic commits and added only the current-beta API adaptations, including passing the real target flash_partials capacity into the Gate 0 oracle.

I also reran the gfx1100 four-arm manual Redline screen with the pinned qwen3.8 draft explicitly forced (the local global config otherwise overrode the harness argument with an older qwen3.6 draft). The loaded draft was SHA-256 d0a74a232a0e2166d889f823e91e0fbf778d21dd9668d7de055cdecb065401bc; the final daemon SHA-256 is 3725b575f31fb5185fe2ab602586fee7c1463f42b92e3a4e2391f7fbadb0503a.

HipAuto, capture-safe direct HIP, recorded HIP, and PM4 remained byte-exact across 12 B16 positions (8176..8352), including tokens/argmax, active KV and guards, hidden/logits, GDN intermediates, recurrent state, and rollback state. Capture had 711 typed launches, 18 AQL contracts, zero untyped launches; 17 replays completed with zero contract/prepare/replay/poison failures. Across five interleaved windows, HipGraph was 45.168 ms median / 45.277 ms p95 and PM4 was 42.714 ms / 43.006 ms (-5.43% median, -5.02% p95). Per docs/VALIDATION.md, I have described this narrowly in the body as a manual capture/shadow parity and timing screen, not a complete product timed-arm certification.

Static/unit status on the final base: dispatch 311 passed / 1 ignored, qwen35 255 / 23, rdna-compute 525 / 17; release CLI+daemon build, env-docs, lifecycle, six crate maps, example check, and diff-check pass. no-gpu-ci.sh now gets past the earlier Gate 0 example API failure; its Python tail remains 553 passed / 21 failed because scripts/hw-gate/select.py shadows stdlib select when review.py is spawned. I retained the small semantic formatting rather than reintroducing the unrelated multi-thousand-line rustfmt churn requested for removal in the prior review.

The PR body is updated with the new base/head, exact fixture identities, current hashes, rerun numbers, and these two disclosed validation limitations.

CI addendum: the two failing GitHub checks reproduce unchanged on beta@175b71fb3 itself (beta run 38014621157). Clippy under Rust 1.99 denies the test fixture range 4..3 in crates/redline-rocr/src/runtime.rs:3480 as clippy::reversed_empty_ranges. The workspace unit job fails only the two hipfire-isa M7 tests whose fixtures attempt to create hard-coded /home/kaden/qcal/release-0.4.2/... directories and receive PermissionDenied (67 passed, 2 failed). The beta workspace build passes. These are shared current-beta CI blockers, not changes introduced by #817, so I have not mixed their fixes into this performance PR.