perf(dflash): widen the verify window past the compact graph's four-row limit

[coderbench]

Summary

The compact verify graph added in #689 only accepts windows of up to four rows, and dflash_generate treats its refusal as a verify failure and stops generating — so raising SPARKINFER_DFLASH_PROPOSALS past 3 collapses DFlash today instead of deepening the block. This keeps the compact graph for the widths it supports, adds a row-batched window for wider blocks, and raises the default proposal depth to 5, which the wider window makes affordable.

Proof of speedup

• Tested on RTX 5090 (sm_120)

Decode tok/s (end-to-end, DFlash eval prompt, 2-rep alternating A/B, same box, same build flags):

	decode tok/s
before (main)	572.71
after (this PR)	631.17

+10.2% DFLASH_TPS. Mean accept length rises 3.657 → 5.333.

Prefill pp tok/s — not targeted by this PR; prefill is unchanged (see the guard tables below).

# main @ 1340617 vs this PR, alternating, /tmp/dflash_eval_ids.txt (seed=fixed, 101-token prompt)
main rep1 METRIC DFLASH_TPS 573.4310   MEAN_ACCEPT 3.6571
PR   rep1 METRIC DFLASH_TPS 630.0493   MEAN_ACCEPT 5.3333
main rep2 METRIC DFLASH_TPS 571.9981   MEAN_ACCEPT 3.6571
PR   rep2 METRIC DFLASH_TPS 632.2938   MEAN_ACCEPT 5.3333

mean: main 572.71 -> PR 631.17   (+10.2%)

What changed

dflash_verify_short_run declines any window past four rows. That refusal reaches dflash_generate as vfail, which breaks the decode loop, so an unsupported width truncates the output rather than degrading. batched_forward now falls back to verify_block instead, and the compact path is gated on kDFlashCompactMaxRows so it is only asked for windows it can serve.

Wider blocks verify their proposal tail as one row-batched window: one CTA per output row holding R accumulators, so each Q4_K/Q8_0 super-block is read once and dotted against every row. The seed token keeps its own forward so it still overlaps the draft. Only the routed experts scale with the window, since each token routes to its own top-8.

The window forwards proposals a rejection will discard, so the recurrent and conv state is snapshotted at the window head and rewound when the tail is not fully accepted, then the accepted prefix is re-advanced through the window rather than a token at a time. KV needs no truncation: later steps overwrite the stale rows and attention only reads seqlen entries.

Why the proposal depth moves 3 -> 5

Measured on the eval prompt with this PR: depth 3 = 584.2, depth 4 = 589.8, depth 5 = 633.6, depth 6 = 600.9, depth 7 = 538.2. Depth 3 continues to run through the compact graph unchanged.

Exactness

Every row is computed with the same operand order, partial-sum order and reduction tree as the single-row kernel, and there is no cross-row reduction, so greedy speculative decoding stays exact.

METRIC SPEC_AGREE 32/32 = 1.0000
VERDICT PASS

ctest --test-dir build — 10/10 passed.

No-regression guard (same PR build)

Each number is the mean of two sweeps run in opposite orders, because a single pass drifts about 1% in favour of whichever build runs first.

Qwen3.6-35B-A3B

ctx	decode before	decode after	ratio	prefill before	prefill after	ratio
128	527.14	527.03	1.000	—	—	—
512	517.68	517.60	1.000	9027.90	9013.36	0.998
4k	496.74	496.64	1.000	24903.92	24890.95	1.000
16k	498.48	498.89	1.001	26139.19	26136.13	1.000
32k	499.05	498.62	0.999	27245.37	27239.30	1.000

Qwythos / Qwen3.5-9B

ctx	decode before	decode after	ratio	prefill before	prefill after	ratio
128	298.40	297.96	0.999	—	—	—
4k	287.33	287.40	1.000	22916.29	22837.94	0.997
32k	287.13	286.50	0.998	23613.25	23535.33	0.997
64k	286.18	286.03	1.000	23447.05	23373.04	0.997
128k	285.75	285.63	1.000	22697.47	22623.48	0.997

Escape hatches

• SPARKINFER_DFLASH_ROWS=0 — restore the token-loop verify for the wide-window path.
• SPARKINFER_DFLASH_COMPACT_VERIFY — unchanged (0=off, 1=force, 2=adaptive).
• SPARKINFER_DFLASH_PROPOSALS=<n> — proposal depth, as before.

[skyrocket2026]

sparkinfer dflash auto-eval — eval-dflash:REJECT

metric	value
label	eval-dflash:REJECT
PR DFlash tok/s	438.40
main DFlash tok/s	479.87
speedup vs main	0.91× (-8.6%)
PR AR tok/s	260.69
DFlash vs AR	1.68×
mean accept τ	5.333
accuracy	METRIC SPEC_AGREE 32/32 = 1.0000
Qwen3.5/3.6 guard	✅ no regression (decode + prefill)
Polaris receipt	59ea9198bad1d4cc… — TDX (Intel hardware attestation)
commit	119c38058

DFlash regression: 438.40 < 98% of main 479.87

_{Scored on pinned RTX 5090 vs same-box origin/main DFlash, gated by a same-build Qwen3.5/3.6 decode+prefill no-regression guard. AR eval:* labels are frozen (casual bidir eval retired).}