qwen36: run the DeltaNet layer end to end on the card (Q36_DN_GPU=1): conv, recurrence and gated norm join the two projections

[kreuzzelg]

With the trunk in VRAM (#1657) a decoded token on the 35B still spent about 8 of 39 ms in the thirty DeltaNet layers' convolution, L2 norms, recurrence and gated norm: dn-sub: proj 7.9 | conv 3.0 | l2n+rec 2.4 | norm+out 2.8. None of that is arithmetic -- the convolution is 33k multiply-adds per token, the recurrence 32 outer products of 128 x 128 -- it is the round trip. The in_proj's result came down to the host, the recurrence ran there, its output went back up for the out_proj: four copies and their synchronisations per layer, thirty layers per token. (Found while reading why Ollama 0.34.4 decodes this model at 36 tok/s against our 25: llama.cpp keeps the whole gated-delta layer on the card.)

What the PR does

Backend (coli_cuda_dn_*, five entry points, optional in the DLL loader): one object per DeltaNet layer holds the short-conv ring, the recurrent state [vh x kdim x vdim], the conv and norm weights and one token's scratch in VRAM. coli_cuda_dn_step runs a decode token end to end: x up, in_proj GEMV, causal conv + SiLU with the ring advancing, per value head the L2 norms (double, eps inside the sqrt as on the CPU), the decay, the delta against the decayed state, the write-back, out = q^T S over the updated state, the gated RMSNorm with silu(z), out_proj GEMV, out down. The gates (a/b, 32 outputs) stay on the CPU and travel as kernel parameters. One upload, one download of 8 KB per layer; the state never crosses the bus. Same arithmetic as deltanet(), so the CPU path stays the reference; the fake backend carries a host-side copy of the steps and can be told to fail.

Tier (qt_dn_gpu_*): where the placer put a layer's in_proj (qt_dnproj_init) and out_proj (qt_dense handle) on one device, the layer joins them there. The host copy of the state stays the engine's, moved with set/get_state; get_state works after a failed step, so the engine continues from what the card holds.

Engine (Q36_DN_GPU=1, opt-in): a decode token computes the two gates and hands x to qt_dn_gpu_step. The host arrays remain canonical, with two flags per layer: the device copy is fresh (nothing to upload before a GPU step) or ahead (download before any CPU use). A prompt (S > 1 keeps the batched CPU path) pulls first, a --pin snapshot pulls all layers, reset_recurrent and a pin restore invalidate the device copy, a failing GPU step pulls what the card holds and continues on the CPU. Prefix reuse and the pin machinery are untouched by construction: they only ever see host arrays that are current.

Placer, two small things the measurement needed: auto puts a layer's dnout on the device its dnproj went to when that device has the room (without it 28 of 30 layers qualified on two cards), and COLI_PLACE accepts experts=all -- the routed experts stay on every COLI_GPUS card even when a hand-written list reserves some for the trunk (the form for "trunk on the fast card, experts on both", which the reservation rule of #1361 could not express).

Measured (RTX 3070, per-row int4 35B, 49-token prompt, 200 decoded tokens, trunk in VRAM, COLI_TIMERS=1, output byte-identical between two GPU runs)

	Q36_DN_GPU=0	Q36_DN_GPU=1
DeltaNet ms/token	16.2	10.8
token ms	39.4 / 39.3	33.9 / 33.3
tok/s (engine line, TTFT included)	21.2 / 21.3	24.1 / 24.4
decode-only tok/s	25.4	30.0
layers on the card	0	30

The 10.8 ms that remain are the two GEMVs (16 and 8 MB per layer) and the two copies. With only lmhead,dnproj,dnout placed the gain is the same (41.5 -> 35.3 ms). On two cards the layer gains the same 6-7 ms (44.3 -> 37.5 with the trunk split, 40.1 -> 32.9 with the trunk on the 3070 and experts=all), but this box's second card is a Quadro RTX 4000 that runs a dense GEMV about 65 % slower (lm_head 7.3 against 4.4 ms), and at 99-100 % residency the MoE phase only drops from 14.0 to 11.2-13.6 ms (take 5.2 ms/token waiting on the slower card's groups), so two cards end level with the 3070 alone. That is a property of this pair of cards, not of the layer.

Not byte-identical to the CPU path (the GEMVs sum in a different order, as for every placed component); two GPU runs are identical to each other. Against Ollama 0.34.4 on the same box (36.4 tok/s decode, Q4_K_M, both cards) the gap goes from 43 % to 20 %.

Verified

• tests/test_qwen36_dn_gpu.c (fake backend, in make check): one in-memory DeltaNet layer driven through deltanet(): GPU outputs equal the CPU outputs; the host copy is left alone while the card advances and reads back the CPU path's state; a two-row CPU prefill after GPU tokens pulls first and continues, the GPU token after it pushes first; a failing step turns the layer off and the CPU continues from the card's state.
• tests/test_qwen36_tier_autoplace.c gains the two placer cases (dnout follows dnproj; experts=all keeps both cards).
• All existing qwen36 tier tests unchanged and green; CPU-only build carries the stubs; colibri (the loader host) builds; the loader ABI tripwire counts the five optional symbols.
• Not exercised on the real container yet: --pin snapshots and prefix reuse under Q36_DN_GPU=1 (the code paths only read host arrays, pulled first, and the fake test covers the pull/push order; a serve smoke on the container is what I would run next if you want it before merging).

Where this goes next

c/delta_attention.h says it: the same gated delta step runs the linear-attention layers of Kimi K3 (KDA), GLM-5.3-Flash and Qwen3.8 as well; only the dimensions and the gate differ, and the gate belongs to the engine. The backend object here takes those dimensions as parameters (vh <= 64, kdim/vdim <= 256), so the other engines can hand their layers to the same five calls. I kept this PR to qwen36 so the numbers are the engine's own; happy to follow with qwen38 (48 value heads) if you want the backend step to be the shared one, or to move it behind delta_attention.h's naming if you would rather it mirror coli_kda_step exactly.

Thanks to Claude Code for the help along the way.

[kreuzzelg]

Withdrawing this for now: the two-card numbers above showed that the layer on the card only pays off fully once the experts are homed per layer range instead of per expert index (no per-layer join on the slower card). I will bring both back together in one PR once that is measured; nothing here is abandoned.