Skip to content

Model M (D3D12): bottleneck kernels at 192 VGPRs under ACO (-110 us GPU time per frame at 1440p on RADV, byte-identical) - #292

Merged
phantoslayer21 merged 2 commits into
phantoslayer21:mainfrom
j0nsh1n:perf/m-bneck-192-aco
Nov 20, 2026
Merged

phantoslayer21 merged 2 commits into
phantoslayer21:mainfrom
j0nsh1n:perf/m-bneck-192-aco

Conversation

@j0nsh1n

@j0nsh1n j0nsh1n commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Two exact changes that bring Model M's bottleneck kernels to 192 VGPRs under RADV/ACO, where they were allocated 256 (5 waves per SIMD, one workgroup per CU). vgpr_limits.json already bounds them at 192 for the Windows compiler, so this is about how ACO allocates them on Linux. Output bytes are unchanged.

  • Attention fragments read in the K loop (bneck.hlsl): attention() read all 16 fq/fv fragments before the two WMMA chains. That one spot, at 193 registers, pushed the whole unfused body over 192. Each K group's two fragments are now read right before their products. Chain order and operands are unchanged.
  • Fused patch-merge prologue (fused_bneck_gen.py, regenerated fused_bneck.hlsl): the second weight stripe's loads take an address term that depends on merge pass 0's first result and is 0 at run time (the same opaque zero as zb), and MB_CH goes from 8 to 4. Accumulation order and operands are unchanged. This needs the first commit: alone it leaves mbo at 256.

Measured on main 11d3bbc with only these two commits, RX 9070 XT, Wine + vkd3d-proton + RADV, synthetic model, overclocked profile. Both arms use one runtime DLL built from main; only the three bottleneck DXIL files are swapped.

  • No-card gfx1201: mb, mbo and nc go from 256 to 192 VGPRs (8 waves/SIMD), no spills.
  • Byte identity: checked at 96x60, 1279x721 and 2560x1440 output, on every dump a frame writes, against two base runs from the same session. A wrong operand in the merge is caught.
  • Timing at 1706x960 -> 2560x1440:
    • summed GPU dispatch time 2609.6 -> 2499.9 us (-109.7 us, -4.2 %);
    • bottleneck 487.2 -> 407.7 us, dec2's xo block 204.2 -> 187.4 us;
    • host frame time with the profiler on -116 us, with it off -36 us (noisier).

Not tested: the Windows driver, where RGA already meets the 192 bound, and RDNA 3. Reviewed before opening by two other models. It is independent of #280 and #290 and applies to main on its own.

A note from checking: fused_bneck_gen.py --check rewrites a stale fused_bneck.hlsl and exits 0, so it can't detect a stale file. That is the same on main; not changed here.

A correction to the second commit's message: its "63 per size" counted three dec1.out dumps, but on these fused pack12 routes dec1 runs inside dec0 and doesn't write out1. The written set is 60 dumps per size, which are all equal; the identity claim holds. The control's count over the 60 was not re-derived, because those dumps are gone.

A note for anyone repeating the byte checks: some dumped buffers hold no data for a frame, depending on the route that actually ran. Examples are b0-b4.out, enc1.merged, dec2.out and post.display on the fused route, and the previous-state buffers on a reset frame. They are allocated without a clear, so they differ from run to run. We compared only dumps that are stable between two base runs from the same session, and dec0.C on its active bytes. compare.py skips them for the routes it tests, but not for every route (for example enc2.merged with WG_SWIN_MID=1, or post.display on a direct frame where D = O).

Checklist

…ll 16 ahead (swin_bneck6_nc 256 -> 192 VGPRs on RADV)

attention() in bneck.hlsl read all 16 fragments fq[8] / fv[8] from LDS
(16 ds_read_b64, 32 registers) before the two WMMA chains. Under RADV/ACO
(CU mode) that is the one spot, 193 registers (ACO_DEBUG=liveinfo), where the
whole unfused body crosses the 192-VGPR line, so swin_bneck6_nc was allocated
256 VGPRs: 5 waves per SIMD and one workgroup per CU instead of 8 waves and
two. clean/dx12/vgpr_limits.json bounds the bneck kernels at 192 for the
Windows compiler (RGA), and bneck.hlsl's header says one workgroup per CU runs
25-27 % slower, so this is a cliff seen under ACO.

Each K group's two fragments are now read right before their products. The
aq and av chains keep their order (kc2 = 0 .. 7, alternating) and operands:
no WMMA accumulation order or operand changes, only when the fragments are
read. fused_bneck.hlsl is regenerated (the committed file equals the
generator's output).

On its own this helps the unfused kernel (WG_D3D12_FUSION without mb / xo);
the fused kernels (mb, mbo: the default) stay at 256 VGPRs because of their
merge prologue, which the next commit addresses. The measured result of both
commits together on main is in the next commit.

Checks (RX 9070 XT, gfx1201, Wine + vkd3d-proton + RADV, synthetic model,
overclocked LACT profile; times are summed GPU dispatch times):
- No-card gfx1201: swin_bneck6_nc 256 -> 192 VGPRs, 8 waves/SIMD, no spills
  (5748 -> 5877 instructions); mb / mbo unchanged at 256.
- Byte identity on the run-stable dumps at 64x40 -> 96x60, 853x481 -> 1279x721
  and 1706x960 -> 2560x1440, with and without the mb / xo fusions; a wrong
  fragment index (kc2 ^ 1) in attention() is caught.
- Timing at 1706x960 -> 2560x1440 with both arms unfused (mb / xo off): the
  bottleneck 477.8 -> 377.4 us, the frame 2419.6 -> 2319.7 us. With the
  default fusion (mbo) 490.6 -> 490.3 us: no effect, no regression.

Not tested: the Windows driver, RDNA 3 (the bneck DXIL does not compile on the
no-card gfx1100 route here, dxil-spirv opcode 79).

Signed-off-by: Jonathan Shin <85975340+j0nsh1n@users.noreply.github.com>
… (mb / mbo 256 -> 192; -110 us GPU time per frame at 1440p on RADV)

The persistent bottleneck kernels with enc2's patch merge fused in
(bneck/swin_bneck6_mb_nc and _mbo_nc; mbo is the default) were allocated 256
VGPRs by RADV/ACO: 5 waves per SIMD and one workgroup per CU, instead of the
192 that clean/dx12/vgpr_limits.json bounds them at for the Windows compiler
(RGA). The peak is not one spot: in the merge phase every fragment batch
(MB_CH = 8 K groups x 2 tiles = 16 ds_read_b64, 32 registers) sits on a base
level of 172-190 registers (zf[24] 48, the second weight stripe sv[6] 24 held
since before the barrier, addresses), reaching 216-230 (ACO_DEBUG=liveinfo).

Two changes to fused_bneck_gen.py, both only about when loads happen:
- the second weight stripe's global loads take an address term that depends
  on merge pass 0's first packed result (tile 0) and is 0 at run time
  (res[0][0].x & (wg_grid_ >> 31), the opaque zero bneck.hlsl already uses as
  zb), so ACO cannot hoist them above that point; it interleaves them with the
  rest of pass 0, which keeps fewer values live;
- MB_CH 8 -> 4 (fragment batches of 8 reads; 4 divides 24).
The 24 K groups of every tile still accumulate in the same order from the
bias, with the same operands: no WMMA accumulation order or operand changes.
fused_bneck.hlsl is regenerated (the committed file equals the generator's
output).

With the previous commit (which removes the body's own 193-register spot) the
kernels compile to 192 VGPRs / 8 waves per SIMD under ACO; this commit alone
leaves mbo at 256, so both are needed.

Checks on main 11d3bbc with these two commits (RX 9070 XT, gfx1201, Wine +
vkd3d-proton + RADV, synthetic model, overclocked LACT profile; one runtime
DLL built from main for both arms, only the three bottleneck DXIL files
swapped):
- No-card gfx1201: mb, mbo and nc 256 -> 192 VGPRs, 8 waves/SIMD, no spills
  (mbo 7278 -> 7495 instructions).
- Byte identity at 64x40 -> 96x60, 853x481 -> 1279x721 and
  1706x960 -> 2560x1440 on every dump a frame writes (63 per size; the
  unwritten fused-route buffers excluded, dec0.C compared on its active
  bytes), against two same-session base runs: equal. A wrong operand index in
  the merge's first accumulator is caught (29/63 equal).
- Timing at 1706x960 -> 2560x1440, 5 interleaved rounds x 60 frames, shader
  clock median 3.05-3.09 GHz, no throttle. Summed GPU dispatch time per frame
  (frames kept when the unchanged input encoder is within 3 % of its run's
  minimum, 267 of 300 per arm): 2609.6 -> 2499.9 us (-109.7 us, -4.2 %;
  unfiltered -114.1 us); the bottleneck 487.2 -> 407.7 us, dec2's xo block
  204.2 -> 187.4 us. Host frame time with the profiler on 2.700 -> 2.584 ms
  (-116 us, faster in 5 of 5 rounds); with it off -36 us (noisy, 4 of 5
  rounds faster).

Not tested: the Windows driver (where RGA already meets the 192 bound), and
RDNA 3 (the bneck DXIL does not compile on the no-card gfx1100 route here).

Signed-off-by: Jonathan Shin <85975340+j0nsh1n@users.noreply.github.com>
@phantoslayer21
phantoslayer21 merged commit 09a7ca9 into phantoslayer21:main Nov 20, 2026
10 checks passed
j0nsh1n added a commit to j0nsh1n/wronggpu that referenced this pull request Nov 20, 2026
phantoslayer21#293); recompute kSourceDigest on the combined sources

phantoslayer21#290 changed the same kSourceDigest line in clean/k/runtime/k_smx_selfcheck.h.
The value is recomputed with clean/k/dx12/smx_selfcheck_digest.py on the
merged tree (stale() is None; test_k_smx_selfcheck_digest passes). No other
conflict.

Signed-off-by: Jonathan Shin <85975340+j0nsh1n@users.noreply.github.com>
@j0nsh1n
j0nsh1n deleted the perf/m-bneck-192-aco branch November 20, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants