Skip to content

Model L: D3D12 port of the C = 32 Swin block, byte for byte against its HIP kernel - #117

Merged
phantoslayer21 merged 5 commits into
mainfrom
model-l/c32-d3d12
Nov 14, 2026
Merged

phantoslayer21 merged 5 commits into
mainfrom
model-l/c32-d3d12

Conversation

@phantoslayer21

@phantoslayer21 phantoslayer21 commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

Model L (#89): the D3D12 port of Model L's C = 32 Swin block, byte for byte against its merged HIP kernel
(l_swin_block_c32_nc of l_swin.co, #102).

  • clean/dx12/kernels/l_swin.hlsl (new): instantiates swin_block.hlsl as l_swin_block_c32_nc:
    • C = 32, 2 heads, MLP 128;
    • HIP's transcendental GELU chain (SW_GELU=0) and HIP's MLP unit cut (SW_HPU=8);
    • the _nc form, as Model M's ports.
  • clean/dx12/kernels/swin_block.hlsl: the RMS-norm scale had only the C = 64 and 96 cases. This adds SW_C == 32
    (scale 1/32). Without it, most output bytes differed. fused_swin.hlsl, generated from it, gets the same two lines
    (fused_swin_gen.py --check: up to date). Model M's DXILs are unchanged (below).
  • clean/dx12/vgpr_limits.json: l_swin_block_c32_nc bounded at 96 VGPRs. clean/dx12/build_dx12.py checks
    l_swin_* kernels too.
  • clean/dx12/tests/kernel_replay.cpp (new): runs single kernels on the native D3D12 device (the project's Device
    API) from a case file of modules, a launch and its arguments.
  • clean/l/runner/l_d3d12_check.py (new): the harness. --prepare (CPU) builds the HIP code objects, the D3D12
    ports and kernel_replay.exe. --run (one GPU slot) runs each case on HIP, then the same inputs through
    kernel_replay on D3D12, and compares the whole output buffers with bytes ==. Counts only, no timing.

What "byte for byte" is checked on:

  • the same synthetic token map (FP8, seeded), window offsets, grid size and packed weight blob (packing.pack_block at
    C = 32, heads 2, MLP 128) on both back ends;
  • the raw downloaded bytes of both outputs compared, the full FP8 output map, with no tolerance. Nothing is decoded
    and encoded again, so for example the two FP8 NaN codes stay distinct;
  • each output buffer filled beforehand with its own non-zero sentinel: HIP 0xA5, D3D12 0x5A (kernel_replay's
    arg out takes a fill byte). A byte that either path, or both, leaves unwritten therefore differs;
  • a negative control: one case again on D3D12 with half its workgroups, which the comparison must catch;
  • Model M's swin_block_c96_nc (HIP swin.co against its existing D3D12 port) as a control on the same harness, so the
    harness itself is proven too.

Refs #89

Tested

Local, Windows, Radeon RX 9000 series card, main cf8e8ac plus this commit:

  • GPU (one GPU slot, 12.5 s), l_d3d12_check.py --run --seeds 6, raw bytes and sentinels (after the review):

    • l_swin_block_c32_nc: 48 of 48 cases identical (6 seeds x 8 grid / offset cases; 23,911,488 output bytes in
      all; 0 differing bytes in every case);
    • the Model M control swin_block_c96_nc: 48 of 48 identical (71,734,464 bytes, 0 differing);
    • since the two sentinels differ, identical outputs mean every byte was written by both back ends;
    • the negative control (half grid) was caught: 1,248 differing bytes, all of them still the D3D12 sentinel;
    • kernel_replay: 0 cases failed.

    The first run, before the review fixes, decoded and re-encoded the HIP output and zero-filled both outputs: also 48 of
    48 and 48 of 48.

  • --prepare (CPU): builds clean. VGPR check: l_swin_block_c32_nc 95 / 96, swin_block_c96_nc 192 / 192.

  • Model M's DXILs unchanged: the 9 Swin kernels of swin_block.hlsl, swin_c64w2.hlsl and fused_swin.hlsl
    (swin_block_c96_nc, swin_block_c96_add_ex_nc, swin_block_c64_nc, swin_block_c96_pm_nc,
    swin_block_c96_pm_rev_nc, swin_block_c96_add_xe_rev_nc, swin_block_c96_add_xe_nc, swin_block_c96_add_xe_xo_nc,
    swin_block_c96_add_xe_xd_nc) were built from main and from this branch under the same output names
    (build_dx12.py --only ... -j 1). DXIL SHA-256: 9 of 9 identical (for example swin_block_c96_nc
    0d8bea45c7803504..., swin_block_c64_nc 9f7c47f23ff1174c..., swin_block_c96_add_xe_xd_nc 2767e993c6ca1d46...).

  • python tools/ci/check_line_endings.py: 0 problems. Public gates (secrets, personal data, licence, chunks) pass.

Not tested

  • Whole-frame compare.py for Model L (waits for the L runtime frame).
  • The FP16-input form of the block (comes with its HIP kernel, then its port).
  • kernel_replay.exe is not built by CI (it needs a GPU to be useful).

…ts HIP kernel

l_swin.hlsl instantiates swin_block.hlsl as l_swin_block_c32_nc (C = 32, 2 heads,
MLP 128, HIP's GELU chain and MLP unit cut), the D3D12 port of #102's HIP kernel.
swin_block.hlsl (and the generated fused_swin.hlsl) gain the RMS-norm scale for
C = 32 (1/32); Model M's DXILs are unchanged. vgpr_limits.json bounds the kernel at
96 VGPRs and build_dx12.py checks l_swin_* kernels too. kernel_replay.cpp runs single
kernels on the native D3D12 device from case files; l_d3d12_check.py drives HIP and
D3D12 on the same synthetic inputs and compares whole output buffers byte for byte,
with Model M's swin_block_c96_nc as a control.

Refs #89

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
…ative control

Review of #117: the HIP output's downloaded bytes are compared as they are (no
decode and encode, so the two FP8 NaN codes stay distinct), and each back end's
output buffer starts filled with its own non-zero sentinel (HIP 0xA5, D3D12 0x5A;
kernel_replay's arg out takes a fill byte), so a byte either path leaves unwritten
differs. A negative control runs one case on D3D12 with half its workgroups and must
be caught.

Refs #89

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
@phantoslayer21
phantoslayer21 marked this pull request as ready for review November 14, 2026 01:36
@phantoslayer21
phantoslayer21 merged commit 4ed7882 into main Nov 14, 2026
9 checks passed
@phantoslayer21
phantoslayer21 deleted the model-l/c32-d3d12 branch November 14, 2026 01:53
j0nsh1n pushed a commit to j0nsh1n/wronggpu that referenced this pull request Nov 14, 2026
…l-linear-d3d12-pr

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant