Repository navigation
Model L: C = 32 Swin block kernel (HIP) from Model M's block template - #102
Merged
Merged
Conversation
…its GPU check clean/l/kernels/l_swin.hip includes clean/kernels/swin.hip for its templates only (WG_SWIN_NO_EXPORTS, as dec1_dec0.hip does) and exports l_swin_block_c32[_nc] (2 heads, MLP 128) with Model M's block arithmetic, packed weights and launch geometry, plus l_swin_block_c64 (the template at Model M's width) for the check that this file builds it as swin.co does. clean/l/runner/l_c32_check.py (GPU, synthetic): the C = 32 kernel against the oracle, _nc against the clamping kernel, and l_swin_block_c64 against Model M's swin_block_c64. Refs #89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
…rence; README rows Refs #89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
This was referenced Nov 13, 2026
phantoslayer21
marked this pull request as ready for review
November 13, 2026 23:04
This was referenced Nov 13, 2026
j0nsh1n
pushed a commit
to j0nsh1n/wronggpu
that referenced
this pull request
Nov 14, 2026
…ts HIP kernel l_swin.hlsl instantiates swin_block.hlsl as l_swin_block_c32_nc (C = 32, 2 heads, MLP 128, HIP's GELU chain and MLP unit cut), the D3D12 port of phantoslayer21#102's HIP kernel. swin_block.hlsl (and the generated fused_swin.hlsl) gain the RMS-norm scale for C = 32 (1/32); Model M's DXILs are unchanged. vgpr_limits.json bounds the kernel at 96 VGPRs and build_dx12.py checks l_swin_* kernels too. kernel_replay.cpp runs single kernels on the native D3D12 device from case files; l_d3d12_check.py drives HIP and D3D12 on the same synthetic inputs and compares whole output buffers byte for byte, with Model M's swin_block_c96_nc as a control. Refs phantoslayer21#89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step P8 of Model L. Refs #89
clean/l/kernels/l_swin.hipincludesclean/kernels/swin.hipfor its templates only (WG_SWIN_NO_EXPORTS, asdec1_dec0.hipdoes) and exportsl_swin_block_c32/l_swin_block_c32_nc(C = 32, 2 heads, MLP 128). The arithmetic, packed weights (clean/runner/packing.pypack_block) and launch geometry are Model M's block kernels'. It also exportsl_swin_block_c64, the same template at Model M's width, used only to check that this file builds the template exactly asswin.codoes.clean/l/runner/l_c32_check.py: the GPU check below.No existing file changes except
clean/l/README.md; Model M's code objects are untouched.What the GPU check proves
Run of 2026-11-13 on an RX 9070 XT:
python clean/l/runner/l_c32_check.py --build <dir> --rocm <ROCm> --seeds 8, synthetic data only (48 cases: map sizes 5x7 to 48x64, window offsets (0,0), (2,0), (0,2), (2,2), random C = 32 weights and FP8 inputs).l_swin_block_c32against the NumPy oracle (clean/l/reference/oracle.pyswin_blockwith the kernels' sigmoid GELU): 6 of 1,407,744 output values differ, each by exactly one FP8 step. No value is more than one step off and none is non-finite. The worst case has 0.0077 % of its values differing, and its mean absolute error is 2.4e-6 of the reference RMS. Model M's per-stage rule (at most 2 % of values more than one step off, mean error at most 1 % of RMS) holds in every case.l_swin_block_c32_nc(no inner clamps, used when the weights' flags prove the range) gives byte-identical output tol_swin_block_c32in all 48 cases.l_swin_block_c64(this file's build of the template) gives byte-identical output to Model M'sswin_block_c64fromswin.coin all 48 cases.What stays unproven while the FP8 WMMA accumulation rule is open
Tested
python -m unittest discover -s clean/l/tests -t .: passes (CPU tests, unchanged by this PR).python tools/ci/check_line_endings.py: 0 problems.Not tested