Repository navigation
Model L kernels: encoder and decoder blocks, input linears, projections, coefficient step (HIP) - #115
Merged
Merged
Conversation
…rid mirror, FP16 output), input linears, both projections, coefficient step; GPU checks Model M's block template gets a gather-source hook (default unchanged; all Model M code objects byte-identical). Refs #89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
…G_ENC_VALUES_OUT, off by default), its GPU check; map-copy test Refs #89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
…column share its depth and motion (width-17 GPU case); FP16 criterion, rounding-boundary inputs and coefficient codes from the real block output in the GPU checks; Model M code-object comparison script Refs #89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
phantoslayer21
marked this pull request as ready for review
November 14, 2026 01:31
This was referenced Nov 14, 2026
Merged
j0nsh1n
pushed a commit
to j0nsh1n/wronggpu
that referenced
this pull request
Nov 14, 2026
…r HIP kernels l_ops_linear.hlsl ports l_linear_32_32_relu and l_linear_32_32_f16 of l_ops.co (clean/l/kernels/l_ops.hip, phantoslayer21#115). No branch around the matrix instructions: a lane past the end of the map stores its clamped pixel's bytes to that pixel (a branch around the store let the compiler sink the WMMAs into it; wrong values in a partly filled wave). l_d3d12_ops_check.py compares any l_ops.co kernel's D3D12 port with its HIP kernel: raw bytes, each back end's output filled with its own sentinel, a negative control. l_d3d12_check.py gains prepare_replay (the ports and kernel_replay.exe), shared by both harnesses. Refs phantoslayer21#89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
j0nsh1n
pushed a commit
to j0nsh1n/wronggpu
that referenced
this pull request
Nov 14, 2026
…eir HIP kernels l_ops_project.hlsl ports l_project_gather and l_dec_project of l_ops.co (clean/l/kernels/l_ops.hip, phantoslayer21#115): two FP8 WMMAs then RN16 per group of 32 inputs, the gather's clamp, FP8 out; no branch (a lane past the last token stores its clamped token's bytes to that token's place). l_d3d12_ops_check.py gains their cases (6 token grids, two of them partly filled waves). Refs phantoslayer21#89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #89
Stacked on #113 (the reference these kernels are checked against). Review the last commit only.
HIP kernels for Model L's encoder and decoder, to the interface in
clean/l/FORMATS.md. Each is checked on the GPUagainst the oracle step it implements.
l_swin_block_c32_enc[_nc]l_swin_block_c32_dec[_nc]l_linear_32_32_relu/_f16l_project_gatherl_dec_projectl_coeflev/input_encoderencoder.hipbuilt withWG_ENC_VALUES_OUT=1(l_enc_values.co): the 20 dithered values of every pixel of the padded grid plus a 2-pixel halo into the halo map (values at the mirrored render position, dither at the unmirrored one), and Model M's motion surfacesModel M's input encoder (
clean/kernels/encoder.hip) gets a build switch,WG_ENC_VALUES_OUT, which is off by default.Model M's block template (
clean/kernels/swin.hip) gets one hook: a gather-source template parameter whose defaultis the existing gather. Model L passes its own sources for the FP16 halo and the token-grid mirror. The FP16 output
uses the existing epilogue hook.
What is proven and what is open
Proven:
clean/l/reference/oracle.py) on synthetic data,within Model M's per-stage rule: at most 2 % of values more than one FP8 step off, and mean absolute error at most
1 % of the reference RMS. The counts are below.
_ncblock kernel is byte-identical to its clamped form whenever the weights allow_nc.build.py --no-runtime --variantsproduces are byte-identical withand without this PR, including the
encoder.hipswitch.surfaces are within Model M's rule for them.
Open:
the WMMA accumulation rule. The probes so far fit no candidate for full-range inputs, and that work is parked.
rounding or as errors. The same holds for the single FP16 C value of
l_coef.l_coefuses the hardware exp2 and reciprocal. The oracle's exponential and tanh are exact, so code-levelidentity there is measured, not proven.
Still to come: the D3D12 port of each kernel (kern2), each in its own PR.
Tested
GPU record: RX 9070 XT, locked GPU slots, 2 seeds, synthetic data, head of this PR.
clean/runner/build.py --no-runtime --variantsfrom this branch and from its base,including the encoder fix: all 84 code objects byte-identical. The comparison is now a script,
clean/runner/m_codeobjects_compare.py --base <ref> --work <dir>.clean/l/runner/l_c32_check.py:off.
l_coefagainst the oracle's functions on that same output: A, B and latent codes 0 differ; 1 of 45,696 Cvalues differs, by 1 FP16 ulp.
(max 1, 1, 2), and 12 of 45,696 C values.
_ncforms identical in 32 of 32 cases. The existing C = 32 block: 0 values differ.clean/l/runner/l_ops_check.py:(0 differ).
and overflow to infinity; FP8 ties and saturation after RN16; the FP16 linear on +-448 inputs with large weights.
l_coefon random FP16 maps: A, B and latent codes 0 differ; 1 of 40,434 C values differs, by 1 FP16 ulp.clean/l/runner/l_enc_check.py, 12 frames, including reset, odd sizes and render width 17 (padded 32, where twohalo columns mirror onto column 0):
reject.
python -m unittest discover -s clean/l/tests -t .(34 tests, one skipped until map v1.1 is indata/).Criteria:
1 % of the reference RMS, and no non-finite value where the reference is finite. The FP16 linear (one rounding)
must be within 1 ulp.
(Model M's encoder test).
(the latent code is 127 tanh), so a code can move by more than 1 where the block differs.
l_coefitself is heldto the strict bound on identical input.
Not tested
float64 oracle with Model M's per-stage rule.
table.UNCONFIRMEDU1): not exercised.