Skip to content

Model L: encoder chain and decoder front kernels (HIP) - #104

Closed
phantoslayer21 wants to merge 4 commits into
mainfrom
model-l/enc-chain-hip
Closed

phantoslayer21 wants to merge 4 commits into
mainfrom
model-l/enc-chain-hip

Conversation

@phantoslayer21

@phantoslayer21 phantoslayer21 commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

Do not merge: superseded. A later answer on Model L's structure moves the encoder's extra network and both width-32 blocks to render resolution and puts the decoder projection before its block. This PR is being reworked to that structure (same building blocks: the C = 32 block of #102, pointwise linears, a 2x2 gather projection). The check records below are for this version.

Step P9 of Model L, stacked on #102 (the C = 32 block, which these chains use). Refs #89

  • clean/kernels/encoder.hip: a compile-time switch WG_ENC_VALUES_OUT (default 0). With the switch on, the kernel stores the quantized values as a token map [fh/2][fw/2][80] instead of projecting them. Model L builds it as its own code object (l_enc_values.co). With the switch off the file preprocesses to the same text, and encoder.co is byte-identical to main's (checked: same output name; the build's -cuid comes from the output file name, so differently named outputs always differ).
  • clean/l/kernels/l_ops.hip:
    • l_linear_80_32 and l_linear_32_32: pointwise FP8-map linears with merge / expand section blobs;
    • l_project_80_32: the output projection over [raw | x | zeros], with an FP16 running sum per group of 32 channels, the clamp, and FP8;
    • l_copy_channels: splits and joins channel ranges for the decoder block.
  • clean/l/runner/l_chain_check.py: the GPU check below.
  • clean/l/FORMATS.md: the kernels, arguments, blobs, buffers and launch order of a Model L frame (Model M's frame with its encoder and decoder steps replaced), and the launch names (l_enc_values.co under the tag lev).

What the GPU check proves

Run of 2026-11-13 on an RX 9070 XT: python clean/l/runner/l_chain_check.py --build <dir> --rocm <ROCm> --seeds 4, synthetic data only. 24 cases: render sizes 40x24, 64x36 and 97x55, both parities, 4 seeds, random Model L weights. Every kernel is run in isolation on the GPU's own output of the step before and compared with the oracle's step on that same input (clean/l/reference/oracle.py, the block with the kernels' sigmoid GELU):

Step Values Differ Largest difference Non-finite
values (input_encoder of l_enc_values.co) 2,293,760 91 1 FP8 step 0
l_linear_80_32 917,504 0 – 0
l_linear_32_32 917,504 0 – 0
l_swin_block_c32 917,504 16 12 FP8 steps 0
l_project_80_32 1,835,008 0 – 0
whole encoder chain (GPU features vs the oracle's encode) 1,835,008 1,775 16 FP8 steps 0
split (l_copy_channels) 917,504 0 – 0
decoder front (split, block, join) vs the oracle 1,835,008 60 6 FP8 steps 0
  • Both linears and the output projection are byte-identical to the oracle in every case. The join gives exactly the block's output and dec1's untouched channels in 24 of 24 cases.
  • Model M's per-stage rule (at most 2 % of values more than one FP8 step off, mean absolute error at most 1 % of the reference RMS) holds in every case and step. The largest share of values more than one step off is 0.18 % of a case (encoder chain); per isolated block it is 0.003 %.
  • The multi-step differences come from the block: an intermediate value that rounds differently in FP32 (kernel) and float64 (oracle) changes the softmax of its window, as described for Model M's blocks in clean/tests/test_kernels_emu.py. The chain carries such differences into the projection.

What stays unproven

  • As in Model L: C = 32 Swin block kernel (HIP) from Model M's block template #102: while the FP8 WMMA accumulation rule is open, there is no byte-exact CPU model of the WMMA sums. The block and the chain are checked against the float64 oracle by Model M's tolerance rule. The linears and the projection happen to match the oracle exactly on these cases; that does not prove they always will.
  • The values kernel is Model M's per-pixel code (unchanged). Its one-step differences are the FP32 vs float64 values before the dither quantization, as in Model M's enc0.

Tested

  • GPU check above; encoder.co of this branch byte-identical to main's (e6500a8), same output name.
  • python -m unittest discover -s clean/l/tests -t .: passes.
  • python tools/ci/check_line_endings.py: 0 problems.

Not tested

  • The runtime does not launch these kernels yet (the runtime frame for Model L is a later step).
  • The host lane emulator (it needs Linux).

… check

clean/kernels/encoder.hip: compile-time switch WG_ENC_VALUES_OUT (default 0)
that stores the quantized values as a token map [fh/2][fw/2][80] instead of
projecting them; built as its own code object for Model L. With the switch
off the file preprocesses to the same text and encoder.co is byte-identical.

clean/l/kernels/l_ops.hip: l_linear_80_32 and l_linear_32_32 (pointwise FP8
map linears, merge / expand section blobs), l_project_80_32 (the output
projection over [raw | x | zeros] with an FP16 running sum per group of 32
channels, clamp and FP8), l_copy_channels (splits and joins channel ranges
of token maps for the decoder block).

clean/l/runner/l_chain_check.py (GPU, synthetic): every kernel in isolation on
the GPU's own previous output against the oracle's step, the whole encoder
chain against the oracle, and the decoder front.

Refs #89

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
Refs #89

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
Refs #89

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
Refs #89

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
@phantoslayer21

Copy link
Copy Markdown
Owner Author

Superseded by #115 (the same kernels, on the confirmed layout), which is merged. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant