Skip to content

Model L on D3D12: the two input linears in one kernel - #160

Merged
phantoslayer21 merged 1 commit into
mainfrom
model-l/lin-fuse
Nov 14, 2026
Merged

phantoslayer21 merged 1 commit into
mainfrom
model-l/lin-fuse

Conversation

@phantoslayer21

Copy link
Copy Markdown
Owner

Model L speed on D3D12, third change: the two input linears in one kernel. l_linear_32_32_fused (D3D12 only) gives
the same bytes as the pair l_linear_32_32_relu -> l_linear_32_32_f16; x1 stays in registers. Nothing is
approximated, and the pair stays as the fallback route.

What changes

  • clean/dx12/kernels/l_ops_linear.hlsl: the fused kernel.
    • The relu linear's lane (t, h) stores x1 as exactly the two FP8 fragments that the f16 linear's lane (t, h)
      loads, so the fragments pass in registers with no shuffles.
    • Its RN16, FP8 pack and FP16 conversions use the driver's instructions (v_cvt_f16_f32, v_cvt_pk_fp8_f32 after
      the clamp; wg_platform.hlsli: the same bits as the software forms).
    • The software forms were most of the pair's instructions: fused with them, the kernel measured 0.236 ms, slower
      than the pair. With the driver's conversions its ISA is 1,928 bytes, against 11,300 for the software form.
    • The pair's own code and DXILs are unchanged (byte-identical to main's).
  • clean/runtime/runtime.cpp: new native D3D12 fusion ll, on by default for Model L.
    • In this route the input encoder writes q to l_narrow. The pair writes x2 over q in l_wide; a fused kernel
      doing that would overwrite q that other workgroups have not read yet.
    • The pair runs (the fallback route) with WG_D3D12_FUSION without ll, or when the manifest lacks the kernel.
    • The kernel report adds l_lin_fused=on|off. compare.py's copy of the default fusion list is updated.
  • clean/l/runner/l_linfuse_check.py (new): the identity check below.
  • Tests: the recorder knows the kernel's arguments. The frame and role checks take either linear form, and the D3D12
    frame test also runs without ll and without the kernel.

Refs #89

Byte identity (clean/l/runner/l_linfuse_check.py, one GPU slot; record records/linfuse-identity.json in the
rr-plugin build folder)

  • The same inputs run three ways: the fused kernel on D3D12, the pair on D3D12 (two kernel_replay passes), and the
    pair on HIP.
  • Raw bytes; 64 bytes of slack after the output, filled with each run's own sentinel (HIP 0xA5, the D3D12 pair 0x5A,
    the fused kernel 0x3C).
  • Cases: 9 map sizes (1 to 8,976 pixels; partly filled waves and workgroups) at 3 weight scales (normal; large:
    results past 448 and past FP16's range; tiny: FP16 subnormal results), every other case with special FP8 codes in
    q (both NaN codes, +-448, +-0, subnormals); 2 seeds.
    • The outputs hold 56,354 infinities, 370,999 FP16 subnormals and 1,060,425 values above 448, and x1 holds 222,520
      bytes at the 448 clamp.
  • 54 of 54 cases identical in all three comparisons (7,731,072 output bytes), and every slack byte kept its own
    sentinel.
  • Negative control (the fused kernel with half its workgroups): caught, 73,680 bytes differ.
  • Model M's swin_block_c96_nc control through the same replay: 8 of 8.

Frame check (clean/l/runner/l_frame_check.py --compare-d3d12, synthetic models; record
records/linfuse-frame-check.json)

  • The default D3D12 run uses the fused kernel (l_lin_fused=on, with l_dec_fold=on); the all-outputs run (fusions
    off) uses the pair.
  • Model L: every stage against the reference 11 of 11 frames; the default kernels against the all-outputs run byte for
    byte 11 of 11; its post against Model M's post reference 11 of 11.
  • Model M: D3D12 against HIP 11 of 11. Pass.
  • Against main's build, every recorded array of both models' D3D12 runs (default and all-outputs, 11 frames each) is
    byte-identical (record records/linfuse-frame-vs-main.json).

Timing (l_d3d12_profile.py, 1080p Quality, synthetic models). Build A is main's build; build B is this change.
Two sets of 3 interleaved rounds, every run in its own GPU slot; records records/linfuse-ab.json and
records/linfuse2-ab.json.

  • The pair: l_linear_32_32_relu + l_linear_32_32_f16 = 0.2161 .. 0.2215 ms per round. The fused kernel: 0.0938 ..
    0.0999 ms (sums of the per-kernel medians 0.2167 -> 0.0956 and 0.2176 -> 0.0938 ms: -121 / -124 us).
  • l_swin_block_c32_enc_nc, which reads x2 next: 0.268 .. 0.271 -> 0.275 .. 0.280 ms in every round (+5 .. +9 us).
  • Whole-frame span: A 2.490 / 2.490, 2.504 / 2.486 / 2.480 ms; B 2.387 / 2.381 / 2.379, 2.375 / 2.385 / 2.375 ms.
    • About -110 us per frame. A's first round of the first set (3.247 ms, with every kernel slower) is left out.

Tested

  • The identity check, the frame check and the timing above.
  • clean/l/runtime/tests/test_l_runtime_frame.py (18 tests); public gates.

Not tested

  • In a game (the next game timing run).
  • Real model weights: the arithmetic is the pair's, byte for byte, including the special values above.
  • No NaN reaches x1 or x2 in the cases (relu takes a NaN accumulator to 0, as in the pair). NaN inputs pass through
    the fused kernel's RN16 conversion before that.

l_linear_32_32_fused (D3D12 only): l_linear_32_32_relu and l_linear_32_32_f16 in one
kernel, x1 kept in registers (the relu linear's lane stores exactly the two FP8
fragments the f16 linear's lane loads), byte for byte the pair. Its RN16, FP8 pack and
FP16 conversions are the driver's instructions (the same bits as the software forms the
pair keeps, which were most of the pair's instructions).

- runtime: native D3D12 fusion "ll" (default on; Model L). The input encoder then writes
  q to l_narrow (x2 over q in l_wide would overwrite q that other workgroups have not
  read yet). Without ll, or without the kernel in the manifest, the pair runs.
- clean/l/runner/l_linfuse_check.py: the fused kernel against the pair on D3D12 and on
  HIP, raw bytes, per-run sentinels in the slack, special FP8 codes and extreme weight
  scales, negative control, Model M control.
- tests: the recorder's signature; the frame and role checks take either linear form.

Refs #89

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
@phantoslayer21
phantoslayer21 marked this pull request as ready for review November 14, 2026 10:32
@phantoslayer21
phantoslayer21 merged commit d79338f into main Nov 14, 2026
9 checks passed
@phantoslayer21
phantoslayer21 deleted the model-l/lin-fuse branch November 14, 2026 10:33
phantoslayer21 added a commit that referenced this pull request Nov 14, 2026
…he Model L set

l_linear_32_32_fused (#160) and l_swin_block_c32_dec_coef12_nc (#159) have no
HIP code object, so the co test alone put them in Model M's set. Model M's set
is unchanged (46); Model L's set is 16. kernel_sets.json regenerated; the
profile tool's docstring names --model-l.

Refs #89

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant