Skip to content

Model L: exact speedup in the hybrid chain ends (-72.75 us at 1080p Q, -112.5 us at 1440p Q) - #284

Merged
phantoslayer21 merged 2 commits into
mainfrom
kl-L/movb16
Nov 18, 2026
Merged

phantoslayer21 merged 2 commits into
mainfrom
kl-L/movb16

Conversation

@phantoslayer21

@phantoslayer21 phantoslayer21 commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

What changed

The low FP16 half is now widened with f16tof32(v) directly instead of f16tof32(v & 0xffff). f16tof32 reads only the low 16 bits, so the result is the same, but the masked form made the driver copy the half into a register first: one v_mov_b16 per FP16 pair at every hybrid chain end and in Model L's own projection, linear and coefficient kernels. The hybrid helpers in wg_hyb.hlsli and the L files (l_coef_px.hlsli, l_lin_px.hlsli, l_ops_linear.hlsl, l_ops_project.hlsl) now call f16tof32 directly. The conversion instruction is unchanged.

wg_platform.hlsli is not touched (its text is part of Model K's softmax self-check digest), so Model M and Model K kernels are not touched.

Diff vs 2b31cbc: 5 files, 22 insertions, 18 deletions.

Byte checks

  • Full build vs the previous build: 21 of 154 DXIL files changed, all Model L (l_* or *_hyb). Model M and K DXIL are byte-identical. Manifest provenance: PASS.
  • 6 final frames (600 to 605) x 2 shapes (1080p Q, 1440p Q): output hashes identical for all 12 cases.
  • 24 timing runs (6 ABBA rounds x 2 shapes): output hashes identical in every run.
  • GPU-run mutant (f16tof32(v ^ 1) in wg_hyb.hlsli): byte check DIFFERENT, mutant caught.
  • Element-wise GPU probe (gelu, rsq, rcp, clamp, chain): 0 differences over 46,776 chain inputs at and around FP8 midpoints, saturation edge and specials. Three probe mutants (gelu, rsq, chain fold) each fail as they should.
  • RGA gate: PASS, 0 FAIL, 3 WARN (VGPR +1 on three hybrid kernels, same WG/CU and LDS, no scratch, no spills).

Timing (paired ABBA, 1500 frames, first 300 dropped, dx12)

shape previous median total this median total paired median (this - previous) faster rounds
1080p Q 2.8498 ms 2.7770 ms -72.75 us 6/6
1440p Q 4.8808 ms 4.7690 ms -112.5 us 6/6

Noise floor (A vs A): 1080p 6.0 us, 1440p 12.0 us. Both deltas are beyond it. An earlier session on the same commit measured -73.3 us (1080p Q) and -118.0 us (1440p Q), 6/6 both.

Evidence records are kept off-repo by the maintainer.

Unit tests

  • clean/l/tests: python -m unittest discover -s clean/l/tests -t . -> 121 tests, OK.
  • clean/dx12/tests: python -m unittest discover -s clean/dx12/tests -> 49 tests, OK.

Follow-up commit d9e9590 adds one-line rationale comments at the direct f16tof32 sites in the L kernels (comment-only; no code or DXIL change).

Scope note: this PR is based on 2b31cbc. Later commits on main are docs only and do not touch these files.

wg_h2f_lo(v) is f16tof32(v & 0xffff). f16tof32 reads only the low 16 bits
anyway, but with the mask the driver copies the half into a register of
its own first: one v_mov_b16 per FP16 pair at every hybrid chain end and
in L's own projection, linear and coefficient kernels. The hybrid
helpers (wg_hyb.hlsli) and those L files now call f16tof32 directly;
the conversion instruction is the same.

wg_platform.hlsli is unchanged (its text is part of Model K's softmax
self-check digest), so Model M and K kernels are not touched.

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
Comment-only change. At each direct f16tof32 site in the L kernels
(l_coef_px.hlsli, l_lin_px.hlsli, l_ops_linear.hlsl, l_ops_project.hlsl)
a one-line comment points to hyb_h2f_lo (wg_hyb.hlsli), so the mask in
wg_h2f_lo is not put back by a later cleanup. No code, no DXIL change.

wg_platform.hlsli and the Model K softmax self-check digest closure are
not touched; smx_selfcheck_digest.stale() is None after the edit.

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
@phantoslayer21
phantoslayer21 marked this pull request as ready for review November 18, 2026 20:03
@phantoslayer21
phantoslayer21 merged commit e1f4679 into main Nov 18, 2026
10 checks passed
@phantoslayer21
phantoslayer21 deleted the kl-L/movb16 branch November 18, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant