Repository navigation
Model L on D3D12: the two input linears in one kernel - #160
Merged
Merged
Conversation
l_linear_32_32_fused (D3D12 only): l_linear_32_32_relu and l_linear_32_32_f16 in one kernel, x1 kept in registers (the relu linear's lane stores exactly the two FP8 fragments the f16 linear's lane loads), byte for byte the pair. Its RN16, FP8 pack and FP16 conversions are the driver's instructions (the same bits as the software forms the pair keeps, which were most of the pair's instructions). - runtime: native D3D12 fusion "ll" (default on; Model L). The input encoder then writes q to l_narrow (x2 over q in l_wide would overwrite q that other workgroups have not read yet). Without ll, or without the kernel in the manifest, the pair runs. - clean/l/runner/l_linfuse_check.py: the fused kernel against the pair on D3D12 and on HIP, raw bytes, per-run sentinels in the slack, special FP8 codes and extreme weight scales, negative control, Model M control. - tests: the recorder's signature; the frame and role checks take either linear form. Refs #89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
phantoslayer21
marked this pull request as ready for review
November 14, 2026 10:32
phantoslayer21
added a commit
that referenced
this pull request
Nov 14, 2026
…he Model L set l_linear_32_32_fused (#160) and l_swin_block_c32_dec_coef12_nc (#159) have no HIP code object, so the co test alone put them in Model M's set. Model M's set is unchanged (46); Model L's set is 16. kernel_sets.json regenerated; the profile tool's docstring names --model-l. Refs #89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Model L speed on D3D12, third change: the two input linears in one kernel.
l_linear_32_32_fused(D3D12 only) givesthe same bytes as the pair
l_linear_32_32_relu->l_linear_32_32_f16; x1 stays in registers. Nothing isapproximated, and the pair stays as the fallback route.
What changes
clean/dx12/kernels/l_ops_linear.hlsl: the fused kernel.loads, so the fragments pass in registers with no shuffles.
v_cvt_f16_f32,v_cvt_pk_fp8_f32afterthe clamp;
wg_platform.hlsli: the same bits as the software forms).than the pair. With the driver's conversions its ISA is 1,928 bytes, against 11,300 for the software form.
clean/runtime/runtime.cpp: new native D3D12 fusionll, on by default for Model L.l_narrow. The pair writes x2 over q inl_wide; a fused kerneldoing that would overwrite q that other workgroups have not read yet.
WG_D3D12_FUSIONwithoutll, or when the manifest lacks the kernel.l_lin_fused=on|off.compare.py's copy of the default fusion list is updated.clean/l/runner/l_linfuse_check.py(new): the identity check below.frame test also runs without
lland without the kernel.Refs #89
Byte identity (
clean/l/runner/l_linfuse_check.py, one GPU slot; recordrecords/linfuse-identity.jsonin therr-plugin build folder)
kernel_replaypasses), and thepair on HIP.
the fused kernel 0x3C).
results past 448 and past FP16's range; tiny: FP16 subnormal results), every other case with special FP8 codes in
q (both NaN codes, +-448, +-0, subnormals); 2 seeds.
bytes at the 448 clamp.
sentinel.
swin_block_c96_nccontrol through the same replay: 8 of 8.Frame check (
clean/l/runner/l_frame_check.py --compare-d3d12, synthetic models; recordrecords/linfuse-frame-check.json)l_lin_fused=on, withl_dec_fold=on); the all-outputs run (fusionsoff) uses the pair.
byte 11 of 11; its post against Model M's post reference 11 of 11.
byte-identical (record
records/linfuse-frame-vs-main.json).Timing (
l_d3d12_profile.py, 1080p Quality, synthetic models). Build A is main's build; build B is this change.Two sets of 3 interleaved rounds, every run in its own GPU slot; records
records/linfuse-ab.jsonandrecords/linfuse2-ab.json.l_linear_32_32_relu+l_linear_32_32_f16= 0.2161 .. 0.2215 ms per round. The fused kernel: 0.0938 ..0.0999 ms (sums of the per-kernel medians 0.2167 -> 0.0956 and 0.2176 -> 0.0938 ms: -121 / -124 us).
l_swin_block_c32_enc_nc, which reads x2 next: 0.268 .. 0.271 -> 0.275 .. 0.280 ms in every round (+5 .. +9 us).Tested
clean/l/runtime/tests/test_l_runtime_frame.py(18 tests); public gates.Not tested
the fused kernel's RN16 conversion before that.