Repository navigation
Model L on D3D12: the projections' conversions by the driver's instructions - #161
Merged
Merged
Conversation
…ctions l_dec_project and l_project_gather round to FP16 after every pair of matrix multiplies and pack to FP8 per tile; both were the software forms (wg_rn16, wg_pack_q8), most of the kernels' instructions. Now RN16 is v_cvt_f16_f32 plus the exact widening and the pack v_cvt_pk_fp8_f32 after the clamp (wg_platform.hlsli: the same bits; the kernels have no branch). Byte for byte against HIP, as before. - clean/l/runner/l_d3d12_ops_check.py: the projection cases at three weight scales (sums past 448 and past FP16's range, FP16 subnormal results) and with special FP8 codes in the input (NaN, +-448, +-0, subnormals). Refs #89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Model L speed on D3D12, fourth change:
l_dec_projectandl_project_gatherdo their FP16 rounding and FP8 packwith the driver's conversion instructions. They used the software forms, which were most of their instructions.
Both stay byte for byte against their HIP kernels; nothing is approximated. The two kernels get the same edit in one
file, so they are one PR, timed separately.
What changes
clean/dx12/kernels/l_ops_project.hlsl: the RN16 after every pair of matrix multiplies is nowv_cvt_f16_f32plusthe exact widening, and the FP8 pack is
v_cvt_pk_fp8_f32after the clamp. Before, these were the softwarewg_rn16/wg_pack_q8.wg_platform.hlslidocuments the same bits; the kernels have no branch, which the driverconversions need.
clean/l/runner/l_d3d12_ops_check.py: the projection cases run at three weight scales: normal; large (sums past 448and past FP16's range: the clamps, infinities); tiny (FP16 subnormal results). Every other case has special FP8
codes in the input (both NaN codes, +-448, +-0, subnormals).
Refs #89
Byte identity (
clean/l/runner/l_d3d12_ops_check.py --kernels l_dec_project,l_project_gather, one GPU slot;record
records/proj-cvt-identity.jsonin the rr-plugin build folder)l_dec_project: 36 of 36 cases identical (1,886,976 bytes). Its outputs include 886,656 NaN codes, 241,038 atthe +-448 clamp and 48,914 subnormals.
l_project_gather: 36 of 36 (943,488 bytes); 884,464 at its +-6.28125 clamp. It cannot output a NaN: the clampdrops it, as in HIP.
swin_block_c96_nccontrol: 12 of 12.Frame check (
clean/l/runner/l_frame_check.py --compare-d3d12, synthetic models; recordrecords/proj-cvt-frame-check.json)byte 11 of 11; its post against Model M's post reference 11 of 11.
byte-identical (record
records/proj-cvt-frame-vs-main.json).Timing (
l_d3d12_profile.py, 1080p Quality, synthetic models). Build A is main's build; build B is main's withthese two DXILs. Two sets of 3 interleaved rounds, every run in its own GPU slot; records
records/projcvt-ab.jsonand
records/projcvt2-ab.json.l_dec_project: 0.1909 / 0.1927 -> 0.0500 / 0.0506 ms (medians of the two sets: -141 / -142 us).l_project_gather: 0.1665 / 0.1679 -> 0.0560 / 0.0564 ms (-111 / -112 us).l_swin_block_c32_enc_nc+2 / +6 us,lev/input_encoder+0 / +11us, the post +4 us,
color_divide-5 / +5 us).out.
Tested
Not tested