Repository navigation
Model L on D3D12: the folded decoder block's epilogue on the driver's conversions - #163
Merged
Merged
Conversation
… conversions l_swin_block_c32_dec_coef12_nc's epilogue converted its 16 values per lane to FP16 and rounded five sigmoids and three products to FP16 with the software forms (wg_to_f16, wg_rn16, wg_mul_f16's wg_f2h). Now they are the driver's v_cvt_f16_f32 (the same bits; the products keep wg_mul_f16's round-to-odd FP32 value). The conversions need wave-uniform control flow, so every lane computes the per-pixel values and only the stores stay per lane. l_coef / l_coef_pack12 and the unfolded block keep their code and DXILs (byte-identical to main's), as does every other kernel built from these files. - clean/l/runner/l_fold_check.py: the cases cycle through large inputs (past the +-20 clamp and tanh's saturation), special FP8 codes (NaN) and extreme S, T (FP16 products from subnormal to infinite). - clean/l/runner/l_d3d12_profile.py: l_linear_32_32_fused in the encoder step. Refs #89 Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Model L speed on D3D12, fifth change: the folded decoder block's epilogue (
l_swin_block_c32_dec_coef12_nc, #159)uses the driver's FP16 conversion instead of the software forms. It stays byte for byte against main's kernel and
against the pair it replaces; nothing is approximated.
What changes
clean/dx12/kernels/swin_block.hlsl(SW_OUT_COEF12withCOEF_CVT_HW) andclean/dx12/hlsl/l_coef_px.hlsli(
l_coef_vals_hw,l_coef_store12).v_cvt_f16_f32(wg_to_f16_ags), as do the five RN16 of sigma (oneconversion) and the three FP16 products (another). Before, these were
wg_to_f16,wg_rn16and thewg_f2hofwg_mul_f16. The products keepwg_mul_f16's round-to-odd FP32 value and its +0 rule.wg_platform.hlslidocuments the same bits for these conversions. They need wave-uniform control flow, so everylane computes the per-pixel values and only the stores stay per lane.
l_coef,l_coef_pack12andl_swin_block_c32_dec_nc(the fallback route and HIP's twins) keep their code.swin_block.hlsland the regeneratedfused_swin.hlsl: the 14 rebuilt,Model M's included.
clean/l/runner/l_fold_check.py: the cases cycle through four kinds:clean/l/runner/l_d3d12_profile.py:l_linear_32_32_fused(Model L on D3D12: the two input linears in one kernel #160) counted in the encoder step.Refs #89
Byte identity (
clean/l/runner/l_fold_check.py, one GPU slot; recordrecords/fold-cvt-identity.jsonin therr-plugin build folder)
l_swin_block_c32_dec_nc->l_coef_pack12on D3D12, and the pair on HIP.
0x3C).
sentinel.
swin_block_c96_nccontrol: 8 of 8.Frame check (
clean/l/runner/l_frame_check.py --compare-d3d12, synthetic models; recordrecords/fold-cvt-frame-check.json)byte 11 of 11; its post against Model M's post reference 11 of 11.
byte-identical (record
records/fold-cvt-frame-vs-main.json).Timing (
l_d3d12_profile.py, 1080p Quality, synthetic models). Build A is main's build; build B is main's withthis DXIL. Two sets of 3 interleaved rounds, every run in its own GPU slot; records
records/foldcvt-ab.jsonandrecords/foldcvt2-ab.json.l_swin_block_c32_dec_coef12_nc: 0.3500 / 0.3526 -> 0.3111 / 0.3136 ms (medians of the two sets: -39 / -39 us).Every round: A 0.348 .. 0.354, B 0.309 .. 0.315 ms.
2.209 ms.
Tested
Not tested