Skip to content

K D3D12 softmax: exact mix-FMA form for the weight rounding (draft) - #230

Closed
phantoslayer21 wants to merge 1 commit into
mainfrom
kspeed/k-mix
Closed

phantoslayer21 wants to merge 1 commit into
mainfrom
kspeed/k-mix

Conversation

@phantoslayer21

Copy link
Copy Markdown
Owner

Draft: work in progress, parked while M and L speed take priority. Not ready for review.

What it does

The K softmax needs A = f16(w * inv) rounded once from the exact product (HIP: v_fma_mixlo_f16 w, inv, 0). The D3D12 port emulates that with a round-to-odd sequence: 7 VALU, 1 convert and 2 VCC hazard waits per weight.

This branch writes the weight as the exact two-term sum p + e (p = w * inv in FP32, e = fma(w, inv, -p), so w * inv = p + e exactly) and feeds it into the driver's round-to-nearest-even matrix conversion. The gfx1201 D3D12 compiler folds that add into v_fma_mixlo_f16 / v_fma_mixhi_f16 p, 1.0, e, whose single rounding of p + e is RN16(w * inv), the same bits as HIP.

KB_SMX 2 (new default) is the mix form, KB_SMX 1 keeps the round-to-odd form for the fallback ports, and KB_SMX_MUT 1 is a negative-control mutant (residual dropped), never shipped.

Rejected shapes (RGA): the product or an fma fed straight into the conversion stays v_mul/v_fma + v_cvt_f16_f32 (double rounding); a native (half) cast gives v_fma_mixlo_f16, but under s_round_mode round-toward-zero.

Where it stands

  • ISA (RGA gfx1201, pinned toolchain), k_stage_d1: VALU 2426 -> 2097 per wave, instructions 4181 -> 3631, 14.2 VALU per WMMA (HIP: 14.3). Per weight: 8 -> 3 VALU, no VCC round trips.
  • GPU byte check against HIP (k_port_check, raw bytes, sentinels 0xA5 / 0x5A, half-grid negative control): narrow family 120/120 identical (111 raw-identical, 9 differ only in a NaN sign), wide family 84/84 (72 / 12). Negative controls caught.
  • The generic port check is too weak for this step: the residual-dropped mutant is caught in only 1 of 12 k_stage_d1 cases and 0 of 12 k_stagew2_e3 cases.

Still to do before review

  • A dedicated adversarial probe (pairs whose FP32 product is exactly an FP16 midpoint with a nonzero residual, both residual signs, FP16-subnormal results), with the residual-dropped mutant as the probe's required catch.
  • A build-time ISA assertion: the build fails if a mix-form port lacks the fold.
  • A runtime probe in the native D3D12 device, falling back to the round-to-odd ports, with one log line per process.
  • The same port check on main's ports, to show that the NaN-sign-only cases already exist there.
  • RGA before/after for all 11 softmax ports.
  • Timing A/B at 1440p Quality (not run yet).

Tested

  • RGA ISA of k_stage_d1 in the trial shapes and the final form.
  • k_port_check narrow + wide on the GPU (one slot each), plus the mutant run.

Not tested

  • Timing, the other ports' ISA, the runtime probe and fallback (not written yet), and an in-game run.

…16), round-odd fallback, negative-control define

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
@phantoslayer21

Copy link
Copy Markdown
Owner Author

Superseded by #254.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant