Skip to content

Frame generation: net1 fused tier B (11 launches), D3D12 within 1.5x of HIP, exact path selectable - #205

Merged
phantoslayer21 merged 3 commits into
mainfrom
fg/net1-fused
Nov 14, 2026
Merged

phantoslayer21 merged 3 commits into
mainfrom
fg/net1-fused

Conversation

@phantoslayer21

Copy link
Copy Markdown
Owner

Refs #96 (frame generation). The second piece of net1's tier-B fast path: the bottleneck and decoders fused, the D3D12 gap to HIP explained and mostly closed, and the exact path selectable as the fallback.

What changes

  • fg_mpair_{b,d4,d3,d2,d1,d0} (clean/fg/kernels/fg_net1_mma.hip, clean/dx12/kernels/fg_mpair.hlsl): two chained 1 x 1 convolutions, kept in registers. A result fragment (lane = pixel, 8 consecutive channels) has exactly the layout of the next convolution's input fragment, so neither LDS nor memory sits between them.
    • The bottleneck (net1_06 + net1_07) is one launch.
    • Each decoder level is one launch: net1's upscale of the coarser level and the skip add are computed per input fragment, with the exact upscale / add arithmetic, then the two convolutions. u, s and e are never stored.
  • net1 in 11 launches (fused) instead of 27: five encoder convolutions with the pool fused, the bottleneck pair, five decoder pairs.
  • fg_leaky_fast16 (clean/fg/kernels/fg_fast16.h, check kernel fg_leaky_fast_f16): the leaky ReLU as one single-precision product with the slope, one float16 rounding and the max. It gives fg_leaky16's bits for every input; the earlier integer version was the bulk of the convolution epilogue's code. fg_mconv uses it too.
  • D3D12: float16 rounding of whole fragments through the driver conversion (wg_to_f16_ags, the bits of HIP's (_Float16)), instead of the software wg_f2h.
  • clean/fg/runner/fg_net1_mma.py --paths fused,tierb,exact: the exact path (Frame generation: net1's 32 steps on the GPU (HIP and D3D12), bit for bit against the CPU reference #191's bit-exact kernels) stays selectable in the same runner, as the validation mode and a cheap fallback (tier C) if the frame gate needs it.
  • clean/dx12/README.md "Driver exceptions": the two findings below, for future ports.
  • clean/fg/records/net1-fused-2026-11-14.json: gate results, report (counts only), timing, and the D3D12 profile per launch.

Why D3D12 was ~2.8x slower than HIP, and what changed

Measured with WG_D3D12_PROFILE GPU timestamps per dispatch against HIP device events per launch (1080p-sized input), plus the RGA ISA:

  1. Software float16 rounding. On this driver a (half) cast rounds toward zero, so wg_f2h is about 20 instructions per value. Tier B rounds every accumulator value after every chunk, which made that the hot loop's main cost; HIP does it with one v_cvt_f16_f32. The driver conversion of a whole fragment replaces it, and the decoders' upscale / add roundings are batched the same way.
  2. The exact integer leaky ReLU in the epilogue: 64-bit chunk arithmetic for 64 values per lane, about 25,000 instructions in the 4-tile convolution kernel. Replaced by fg_leaky_fast16, which halved that kernel's code.
  3. The kernel_replay wall time includes host submission and the fence wait of every run, about 1.6 ms per run at 11 launches. That is not GPU work.

Fused path, D3D12 GPU time summed over its launches: 10.40 ms before the driver conversion, 3.99 ms after; HIP 2.72 ms (1.47x). net1_01 (the largest layer) is 1.18 ms on D3D12 against 0.74 ms on HIP.

A port bug found and fixed on the way: lanes past the end of the grid corrupted other lanes' matrix results on D3D12. The kernel used a per-lane validity array with select() for the pixel and a guarded store. With part of the last 16-pixel group past the end (odd level sizes), valid lanes got NaN in exactly the output rows whose weight-fragment row sat in an invalid lane, while HIP was right. Every lane now computes a clamped pixel and only the store tests the range. The 70 x 90 case caught it; it is in the README's driver exceptions.

Gates (the port standard)

  • HIP == D3D12, byte for byte, for every buffer every path writes: sentinels 0xA5 / 0x5A, 64 slack bytes, special values. There is a half-grid negative control per kernel, plus a 768 x 768 case where every kernel, including the small-level pairs, has two or more workgroups and so its own negative control.
  • fused == tierb, byte for byte, on all common buffers: the fusion computes the same arithmetic.
  • Fast leaky ReLU on all 65,536 float16 inputs: HIP == D3D12 == the CPU reference's leaky.

Report against the exact CPU reference (record above; not a gate)

Converted real weights, fixed 64 x 64 input (execute-only: converter pass, 42/42 stage digests, converted file deleted, no output digest in public):

path output > 1 step > 4 steps max step max |diff| max |ref| max |diff| / max |ref|
fused 7,410 / 16,384 2,881 14,732 0.070 29.3 2.4e-3
tierb the same (byte-identical)
exact 0 0 0 0 0

Per output value (fused), max |diff| / max |ref|: 2.2e-3, 2.2e-3, 2.4e-3, 3.9e-3. The step counts come from small and near-zero values. The absolute error stays at about 10⁻³ of each buffer's range through the network; see the drift analysis that preceded #199's merge.

Timing (information only; synthetic weights, mean of 10 runs after a warm-up; HIP device events, D3D12 wall time per synchronised run including host submission and the wait)

path input HIP D3D12
fused (11 launches) 1080p 2.48 ms 5.59 ms
fused 1440p 3.57 ms 7.23 ms
tierb (27 launches) 1080p 3.22 ms 7.12 ms
tierb 1440p 5.35 ms 11.92 ms

The longest single launch was 1.26 ms. For comparison, #199 measured tierb at 4.46 / 7.10 ms (HIP) and 12.47 / 20.46 ms (D3D12).

Tested

  • GPU, each run in one GPU slot under its cap:
    • Gate run: 18 cases (synthetic 64 x 64, 70 x 90, NaN/inf specials, finite specials, 768 x 768, converted weights) x 3 paths, all passing with every negative control caught; the leaky check passed.
    • Timing: the runs above; output byte-identical HIP to D3D12 at both sizes on both paths.
    • Profiles: before and after, in the record.
  • python -m unittest discover -s clean/fg/tests -t .: 85 OK, including new tests: the 11-launch plan (buffers, launch arguments, every kernel has a half grid at 768 x 768) and the fast leaky formula on all 65,536 inputs. clean/dx12/tests/test_kernel_sets.py: 4 OK.
  • public_gates.py: must-pass gates pass; the LOOK lines are at main's baseline.

Not tested

  • The generated-frame gate against the oracle: that needs part 3's non-network stages.
  • Other GPUs and drivers.

…of HIP, exact path selectable

- fg_mpair_{b,d4..d0} (HIP + fg_mpair.hlsl): the bottleneck and each decoder level as two chained 1 x 1
  convolutions in registers, the decoders' upscale and skip add computed per input fragment
- fg_fast16.h: leaky ReLU with fg_leaky16's bits from one product and one rounding (checked on all 65,536 inputs
  on HIP, D3D12 and the CPU), the skip add; fg_mconv's epilogue uses it
- D3D12: float16 rounding of whole fragments by the driver conversion instead of the software wg_f2h; lanes past
  the grid end compute a clamped pixel (a validity array corrupted other lanes' matrix rows); both in the README's
  driver exceptions
- runner/fg_net1_mma.py: paths fused / tierb / exact; gates HIP == D3D12 (sentinels, slack, half-grid negative
  control per kernel incl. a 768 x 768 case), fused == tierb, the leaky check; report with absolute and relative
  differences; timing; records/net1-fused-2026-11-14.json

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
@phantoslayer21
phantoslayer21 marked this pull request as ready for review November 14, 2026 21:47
@phantoslayer21
phantoslayer21 merged commit 8774be5 into main Nov 14, 2026
9 checks passed
@phantoslayer21
phantoslayer21 deleted the fg/net1-fused branch November 14, 2026 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant