Skip to content

Frame generation: net1 fast path, tier B on the matrix instructions (HIP and D3D12 byte-identical) - #199

Merged
phantoslayer21 merged 1 commit into
mainfrom
fg/net1-mma
Nov 14, 2026
Merged

phantoslayer21 merged 1 commit into
mainfrom
fg/net1-mma

Conversation

@phantoslayer21

Copy link
Copy Markdown
Owner

Refs #96 (frame generation). The net1 fast path, tier B as decided after the probe (#193): v_wmma_f32_16x16x16_f16 with float16 rounding per chunk. This first piece covers every convolution on the matrix instructions, with the encoders' pools fused; the decoder and bottleneck fusion comes in the next PR. The exact kernels are unchanged and stay as the validation mode.

What changes

  • clean/fg/kernels/fg_net1_mma.hip (code object fg_net1_mma.co) and clean/dx12/kernels/fg_mconv.hlsl add fg_mconv1/2/4, one per number of 16-channel output tiles per wave:
    • The spec's chunk structure: the accumulator starts at the bias, and each chunk (16 input channels of one tap, in the layer's chunk order) is one v_wmma_f32_16x16x16_f16. Its C is the float16 accumulator; its result is rounded to float16 (nearest even) before the next chunk.
    • Leaky ReLU and, for net1_01..05, the 2 x 2 max-pool run in the epilogue, using the exact helpers of fg_exact16.h.
    • Tensors are NHWC float16, so 8 consecutive channels of a pixel are one 16-byte fragment load or store.
    • Weights are arranged on the host into A fragments in chunk order.
    • A workgroup is 8 waves over a 16 x 16 pixel tile.
    • D3D12 goes through the driver channel and follows the README's driver rules: loads before the matrix calls, branchless selects, an opaque loop bound, store pointers through wg_uniform().
  • fg_upscale_hwc_f16 (HIP, plus fg_upscale_hwc.hlsl): the exact upscale's arithmetic on NHWC tensors. The adds reuse fg_add_f16.
  • clean/fg/runner/fg_net1_mma.py runs net1 in 27 launches on both back ends, then:
    • Gate: every buffer (a_k, p_k, b5, d5, u_k, s_k, e_k, d_k, out) is byte-identical between HIP and D3D12, with sentinels 0xA5 / 0x5A and 64 slack bytes. There is one half-grid negative control per kernel, and the cases include special values.
    • Report: the CPU reference comparison: largest distance in float16 steps, plus the count of values more than 1 and more than 4 steps off, per buffer and per output value.
    • Timing: --timing (information only) runs a launch-length probe first, then 1080p- and 1440p-sized inputs.
  • clean/fg/records/net1-tierb-2026-11-14.json: gate results, the report (counts only) and the timing.
  • Signed zero: tier B already gives +0 for exact-zero sums, the reference's convention, at no cost. The probe's only zero-sign difference for this form was a tiny term the matrix unit dropped.

Report against the exact CPU reference (record above; not a gate)

On the converted real weights, with the fixed 64 x 64 input (execute-only: converter pass, 42/42 stage digests, converted file deleted, no output digest kept in public):

  • Network output: 16,384 values; 7,410 more than 1 step off, 2,881 more than 4; max 14,732 steps.

  • Per output value (4,096 each):

    Value > 1 step > 4 steps Max
    0 2,257 849 2,705
    1 2,522 1,110 14,732
    2 748 42 9
    3 1,883 880 13,427
  • The first layer is at the probe's per-chunk level: 15 of 32,768 values more than 1 step off. The difference then grows roughly 6–10× per encoder level. So the probe's per-chunk figure (about 0.002 % of chunks more than 1 step off) does not carry over to net1's output.

  • For comparison, float16 accumulate on the same input put 13,965 values more than 1 step off and 10,129 more than 4.

Timing (information only; synthetic weights, mean of 10 runs after a warm-up; HIP device events, D3D12 wall time per synchronised run)

Input size HIP D3D12
1080p 4.46 ms 12.47 ms
1440p 7.10 ms 20.46 ms

The longest single launch was 1.72 ms. For reference, the exact kernels took 169 ms on D3D12 at 1080p (#191). D3D12 is slower than HIP here; the next PR (fused decoders and bottleneck, D3D12 code inspection) targets that.

Tested

  • GPU, each run in one GPU slot under its cap:
    • all 5 cases: synthetic 64 x 64, 70 x 90, NaN/inf specials, finite specials, converted weights. Every buffer is byte-identical HIP to D3D12, and 5 of 5 negative controls were caught per case;
    • the timing run above, where the output was also byte-identical at both sizes.
  • python -m unittest discover -s clean/fg/tests -t .: 83 OK, including 2 new plan tests: every weight fragment element against weights, chunk order and tap; the fused plan's buffers and launches. clean/dx12/tests/test_kernel_sets.py: 4 OK.
  • public_gates.py: must-pass gates pass; the LOOK lines are at main's baseline.

Not tested

  • The generated-frame gate against the oracle: that needs part 3's non-network stages.
  • Other GPUs and drivers.

…HIP and D3D12 byte-identical)

- kernels/fg_net1_mma.hip, clean/dx12/kernels/fg_mconv.hlsl: fg_mconv1/2/4, one v_wmma_f32_16x16x16_f16 per spec
  chunk in the layer's chunk order, the float16 accumulator from the bias as C, rounded to float16 after every
  chunk; exact leaky ReLU and fused 2 x 2 pool in the epilogue; NHWC float16
- fg_upscale_hwc_f16 (HIP and fg_upscale_hwc.hlsl): the exact upscale on NHWC tensors
- runner/fg_net1_mma.py: net1 in 27 launches; HIP == D3D12 for every buffer (sentinels, slack, half-grid negative
  control per kernel, special values); report against the CPU reference; timing (information only)
- records/net1-tierb-2026-11-14.json: gate results, report (counts only) and timing
- Exact kernels unchanged (validation mode); fg_ kernels still built only through --only

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
@phantoslayer21
phantoslayer21 marked this pull request as ready for review November 14, 2026 20:49
@phantoslayer21
phantoslayer21 merged commit 2829815 into main Nov 14, 2026
9 checks passed
@phantoslayer21
phantoslayer21 deleted the fg/net1-mma branch November 14, 2026 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant