Skip to content

HMMA with bf16 and tf32 sources translates, bit-exact with a Jetson Orin (chain change) - #2062

Open
Know-It-All-Brother wants to merge 4 commits into
seedds:mainfrom
Know-It-All-Brother:ampere-hmma-bf16-tf32
Open

Know-It-All-Brother wants to merge 4 commits into
seedds:mainfrom
Know-It-All-Brother:ampere-hmma-bf16-tf32

Conversation

@Know-It-All-Brother

@Know-It-All-Brother Know-It-All-Brother commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Part of the Ampere accuracy series (#2040). It depends on #2041 for the test and builds on #2061 (the f32-accumulator model): the commits before this PR's own are #2041's and #2061's and drop out once those land, so this PR's own change is the last commit.

What and why

HMMA with bf16 sources (mma.sync m16n8k16 .f32.bf16.bf16.f32) and with tf32 sources (m16n8k8 .f32.tf32.tf32.f32) failed to translate: AHmma only took f16 sources.

This adds both source types and routes them through #2061's tensor-core model, which matches the Orin exactly for these too:

  • bf16 uses the f16 m16n8k16 fragment layout. Each element is its bits moved up 16. Blocks of 8 products, as for f16.
  • tf32 has its own m16n8k8 layout, one element per register (A: (g, t), (g+8, t), (g, t+4), (g+8, t+4); B: k = t and t+4 in column g). The tensor core ignores the low 13 bits of each tf32 input, and with them kept, no output matched. Blocks of 4 products.

To do this, AHmmaF32 now takes the source type: it sets the block size, the exponent width and how each element is unpacked. bf16 and tf32 need an f32 accumulator, tf32 needs m16n8k8, and the compute shaders' simdgroup_matrix route still takes f16 only.

One change touches #2061's code: the model's final right shift is clamped to 31. A sum below f32's denormals from an 8-bit-exponent source could ask for a shift of 32 or more, which is undefined in MSL. For f16 sources the results are unchanged, at the cost of one min per block.

These are heavy instructions: about 1,200 MSL statements for a tf32 HMMA and 1,900 for a bf16 one. In Ocarina of Time's shader code, every HMMA but 2 has f16 sources and an f16 accumulator, and that path doesn't change.

Reference entry: "HMMA with bf16 and tf32 sources" in docs/reference/shaders.md, which links the f32-accumulator entry.

How to verify

DMSH_TEST_XLATE=dm/gfx/tools/dmsh_xlate_bin/dmsh_xlate AMPERE_ORIN_ONLY=tensor python3 dm/gfx/tests/test_dmsh_ampere_orin.py
Op (outputs of 4,096) main with #2061 alone this PR
hmma_16816_bf16 failed to translate failed to translate 0
hmma_1688_tf32 failed to translate failed to translate 0
hmma_16816_f32 2,693 0 0

Both come out of KNOWN. Before applying #2061's model, a plain FMA chain for these left 1,334 and 2,189 outputs differing. The fixture's bf16 and tf32 inputs all have ordinary exponents and every output is normal. So these aren't checked against the GPU: product overflow, sums below f32's denormals, denormal / Inf / NaN sources, and a tf32 NaN whose mantissa bits are all in the ignored low 13 (it reads as Inf).

Checked

  • Tests: test_dmsh_ampere_orin.py, test_dmsh_hmma.py; every other test_dmsh_* gives the same result as on main (test_dmsh_ampere_mem.py, test_dmsh_meshstage.py and test_dmsh_pairs.py fail on main the same way); test_nsp_chain.py, test_xlate_anycpu.py, test_docs.py. lint_python.py shows no new findings.
  • Translator: the dlls are rebuilt from this tree with .NET 10; tool_lock.py --update-lock and dmsh_chain.py --header were run (chain b0cbe916d032).
  • Devices and OS: MacBook Pro M5 Max, macOS 27.
  • regress.py: not run. My only local Switch 2 build's shader package comes from an older shader split than main's map, so main's chain can't build it.
  • Games run: none.

Docs

  • docs/reference/shaders.md has the entry, and python3 dm/tools/docs_symptoms.py was run
  • A chain tool changed: dmsh_chain.py --header was run, and dmsh_chain.h, the translator dlls and the lock are part of the change
  • No game files, keys or data extracted from a game

…re_orin.py runs about 190 operations, each the GPU compiler's own machine code for one PTX operation, through the translator, spirv-cross, the chain's MSL rewrites and Metal on this Mac's GPU, and compares every output word with what a Jetson Orin (sm_87) computed from the same 1,024 input rows. The translator tests checked the MSL a few encodings give, not the values; this one finds the FSETP.NE, SHFL lane, F2FP, rounding-mode and signed-zero differences the tracking issue lists, which KNOWN names until each fix takes its entry out. dm/gfx/tools/ampere_orin/capture.py records the fixture on any Linux machine with an Ampere GPU and NVIDIA's driver, with no toolkit or compiler on either side
…ules with its functions, so a rewrite that uses one (struct in PR seedds#1902's hfma_msl) runs as it does in a build instead of raising NameError
…core does: the translator added the 16 products to C with a chain of FMAs rounded to nearest, while the tensor core aligns each block of 8 exact products and C to the largest exponent with one bit more than f32's fraction, truncates the bits below, adds them exactly and truncates the sum to f32 (Khattak and Mikaitis's A100 model, MATLAB-tensor-core's A100TC). AHmmaF32 computes that model with integer operations on the bits, so Metal's fast math can't change it, and an Inf or NaN operand keeps the FMA chain's result; it costs about 1,640 more MSL statements per HMMA. An exact Python version of the model matches all 4,096 of a Jetson Orin's outputs (blocks of 4 or 16, other extra-bit counts or a final rounding to nearest match at most 75 %), and with blocks of 8 for bf16 and 4 for tf32 it also matches all of hmma_16816_bf16 and hmma_1688_tf32; test_dmsh_ampere_orin.py's hmma_16816_f32 went from 2,693 of 4,096 lanes differing to none and leaves KNOWN; chain 44897f07bf75
…ore does: HMMA.16816.F32.BF16 and HMMA.1688.F32.TF32 stopped the translator (HMMA size 1 source type 1, size 0 source type 2), since AHmma only knew f16. A bf16 is now read in m16n8k16's f16 fragment layout and a tf32 in m16n8k8's own layout (one element a register: A at rows g and g + 8, columns t and t + 4; B at k t and t + 4), with its low 13 bits ignored as the tensor core does (with them kept, no output of the Orin's 4,096 matches). AHmmaF32's model takes both, in blocks of 8 for bf16 and of 4 for tf32, with the 8-bit exponents brought to its scale, and a right shift past 31 that only those exponents reach is clamped. test_dmsh_ampere_orin.py's hmma_16816_bf16 and hmma_1688_tf32, which the translator refused, now match all 4,096 outputs each (an FMA chain left 1,334 and 2,189 differing) and leave KNOWN; chain b0cbe916d032
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders labels Nov 9, 2026
Know-It-All-Brother added a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…s branch's translator rebuilt from its own source; chain 3c5093528f44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant