Skip to content

bf16 half pairs (HFMA2.BF16_V2, HMNMX2.BF16_V2) and HMNMX2.NAN translate (chain change) - #2060

Open
Know-It-All-Brother wants to merge 3 commits into
seedds:mainfrom
Know-It-All-Brother:ampere-bf16-pairs
Open

Know-It-All-Brother wants to merge 3 commits into
seedds:mainfrom
Know-It-All-Brother:ampere-bf16-pairs

Conversation

@Know-It-All-Brother

@Know-It-All-Brother Know-It-All-Brother commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Part of the Ampere accuracy series (#2040). It depends on #2041 for the test: the commits before this PR's own are #2041's and drop out once it lands, so this PR's own change is the last commit.

What and why

Three half-pair forms failed to translate: HMNMX2 .NAN (min.NaN.f16x2), and the bfloat16 forms HMNMX2 .BF16_V2 (min / max.NaN on bf16x2) and HFMA2 .BF16_V2 (fma.rn.bf16x2). The half-pair helpers only knew f16 lanes. The fields from the Orin's encodings are .NAN at bit 81 and .BF16_V2 at bit 85.

  • bf16 lanes unpack as bits << 16 (low) and bits & 0xffff0000 (high), and pack by rounding to nearest even with integer operations. A NaN packs to 0x7fff, as the Orin writes it whatever its sign.
  • HMNMX2 on bf16 compares integer keys that order like the floats with −0 below +0, as the Orin does. Without .NAN, one NaN gives the other operand and two give NaN; with .NAN, any NaN gives NaN.
  • HMNMX2 .NAN on f16 keeps the existing min / max and overrides a lane with NaN where an operand is NaN. It doesn't collide with FMNMX and HMNMX2 order -0 below +0, as the GPU does (chain change) #2047's ±0 change.
  • Still refused: HADD2 / HMUL2 .BF16_V2, which PTX only offers on sm_90 so the fixture can't check them, and HMNMX2 .F32 / .XORSIGN.

These forms aren't hot. A bf16 FMA costs about 8 integer operations per lane for the rounding, and a bf16 min/max about 12.

A change to the test, which I want to be upfront about: compare.py now treats a bf16 denormal like an f32 one. Metal computes bf16 in f32, so it flushes bf16 denormals just as it flushes f32 ones. All 31 lanes bfma2 still differs in are of that kind (for example inf × the bf16 denormal 0x0001 is inf on the Orin and NaN on Metal). The change only reclassifies bfma2 and f2fp_pack_bf16x2, which is still listed. If you'd rather not widen the rule, bfma2 can go back into KNOWN with the reason "bf16 denormals flushed by Metal".

Reference entry: "bf16 half pairs and HMNMX2.NAN" in docs/reference/shaders.md.

How to verify

DMSH_TEST_XLATE=dm/gfx/tools/dmsh_xlate_bin/dmsh_xlate AMPERE_ORIN_ONLY=hmin2_nan,bmin2,bmax2_nan,bfma2 python3 dm/gfx/tests/test_dmsh_ampere_orin.py
Op (halves of 2,048) main this PR
hmin2_nan failed to translate 0
bmin2 failed to translate 0
bmax2_nan failed to translate 0
bfma2 failed to translate 31, all bf16 denormals (30 inputs, 1 result)

All four come out of KNOWN. The bf16 FMA computes in f32 and then rounds to bf16, which is two roundings. A model of that matches all 1,533 finite fixture lanes, but a case where the f32 result lands exactly on a bf16 tie that the exact result wasn't on would round wrong, and the fixture has none.

Checked

  • Tests: test_dmsh_ampere_orin.py; every other test_dmsh_* gives the same result as on main (test_dmsh_ampere_mem.py, test_dmsh_meshstage.py and test_dmsh_pairs.py fail on main the same way); test_nsp_chain.py, test_xlate_anycpu.py, test_docs.py. lint_python.py shows no new findings.
  • Translator: the dlls are rebuilt from this tree with .NET 10; tool_lock.py --update-lock and dmsh_chain.py --header were run (chain 7b4846c830cd).
  • Devices and OS: MacBook Pro M5 Max, macOS 27.
  • regress.py: not run. My only local Switch 2 build's shader package comes from an older shader split than main's map, so main's chain can't build it. Ocarina of Time's build already translates every shader, so it can't contain this instruction.
  • Games run: none.

Docs

  • docs/reference/shaders.md has the entry, and python3 dm/tools/docs_symptoms.py was run
  • A chain tool changed: dmsh_chain.py --header was run, and dmsh_chain.h, the translator dlls and the lock are part of the change
  • No game files, keys or data extracted from a game

…re_orin.py runs about 190 operations, each the GPU compiler's own machine code for one PTX operation, through the translator, spirv-cross, the chain's MSL rewrites and Metal on this Mac's GPU, and compares every output word with what a Jetson Orin (sm_87) computed from the same 1,024 input rows. The translator tests checked the MSL a few encodings give, not the values; this one finds the FSETP.NE, SHFL lane, F2FP, rounding-mode and signed-zero differences the tracking issue lists, which KNOWN names until each fix takes its entry out. dm/gfx/tools/ampere_orin/capture.py records the fixture on any Linux machine with an Ampere GPU and NVIDIA's driver, with no toolkit or compiler on either side
…ules with its functions, so a rewrite that uses one (struct in PR seedds#1902's hfma_msl) runs as it does in a build instead of raising NameError
…he Orin's results: the half-pair emitters only unpacked and packed f16 lanes and HMNMX2 had no NaN mode, so the translator refused all three. A bf16 lane now unpacks to its f32 with a shift or a mask, HFMA2 computes it with an f32 FMA and rounds it to bf16 with integer arithmetic (ties to even, a NaN gives 0x7fff), HMNMX2.BF16_V2 compares its lanes as integers so fast math can't fold it and -0 orders below +0 as on the Orin, and with .NAN a NaN in either operand gives 0x7fff in that lane; HMNMX2 .F32 and .XORSIGN and HADD2 / HMUL2 .BF16_V2 stay refused. compare.py counts a bf16 lane's denormals as Metal's f32 flush, since bf16 has f32's exponent. test_dmsh_ampere_orin.py: hmin2_nan, bmin2 and bmax2_nan went from refused to 0 of 2,048 lanes differing, bfma2 to 31 of 2,048, all flushed bf16 denormals, and all four are out of KNOWN; chain 7b4846c830cd
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders labels Nov 9, 2026
Know-It-All-Brother added a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…branch's translator rebuilt from its own source; chain ba6861aa2b18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant