Skip to content

FMNMX and HMNMX2 order -0 below +0, as the GPU does (chain change) - #2047

Open
Know-It-All-Brother wants to merge 2 commits into
seedds:mainfrom
Know-It-All-Brother:ampere-minmax-zero
Open

Know-It-All-Brother wants to merge 2 commits into
seedds:mainfrom
Know-It-All-Brother:ampere-minmax-zero

Conversation

@Know-It-All-Brother

@Know-It-All-Brother Know-It-All-Brother commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Part of the Ampere accuracy series (#2040). It depends on #2041 for the test: the commits before this PR's own are #2041's and drop out once it lands, so this PR's own change is the last commit.

What and why

Ampere's FMNMX and HMNMX2 order -0 below +0, so max(+0, -0) is +0 and min(+0, -0) is -0, in either operand order. Metal's fmin / fmax may return either zero. On a Jetson Orin, max(0x00000000, 0x80000000) gives 0x00000000, and we gave 0x80000000. A sign like that matters downstream: 1 / max(x, 0) turns into -inf.

A new helper, FPMinMax, keeps Metal's min / max and fixes up the one case where they disagree. When both operands are zeros, tested with integer operations on their bits (((a | b) << 1) == 0), the result is a | b for min and a & b for max. I deliberately didn't use a == b: under fast math it can compile as an unordered compare, and then a NaN operand would take the zero path. The integer test can't be folded. FMNMX on Ampere and Maxwell and HMNMX2 (per half) use the helper; Maxwell's DMNMX keeps plain min / max.

The cost is 4 extra integer operations per min and 5 per max (OR, shift, compare, select, plus an AND for max), twice for HMNMX2. FMNMX is common, so this touches a lot of shaders, but each instance costs only a few ALU operations next to the min / max itself. I counted the MSL and haven't measured frame time.

Reference entry: "FMNMX and HMNMX2 order -0 below +0" in docs/reference/shaders.md.

How to verify

DMSH_TEST_XLATE=dm/gfx/tools/dmsh_xlate_bin/dmsh_xlate AMPERE_ORIN_ONLY=fmax,hmax2,fmin,hmin2,fmin_ftz python3 dm/gfx/tests/test_dmsh_ampere_orin.py
Op main this PR
fmax (of 1,024) 2: one zero-sign, one f32 denormal 1: the f32 denormal, which is Metal's own
hmax2 (halves of 2,048) 7: one zero-sign, six NaN bit patterns 7: all NaN bit patterns, which are Metal's own

Both come out of KNOWN. Every ±0 pair in the fixture follows the rule, including in fmax_nan, fmin_nan, bmin2 and bmax2_nan, which are still in KNOWN for other reasons. The min half of the rule rests on that data and on reading the code: none of the ops that run now has a ±0 pair as input. Maxwell's ordering is assumed to be the same as Ampere's, since I have no Maxwell hardware to check. With both halves NaN, HMNMX2 now gives Metal's canonical NaN rather than one input's, which the test counts as a NaN bit difference, as it did before.

Checked

  • Tests: test_dmsh_ampere_orin.py; every other test_dmsh_* gives the same result as on main (test_dmsh_ampere_mem.py, test_dmsh_meshstage.py and test_dmsh_pairs.py fail on main the same way); test_nsp_chain.py, test_xlate_anycpu.py, test_docs.py. lint_python.py shows no new findings.
  • Translator: the dlls are rebuilt from this tree with .NET 10; tool_lock.py --update-lock and dmsh_chain.py --header were run (chain 3483a0096b4a).
  • Devices and OS: MacBook Pro M5 Max, macOS 27.
  • regress.py: not run. My only local Switch 2 build's shader package comes from an older shader split than main's map, so main's chain can't build it. I'll add a report once I have a fresh build.
  • Games run: none.

Docs

  • docs/reference/shaders.md has the entry, and python3 dm/tools/docs_symptoms.py was run
  • A chain tool changed: dmsh_chain.py --header was run, and dmsh_chain.h, the translator dlls and the lock are part of the change
  • No game files, keys or data extracted from a game

…re_orin.py runs about 190 operations, each the GPU compiler's own machine code for one PTX operation, through the translator, spirv-cross, the chain's MSL rewrites and Metal on this Mac's GPU, and compares every output word with what a Jetson Orin (sm_87) computed from the same 1,024 input rows. The translator tests checked the MSL a few encodings give, not the values; this one finds the FSETP.NE, SHFL lane, F2FP, rounding-mode and signed-zero differences the tracking issue lists, which KNOWN names until each fix takes its entry out. dm/gfx/tools/ampere_orin/capture.py records the fixture on any Linux machine with an Ampere GPU and NVIDIA's driver, with no toolkit or compiler on either side
… and -0 is -0 and the maximum +0, in either operand order, where Metal's fmin and fmax may return either zero. Where both operands are zeros, found by an integer test on their bits (((a | b) << 1) == 0) that fast math can't fold and a NaN never passes, the minimum is their bits ORed and the maximum their bits ANDed; Ampere's FMNMX and HMNMX2 and Maxwell's FMNMX share the helper, DMNMX is unchanged, and with a constant predicate an FMNMX adds an OR, a shift, an integer compare and a select, plus an AND for a maximum. test_dmsh_ampere_orin.py takes fmax (2 differing lanes of 1,024 to 1, a denormal input) and hmax2 (the zero-sign lane gone, NaN bits only) out of KNOWN, and the other translator tests are as on main; chain 3483a0096b4a
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders labels Nov 9, 2026
Know-It-All-Brother added a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…'s translator rebuilt from its own source; chain 4a7002853654
AmateurWizrd added a commit to AmateurWizrd/nxnative that referenced this pull request Nov 10, 2026
…he chain batch (chain 483568384b04)

Rebuilt from the in-repo source after seedds#2042, seedds#2047, seedds#2048 (translator), seedds#1846, seedds#1797 and 064a8cc (dmsh_build.py).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant