Skip to content

FADD, FMUL and FFMA apply their .RZ / .RM / .RP rounding, as the GPU does (chain change) - #2051

Open
Know-It-All-Brother wants to merge 2 commits into
seedds:mainfrom
Know-It-All-Brother:ampere-directed-rounding
Open

Know-It-All-Brother wants to merge 2 commits into
seedds:mainfrom
Know-It-All-Brother:ampere-directed-rounding

Conversation

@Know-It-All-Brother

@Know-It-All-Brother Know-It-All-Brother commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Part of the Ampere accuracy series (#2040). It depends on #2041 for the test: the commits before this PR's own are #2041's and drop out once it lands, so this PR's own change is the last commit.

What and why

FADD, FMUL and FFMA carry a rounding mode (bits 78–79: .RM, .RP, .RZ), and we dropped it: Metal only rounds to nearest. On a Jetson Orin, 30 to 45 % of the directed-rounding cases in the fixture came out one ULP off, at the round-to-nearest value.

The fix keeps the round-to-nearest result and computes the sign of its exact error. Where the mode excludes that side of the true value, it steps the result one ULP on its bits. The error comes from:

  • FADD: TwoSum. Operands below 2^-63 are scaled by 2^100 first, so the error isn't flushed.
  • FMUL: fma(a', b', -(a'·b')) on the significands, so it's never a denormal. .D2/.M4 scaling is applied before the step, and .FMZ/.SAT after, as before.
  • FFMA: Boldo–Muller ErrFma on a copy scaled by a power of two.

An infinity from finite inputs is an overflow (RZ gives the largest finite, RM/RP by sign), and an exact zero sum is −0 under RM. The error terms are marked precise, so spirv-cross writes them as its optnone wrappers and Metal's fast math can't fold the TwoSum. Without that, FADD and FFMA stayed at round-to-nearest. Maxwell's FADD/FMUL/FFMA had the same bug and use the same helpers.

The cost lands only on instructions with a directed mode. Plain round-to-nearest FADD, FMUL and FFMA emit exactly the MSL they did before.

Directed instruction Extra MSL statements optnone calls
FMUL about 27 none
FADD 32 (RZ) to 49 (RM) 6
FFMA 82 (RP) to 96 (RM) 16

One exemption: ptxas puts FMUL.RZ x, 1/(2π) in front of every MUFU.SIN and MUFU.COS. Ocarina of Time has about 430,000 of them, so emulating them would cost every sin and cos for nothing: Metal's sin and cos are their own approximation, whose error is far larger than one ULP of the argument. An FMUL by that exact immediate (0x3e22f983) stays round-to-nearest. Directed FADD and FFMA are rare (about 11,000 each in Ocarina of Time's shader code, out of about 34 million instructions).

Reference entry in docs/reference/shaders.md.

How to verify

DMSH_TEST_XLATE=dm/gfx/tools/dmsh_xlate_bin/dmsh_xlate AMPERE_ORIN_ONLY='f32 arith' python3 dm/gfx/tests/test_dmsh_ampere_orin.py
Op (lanes of 1,024) main this PR
fadd_rz / _rm / _rp 381 / 357 / 413 67 / 53 / 64
fmul_rz / _rm / _rp 318 / 374 / 336 64 / 91 / 71
ffma_rz / _rm / _rp 461 / 438 / 468 92 / 95 / 82

Every remaining lane is a NaN bit pattern or an f32 denormal, which Metal gives itself and the test allows, and no round-to-nearest result is left. All nine come out of KNOWN. Every other op gives the same lanes as before; mufu_sin and mufu_cos match more closely with full emulation (644 and 539 "other" lanes against 678 and 564), the gain the exemption gives up.

Not checked against the Orin: Maxwell (no fixture), FFMA whose product overflows while the result doesn't (the error comes out NaN and no step is applied), and FP64 DADD/DMUL/DFMA, whose mode is still dropped.

This doesn't overlap #1800's directed I2F / F2F roundings. Both PRs edit InstEmitAmpereAlu.cs, but in different functions.

Checked

  • Tests: test_dmsh_ampere_orin.py; every other test_dmsh_* gives the same result as on main (test_dmsh_ampere_mem.py, test_dmsh_meshstage.py and test_dmsh_pairs.py fail on main the same way); test_nsp_chain.py, test_xlate_anycpu.py, test_docs.py.
  • Translator: the dlls are rebuilt from this tree with .NET 10; tool_lock.py --update-lock and dmsh_chain.py --header were run (chain 279009302576).
  • Devices and OS: MacBook Pro M5 Max, macOS 27.
  • Measurements: I counted the MSL; I haven't measured frame time. Since plain arithmetic and the sin/cos idiom are untouched, I don't expect a visible change.
  • regress.py: not run. My only local Switch 2 build's shader package comes from an older shader split than main's map, so main's chain can't build it. I'll add a report once I have a fresh build.
  • Games run: none.

Docs

  • docs/reference/shaders.md has the entry, and python3 dm/tools/docs_symptoms.py was run
  • A chain tool changed: dmsh_chain.py --header was run, and dmsh_chain.h, the translator dlls and the lock are part of the change
  • No game files, keys or data extracted from a game

…re_orin.py runs about 190 operations, each the GPU compiler's own machine code for one PTX operation, through the translator, spirv-cross, the chain's MSL rewrites and Metal on this Mac's GPU, and compares every output word with what a Jetson Orin (sm_87) computed from the same 1,024 input rows. The translator tests checked the MSL a few encodings give, not the values; this one finds the FSETP.NE, SHFL lane, F2FP, rounding-mode and signed-zero differences the tracking issue lists, which KNOWN names until each fix takes its entry out. dm/gfx/tools/ampere_orin/capture.py records the fixture on any Linux machine with an Ampere GPU and NVIDIA's driver, with no toolkit or compiler on either side
…ters dropped the mode and Metal rounds to nearest, so about a third of the Orin test's lanes were one ULP off. The shared helpers in InstEmitAluHelper.cs (used by Ampere's AFadd, AFmul and AFfma and by Maxwell's EmitFadd, EmitFmul and EmitFfma) compute the result to nearest and the sign of its exact error, then step the result one ULP on its bits where the mode excludes that side, with overflow and RM's -0 handled. FADD's error is a TwoSum, FMUL's is an fma on the two significands, and FFMA's is ErrFma on a copy scaled so no step is a denormal. The TwoSum's operations are marked precise so spirv-cross writes them as optnone spvFAdd and spvFSub, because fast math folds them otherwise. Plain round-to-nearest instructions are unchanged, and so is the compiler's sin and cos argument scale, FMUL.RZ by the immediate 1/2pi (0x3e22f983, about 430,000 in Ocarina of Time), because Metal's sin and cos err far more than one ULP of the argument. fadd, fmul and ffma in rz, rm and rp went from 254 to 383 round-to-nearest lanes each out of 1,024 to none, so they leave KNOWN. mufu_sin and mufu_cos keep their 780 and 606 differing lanes; emulating that FMUL.RZ would have brought them to 746 and 581. No other op or translator test changed; chain 279009302576
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders labels Nov 9, 2026
Know-It-All-Brother added a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
), with this branch's translator rebuilt from its own source; chain ae889ee4a3e1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant