Repository navigation
FSETP.NE and FSET.NE are false with a NaN operand, as NVIDIA's are (chain change) - #2042
Open
Know-It-All-Brother wants to merge 2 commits into
Open
Know-It-All-Brother wants to merge 2 commits into
Know-It-All-Brother wants to merge 2 commits into
Conversation
…re_orin.py runs about 190 operations, each the GPU compiler's own machine code for one PTX operation, through the translator, spirv-cross, the chain's MSL rewrites and Metal on this Mac's GPU, and compares every output word with what a Jetson Orin (sm_87) computed from the same 1,024 input rows. The translator tests checked the MSL a few encodings give, not the values; this one finds the FSETP.NE, SHFL lane, F2FP, rounding-mode and signed-zero differences the tracking issue lists, which KNOWN names until each fix takes its entry out. dm/gfx/tools/ampere_orin/capture.py records the fixture on any Linux machine with an Ampere GPU and NVIDIA's driver, with no toolkit or compiler on either side
…e: the ordered not-equal went to SPIR-V as OpFOrdNotEqual, which spirv-cross writes as MSL's unordered !=, so a NaN guard let the NaN through. (a < b) || (a > b) does not help, since Metal's fast math folds it back to !=; GetFPComparison now writes a != b && !isnan(a) && !isnan(b), and Metal keeps isnan. The helper is shared, so Maxwell's FSETP and FSET change too. test_dmsh_ampere_orin.py's fsetp and fsetp_ftz match a Jetson Orin (800 of 16,384 cases each differed before); chain 230bb40d4e58
Know-It-All-Brother
added a commit
to Know-It-All-Brother/nxnative
that referenced
this pull request
Nov 9, 2026
…, with this branch's translator rebuilt from its own source; chain c5573a16e3a3
AmateurWizrd
added a commit
to AmateurWizrd/nxnative
that referenced
this pull request
Nov 10, 2026
…he chain batch (chain 483568384b04) Rebuilt from the in-repo source after seedds#2042, seedds#2047, seedds#2048 (translator), seedds#1846, seedds#1797 and 064a8cc (dmsh_build.py).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the Ampere accuracy series (#2040). It depends on #2041 for the test: the commits before this PR's own are #2041's and drop out once it lands, so this PR's own change is the last commit.
What and why
.NEis the ordered not-equal: it's false when either operand is NaN (.NEUis true then). The translator emits SPIR-V'sOpFOrdNotEqual, which is right, but spirv-cross writes it as MSL's!=, which is unordered. SoFSETP.NEheld for a NaN operand, and a NaN guard let the NaN through. ptxas also buildssetp.equfrom the negatedFSETP.NE, so that was wrong too.The obvious rewrite,
(a < b) || (a > b), doesn't work: Metal's fast math assumes no NaN and folds it straight back to!=. I checked that on the GPU.GetFPComparisonnow writesa != b && !isnan(a) && !isnan(b), and Metal keepsisnanunder fast math, as the unordered comparisons already rely on.The helper is shared, so Maxwell's FSETP and FSET get the fix too.
Reference entry: "FSETP.NE and FSET.NE held with a NaN operand" in
docs/reference/shaders.md.How to verify
fsetp(14 compares)fsetp_ftzOn the full 16,384-case run, 800 cases of each differed before, every one with a NaN operand. The test takes
fsetpandfsetp_ftzout ofKNOWN, so it fails on main's translator and passes here.Checked
test_dmsh_ampere_orin.py, and every othertest_dmsh_*gives the same result as on main (test_dmsh_ampere_mem.py,test_dmsh_meshstage.pyandtest_dmsh_pairs.pyfail on main the same way).test_nsp_chain.py,test_xlate_anycpu.py,test_docs.py.tool_lock.py --update-lockanddmsh_chain.py --headerwere run (chain 230bb40d4e58).regress.py: not run. My only local Switch 2 build's shader package comes from an older shader split than main's map, so main's chain can't build it. I'll add a report once I have a fresh build. A scan of Ocarina of Time's shader code finds only a handful of FSETP.NE.Docs
docs/reference/shaders.mdhas the entry, andpython3 dm/tools/docs_symptoms.pywas rundmsh_chain.py --headerwas run, anddmsh_chain.h, the translator dlls and the lock are part of the change