Skip to content

SHFL takes its lane from the low 5 bits of a register, as the GPU does (chain change) - #2044

Open
Know-It-All-Brother wants to merge 2 commits into
seedds:mainfrom
Know-It-All-Brother:ampere-shfl-lane
Open

Know-It-All-Brother wants to merge 2 commits into
seedds:mainfrom
Know-It-All-Brother:ampere-shfl-lane

Conversation

@Know-It-All-Brother

@Know-It-All-Brother Know-It-All-Brother commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Part of the Ampere accuracy series (#2040). It depends on #2041 for the test: the commits before this PR's own are #2041's and drop out once it lands, so this PR's own change is the last commit.

What and why

SHFL takes its lane from the low 5 bits of the register (PTX: bval = b[4:0]), in every mode. Our shuffle helpers took the whole register. In IDX and BFLY mode the high bits made the source lane fail the clamp check, so the thread got its own value back. On a Jetson Orin, lane 2 shuffling with lane 0xdaa06cd7 reads lane 23 (0xd7 & 31); we returned lane 2's own value.

The fix is in the four shared helpers in HelperFunctionManager.cs (IDX, UP, DOWN, BFLY), which mask the lane to & 31 first. Both the Ampere (AShfl) and Maxwell (InstEmitWarp.Shfl) emitters go through them, so Switch 1 games get it too. It costs one integer AND per shuffle, and the c operand's clamp and segment mask were already right.

Reference entry: "SHFL takes its lane from the low 5 bits" in docs/reference/shaders.md.

How to verify

DMSH_TEST_XLATE=dm/gfx/tools/dmsh_xlate_bin/dmsh_xlate AMPERE_ORIN_ONLY=warp python3 dm/gfx/tests/test_dmsh_ampere_orin.py
Op (lanes of 1,024) main this PR
shfl_idx 207 0
shfl_bfly 196 0
shfl_idx_seg (segment mask 0x18, clamp 7) 179 0

shfl_up, shfl_down and shfl_bfly_imm matched before and still do. The test takes the three ops out of KNOWN, so it fails on main's translator and passes here.

One thing this doesn't cover: SHFL.UP with a non-zero clamp should check against maxLane, not minLane. #1800 fixes that, and the fixture's SHFL.UP uses a zero clamp, so I left it alone here.

Checked

  • Tests: test_dmsh_ampere_orin.py; every other test_dmsh_* gives the same result as on main (test_dmsh_ampere_mem.py, test_dmsh_meshstage.py and test_dmsh_pairs.py fail on main the same way); test_nsp_chain.py, test_xlate_anycpu.py, test_docs.py.
  • Translator: the dlls are rebuilt from this tree with .NET 10; tool_lock.py --update-lock and dmsh_chain.py --header were run (chain 8791889b7d22).
  • Devices and OS: MacBook Pro M5 Max, macOS 27.
  • regress.py: not run. My only local Switch 2 build's shader package comes from an older shader split than main's map, so main's chain can't build it. I'll add a report once I have a fresh build.
  • Games run: none.

Docs

  • docs/reference/shaders.md has the entry, and python3 dm/tools/docs_symptoms.py was run
  • A chain tool changed: dmsh_chain.py --header was run, and dmsh_chain.h, the translator dlls and the lock are part of the change
  • No game files, keys or data extracted from a game

…re_orin.py runs about 190 operations, each the GPU compiler's own machine code for one PTX operation, through the translator, spirv-cross, the chain's MSL rewrites and Metal on this Mac's GPU, and compares every output word with what a Jetson Orin (sm_87) computed from the same 1,024 input rows. The translator tests checked the MSL a few encodings give, not the values; this one finds the FSETP.NE, SHFL lane, F2FP, rounding-mode and signed-zero differences the tracking issue lists, which KNOWN names until each fix takes its entry out. dm/gfx/tools/ampere_orin/capture.py records the fixture on any Linux machine with an Ampere GPU and NVIDIA's driver, with no toolkit or compiler on either side
…s: on a Jetson Orin, lane 2 shuffling with lane 0xdaa06cd7 reads lane 23, and the translation gave lane 2 its own value. The shuffle helpers the translator builds for every SHFL mode (HelperFunctionManager.cs) took the whole register as the lane, so in IDX and BFLY mode the high bits failed the clamp check and the helper kept the thread's value. Each helper now masks the lane to & 31 first, which covers both the Ampere and the Maxwell emitter (one more AND per shuffle; the c operand's clamp and segment mask were already right). test_dmsh_ampere_orin.py: shfl_idx 207 of 1,024 lanes differed and now 0, shfl_bfly 196 and 0, shfl_idx_seg 179 and 0, all three out of KNOWN; shfl_up, shfl_down and shfl_bfly_imm still match, and the other dmsh tests are as on main; chain 8791889b7d22
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders labels Nov 9, 2026
Know-It-All-Brother added a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…, with this branch's translator rebuilt from its own source; chain 1e8f104062cf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant