Skip to content

Float16 targets: limit colour outputs written from a constant buffer (A19 Pro stores overflow as infinity) - #1128

Open
yaasin-raki2 wants to merge 1 commit into
seedds:mainfrom
yaasin-raki2:float16-target-limit
Open

yaasin-raki2 wants to merge 1 commit into
seedds:mainfrom
yaasin-raki2:float16-target-limit

Conversation

@yaasin-raki2

Copy link
Copy Markdown

What and why

Symptom. Super Mario Odyssey 1.4.1, Sand Kingdom at night, A19 Pro (iOS 27): screen-wide white blobs along the horizon. Not reproduced on the M1 Pro.

Localization (on device, DM_CTL).

  • A one-pixel line of far-terrain pixels (depth 0.99998; sky 1.0) reaches 2048 in the pass that opens with alDeferredShading (00fea7508a519dd7). The bloom chain spreads it.
  • DM_NVN_UB attribution: dropping 00fea7508a519dd7 removes the line. Dropping the cloud layer or any of the pass's other 13 fragment shaders does not.

Cause.

  • alRenderSky (9047ce47645d5840) writes a far-distance sentinel straight from a constant buffer to the G-buffer's R16F target.
  • The value exceeds the float16 range. The M1 Pro stores 65504; the A19 Pro stores +infinity.
  • The deferred shader reconstructs a position from that value: inf * 0 = NaN, and precise::min(NaN, 2048.0) returns 2048. DM_NVN_MAGSCAN therefore reports no non-finite pixel.

Fix (general).

  • dmsh_limit16.py builds a side library <key>_bs for each pixel shader with a colour output written directly from a constant buffer (470 of Odyssey's 20,522 pixel shaders). Those outputs are clamped to ±65504, and reflection limit16.colors lists them.
  • lim_key selects the library only for a pipeline with such an output on a float16 target and none on a 32-bit float target. It is encoded as bit 1 of pso_key.atest, so the key layout and prewarm rows are unchanged.
  • The alpha test and the limit share one loader (side_fn).
  • Computed outputs are not clamped. An alpha-tested pipeline keeps the alpha test library.

Reference entry: Float16 target overflow stored as infinity on the A19 Pro. The game note has a resolved entry.

Checked

  • A19 Pro, iOS 27: a build of this change on 0c94284 (chain 5bee4f71eb9d) removes the line from the same save state. This rebase was not run on a device.
  • M1 Pro, macOS 27, Xcode 27.2 (Metal 32023.922): every job of tests.yml passes: host tests, TSan, ASan/UBSan, thermal, CPU texture, CPU-only gfx tests, GL imports, dmhost for the three CI profiles, all gfx tests compiled, iOS compile check, unsigned iOS app, and the dm/tools, dm/ios and dm/gfx tool tests.
  • GPU tests: check_t_limit16, check_t_alpha and check_t_logic pass.
  • Lint: lint_python.py --against main reports 0 new findings. clang-tidy 23.1.4 reports 0 new findings from this change. The local SDK reports 5 findings above the baseline in untouched files (t_dlss_mfx.m, t_reorder_chain.m, t_upscaler_mfx.m, t_home_quit_mac.m), identical on an unmodified main.
  • Not checked: scripts/regress.py check --games (no Mac baselines). The change reaches every game's float16 pipelines with a constant-buffer colour output, and the M1 Pro does not produce the fault.

Docs

  • The finding is in its game note and its docs/reference/shaders.md entry, and python3 dm/tools/docs_symptoms.py was run
  • A chain tool changed: dmsh_chain.py --header was run and dm/gfx/nvn/dmsh_chain.h is part of the change
  • No game files, keys or data extracted from a game in the change or the attachments

…(A19 Pro stores overflow as infinity)

Symptom: Super Mario Odyssey 1.4.1, Sand Kingdom at night, A19 Pro (iOS 27): screen-wide
white blobs along the horizon; not reproduced on the M1 Pro. A one-pixel line of far-terrain
pixels (depth 0.99998, sky 1.0) reaches 2048 in the pass that opens with alDeferredShading
(00fea7508a519dd7); the bloom chain spreads it. Dropping that shader with DM_NVN_UB removes
the line; dropping the cloud layer or the pass's other 13 shaders does not.

Cause: alRenderSky (9047ce47645d5840) writes a far-distance sentinel from a constant buffer
to the G-buffer's R16F target. The value exceeds the float16 range: the M1 Pro stores 65504,
the A19 Pro stores +infinity. The deferred shader reconstructs a position from it,
inf * 0 = NaN, and precise::min(NaN, 2048.0) returns 2048, so DM_NVN_MAGSCAN reports no
non-finite pixel.

Fix: dmsh_limit16.py builds a side library <key>_bs for each pixel shader with a colour
output written directly from a constant buffer (470 of Odyssey's 20,522 pixel shaders), with
those outputs clamped to +-65504; reflection limit16.colors lists them. lim_key selects the
library only for a pipeline with such an output on a float16 target and none on a 32-bit
float target, as bit 1 of pso_key.atest, so the key layout and prewarm rows are unchanged.
The alpha test and the limit share one loader (side_fn). Computed outputs are not clamped;
an alpha-tested pipeline keeps the alpha test library.

Checked: A19 Pro, iOS 27: a build of this change on 0c94284 (chain 5bee4f71eb9d) removes
the line; this rebase was not run on a device. M1 Pro, Xcode 27.2: every tests.yml job,
check_t_limit16, check_t_alpha and check_t_logic pass; lint_python and clang-tidy 23.1.4
report no new findings from this change. regress.py was not run; the M1 Pro does not
produce the fault.
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders game: 0100000000010000 Super Mario Odyssey labels Nov 3, 2026
@seedds

seedds commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Thanks for the on-device localization; the inf × 0 → NaN → min(NaN, 2048) chain is convincing. I'm not landing it in tonight's unattended run. It's a chain change, and it adds another side library (_bs) on top of _ba / _bl, the shader-build cost #757 already flags. That's the maintainer's call, and it's the same question as #1032. One design point for that discussion: the game works on a Switch, so NVIDIA's float32 → float16 render-target conversion presumably saturates to ±65504, as the M1 does. If so, the general rule is "a float16 target saturates", for every output and not only those written straight from a constant buffer. Could you check whether a computed out-of-range output reaches an R16F target anywhere in Odyssey? Whether this stays narrow or becomes a per-target clamp depends on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0100000000010000 Super Mario Odyssey nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants