Skip to content

A mesh shader gets the vertex outputs its pixel shader reads and its SPH does not enable (Ocarina of Time's foliage mesh pipelines failed to link) - #1628

Closed
Romen-Adama-Dev wants to merge 1 commit into
seedds:mainfrom
Romen-Adama-Dev:fix/mesh-fragment-pad
Closed

Romen-Adama-Dev wants to merge 1 commit into
seedds:mainfrom
Romen-Adama-Dev:fix/mesh-fragment-pad

Conversation

@Romen-Adama-Dev

Copy link
Copy Markdown
Contributor

What

Two of Ocarina of Time's Foliage.Product task / mesh pipelines never build:

[RED] task / mesh pipeline: task - mesh a203347bb7bdbd03_b fragment cd609edc78b467d8_b, 17.7 ms in the background: Fragment input(s) `user(locn5)` mismatching mesh shader output type(s) or not written by mesh shader
[RED] task / mesh pipeline: task - mesh 3a6340a04bf09797_b fragment 2a7c48d595feaaeb_b, …

The mesh shaders' SPH enables the position alone, so dmsh_meshstage.to_stage declared dm_vertex_t with the position only; their pixel shaders read locn5 (x, y and z). On the hardware an attribute the mesh shader never wrote reads a default; Metal refuses to link the pipeline, the host reports it once and skips every draw of the pair: foliage missing from Hyrule Field.

This is the mesh-shader form of pixel shader inputs the vertex shader does not write: vs_pad gives a vertex shader the outputs its pixel shaders read as (0, 0, 0, 1), but the pair tables hold vertex / pixel pairs only, so a mesh shader was never padded.

Fix (chain b488b8d163b2, on main 581f9c1)

  • dmsh_build.py: mesh_pairs pairs each mesh shader with the pixel shader of its BNSH variation from shader_map.tsv's source column (the same #varN the split pairs a vertex shader by), and adds the pairs to PAIRS, so a mesh shader is built after its pixel shader and keyed with it. ms_pad takes the locations the pixel MSL reads and the SPH does not enable.
  • dmsh_meshstage.py: to_stage writes those locations (i['pad']) as float4(0, 0, 0, 1) vertex outputs; the reflection's vs_pad names them.
  • dm/gfx/nvn/dmsh_chain.h regenerated; reference entry, game note and symptom index.

Verified

The four shaders rebuilt with the change into a scratch cache (dmsh_build.py <pkg> <out> <4 keys>); the mesh sources now declare float4 a5 [[user(locn5)]] and set it in main0 and main0_z. A Metal mesh pipeline built from each pair's metallibs (newRenderPipelineStateWithMeshDescriptor, RGBA8 target, Depth32Float):

pair the app's libraries rebuilt
a203347bb7bdbd03 + cd609edc78b467d8 Fragment input(s) user(locn5) … not written by mesh shader built (main0 and main0_z)
3a6340a04bf09797 + 2a7c48d595feaaeb same built

The game's full rebuild with the new chain and an eye check in Hyrule Field are pending (about 75 minutes of shaders and archive on this Mac).

Tests

  • test_dmsh_meshstage.py (28, with the new test_padded_outputs), test_nsp_chain.py (17), test_docs.py: pass.
  • test_dmsh_pairs.py: 2 errors, the same on main since 9c2fb0c (vs_pad now calls output, which the test's builder() does not exec); not touched here.
  • Reach: every game with task / mesh shaders (Mario Kart World, Ocarina of Time). A mesh shader whose pixel shader reads only what it writes gets no pad and the same source as before, apart from the chain version; regress.py check --games cannot run on this Mac (one local game).

…SPH does not enable, as (0, 0, 0, 1), like a vertex shader's vs_pad: Ocarina of Time's two Foliage.Product mesh pipelines failed to link on locn5 and their draws were skipped in Hyrule Field

dmsh_build.py pairs each mesh shader with the pixel shader of its BNSH variation from shader_map.tsv's source column (mesh_pairs, the pair tables hold vertex / pixel pairs only) and ms_pad takes the locations the pixel MSL reads that the SPH does not enable; dmsh_meshstage.to_stage declares and writes them. The two pairs rebuilt link as Metal mesh pipelines where the app's libraries fail. test_dmsh_meshstage checks the pad; reference entry, game note, symptom index.
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders game: 0400486024DEC000 A game labels Nov 7, 2026
@lunks

lunks commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Thanks. We carry the same padding on our downstream stack: all 23 triples Metal refused in Ocarina of Time link on the device with it. One case you may meet: one mesh shader was over Metal's 32 KB mesh limit with the padding, so there the pixel shader's missing inputs become constants instead.

@seedds

seedds commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Thanks, this is the right fix. The same change landed on main in 5e344a8 (chain 5616dbb62ec3, from #1590) in nearly the same form: to_stage pads to (0,0,0,1) what the paired pixel shaders read. The pairs come from shader_split.py var_pairs into shader_pairs.tsv rather than from a build-time pairing, so the build, pso_offline and the host's prebuild share one table. All 1,284 of Ocarina's mesh pairs were built against it, and 1,283 link. Closing as superseded.

The one left, 626fbf48 + c5eb985e at 35,840 bytes, is the 32 KB case @lunks describes; I'm working on it next.

@seedds seedds closed this Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants