[FifteenPen]

What and why

Fixes #1753: Nintendo Switch Sports Resort (04008E001F098000, v65536) builds 11,053 of its 11,059 shaders. The other six are refused with mesh stage failed: shared memory of its own, or stage memory past Metal's limit:

• task shaders 4d57f62a663c4cc9, 3c8eaa19f720a02f, 07abb052a670b8ba, 8c49b3da7fd802df;
• the mesh shaders they launch, d60092d7424cb5b4 and a68f2301bbaceb8c.

Cause. Metal allows an object function's payload up to 16 KB and a function's threadgroup memory up to 32 KB. These task shaders hand their mesh shaders 21–28 KB:

• The payload went through Metal's payload, which is too small for it.
• The mesh shader copied it into its stage memory (the input area, ISBERD 0x38) on top of its attributes and primitives, about 33 KB.
• The task shader's stage memory also reserved an input area as large as its payload, although a task shader reads no input. That made 42–56 KB.

Fix. A payload past DMSH_MESH_PAYLOAD_MAX (16 KB) goes through a device buffer:

• Host (nvn_geometry.inc mtk_encode): binds the buffer at DMSH_MESH_SPILL_SLOT (11) of the object and mesh tables. Word 0 holds the number of slices, followed by one slice of the payload's words per task workgroup. Metal's payload is then 16 bytes.
• Task shader (dmsh_meshstage.to_stage): stores its payload in its workgroup's slice and passes the slice's number in Metal's payload. A workgroup past the host's slices launches no mesh workgroups.
• Mesh shader: its input becomes a shared memory of its own (the job's fifth stage memory entry). The translator loads ISBERD 0x38 from it (InstEmitAmpereMem.cs AIsberd, ResourceManager.GetStageInputMemoryId), and the wrapper backs it with the slice. The input is therefore no longer part of the mesh shader's stage memory.
• Task stage memory: has no input area (dmsh_meshstage.memory).
• Limits: a direct draw gets a slice per workgroup up to 1,024 (MTK_SPILL_MAX); past that the remaining workgroups launch no mesh workgroups, with a [RED] line. An indirect draw always gets 1,024, since its count is written on the GPU. The host logs the first draw that uses the buffer.

Payloads up to 16 KB take the old path unchanged: the same Metal payload and stage memory layout. The task shaders' smaller stage memory (no input area) applies to every task shader.

Chain change: eb9a3f4c3394 (the translator, dmsh_meshstage.py).

Checked

• The game's shaders, main + this PR: 11,059 of 11,059 build, with no shader regressions in the build's comparison against the previous cached chain. On main alone it's 11,053 (6 Mac mesh-stage shaders failed to compile for Nintendo Switch Sports Resort (04008E001F098000, v65536) #1753).
• test_dmsh_meshstage.py (new tests):
  • the task and mesh layouts with the spill;
  • the task's slice store and its no-launch past the slices;
  • the mesh shader reading its input from the slice;
  • a spill mesh shader without its input array is refused;
  • Metal compiles of a spill task and a spill mesh shader.
• test_dmsh_ampere_mem.py and test_dmsh_ampere_alu.py pass with the rebuilt translator.
• Host checks (Ocarina of Time's profile):
  • t_geometry: 1,929 checks;
  • t_geometry_feedback: 110;
  • t_geometry_hazards: 61;
  • t_geometry_mesh: 23;
  • t_mesh_first_use: 53;
  • all pass.
• test_docs.py and the symptom index pass.
• lint_c.py tidy: no new findings. The three above the baseline are in t_pso_archive.m, which this PR doesn't touch.

Not checked

• No draw through the buffer has run in the game. The host logs the first one, and none of our Switch Sports Resort sessions has logged it, so I don't know which scene uses these shaders. This makes them build; whether they draw correctly isn't shown yet.
• Other mesh-shader games (Ocarina of Time, Yoshi and the Mysterious Book, Mario Kart World) rebuild with the chain change. Any task shader of theirs with a payload up to 16 KB keeps the old path apart from its smaller stage memory. I didn't check their payload sizes or run them in-game.

The game's profile is in #2107.