[nafields]

What and why

Symptom. Ocarina of Time's culling task shaders (f75708d176bb6591, 47f9b38490dcf9a2, 99280b7b8aae78df and five more) hang the GPU when the pool their range pointer names is missing from the raw pool table. Any raw load into such a pool reads 0. The table lists only pools the game was handed a GPU address for (the #1394 filter, in main), and that leaves out 172 of Ocarina's pools (3.7 GB).

Cause. These tasks read both ends of a range through a pointer loaded from a storage buffer, then loop (min(end - start, 256) - 1) >> 5 times. A miss reads both ends as 0, which makes 134,217,727 iterations. 99.5% of the raw-table accesses of Ocarina's 2,247 raw shaders take their pointer from a fixed byte offset of a bound storage buffer. Every model shader, for example, has five pointers at 0x0–0x20 of one slot. So the CPU can read the pointer before the draw.

Change.

• GlobalToStorage.FindPointerSite numbers such loads per shader: at most DMSH_GPTR_MAX = 64, noted gptr site k: c<slot>[<word>] +<byte>.
• The raw-load helper tests the site's entry {start, end, delta} (DMSH_KIND_GPTR) before main's last-entry check, hint and search.
• dmsh_build.py writes gptr_sites as [NVN storage slot, byte].
• encode_args (NVN_RB_GPTR) reads each pointer word and fills the entry from mtl_gpu_addr, which also makes the pool resident.
• The shader still checks the address against the entry, so a stale or empty entry costs only the search it would have done anyway.
• Stores and atomics are unchanged: the write log is per table entry.

Chain 1ffcb6b77740. Reference entry: Raw global memory through a pointer read when the draw is bound. The game note has a line for it.

What this builds on, and what it doesn't take:

• Raw global memory's pool table lists only the pools the game took a buffer address in: every pool was wired for it, which slowed Ocarina of Time's frames to 60-65 ms #1394's address filter is what leaves pools out of the table. It stays: this makes the filter safe for pointer reads, rather than widening the table back to every pool (the old GMEM_ALL route, 3.7 GB more resident on Ocarina).
• The existing last-entry cache and hint (Reduce Ocarina of Time compute cost with shared raw-memory lookups #1405, Raw global memory lookups reuse the range they last chose: about 3 ms less GPU a frame in Ocarina of Time's Kokiri Forest #1464) still run after the site test, unchanged.
• The Legend of Zelda: Ocarina of Time (Switch 2, 0400486024DEC000, v0 & v131072): playable at 60 fps through Dodongo's Cavern #1444's 262,144-iteration loop guard also ends this loop. It isn't taken: it caps the symptom, and the shader would still read 0 for the range.
• The idea of reading the pointer on the CPU at bind time follows how desktop emulators handle storage reached through constant-buffer pointers (Ryujinx's GlobalToStorage, which this extends, and Lichii's indirect offset).

What it doesn't claim. In the game, the remaining task/mesh hangs on main were cured by an encoder boundary after the task/mesh run, not by this. That's a separate change, still being narrowed. This PR removes the table-miss path, which the standalone test shows hangs every time.

Checked

• Translation of the 4,680 raw-table jobs gives sites to 2,173 shaders.
• The generated helper of 0031f9442b11b682, run on the GPU against an 8,191-entry table: 11 of 11 cases. The cases are site path, table path, empty, no site, range ends, no pool and the last-entry cache (dm/gfx/tests/probes/raw_sites.m in Standalone Metal probes: heap aliasing, render-target sizes, mesh index ranges, mesh pipeline archive, mesh/task sweep, task-shader hang #1583).
• The eight task shaders, each run alone (task_hang.m in Standalone Metal probes: heap aliasing, render-target sizes, mesh index ranges, mesh pipeline archive, mesh/task sweep, task-shader hang #1583):
  • pool in the table: completes;
  • pool left out, no site: hangs every time (about 22 ms of GPU time);
  • pool left out, site filled: completes.
• Game: Ocarina of Time (0400486024DEC000, v131072), Kokiri Forest route on an M2 Pro 16 GB. 1,091,044 sites filled by frame 2,700, all naming a pool; 16,250 of 258,329 raw-load bindings had no site.
• Tests passed: test_dmsh_reflect, t_refl (52 checks), test_nsp_chain, test_xlate_anycpu, test_docs, make -C dm/host GAME=0400486024DEC000, make -C dm/host/tests GAME=0400486024DEC000 -j run.
• make -C dm/gfx/tests GAME=0400486024DEC000 run: t_nvn and t_gfx_suspend don't link on main b2d46b8 either (_dm_vi_frame_queued undefined: their link lines leave out t_host_stubs.c). The other 42 checks were run one by one: all pass except t_fuse_check (2 of 10, "the later pass's logical target is kept afterwards"), which fails the same way on main b2d46b8.
• Devices: M3 (standalone GPU tests, translation); M2 Pro 16 GB, macOS 26 (game).
• scripts/regress.py check wasn't run: only Ocarina of Time is available here. regress.py impact puts the reach at every game with raw global memory (the NVN entry points through encode_args). A shader with no fixed-offset pointer gets no site and translates as before.

Docs

• The finding is in docs/reference/shaders.md and the game note, and python3 dm/tools/docs_symptoms.py was run
• A chain tool changed: dmsh_chain.py --header was run and dm/gfx/nvn/dmsh_chain.h is part of the change
• No game files, keys or data extracted from a game in the change or the attachments

[Amost-]

Tested this branch in game on an M2 Pro. The pointer sites work as described, but the GPU hangs still come: 5 of 6 runs hung.

Setup: MacBook Pro M2 Pro, 32 GB, macOS 27.2. Ocarina of Time (0400486024DEC000, v131072, from the NUP), Mac app. The branch is cherry-picked last onto main 863bc6b, together with #1331, #1368, #1176 and a few host changes of ours. The only conflict was dmsh_chain.h: recomputed, chain 3f146103c47b, tool_lock.py --check-tools passes. Full build: 42,848 of 42,848 shaders. t_refl 52 checks, 0 failed; test_dmsh_reflect OK.

The sites: each of the three culling task shaders gets one site, c0[412] +0. Over a run:

bind-time pointers up to frame 5100: 983213 sites encoded, 983213 of them named a pool (1000 per mille); 241033 raw-load shader bindings, 17408 of them with a site empty or no site (they rely on the table)

The hangs: six headless runs from the Link's-house autosave, down the ladder towards the cave pond by Mido's gate. The forest load happens at about frame 3000. Runs 4 to 6 used a host without one of our pool-table changes, to rule it out; it made no difference.

Run	Length	Failed command buffers	First at frame	Outcome
1	10 min	0	–	clean, forest p50 about 27 ms
2	228 s	5	3120	GPU access revoked, stopped by #1332's path
3	5 min	3	6842	recovered
4	4 min	3	3731	GPU access revoked
5	4 min	1	3038	recovered
6	4 min	2	3515	recovered

No page fault in any run. Four of the five hanging runs failed first within about 700 frames of the load. Run 1's route drifted, and Link spent the time elsewhere.

What the failed buffers had in common:

• f75708d176bb6591, 47f9b38490dcf9a2 and 99280b7b8aae78df were in every failed buffer. They run every frame, so that alone says little.
• The most specific shaders by failed/ran differ in every run:

Run	Top pair (failed/ran)	What it is
2	eea3629af1ba8569_b 4/473	Model.Product.110 vertex shader, no raw memory
3	a0fcac5b97dfc785_b / a9458a69d267e757_b 3/1236	Model.Product.110 material pair reading the texture pool
4	6bb6772fb6988a3f_b / b28cb32fc48ef9b0_b 2/12	Model.Product.110 pixel shaders
5	1cf6b315c28eb7ac_b / 6fe71b3f49cfd816_b 1/4	effect EftLoc_KokiriForest_WaterFall_DryIceGround_00
6	045b6c1fba990ab4_b / 7f0c7d09bca38025_b 2/71	effect static.vsetb var1168

The two effect pixel shaders have the light loop described in #1326 (min(s0[...], c15[113].x)), but many pixel shaders share it, so I would not read much into that.

So either a case of the range loop is still left uncovered, or the remaining hang is another loop. I did not run without this branch: that needs another chain and a full rebuild. The six logs are available as a bug_report.py bundle if they help.

[reefbluesky]

A data point for the remaining task/mesh hangs, on a MacBook Pro 14" M2 Pro (32 GB), macOS 27.2.

Build: main bd979ced with #1368 merged locally and two local cuts in the game's code (the movie player always takes its plain path, without NDVE; the model GPU heap doubled), chain 3ee52af3ca18. Ocarina of Time v131072, new game, same route each time.

• Two normal runs: kIOGPUCommandBufferCallbackErrorHang (no page fault) at 432 s and 425 s after launch, then SubmissionsIgnored and A GPU whose access macOS revoked stops the game instead of leaving it running with nothing drawn #1332's alert. Every failed command buffer the log lists holds 25-33 task/mesh shaders, among them the three culling tasks f75708d176bb6591, 47f9b38490dcf9a2 and 99280b7b8aae78df.
• Same route with DM_NVN_SKIP_MESH=1: 569 s with no GPU error. In the forest, frame time p50 went from 25.5 to 19.9 ms.

You wrote that the remaining task/mesh hangs on main were cured by an encoder boundary after the task/mesh run. Is that change available somewhere?

[lunks]

@reefbluesky The sentence you quote is in this PR's description, written by its author (@nafields). It is not ours, so I can't point you to that change. I found no merged or open PR with an encoder boundary after the task/mesh run; the description calls it "a separate change, still being narrowed".

The nearest open change I know of is #1707: game-mode draw passes hold the next pass's task and mesh shaders until their fragments are done. It is a different change, and I have not tested it against a hang like yours.

[LeonardoBai12]

A data point from an M1 Pro (16 GB, macOS 26.6.1): with this PR the hang came much earlier.

Build: main bd979ce plus this PR, rebased onto it (one conflict in the memory up to frame log line, resolved by keeping main's line and adding this PR's bind-time pointers line; dmsh_chain.py --header gives chain 9eb5e5a9a0ec here). On top: #1543, #1497, #1331, #1453, #1377 and #1379. All 42,848 shaders built. Handheld, DM_UPSCALER=game.

Result: three sessions, all ended by a GPU hang within the first minute, then GPU access revoked and exit 70. The two logs I kept hang at 48 s (frame 1698) and 26 s (frame 910), each with 12 × ErrorHang.

For comparison, the same Mac on main 24fe1e5 with the same PRs but without this one: five sessions. Three ended in a fatal hang, 16 to 44 min in, and two recovered from every hang. One session also had a hang at 36 s that the game recovered from. So the base moved too (24fe1e5 → bd979ce), and this is not a clean A/B.

What the logs show

• bind-time pointers up to frame 1800: 302467 sites encoded, 302467 of them named a pool (1000 per mille); 82761 raw-load shader bindings, 7600 of them with a site empty or no site (they rely on the table)
• The culling tasks are still in the failed command buffers: 47f9b38490dcf9a2 and 99280b7b8aae78df in all three failed buffers of the second log, f75708d176bb6591 in one.
• Two task/mesh pipelines failed to build after about 22 s with Compilation failed due to an interrupted connection: XPC_ERROR_CONNECTION_INTERRUPTED (mesh 4c7d6be5a7717c8d with fragment 4b825abdc0916973; task a7f22b4a795ff4d1, mesh f85bb2e8d78b864c, fragment 72fba2ec8558f6d4). The foliage was then missing, drawn as white. I don't know whether the compiler service failed because of this PR's shaders or because of the hangs. No such line appears in my logs without this PR.

DM_PSO_MTL4=0 was set in the launcher in every run on both builds, so pipelines went through the MTLDevice.

I'm going back to main without this PR, with #1707, and can test a newer revision of this PR on this Mac if that helps.

[nafields]

@Amost- Thanks for the six runs! That's more than I had on this branch.

The sites are doing what they should: They all named a pool. The bindings with an empty site or none fallback to the table.

They should hang only if their pool is missing from it.

Im fairly confident that a table miss looks different from what you saw: the standalone test (task_hang.m in #1583) hangs on every run of the shader, ~22ms of GPU time. AND it hangs at the first frame that runs it.

It shouldn't hang a 700 one time, and 3,200 the next. The failed buffers also hold the three culling tasks every time but a different "most specific" pair in every run, which is what a buffer-level hang looks like, not a shader-level one. So I read your runs as the other hang, the one the description calls separate, and I should have been explicit there.

What I know about that, from my own runs on the same route (M2 Pro 16 GB), so far:

•	It needs one task shader's run and the mesh draws that read its output to share a command buffer with the rest of the frame. 

With a command-buffer cut around that run (DM_NVN_ISOLATE_TASK, a scratch switch in my tree, not in this PR), 4 of 4 warmed runs went past f6,600 with no hang; without it 6 of 6 hung, two or three times each. A cut-only variant, which split the run from the work before it in only about 30 % of cases, also had no hang in 2 runs.
• Full barriers between every draw pass (DM_NVN_BARRIER=0) did not help: 3 hangs in one run. So the relaxed pass-end barriers of this profile are not the cause.
• In the hung buffers the mesh draws read a storage buffer the GPU wrote earlier in the same submission. Whether the missing piece is an intra-encoder barrier between fragment writes and object/mesh reads, or a per-buffer effect such as the tiler's parameter buffer running out, I cannot yet say; the cut-only result leans toward the second.

Yes to the bundle, please; attach it here. Two things I will look for in it: whether any failed buffer's log line names a raw load whose address fell outside its site entry (a word the GPU rewrote after bind would do that; the culling tasks' site is c0[412] +0 and the comment in nvn_mtl.m names FoliageShader as the case), and which shaders make the 17,408 short bindings.

On the chain: 3f146103c47b against 1ffcb6b77740 is expected once the tool files differ after a cherry-pick; dmsh_chain.py --header gives the value for the tree it runs in.

@reefbluesky No, and the sentence in the description promised more than exists. There is no PR with an encoder boundary after the task/mesh run. What I have is the isolation switch described above, which forces a command-buffer cut around one task shader's draws; it prevents the hang in every run so far, but it is a diagnostic that says where the hang lives, not a fix, because the cause is not yet named. I will reword the description, and I will put the switch up as a diagnostic in a PR of its own so the hang can be reproduced and bisected on other machines if people will find that helpful.

Your two runs match mine and Amost-'s in every respect that matters (no page fault, the three culling tasks in every failed buffer, the forest). The frame-time drop with DM_NVN_SKIP_MESH=1 (25.5 → 19.9 ms p50) is a useful number: it bounds what the task/mesh path costs here. One note on reading them against this PR: your build list (main bd979ce, #1368, two local cuts) does not include #1585, so those hangs are main's, and they say nothing about the change here either way; they do say the hang is not something this branch introduced.

@lunks Thank you for the pointer to #1707! It is a different mechanism (holding the next pass's task and mesh shaders until the fragments are done), but it acts at the same place. It is worth a run against this hang.

[LeonardoBai12]

A correction to my run above. I wrote that the XPC_ERROR_CONNECTION_INTERRUPTED lines for task a7f22b4a795ff4d1 never appeared without this PR. They do: on a build without it (main e940e1f plus #1707 and the same other PRs), the host logged the same failure for task a7f22b4a795ff4d1_b (mesh ac2f3feb11427426_b, fragment eddc12907cefc9af_b). macOS wrote 11 MTLCompilerService crash reports in those sessions, all EXC_BAD_ACCESS in libLLVM.dylib under AGXCompilePlan::execute. So Apple's compiler crashes on that task shader with or without this PR, and those lines say nothing about it.

The hang on that build still came at about 7 min, with 47f9b38490dcf9a2 and 99280b7b8aae78df in every failed buffer, which fits your reading that this is the other hang.

[Amost-]

@nafields here are the six runs, redacted the way scripts/bug_report.py redacts its bundle: https://gist.github.com/Amost-/2809c6a5fd244b8f5e4861cd6682bf92

File	Run in my table above
1-run1-10min-clean.log	1, no failure
2-run2-228s-revoked.log	2, revoked
3-run3-5min-recovered.log	3, recovered
4-run4-4min-revoked.log	4, revoked
5-run5-4min-recovered.log	5, recovered
6-run6-4min-recovered.log	6, recovered

They are the host's own logs of headless runs, so there is no report form with them. Runs 4–6 are the host without one of our pool-table changes. Since then the same tree, plus two movie-timing PRs, hung in 21 of 21 runs on that route, with full barriers, a lighter GPU load or a fresh save as well (#1326), so I can run your isolation switch as soon as it is up.

[ycarogabriel-lgtm]

Independent base M1 result for the isolation nafields describes (MacBook Pro M1, 16 GB, macOS 26.6.2; OoT v131072; local branch of open PRs on an older main, not this PR).

Measured: a host switch that ends the command buffer before every draw of a5288d0a6c4971fd (off by default) gave 4 of 4 clean 420 s Kokiri Forest walks, against hangs in nearly every walk without it (our estimate, about 15 of 16, not a controlled count). Each run cut 175,001-195,001 draws, 100% with the cut before them. GPU +0 to 2 ms early on; 31-44 ms after 180 s, similar to uncut runs.
Measured, not curing it: full barriers, mesh grid size, all pools in the raw table, no pool chunks, guard/guard0 mesh variants. A mesh variant that runs the body but emits nothing was clean 3 of 3.
Hypothesis, untested: tiler state when task/mesh draws share a command buffer with the rest of the frame. We do not know the mechanism and the cut hides it rather than explains it.
Offer: we can run the switch from your branch on the M1 once it is published, with the same walk.