Repository navigation
Ocarina of Time (0400486024DEC000) loses its forest haze: bounded uniform reads (26062f9e, 918e9f34 / #2127) miscompile in Apple's Metal compiler at -O1 #2425
Description
Summary
On Ocarina of Time (0400486024DEC000 v131072, Switch 2, NVN2), the haze and light shafts of Kokiri Forest are gone since the chain that bounds uniform reads at computed offsets (ecd9cb97f72c, against bf6035002b98 before it).
The cause is one compute shader, the froxel light injection 9ed48d1011f1b48b. As translated with the bounds, it writes only zeros into its 128x72x96 froxel volumes. The volumetric haze pass that reads them then adds nothing.
- The translation is correct: the same MSL compiled at
-O0gives the right picture. - It is Apple's Metal compiler (32023.922) at
-O1that miscompiles the shader whenever its 195 uniform reads at computed offsets carry an upper bound, whatever the form of the bound.
Environment
- Apple M2 (MacBook Air, 16 GB), macOS 27.0 (26A428).
- Apple metal version 32023.922 (metalfe-32023.922.4).
- dmsh_build's Mac flags:
-O1 -mmacosx-version-min=26.6 -fpreserve-invariance -fwrapv -fbracket-depth=1024.
How it was found
- Scene: Kokiri Forest, at the load point of a save there, camera still.
- Method: one frame's passes and colour targets were dumped raw (
DM_NVN_PASSDUMP,texdumpwithDM_NVN_SNAPRAW,texraw), on chain ecd9cb97f72c and on bf6035002b98, and compared target by target. The whole scene's upper half is 3.7× darker. - Inputs that match byte for byte:
- the 256x256 sky/ambient pair and the SH producer's buffer;
- the R16 1360x1000x8 array;
- the probe, the shadow mask and the 64x36 ambient volumes.
- First divergent data: the 128x72x96 froxel volumes (six 3D storage images, two ping-pong sets) are all zero. They are written by
9ed48d1011f1b48band integrated by5b2bb0924ecc403b, whose translation is unchanged between the two chains. - The shader's uniform bindings are the same on both chains: c3 13568 bytes, c4 16960, c5 14848. The size table (bank 25) holds them.
The split
Every variant of 9ed48d1011f1b48b was given to the unchanged ecd9cb97f72c host through a library overlay. Ratios are against bf6035002b98 (scene upper half / haze volume / final image); 1.0 is correct and about 0.26 is the regression.
| Variant | Result |
|---|---|
| bf6035002b98's translation | 1.01 / 1.02 / 1.03 ✓ |
| ecd9cb97f72c's translation as shipped (local compile, control) | 0.27 / 0.35 / 0.38 ✗ |
| … with the 15 new threadgroup fences removed | 0.27 / 0.34 / 0.38 ✗ |
| … with the size table reporting 64 KB for every bound slot (host switch) | 0.27 / 0.35 / 0.38 ✗ |
… the 195 bounded reads as plain reads cb[i][c] |
1.00 / 1.00 / 1.01 ✓ |
| … every guard compared with a constant 64 KB (always true here) | 0.26 / 0.32 / 0.38 ✗ |
… select(0, cb[i][c], cond) (plain index) |
0.25 / 0.34 / 0.38 ✗ |
… cb[cond ? i : 0][c] (no select) |
0.26 / 0.32 / 0.38 ✗ |
… the guard reduced to i >= 0 (no upper bound) |
1.00 / 1.03 / 1.02 ✓ |
… the guard as uint(i) < size >> 4 (row compare) |
0.26 / 0.35 / 0.38 ✗ |
| … a bitwise mask of the load instead of select | 0.26 / 0.33 / 0.38 ✗ |
… the shipped MSL unchanged, compiled at -O0 |
0.99 / 1.02 / 1.03 ✓ |
Why the bounded and plain forms must agree here
Statically, every guarded read is a small base plus a clamped index:
- c3 at 12352–12408 bytes plus a cascade index (0–3) × 64;
- c5 at 944–1832 bytes plus
min(max(i, 0), 4)× 112, and similar.
Against bound sizes of 13568 and 14848 bytes, every guard is true. So select(0, read, cond) must equal the plain read. With a constant 64 KB guard it cannot even depend on the table, and it still fails at -O1.
Possible upstream fixes (for the maintainer)
- Drop the upper bound and keep
>= 0. This gives up the protection 26062f9 added for Paper Mario: The Thousand-Year Door's reads past a 16-byte binding (Paper Mario: TTYD 1.0.1 — GPU hangs during Peach’s invisibility-potion sequence on Mac and iPad #1860). Not recommended alone. - Bound on the host instead of in the shader: bind short uniform slots padded with zeros up to what the shader can read, as e45694b does for fixed offsets, and emit plain reads. It keeps Maxwell's semantics and avoids the compiler, at a bandwidth cost of up to 64 KB per slot read at computed offsets, per draw or dispatch.
- (Narrow) Emit the plain read where the translator can prove a read in bounds: a clamped index times a constant stride plus a constant base, within the slot's fixed extent. That would have covered this shader, and it keeps the bound elsewhere.
Workaround in use
Our OoT build (cut8h) replaces this one shader's library with the plain-read form, compiled at -O1. It is the "plain reads" row above: every read is statically in bounds here, so it means the same. It is a one-shader replacement in the app's library image, reapplied by a script after each library rebuild.
Open
- Other games: whether other shaders on other games hit the same miscompile is unknown.
- How widespread in OoT: 329 of its 42,848 translated shaders carry bounded computed-offset uniform reads: 165 compute kernels and 164 others, about 23,000 guarded reads in all, up to 840 in one shader. Only 9ed48d10 is known to be miscompiled. The others are unchecked; comparing their
-O0and-O1output on captured frames would check them. - Apple: a reduced case (the MSL, the
-O1and-O0libraries, flags, expected and actual output) is ready for Apple Feedback.
Activity
seedds commentedon Nov 12, 2026
Thanks for the careful split. I checked it on an M4 Pro (20-core GPU), macOS 27.2 (26B123), with the same Metal compiler (32023.922), and could not reproduce it there:
9ed48d1011f1b48bis byte-identical to the shader in the current select-form library (chain8d9afacc48e2). Its-O1LLVM IR is what the source says: each guarded read is a load at the chosen row and a select. There is nounreachableor added poison, and all six froxel writes are kept.-O0and the-O1library (largest difference 1.5e-4).-O0, the upper half's brightness ratio is 0.98–1.01 (two runs of the shipped form), where your M2 gave 0.27.So the translation looks right, and the fault looks like Apple's GPU back end on older GPUs, as in #2127 (M1 only).
One question decides the next step: does the haze still go missing on your M2 after updating to macOS 27.2 (or later), with the shipped library and no overlay? The GPU compiler ships with the OS, and you are on 27.0. Your library-overlay ratios against bf6035002b98 would be enough, as before.
If it is still broken on 27.2, the plan is your option 3 in general form: the translator proves the index range of a computed-offset read (a clamp times a constant stride plus a constant base, ranges merged through phis), and a proven read is bounded by the host's padding, as fixed-offset reads are since e45694b, and emitted as a plain read. Only unproven reads keep the in-shader bound. For this shader that should give your "plain reads" row. We would then need you to check it on the M2.
reefbluesky commentedon Nov 12, 2026
@seedds @dpadr answering the question about macOS 27.2, with one more data point.
Still broken on 27.2. MacBook Pro M2 Pro, macOS 27.2 (26B123), Metal 32023.922, Ocarina of Time
0400486024DEC000v131072: with the shipped library (main0d84a3f7+ local commits that do not touch drawing, chain2dadab744059) the haze and light shafts at the Kokiri Forest exit and in Hyrule Field are gone. A build from mainab41d6a1(+ #2030 and2f131edc) draws them; the commit where they vanish for me is918e9f34.One-shader overlay. I took only
9ed48d1011f1b48b(library and reflection, the reflection'schainfield set to the host's) from a tree where its computed-offset reads are written ascond ? read : 0(main0d84a3f7withe45694b7,918e9f34andadaff735reverted, chain204d2bdb369d; 599 such reads against 236selects in the shipped one) and put it in an overlay folder over the shipped library, withDM_PSO_ARCHIVE=off DM_PSO_BUNDLE=off(log: overlay folder loaded,0 from the archive, no chain rejection, no[RED]). The haze and light shafts come back, at both spots, with the same save and camera.So on this M2 Pro the shader written with the branch form
cond ? read : 0works, while the shipped one with theselectform fails, which is a form missing from the table in the issue (that one listsselect,cb[cond ? i : 0][c], a row compare and a bitwise mask). The branch form was what 26062f9 produced until 918e9f3 changed it.A caveat on what this shows: the overlay differs from the shipped shader in two ways, not one. Besides the 195 computed-offset reads (same count in both, branch form against
select), it also guards the fixed-offset reads (they are bare in the shipped one), so I have not isolated which of the two matters, and I have not compiled at-O0here, so "Apple's compiler" is your inference and the M4 Pro result, not something I measured. I then ran that overlay: I took the shipped9ed48d1011f1b48band changed only its 195 uniform reads with a computed offset, from theselectform back to the branch form (cond ? read : 0), leaving everything else untouched (the fixed-offset reads stay as they are). The mist comes back in the Hyrule field on the M2 Pro (macOS 27.2). As a control, recompiling the unmodified source reproduces the shipped library byte for byte. So the trigger seems to be theselectform on those computed-offset reads, not the fixed reads. This is one short session and one screenshot, so please treat it as a strong hint, not a proof of a compiler bug. A minimal fix could be to emit those reads in the branch form again.This is the same symptom as #2276 (fog, light shafts and glow gone on an M1 Max, and the same M2 Pro here): the regression window and the
918e9f34bisect there point at the same change, and the three mist shaders discussed in that thread (task, mesh and fragment) are not the cause: seven variants of them, an exact-form overlay of the first two and a diff of their MSL changed nothing here, while this one-shader overlay brings the mist back. Logs and the overlay are available.computerguy526 commentedon Nov 12, 2026
I think the M2/M4 split is a threadgroup-size limit, not a miscompile. Data from an M4 Pro (macOS 26.6.2), with offline compiles for the M2 using the Metal toolchain's
applegpu-nt(32023.922).What happens. On GPUs without Dynamic Caching (
applegpu_g13*/applegpu_g14*: M1, M2), a compute pipeline'smaxTotalThreadsPerThreadgroupdrops as the kernel uses more registers:The GPU compiler picks the register count without knowing the threadgroup size.
nvn_mtl.mdispatches the shader's local size and never compares it with the pipeline's limit.9ed48d1011f1b48bis 8x8x8 = 512 threads.9ed48d1011f1b48bonapplegpu_g14gat-O1with dmsh_build's flags (registers / spill bytes per thread):cond ? read : 0i >= 0)-O0[[max_total_threads_per_threadgroup(512)]]At 128 registers the pipeline allows 384 threads. A 512-thread dispatch is then over the limit, no invocation runs, and the six volumes stay zero. This matches every row of the split table. The working forms all come in at 104 registers or fewer, and plain reads land exactly on the 512-thread budget. Every upper-bound form comes in at 128, which is why the form of the bound didn't matter.
-O0works because it spills down to 48 registers. M3/M4 (g15/g16) top out at 96 registers and allow 1,024 threads, which explains why the M4 Pro could not reproduce it.The front end's
-O1IR matches seedds' reading: nounreachable, only benignpoison, and all sixwrite_texture_3dcalls kept.Other froxel kernels. Seven more 8x8x8 froxel kernels in the game are at 124–128 registers on g14g with the select form:
f2e5a4e0,6b65ea05,2503f644,109fc917,2445e600,3044d755,f01717f6. Under the branch form all are at 72–96. Across four of our main-based libraries, 123 of 363 kernels with more than 384 threads exceed their size's budget on g14g:On g16s none of them changes.
Proposed fix (translator, general). Emit
[[max_total_threads_per_threadgroup(N)]]on every compute kernel whose local size is above 384. Below that size no register count can make the pipeline refuse the dispatch. The front end passes it to the GPU compiler asair.max_work_group_size, and the compiler fits the registers to N, spilling the rest. On g16s the GPU code is byte-identical with or without it, for all 363 large kernels and a 120-kernel sample of small ones. On g13/g14 the attribute also changes the code of about half the kernels that already fit, which is why sizes of 384 and below are left alone. Task and mesh threadgroups here are 32 threads or fewer.In game, M4 Pro (Kokiri Forest exit save, main-based build): the haze and light shafts are present, and
9ed48d10dispatches 16x9x12 groups of 512 threads.With these eight kernels replaced by copies that return at once (what an over-limit dispatch does), the haze and light shafts vanish on the M4 too, and the scene's upper half drops to 0.64–0.66 of normal. With the attribute added to the same eight kernels through a side library, the picture is unchanged (0.99–1.01) and frame times are unchanged, as expected from identical GPU code.
The fix still needs a check on an M1 or M2 (dpadr, reefbluesky): the shipped library plus a one-shader overlay of
9ed48d1011f1b48bbuilt with the attribute added beforekernel void main0(. A host-side check of each dispatch againstps.maxTotalThreadsPerThreadgroup, with a red line when it is exceeded, would also make this visible in the log.reefbluesky commentedon Nov 12, 2026
@computerguy526 confirmed on an M2 Pro (macOS 27.2, Metal 32023.922).
I took the shipped
9ed48d1011f1b48b(still the select form) and only added[[max_total_threads_per_threadgroup(512)]]beforekernel void main0(, as a one-shader overlay with the reflection'schainset to the host's. The haze and light shafts come back in Hyrule Field.That fits your threadgroup-limit explanation better than what I wrote above. My earlier overlay, which turned the 195 computed-offset reads from
selectback into the branch form, also brought the mist back, but I now read that as the lower register count doing it, not the select form itself. So please disregard my "the trigger seems to be the select form" line.This is one short session on one machine, so please read it as a single confirmation, not a sweep. I can run other kernels or a host-side check if that helps.
computerguy526 commentedon Nov 12, 2026
@reefbluesky thanks, that's the confirmation the offline numbers needed: the select form unchanged, only the attribute added, and the haze is back on the M2 Pro.
The general fix is up as #2487. The translator adds
[[max_total_threads_per_threadgroup(N)]]to every compute kernel whose local size is above 384, and leaves smaller kernels alone. On g16s the GPU code is byte-identical for every kernel I measured, so M3/M4 shouldn't change.If you have time for one more check, a library built from #2487 (it changes the chain, so it's a full rebuild) would cover the other seven 8x8x8 froxel kernels as well, not just
9ed48d10. Any spot where volumetrics or light shafts were missing on the M2 since918e9f34would be worth a look. Hyrule Field and the Kokiri Forest exit are the two we know.On the host-side check: a red line when a dispatch's threadgroup is larger than the pipeline's
maxTotalThreadsPerThreadgroupwould have named this at once. I'll keep it out of #2487 so that stays one topic, and can follow up with it separately if seedds wants it.emmanuel-ghz commentedon Nov 12, 2026
#2487's attribute fixes the missing fog on an M1 Pro as well.
applegpu_g13s), 16 GB, macOS 27.0.1, Metal 32023.922.0400486024DEC000v131072, chainb-8c0bedfbc881(main near224ecb18).What was done. I did not rebuild the chain. Instead:
shader_facts.tsv), each shipped<key>_b.metalwas taken fromcas.db.[[max_total_threads_per_threadgroup(N)]]was added beforekernel void main0(, as inmax_threads.MAC_METAL+-fbracket-depth=1024. As a check, an unmodified shader recompiled with these flags came out byte-identical to the cached library.Contents/Resources/msl/b-8c0bedfbc881/as a folder overlay, and the app was re-signed ad-hoc.maxTotalThreadsPerThreadgroupmeasured on this GPU (shipped → with the attribute; function constants zeroed):109fc917f6f88db52445e600f3887bc22503f644249ad3873044d7555ebdd4276b65ea05fc1e816c9ed48d1011f1b48bf01717f643536e86f2e5a4e0899b3013990c4f53,739ed280,21d9f83eat 1024, …)All 8 failing kernels are from
AglShader.sharcb.Result in play. Only those 8 were overlaid. Haze, mist and light shafts came back in Kokiri Forest and Lost Woods (the maze's darkness), and the player reports them in almost every area visited. No
[RED]lines.Side effect on a 16 GB Mac. With the effects drawing again, footprint in Kokiri Forest rose from about 11 to about 13 GB (
DM_GFX_SCALE=1, handheld), and the machine started swapping. Going back to Hyrule Field did not recover the frame rate. That fits the earlier observation in #2276 that the broken build used about 3.5 GB less.DM_GUEST_FILE=1brought footprint to about 6.4 GB and kept ~30 fps. It costs heavy file writes, though: macOS flagged about 2 GB of dirtied file-backed memory in 40 s.Separately, with the frame cap at 60 the player had GPU stalls and one full system freeze that needed a forced power-off. The log was lost with the reboot, so this is only a report. At a 30 fps cap it did not happen.
reefbluesky commentedon Nov 12, 2026
@computerguy526 checked #2487 with a full rebuild on an M2 Pro (macOS 27.2, Metal 32023.922), no overlay.
The tree is main
0d84a3f7plus #2271 plus the commit of #2487, plus a few local commits that do not touch the shader translation (my base differs from the PR's, sodmsh_chain.hwas regenerated: chain99737dc05205). The haze and light shafts are back at both places we know, the Kokiri Forest exit and Hyrule Field. The log has no[RED]orErrorHang, and the game holds 30 fps (p99 33.3 ms over 300-frame windows) at about 10 GB footprint on a 32 GB Mac.One session on one machine, and I only looked at those two places, so I did not check the other seven froxel kernels one by one. On my older base 4 of the 5 tests in the PR's test file pass; the fifth uses
dmsh_chain.MAC_METAL, which my base does not have, so it is not about the patch.chain/dmsh_rewrite: a compute kernel above 384 threads carries its lo…