Skip to content

Ocarina of Time's prewarm list gets 41 more task / mesh pipelines from later play - #2083

Closed
FifteenPen wants to merge 1 commit into
seedds:mainfrom
FifteenPen:pr/oot-mesh-rows
Closed

FifteenPen wants to merge 1 commit into
seedds:mainfrom
FifteenPen:pr/oot-mesh-rows

Conversation

@FifteenPen

Copy link
Copy Markdown
Contributor

What and why

Ocarina of Time's pso_manifest.tsv has 123 M rows (native task / mesh pipelines, PR #1368 / #1488 / bf420fd), recorded in Kokiri Forest, Hyrule Field, Castle Town and Hyrule Castle. Task / mesh pipelines are not in the bundled offline archive (pso_offline.py leaves mesh pairs to the draw), so one outside the list is compiled on the submit thread at its first draw. On a cold Metal cache that is 50 ms to 1.7 s per pipeline.

This adds 41 M rows: 39 model pipelines behind the task shader a5288d0a6c4971fd_b and 2 foliage pipelines without one. They come from a play log of later areas, Death Mountain Trail among them, recorded by a host whose M rows match main's byte for byte (118 of main's 123 rows appear in that log as they are), and 11 of them were also recorded by main's own host on Death Mountain Trail in the runs below. All 164 rows parse and build in the prepare step (pipeline prewarm: 164 native task / mesh pipelines; … 164 built and captured).

No code change; the game note's entry on native mesh pipelines gets the numbers.

Checked

Cold first start as the Mac app starts it (DM_PSO_PREPARE=1, the prepare step before the first frame), Metal's shader cache and the device pipeline archive emptied before every run, a build of main 04811a6 with other open PRs applied (none of them touches the prewarm or task / mesh pipelines) and its bundled archive, bench.py route field-run from a Death Mountain Trail save (window 85-130 s), M4 Pro, macOS 27.2, alternating:

run prepare step task / mesh pipelines built at their first draw worst frame fps
123 rows 29.6 s 9 (2.7 s, worst 1,742 ms) 1,775 ms 57.4
164 rows 32.1 s 0 25.6 ms 60.0
123 rows 29.3 s 10 (4.2 s, worst 1,627 ms) 1,657 ms 54.7
164 rows 31.0 s 0 18.5 ms 60.0
  • With a warm Metal cache the same pipelines build at the draw in 0.1-2.3 ms, so the gain is a first start's (or a first visit's) on a device.
  • Tests: test_game_profile.py, test_docs.py; docs_symptoms.py --check.
  • Not run: regress.py check (a profile list only, no shader or host change), iOS.

Docs

  • The finding is in its game note (the "Native mesh pipelines compiled at their first draw" entry under Found on this game).
  • No game files, keys or data extracted from a game in the change or the attachments.

…m later play: a cold first start on Death Mountain Trail no longer builds 9-10 of them at their first draw (frames of 1.7 s)
@github-actions github-actions Bot added docs Documentation game: 0400486024DEC000 A game labels Nov 9, 2026
@FifteenPen

Copy link
Copy Markdown
Contributor Author

This PR has a substantial playability benefit. The vast majority of pre-existing stutters are eliminated with this. Things that reliably cause stuttering (like moving the camera around while standing still in Hyrule Castle Town) will no longer do that.

seedds added a commit that referenced this pull request Nov 10, 2026
…m later play: a cold first start on Death Mountain Trail no longer builds 9-10 of them at their first draw (frames of 1.7 s), at about 2 s more in the prepare step (PR #2083)

39 model pipelines behind the task shader a5288d0a6c4971fd_b and 2 foliage pipelines without one; 164 M rows in all. The shader reference's seed-rows line links the game note for the count instead of repeating it.

Co-authored-by: FifteenPen <354289956+FifteenPen@users.noreply.github.com>
@seedds

seedds commented Nov 10, 2026

Copy link
Copy Markdown
Owner

Landed as dbf8499 on main, with you as co-author: your 41 rows and the game-note entry as they were, plus one line in docs/reference/shaders.md that now links the game note for the M-row count instead of saying 123. Thanks for the alternating cold-cache runs; they made this an easy call. Building these mesh pipelines into the bundled archive (pso_offline.py) would remove the extra ~2 s in the prepare step; that stays open in the reference entry.

@seedds seedds closed this Nov 10, 2026
oliverames pushed a commit to oliverames/nxnative that referenced this pull request Nov 10, 2026
…corded: 7,647 rows from Kokiri Forest, Hyrule Field, Castle Town and Hyrule Castle

The list on main came from a Mac (9,625 rows plus the M rows). HookLink recorded 3,192 render pipelines, 4,049 shaders and 406 compute pipelines more on an iPhone 18 Pro Max (iOS 27.2) on v131072, built on the spot in those four areas. They are appended after the existing rows, so a first start prepares them before the first frame. With the 41 M rows main took in PR seedds#2083, the list is now 17,436 rows (12,425 S, 4,304 R, 543 C, 164 M).

Checked: no duplicate row in the file or against main's rows; the existing rows are unchanged and in place; every row has the column count its kind has; all 4,049 new S keys and 406 new C keys are in shader_map.tsv; dm/tools/tests/test_pso_offline.py passes. Not run: the longer list on a device or Mac, so the extra prepare time and what it saves are not measured.

Ported from HookLink's commit ed56d6a (https://github.com/dsi-hub/nxnative/tree/hooklink).

Co-authored-by: dsi-hub <260802842+dsi-hub@users.noreply.github.com>
Fasen24-AI added a commit to Fasen24-AI/swordboiMacNative that referenced this pull request Nov 10, 2026
seedds#1901 seedds#1459 seedds#1928 seedds#1942 seedds#1940 seedds#1952 seedds#1611 seedds#2065 seedds#2083 seedds#2008 seedds#2079 seedds#2022 seedds#2075 seedds#2077 seedds#2020 seedds#2021 onto main 04811a6

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014eD437mxrMnW2UwLZhpwc7

Upstream pull requests applied in this commit (their authors' work, brought onto this tree with a three-way merge; the description is in each PR):
- seedds#1459 by PromethOps (open): DM_UPSCALER=metalfx takes agl's agl_ndsr_vr.pck networks too, and a variant with no curve and no power is drawn as one colour space: Ocarina of Time's upscaler is MetalFX's
- seedds#1611 by Esteban-V (open): An argument block whose only change is a uniform buffer is patched instead of encoded again
- seedds#1774 by uncraftablexd (closed): Compute threads that exit before a barrier keep arriving at it, and every shader loop has a cap (GPU hang defences), chain change
- seedds#1824 by cocoman93 (open): Clamp texel fetches past a sampled 2D texture's edge: no hard edge in Ocarina of Time's mist
- seedds#1901 by FifteenPen (open): The game's compute dispatches with no game barrier between them run side by side (profile key dispatch_overlap, on for Ocarina of Time): Kokiri Forest GPU p50 15.63 → 14.76 ms
- seedds#1928 by FeReuss (open): Ocarina of Time: enc_thread on, the replay encodes on a thread of its own
- seedds#1929 by FeReuss (open): Unused border corrections are compiled out of pipelines that bind no sampler with a twin (function constant DMSH_BORDER_FC)
- seedds#1930 by FeReuss (open): A float add to storage memory with no result read is one native atomic add instead of a compare-and-swap loop
- seedds#1940 by uncraftablexd (open): A prewarm list's lines after its #@after_first_frame line are built after the first frame, in the background, instead of in the prepare step (issue seedds#1747)
- seedds#1942 by uncraftablexd (open): Frame step: late frames with the GPU idle do not step up, and a longer hold the GPU does not explain is probed
- seedds#1952 by AdyaGMD (open): Compute moves ahead of scene passes past occlusion queries and conditional rendering; clears wait across dispatches
- seedds#2008 by bquilty25 (open): nvn: reactive memory watchdog, 16-way view cache, and texture upload batching
- seedds#2020 by padsiwono (open): Ocarina of Time v131072: Output Native draws the full output size instead of the game's 80 % frame (`DM_OUTPUT`)
- seedds#2021 by padsiwono (open): Ocarina of Time v131072 renders at the launcher's Resolution (`DM_RENDER_SIZE`), docked
- seedds#2022 by sleepyeyedjoe (closed): Ocarina of Time: large models no longer go black for one frame twice, 4–5 frames apart, when the game streams new textures into them (a texture ID a shader has read keeps its read mark when it gets another texture)
- seedds#2030 by humanperzeus (open): A NaN colour at alpha 0 is returned as 0, as NVIDIA's blender takes a zero factor: Ocarina of Time's white clouds over the ferns are gone
- seedds#2035 by Know-It-All-Brother (open): The shader build runs as worker processes, and no shader waits for a batch (Ocarina of Time's cold Mac map: 3,002 s to 1,298 s)
- seedds#2065 by Raymon82777 (closed): A vertex library with layer entry points of its own gets bases for its main0 pairs: Ocarina of Time's 230 ms camera-turn hitch in Hyrule Field goes to 39 ms
- seedds#2075 by sleepyeyedjoe (closed): Ocarina of Time: a patch of scenery no longer draws dark for one frame when the game gives it a sharper texture (a pool texture the walk does not make holds a stand-in of its own colours until its first read)
- seedds#2077 by roandegraaf (closed): Ocarina of Time: fix the "plaid" terrain / missing texture tiles (GPU-drawn textures now reach the storage the game copies)
- seedds#2079 by uncraftablexd (open): Render pipelines that share a vertex function are taken from the archive or compiled one at a time: the AGX driver crashes in createVertexProgramVariant's completion block
- seedds#2083 by FifteenPen (closed): Ocarina of Time's prewarm list gets 41 more task / mesh pipelines from later play
oliverames pushed a commit to oliverames/nxnative that referenced this pull request Nov 10, 2026
…corded: 7,647 rows from Kokiri Forest, Hyrule Field, Castle Town and Hyrule Castle

The list on main came from a Mac (9,625 rows plus the M rows). HookLink recorded 3,192 render pipelines, 4,049 shaders and 406 compute pipelines more on an iPhone 18 Pro Max (iOS 27.2) on v131072, built on the spot in those four areas. They are appended after the existing rows, so a first start prepares them before the first frame. With the 41 M rows main took in PR seedds#2083, the list is now 17,436 rows (12,425 S, 4,304 R, 543 C, 164 M).

Checked: no duplicate row in the file or against main's rows; the existing rows are unchanged and in place; every row has the column count its kind has; all 4,049 new S keys and 406 new C keys are in shader_map.tsv; dm/tools/tests/test_pso_offline.py passes. Not run: the longer list on a device or Mac, so the extra prepare time and what it saves are not measured.

Ported from HookLink's commit ed56d6a (https://github.com/dsi-hub/nxnative/tree/hooklink).

Co-authored-by: dsi-hub <260802842+dsi-hub@users.noreply.github.com>
Aznracer pushed a commit to Aznracer/Mighty-Green-Kokiri-Kid that referenced this pull request Nov 11, 2026
…ecorded in a sweep of 33 areas and in long play on an M3 Max (not measured on main)

166 pipelines behind the task shader a5288d0a6c4971fd_b and 5 behind three others (b43c34b2a11c2142_b, efaa4fa6a80ba719_b, 2d3240242e2ec411_b); 335 M rows in all. A task / mesh pipeline that no list names is compiled at its first draw (Native mesh pipelines compiled at their first draw). These were recorded on v131072 by a fork whose host writes the same rows (the key without the border field, the length of main's 164 M rows) as it built them. One of them, task a5288d0a6c4971fd / mesh 53dc93ba404ae78c / pixel 1ac5c1ed2feed1bc, froze adult Zora's Domain for 824 ms at its first draw on that fork.

Checked against main: every row parses with pso_mesh_skey_parse and is byte for byte its normalized form; none is on main already or repeats another (task key and normalized key compared); every task, mesh and pixel key is in shader_map.tsv with that stage; none is among PR seedds#2280's 15 rows. Tests: test_docs, test_game_profile, test_pso_offline, test_nsp_incremental; t_pso_skey and t_shmap_warm (make -C dm/gfx/tests GAME=0400486024DEC000); the Mac host builds. Not run: a game on main, so neither the stalls the rows avoid nor the prepare step's added time is measured (PR seedds#2083's 41 rows added about 2 s on an M4 Pro); iOS.

(cherry picked from commit 81b3971)
UserDerezzed pushed a commit to UserDerezzed/nxnative that referenced this pull request Nov 12, 2026
…ecorded in a sweep of 33 areas and in long play on an M3 Max (not measured on main)

166 pipelines behind the task shader a5288d0a6c4971fd_b and 5 behind three others (b43c34b2a11c2142_b, efaa4fa6a80ba719_b, 2d3240242e2ec411_b); 335 M rows in all. A task / mesh pipeline that no list names is compiled at its first draw (Native mesh pipelines compiled at their first draw). These were recorded on v131072 by a fork whose host writes the same rows (the key without the border field, the length of main's 164 M rows) as it built them. One of them, task a5288d0a6c4971fd / mesh 53dc93ba404ae78c / pixel 1ac5c1ed2feed1bc, froze adult Zora's Domain for 824 ms at its first draw on that fork.

Checked against main: every row parses with pso_mesh_skey_parse and is byte for byte its normalized form; none is on main already or repeats another (task key and normalized key compared); every task, mesh and pixel key is in shader_map.tsv with that stage; none is among PR seedds#2280's 15 rows. Tests: test_docs, test_game_profile, test_pso_offline, test_nsp_incremental; t_pso_skey and t_shmap_warm (make -C dm/gfx/tests GAME=0400486024DEC000); the Mac host builds. Not run: a game on main, so neither the stalls the rows avoid nor the prepare step's added time is measured (PR seedds#2083's 41 rows added about 2 s on an M4 Pro); iOS.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants