Skip to content

games/0400486024DEC000: update Ocarina of Time pso_manifest.tsv, shader_seen.tsv and notes - #1732

Open
ksolenk38ze wants to merge 1 commit into
seedds:mainfrom
ksolenk38ze:pr/oot-profile-and-notes
Open

ksolenk38ze wants to merge 1 commit into
seedds:mainfrom
ksolenk38ze:pr/oot-profile-and-notes

Conversation

@ksolenk38ze

@ksolenk38ze ksolenk38ze commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

What and why

Updates the The Legend of Zelda: Ocarina of Time (0400486024DEC000) prewarm pipeline manifest (pso_manifest.tsv, 40,154 entries regenerated on current main covering Kokiri Forest, the Great Deku Tree, Hyrule Field, Kakariko Village, Dodongo's Cavern, Zora's Domain, Jabu-Jabu's Belly, and the Forest Temple through Phantom Ganon), expands shader_seen.tsv (7,278 -> 25,261 shader keys, covering all keys referenced by pso_manifest.tsv plus all 520 mesh/task shaders and their 1,000 paired pixel shaders from shader_pairs.tsv so --seen-shaders partial Mac builds cover the same route and mesh pairs), and updates the game notes (README.md).

Related

Checked

  • Tests: python3 dm/tools/docs_symptoms.py, python3 -m unittest dm/tools/tests/test_game_profile.py dm/tools/tests/test_docs.py (18/18 OK)
  • Games run, with title ID and version: The Legend of Zelda: Ocarina of Time (0400486024DEC000, v131072, SDK 22.2.8), verified prewarm archive generation and 60 fps gameplay on Mac.
  • Devices and OS: Apple Silicon Mac, macOS 26.2

Docs

  • The finding is in its game note or its docs/reference/<subsystem>.md entry, and python3 dm/tools/docs_symptoms.py was run (the symptom index)
  • No game files, keys or data extracted from a game in the commit

@github-actions github-actions Bot added docs Documentation game: 0400486024DEC000 A game labels Nov 7, 2026
@ksolenk38ze
ksolenk38ze force-pushed the pr/oot-profile-and-notes branch from 64a4c47 to b920949 Compare November 7, 2026 23:42
@ksolenk38ze ksolenk38ze changed the title games/0400486024DEC000: update Ocarina of Time profile, manifest, and notes games/0400486024DEC000: update Ocarina of Time pso_manifest.tsv and notes Nov 7, 2026
@ksolenk38ze
ksolenk38ze force-pushed the pr/oot-profile-and-notes branch from b920949 to cce1ad9 Compare November 7, 2026 23:55
@ksolenk38ze ksolenk38ze changed the title games/0400486024DEC000: update Ocarina of Time pso_manifest.tsv and notes games/0400486024DEC000: update Ocarina of Time pso_manifest.tsv, shader_seen.tsv and notes Nov 7, 2026
@ZhaowTao

ZhaowTao commented Nov 8, 2026

Copy link
Copy Markdown

A partial build on a wider route misses 2,614 keys, and this list covers all of them

Independent playtest data, in case it helps judge whether 25,261 is enough.

Setup. Mac app built with --seen-shaders at chain adec7d81d970, engine 641d1480 (2026-11-03). Its partial library is 8,648 keys in a 3.59 GB b-adec7d81d970.img, and its pso_manifest.tsv has 2 rows, so no pipeline is prewarmed. Apple M5, 24 GB, macOS 26.6.2, v131072.

Method. Two ordinary play sessions, about 11 and 29 minutes of play before a GPU address fault stopped the picture (the host kept presenting afterwards: issue 1326). I scanned both host logs for the lines the missing library produces:

[nvn-mtl] [RED] failed to load shader <key>_b: cannot open
    .../msl/b-adec7d81d970/<key>_b.metallib

Result. 2,358 keys in the first log, 1,391 in the second, 2,614 unique keys together. Every one of them is a key of the v131072 shader_map.tsv (checked against the map at main, 42,848 keys). By stage:

stage keys
vertex 1,204
pixel 1,151
compute 259

Coverage against the two lists.

list of the 2,614
main's shader_seen.tsv (7,278 keys) 0
this PR's shader_seen.tsv (25,261 keys) 2,614
in neither 0

What the gap is made of. Not places, effect and enemy tables:

  • by package: vs_effect.bin 813, ps_shader.bin 683, ps_effect.bin 464, vs_shader.bin 387, cs_effect.bin 252, cs_shader.bin 7, vs_lib.bin 4, ps_lib.bin 4
  • by source column: 1,529 under /Effect, 1,077 under /Shader
  • enemies: Enemy_Peahat 119, Enemy_Stalchild 58, Enemy_Keese 48, Enemy_Peahat_Larva 32, Enemy_Guay 30, Enemy_ReDead 19, Enemy_Gibdo 15
  • weather and fog: EftLoc_Hyrule_Fog_HyruleLakeArea 50, EftLoc_Kakariko_Fog 26, ..._DeathMountain 24, ..._HyruleCastle 22, ..._ZoraArea 17
  • water: EftLoc_HrlField_WaterFall_ZoraTerrain_A 21, ..._WaterFallBottom 19

Why I think this supports the PR. The 7,278 list is the main route. A player who fights at night, crosses the lake in fog or walks the Zora waterfalls leaves it almost at once, and the picture loses those draws until someone rebuilds. This PR's list does not depend on a route at all: it is the keys pso_manifest.tsv references plus every mesh and task shader with its pairs, so it happens to cover the effect tables a route-based list never asks for. Our 2,614 missing keys came out of it with no additions, which is the first evidence I have that the "whole map, not a walk" rule is the right one for this title.

One observation I can only report, not explain: of the 259 missing compute keys, cs_effect.bin holds 252, and in those sessions the picture showed wrong-colour patches (magenta and a red glow) rather than dropping those draws. That is the opposite of what issue 1519 describes for a missing shader, so I am not claiming a cause. DM_NVN_ZBLOCK=1 on the next build should say whether the textures those passes fill are all-zero blocks.

A doc nit: the game note still reads listed in shader_seen.tsv (7,276; building them took about 10 minutes when most came from the previous chain's libraries), while this PR takes the list to 25,261. The build time of the new list is also much larger than ten minutes, so that sentence may want the new figure.

I am not asking for changes to the list; I wanted to hand over a route that reaches past it, since 25,261 is a big number and this says where it stops being one.

…er_seen.tsv and notes

Expand dm/games/0400486024DEC000/pso_manifest.tsv (40,154 rows), update
dm/games/0400486024DEC000/shader_seen.tsv (25,261 keys, including the
playthrough's shader keys and all 520 mesh/task shaders with their pixel
partners for --seen-shaders builds), and document the renderer findings
verified across the playthrough through the Forest Temple in
dm/games/0400486024DEC000/README.md.
@ksolenk38ze

Copy link
Copy Markdown
Contributor Author

Thank you @ZhaowTao for the independent verification against your 2,614 missing shader keys! That is a great confirmation that deriving shader_seen.tsv from the full pso_manifest.tsv plus all 520 mesh/task shaders and their 1,000 paired pixel shaders covers the effect, weather, fog, and night enemy tables outside a single walk.

We also updated line 26 of dm/games/0400486024DEC000/README.md to reflect the new 25,261 figure as noted.

@ksolenk38ze
ksolenk38ze force-pushed the pr/oot-profile-and-notes branch from cce1ad9 to 96a61bb Compare November 8, 2026 10:25
@ycarogabriel-lgtm

ycarogabriel-lgtm commented Nov 8, 2026 •

Copy link
Copy Markdown

Base M1 data on the shader key list (MacBook Pro M1, 16 GB, macOS 26.6.2; OoT v131072; local branch, chain 90e8d82142f4).

Measured: our shader map has 42,848 keys; the app library we built from a seen-list had 18,563 of them. One 25-minute session logged 592 failed to load shader keys, none benign (enemy and object effects in the Deku Tree, 22 Model.Product ones), and the repo's shader_seen.tsv of that time covered 0 of the 592. Separately, the foliage compute e28c9a738fbd99fe and 3 Weapon_Sword shaders were missing; the host skips the dispatch ("the dispatched program has no compute shader"), so foliage disappears for about 3 frames at each sword hit. We built those 33 as an overlay.
Hypothesis/plan, not yet done: we are building the other 23,322 keys. We have not checked whether the 25,261 keys here cover our 592; we can post that overlap. We would share the key list and method, not compiled libraries, which are chain-specific.

@ycarogabriel-lgtm

ycarogabriel-lgtm commented Nov 8, 2026 •

Copy link
Copy Markdown

A data point on how far the list can go, from a base M1 (MacBook Pro, 16 GB, macOS 26.6.2), measured against this PR's head (shader_seen.tsv = 25,261 unique keys).

  • We built the whole dm/games/0400486024DEC000/shader_map.tsv offline: 42,848 unique keys, of which 42,847 produced a metallib. The one failure is c58be4c13dd41495 (listed in shader_map.tsv as agl_common.sharcb#BAHS): the translator stops with "Ampere instruction 481 not implemented". The word it chokes on, 020c0481, looks like the first word of a vertex shader's header rather than an instruction, so this is probably a false BAHS entry of the kind the shader splitter already filters (the same word shows up in Mario Kart World 1.8.0 (v917504): plays on the Mac; iOS kills it for memory when the first race loads (iPhone 18 Pro Max) #1237 for another game); we have not re-run the split to check that the current rule removes this key, so treat it as a lead, not a finding.
  • Cost on that machine: about 4 hours of wall time including pauses for memory guards (12 batches of 2,000 keys, 3 to 7 workers), and a 9 GB library. With only the build running, memory stayed comfortable at those worker counts (free memory above 60% in the batches we sampled); running a game next to it pushed swap to about 4 GB and we paused the build.
  • 17,587 keys of shader_map.tsv are not in this PR's shader_seen.tsv; 0 keys in the PR's list are absent from shader_map.tsv, so the list has no stray entries. In one 25-minute Deku Tree walk with an app library of 18,563 keys, we logged 592 failed to load shader keys; the walk stopped logging any once the library covered the whole map (a 55-minute session afterwards: 0 such lines).
  • The 17,587 are simply the rest of the existing shader_map.tsv, so a complete list needs no new information, only a decision to include them. We can share the plain key list (one hash per line) if useful. We are not sharing binaries, metallibs or MSL, since they derive from the game.

Aeonitis pushed a commit to Aeonitis/nxnative that referenced this pull request Nov 9, 2026
… rows, joined with this fork's) and shader_seen.tsv (25,261 keys)
Know-It-All-Brother added a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…the list keeps seedds#1663's rows it lacks, its native mesh (M) rows among them
AmateurWizrd added a commit to AmateurWizrd/nxnative that referenced this pull request Nov 9, 2026
# Conflicts:
#	dm/games/0400486024DEC000/pso_manifest.tsv
FeReuss pushed a commit to FeReuss/nxnative-oot that referenced this pull request Nov 10, 2026
…fter the first frame, in the background, instead of in the prepare step (issue seedds#1747)

The prepare step built every line of the prewarm lists before the game's first frame. A seed list recorded over a whole playthrough holds pipelines for scenes hours away: Ocarina of Time's 40,155-line list (PR seedds#1732) compiled 10,737 bases in 170 s on an M4 Pro before the title, against 880 bases in 48 s for main's 9,625 lines (title and Kokiri Forest).

- nvn_mtl.m warm_parse: a list's #@after_first_frame line (WARM_LATER_MARK, matched without trailing blanks or CR) puts its later lines behind every list's lines before such a line, so this device's pso_user.tsv stays in the step. A line the first part already has stays there; a pair's later render rows join its task (warm_lines counts them)
- warm_start: the first part is prepared and the prebuild makes the pulling bases, then warm_later builds the rest on half the performance cores at utility QoS (as pso_prebuild_bg), with warm_finish at its end. Draws whose pipeline is not built yet use their pair's pulling pipeline (hybrid). A pair that only the later lines name and no pair table lists has no pulling base and is built when first drawn
- Not split (everything in the step, as before): without hybrid draws, and on a start that compacts the pipeline archive (a compaction recaptures every base the prewarm finds before it removes the old files; such starts find them all in the archive)
- A list without the line prepares as before; the background prewarm (no prepare step) ignores it. No list on main has the line yet: it belongs after main's 9,625 rows in PR seedds#1732's list
- t_warm_later: the two parts (order, the user list, a pair's later row, two marks, CRLF, stale keys)
- docs: development.md (the prepare step), reference "A first start prepares a whole playthrough's pipelines before the title"
Fasen24-AI added a commit to Fasen24-AI/swordboiMacNative that referenced this pull request Nov 10, 2026
…_manifest.tsv, shader_seen.tsv and notes

Upstream pull requests applied in this commit (their authors' work, brought onto this tree with a three-way merge; the description is in each PR):
- seedds#1732 by ksolenk38ze (open): games/0400486024DEC000: update Ocarina of Time pso_manifest.tsv, shader_seen.tsv and notes
Fasen24-AI added a commit to Fasen24-AI/swordboiMacNative that referenced this pull request Nov 10, 2026
…r pipelines, 4 shaders, 3 compute) not in seedds#1732

Upstream pull requests applied in this commit (their authors' work, brought onto this tree with a three-way merge; the description is in each PR):
- seedds#1732 by ksolenk38ze (open): games/0400486024DEC000: update Ocarina of Time pso_manifest.tsv, shader_seen.tsv and notes
@XIIYAZ

XIIYAZ commented Nov 13, 2026

Copy link
Copy Markdown

This PR conflicts with main in pso_manifest.tsv and the README. How I merged it for my Ocarina of Time build:

The game ran with that list for several days on an M4 Max: no pipeline stalls I could see, in areas that stuttered before.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants