Skip to content

Ocarina of Time's first start no longer shows a black picture with the title music for minutes: pso_manifest.tsv seeds the prepare step - #1493

Closed
itsyunusacar wants to merge 1 commit into
seedds:mainfrom
itsyunusacar:oot-pso-manifest
Closed

itsyunusacar wants to merge 1 commit into
seedds:mainfrom
itsyunusacar:oot-pso-manifest

Conversation

@itsyunusacar

Copy link
Copy Markdown
Contributor

What and why

Fixes #1477. A first start of Ocarina of Time (0400486024DEC000 v131072) on a Mac with no pso_user.tsv yet and a cold Metal shader cache shows a black picture with the title music for minutes: the first title frame builds 11 pipelines and loads 12 shader libraries inside one submission, 252.6 s on an M3 Pro (macOS 27.2) with MTLCompilerService at 100 %, while the window keeps presenting the last frame. The profile's pso_manifest.tsv was header-only, so the prepare step had nothing to build before the first frame.

This seeds dm/games/0400486024DEC000/pso_manifest.tsv with the list this Mac recorded on v131072 (chain adec7d81d970) with no lists present, so everything was built on the spot: 9,620 lines (8,376 shaders, 1,112 render pipelines, 132 compute pipelines) over boot, the title and its movie, a new game and the opening, Link's house, Kokiri Forest and a save load. The game note has the finding.

Checked

  • Tests: python3 -m unittest dm/tools/tests/test_nsp_incremental.py (the manifest's derivation into the build).
  • Games run: Ocarina of Time 0400486024DEC000 v131072, Mac app on 671c68e / adec7d81d970. Cold start (lists and Metal cache set aside): frame 108 whole frame 252618.4 ms … new pipelines 11 + shader libraries 12, 252581.4 ms in all, then the title at 287 s. Same start with the list in place: the title in about 20 s, screenshots at frames 300/800/1500 full range, GPU 25 ms a frame in a window. Not checked: a rebuilt app with this manifest bundled (the list was tested through pso_user.tsv, which the prepare step reads the same way), and the first-start time on an M4 Max (the other report in Ocarina of Time (0400486024DEC000 v131072, Mac M3 Pro): black picture with sound from the first frame; Game's upscaler = MetalFX (DM_UPSCALER=metalfx) brings the picture back #1477).
  • Devices and OS: MacBook Pro M3 Pro (18-core GPU, applegpu_g15s), 18 GB, macOS 27.2, Xcode 27.2.
  • Not a chain, host or tool change: no regress.py run.

Docs

  • The finding is in its game note (dm/games/0400486024DEC000/README.md); no docs/reference entry, so docs_symptoms.py was not needed
  • No chain tool changed
  • No game files, keys or data extracted from a game in the change or the attachments (the list holds shader content keys only)

🤖 Generated with Claude Code

…e title music for minutes: pso_manifest.tsv seeds the prepare step

On a Mac that has no pso_user.tsv yet and a cold Metal shader cache, the first title frame built 11 pipelines and loaded 12 shader libraries inside one submission: 252.6 s on an M3 Pro (macOS 27.2) with MTLCompilerService at 100 %, while the window kept presenting the last frame, black, and the title music played (issue seedds#1477; the same symptom reported on an M4 Max). The profile's seed list was header-only, so the prepare step had nothing to build before the first frame.

The seed list is this Mac's pso_user.tsv, recorded on v131072 (chain adec7d81d970) with no lists present, so everything was built on the spot: 9,620 lines (8,376 shaders, 1,112 render pipelines, 132 compute pipelines) over boot, the title and its movie, a new game and the opening, Link's house, Kokiri Forest and a save load. With those built behind the Preparing-shaders screen, or in the archive, the same start reaches the title in about 20 s.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jonpage0

jonpage0 commented Nov 6, 2026

Copy link
Copy Markdown

Same comparison as on #1488, since the two lists conflict on one file:

Both lists compared, since they conflict on the same file: #1488 (9,784 rows: S 8,355, R 1,227, C 144, M 58) and #1493 (9,620 rows: S 8,376, R 1,112, C 132) share 9,558 rows; #1493 adds 21 shaders (its S set is a superset of #1488's) and 40 render pipelines #1488 lacks, #1488 adds 155 render pipelines, 12 compute pipelines and the 58 M rows. A third list, this Mac's own pso_user.tsv after a few hours of Kokiri Forest routes (9,102 rows), lies entirely within #1488. Their union is 9,846 rows (S 8,376, R 1,267, C 145, M 58); we build with that.

Cost of the prepare step with the union list, Ocarina of Time v131072 Mac app on main bd9f422 + #1368 + #1307, M3 Max (36 GB, macOS 27.0.1), the device archive empty each time:

@PucaVaz

PucaVaz commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Compared both lists on a true first start, M4 MacBook Air (24 GB, macOS 27.2, 10 cores), Mac app from main b2d46b8 (+ #1544, #1543, #1555), --seen-shaders build (about 6,930 of each list's ~8,370 shaders are in its library). Each list was given with DM_PSO_MANIFEST=<file> (main's bundled list is header-only). Each run started with no pso_user.tsv, an empty pipeline archive, and the app's Metal caches (com.apple.metal, metalfe, gpuarchiver under the user cache folder) set aside. One scripted run per list from launch through the title, a save load into Link's house, out of the door and Saria's greeting.

#1493 #1488
prepare step 9,620 lines in 134.7 s (887 bases compiled, 29 from the bundled archive) 9,726 lines in 150.4 s (1,001 compiled, 44 from the archive; the 58 M rows skipped)
host start → first frame 135.9 s 151.7 s
first frame → save loaded: frames over 0.25 s 0 0
stalls over 1 s 3.7 s + 2.9 s at Saria's greeting 3.4 s + 2.7 s at Saria's greeting
built on the spot 7 render, 5 compute pipelines 5 compute pipelines

Both lists miss the same five compute pipelines: C rows for 2de44613e731e867, e28d865e2cc82d36, 983d63a70f1d7fcc (Enemy_Scrub_Deku), 50bb5cb052951045 (Enemy_Scrub_Business) and fef9f883f289b61b (Enemy_Dekubaba). Both have them as S rows (stage 5), so the shaders load early, but the pipelines are built when Saria's greeting starts: about 6 s of frozen picture on a first start with either list. Adding those five C rows to whichever list lands should remove it.

The 7 render pipelines #1488 has and #1493 lacks on this route never made a frame over 100 ms (hybrid pipelines cover them), and #1488's longer list costs about 16 s more before the first frame here. On this route, then, #1493's list (or the union, plus the five C rows) gives the shorter wait. I haven't covered later areas, where #1488's iPad rows may matter, or a no-list cold start on this Mac.

@seedds

seedds commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Compute pipelines are now in the offline archive (30fb213), so the title's kernels behind #1477's black screen come from the bundle on a first start. This list (the first-scenes route, as docs/development.md asks) plus the five C rows PucaVaz found at Saria's greeting is the planned seed list. It's waiting on a true first-start measurement on main now that compute is archived, and the result decides whether it lands.

seedds added a commit that referenced this pull request Nov 7, 2026
…ipelines before the first frame instead of building them at the title and in play (PRs #1493, #1488)

pso_manifest.tsv was header-only, so a first start with a cold Metal shader
cache built the title's pipelines in one submission: 252.6 s of black picture
on an M3 Pro (issue #1477). The list is #1493's, recorded as
docs/development.md asks (boot, the title and its movie, a new game and the
opening, Link's house, Kokiri Forest, a save load; 8,376 shaders, 1,112 render
and 132 compute pipelines), and the five compute pipelines Saria's greeting
builds, which PucaVaz found on #1488. 9,625 rows; every shader is in the map.

#1488's longer list (hours of iPad play, 58 M rows main's prewarm skips)
prepared 16 s longer on an M4 MacBook Air and avoided only 7 render builds
that made no frame over 100 ms. Open, in the game note: the first start with
compute pipelines in the offline archive (30fb213) and on iOS, where this
game has no bundled archive, and the render bases the R rows need.

Co-authored-by: Yunus ACAR <125305588+itsyunusacar@users.noreply.github.com>
@seedds

seedds commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Landed in 540f797 with credit: this list unchanged, plus the five C rows PucaVaz found at Saria's greeting (on #1488), 9,625 rows. Thank you for the recording and for tracking #1477 down to the title's pipeline builds. The first start on main (compute pipelines are in the offline archive since 30fb213) is not measured yet; the game note lists that as open.

@seedds seedds closed this Nov 7, 2026
ksolenk38ze added a commit to ksolenk38ze/nxnative that referenced this pull request Nov 7, 2026
…loading

Merge adjacent GPU memory pools with contiguous guest virtual addresses and
matching GPU address deltas into single gmem_map run entries (up to
GMEM_RUN_MAX = 1,024 runs) backed by a reusable MTLBuffer ring
(gmem_table_take / gmem_table_give), and reuse g_gmem_buf across encode_args_
calls within a submission since gmem_commit_patch updates the table at commit
time. Also zero-fill
unmapped virtual pool spans with PROT_READ | PROT_WRITE in vpool_map so late
CPU reads do not fault.

For MTL4Archive pipeline caches (related to PRs seedds#1307, seedds#1488, seedds#1493, seedds#1546),
separate vertex-descriptor (vd) and pulling (non-vd) archive epochs so
specialized vertex-descriptor pipelines are not written into the pulling archive
set; guard against macOS 27's crash in MTL4ArchiveImpl::newArchiveReplyForKey
when deserializing an archive entry with a zero-length function reflection
string; preserve the active OP_CALL return stack when starting
shader_prefetch_rest and pso_predict_rest_; and support falling back to the
bundled shader chain directory when DMSH_CHAIN advances.

Verified in The Legend of Zelda: Ocarina of Time (0400486024DEC000) on Apple
M-series (macOS).
Aznracer pushed a commit to Aznracer/Mighty-Green-Kokiri-Kid that referenced this pull request Nov 8, 2026
…ipelines before the first frame instead of building them at the title and in play (PRs seedds#1493, seedds#1488)
Know-It-All-Brother pushed a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…ipelines before the first frame instead of building them at the title and in play (PRs seedds#1493, seedds#1488)

pso_manifest.tsv was header-only, so a first start with a cold Metal shader
cache built the title's pipelines in one submission: 252.6 s of black picture
on an M3 Pro (issue seedds#1477). The list is seedds#1493's, recorded as
docs/development.md asks (boot, the title and its movie, a new game and the
opening, Link's house, Kokiri Forest, a save load; 8,376 shaders, 1,112 render
and 132 compute pipelines), and the five compute pipelines Saria's greeting
builds, which PucaVaz found on seedds#1488. 9,625 rows; every shader is in the map.

seedds#1488's longer list (hours of iPad play, 58 M rows main's prewarm skips)
prepared 16 s longer on an M4 MacBook Air and avoided only 7 render builds
that made no frame over 100 ms. Open, in the game note: the first start with
compute pipelines in the offline archive (bb0ce6c) and on iOS, where this
game has no bundled archive, and the render bases the R rows need.

Co-authored-by: Yunus ACAR <125305588+itsyunusacar@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game

Projects

None yet

4 participants