Repository navigation
nvn: merge contiguous gmem pool runs and harden MTL4Archive pipeline loading - #1730
Open
ksolenk38ze wants to merge 1 commit into
Open
ksolenk38ze wants to merge 1 commit into
ksolenk38ze wants to merge 1 commit into
Conversation
github-actions
Bot
added
nvn
…loading Merge adjacent GPU memory pools with contiguous guest virtual addresses and matching GPU address deltas into single gmem_map run entries (up to GMEM_RUN_MAX = 1,024 runs) backed by a reusable MTLBuffer ring (gmem_table_take / gmem_table_give), and reuse g_gmem_buf across encode_args_ calls within a submission since gmem_commit_patch updates the table at commit time. Also zero-fill unmapped virtual pool spans with PROT_READ | PROT_WRITE in vpool_map so late CPU reads do not fault. For MTL4Archive pipeline caches (related to PRs seedds#1307, seedds#1488, seedds#1493, seedds#1546), separate vertex-descriptor (vd) and pulling (non-vd) archive epochs so specialized vertex-descriptor pipelines are not written into the pulling archive set; guard against macOS 27's crash in MTL4ArchiveImpl::newArchiveReplyForKey when deserializing an archive entry with a zero-length function reflection string; preserve the active OP_CALL return stack when starting shader_prefetch_rest and pso_predict_rest_; and support falling back to the bundled shader chain directory when DMSH_CHAIN advances. Verified in The Legend of Zelda: Ocarina of Time (0400486024DEC000) on Apple M-series (macOS).
ksolenk38ze
force-pushed
the
pr/nvn-gmem-runs-and-pso-archive
branch
from
November 7, 2026 23:37
c002240 to
ac46095
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
gmempool run merging: Shaders that access guest memory through raw 64-bit pointers search thegmempool table (pool_gpu_raw) by(guest_va, len, gpu_va). When a game allocates many adjacent NVN memory pools inside one virtual reservation, thegmemtable grows and its binary search deepens. Merging contiguous(lo, hi)pool ranges whose MetalgpuAddressvalues are equally contiguous compresses adjacent runs into a single entry without altering address translation.MTL4Archiveretention and zero-length slice guard: Whendm_pso_prewarmbuilds vertex-pulling base render pipelines (pull=1) into a.mtl4aarchive, storing only the derived state or mixingvd == 0andvd != 0epochs caused pulling bases to be dropped on compaction or recompiled on reload, and.mtl4aarchives with a zero-length reflection slice crashedMTL4ArchiveImpl::newArchiveReplyForKey. SeparatingArchEpochbyep->kind, probing pulling bases inpso_prebuild, carrying theOP_CALLreturn stack acrossshader_prefetch_rest/pso_predict_rest_, and validating_archiveReaderslices on open (arch_entries_ok) avoids startup stalls and archive crashes.### Contiguous gmem pool run merging and pulling-base archive retentionindocs/reference/rendering.mdand updatesdocs/agents/symptoms.md.Related
gmemraw global memory pool table), Switch 2 04004860224DEC000 (Zelda NSW2 1.1.1): boots to nnMain, streams assets, then aborts on an async null; plus AirNT archive crash #1431 (MTL4Archive/ AirNT archive crash), Ocarina of Time (0400486024DEC000 v131072, Mac M3 Pro): black picture with sound from the first frame; Game's upscaler = MetalFX (DM_UPSCALER=metalfx) brings the picture back #1477, Ocarina of Time: a prewarm seed list (9,784 rows), so a first start prepares its pipelines instead of freezing for them in play #1488, and Ocarina of Time's first start no longer shows a black picture with the title music for minutes: pso_manifest.tsv seeds the prepare step #1493 (pipeline prewarm and archive retention).Checked
make -C dm/gfx/tests GAME=0400486024DEC000 t_pso_archive,make -C dm/gfx/tests GAME=0400486024DEC000 run,python3 dm/tools/docs_symptoms.py0400486024DEC000, v131072, SDK 22.2.8), verified cold and warm startup with 40,154 prewarmed pipelines and 60 fps gameplay.Docs
docs/reference/<subsystem>.mdentry, andpython3 dm/tools/docs_symptoms.pywas run (the symptom index)