Repository navigation
DM_NVN_TEX_BREAKDOWN: shared render targets against the guest storage they stand for ([rtbreakdown] on DM_CTL restop, offline report) - #2313
Open
Fasen24-AI wants to merge 1 commit into
Conversation
…ools against the guest storage they stand for: [rtbreakdown] lines on DM_CTL restop and an offline report How much memory the shared render targets of a game like Ocarina of Time could give back depends on two numbers the host did not log: the Metal memory those targets hold, and the guest storage they stand for once overlapping intervals are counted once. The memory line and restop sum allocated sizes; several targets can sit in one family heap, and on NVN several descriptions overlap in the same pool bytes, so neither sum says what the targets need. DM_NVN_TEX_BREAKDOWN=1 (off by default) makes DM_CTL restop log one [rtbreakdown] line per shared render target of a GPU-only or virtual pool: pool, offset, guest size, Metal format, sample count, the texture and its allocated size, its heap and whether that is a resolution family's, and the frame of its last recorded GPU write (mtl_tex_gpu_written, kept apart from the lookup stamp so a descriptor lookup does not count as a write). dm/tools/nvn_rt_breakdown.py reads a log and reports nominal texture bytes, distinct Metal allocation bytes, the union of guest intervals per pool, format and sample count, what family heaps already cover, and an estimate of the entries a newer write elsewhere has overwritten (NX2navi's rule; equal offsets, a writer inside the older range, unknown write times and targets written in the last two frames are left out). It changes no allocation or residency: it measures the opportunity, not the saving. Measured on an M5 with 24 GB, on a branch that also carries PR seedds#1453 and PR seedds#1731 (so not main's numbers), the scripted Castle Town walk from one save: at its peak 3,950 MiB of distinct shared-target allocations against 841 MiB of guest unions, 469 MiB of them already in family heaps; RGBA16F alone 757 MiB against 37 MiB. The overwrite estimate at that snapshot was 666 MiB. With the diagnostic on and off on the same host (one run each), neither run had a 300-frame window over 17.5 ms; footprint 12,917 and 12,472 MiB. On that branch the lines went to the texstat command of PR seedds#1436; here they go to restop, which main has. Checked: t_rt_breakdown with the switch on and off under Metal validation (only the enabled diagnostic stamps writes, finalizing the pool removes them, the snapshot makes no pool buffer), test_nvn_rt_breakdown (interval unions, allocation deduplication, overwrite exceptions, shared heaps), test_docs; the Mac host builds. Not run: the game on main, whose shaders do not convert there since 0662f9d (issue seedds#2272). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014eD437mxrMnW2UwLZhpwc7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
How much memory the shared render targets of a game like Ocarina of Time could give back depends on two numbers the host did not log: the Metal memory those targets hold, and the guest storage they stand for once overlapping intervals are counted once. The memory line and
restopsum allocated sizes, but several targets can sit in one family heap, and on NVN several descriptions overlap in the same pool bytes. So neither sum says what the targets need.DM_NVN_TEX_BREAKDOWN=1(off by default) makesDM_CTL restoplog one[rtbreakdown]line per shared render target of a GPU-only or virtual pool: pool, offset, guest size, Metal format, sample count, the texture and its allocated size, its heap and whether it is a resolution family's, and the frame of its last recorded GPU write. The write stamp is kept apart from the lookup stamp, so a descriptor lookup does not count as a write.dm/tools/nvn_rt_breakdown.py <log>then reports:It changes no allocation or residency: it measures the opportunity, not a saving. Reference entry: Render-target allocation and guest storage census.
What I found with it on Ocarina of Time, on a branch that also carries #1453 and #1731, so not main's numbers: on the scripted Castle Town walk, at the peak, 3,950 MiB of distinct shared-target allocations stood for 841 MiB of guest storage. 469 MiB of the targets were already in family heaps. RGBA16F alone was 757 MiB against 37 MiB. The overwrite estimate at that snapshot was 666 MiB. So heaps per pixel format at the guest offset would reach more than the overwrite rule, at least on this walk.
On my branch the lines went to the
texstatcommand of #1436; here they go torestop, which main has.Checked
make -C dm/gfx/tests GAME=0400486024DEC000 check_t_rt_breakdown, with the switch on and off, under Metal validation: only the enabled diagnostic stamps writes, finalizing the pool removes them, and the snapshot makes no pool buffer.test_nvn_rt_breakdowncovers interval unions, allocation deduplication, overwrite exceptions and shared heaps.test_docspasses. The Mac host builds.0400486024DEC000, v131072), on my branch, with the diagnostic on and off on the same host, one run each. Neither run had a 300-frame window over 17.5 ms; footprint 12,917 and 12,472 MiB. Not run on main: this game's shaders do not convert there since 0662f9d (Ocarina of Time (0400486024DEC000 v131072, Mac): 22,296 shaders fail to compile in Metal since 0662f9d, dm_blend0'smain0_out&has no address space qualifier #2272).Docs
docs/reference/memory.mdentry, andpython3 dm/tools/docs_symptoms.pywas run🤖 Generated with Claude Code
https://claude.ai/code/session_014eD437mxrMnW2UwLZhpwc7