[Fasen24-AI]

What and why

Walking into another area of Castle Town in Ocarina of Time, on my 24 GB M5, the textures in the residency set went from 2.3 GB to 5.4-6.5 GB. About 4.8 GB of them were render targets the game had already let go (1,744 MB Depth32Float, 624 MB RGBA16Float, while the live targets took 1.5 GB). The Mac ran out of free pages, compressed 5 GB in 10 s, and frames took 62-92 ms for tens of seconds.

The game lays a frame's render targets out with a linear allocator in one GPU-only pool, so when one changes size the ones after it move, and each move is a new sharing entry with a Metal texture of its own. Three things kept the old ones:

1. The ghost ring. A finalized serial that a replayed record resolved keeps its texture until the ring turns. At 512 entries the ring is too small for this game: under about 10,000 entries, the serials its registered descriptors still read are evicted and entered again every frame. That gave 59 to 155 million puts in 100 s and a submit CPU from 7 to 16-23 ms; the bursts in Ocarina of Time on an M1 Pro: slow motion after ~80 min on main 25c2546, finalized-texture table puts jump from <1 to thousands a frame #2213 look the same to me, but I have not reproduced that issue. At 32,768 entries (nvn: preserve texture upgrades, BC render targets, and retired descriptor bounds #1731) the ring takes about 80 s to turn at this game's 400 finalized textures a second, and holds what the game let go for that long.
2. Nothing let an entry go by its age. Only a takeover over the same storage or the pool's Finalize removed one.
3. The takeover's 16-submission spare was too short for compressible targets. At full screen, while the dynamic resolution moves, the game's alternating layouts were made again every frame: 16,819 textures (30 GB of allocations) in 300 frames, and frames of 34 ms.

The change, all in nvn_mtl.m:

• the ring holds 32,768 entries, and an entry that leaves the sharing table takes its ghost serials with it (ghost_purge; DM_NVN_GHOST_TAKEOVER=0 restores the old behaviour, DM_NVN_GHOST_N sets other sizes for measuring);
• every 120 frames, a sweep lets go of the entries nothing holds and no lookup found for 600 submissions (alias_sweep, with GPU memory guard, the system's side: stop before the Mac runs out of RAM (kern.memorystatus_level, DM_NVN_SYS_MARGIN_MB; on #1482) #1589's idle eviction; DM_NVN_ALIAS_SWEEP, DM_NVN_ALIAS_SWEEP_IDLE);
• a compressible entry outlives a takeover for 1,800 submissions while the GPU's allocation is under half of Metal's working set (nvn: preserve texture upgrades, BC render targets, and retired descriptor bounds #1731's hold; DM_NVN_ALIAS_KEEP_MB), and for 600 over it. Ending the hold at the bound instead was a cliff: a session that crossed it made 23,100 textures in 300 frames, with frames of 93 ms.

From #1731 and #1589 this takes only the ring size, the hold and the idle eviction it needs, so it stands on main by itself.

One behaviour changes for a texture ID that names a finalized texture. When a takeover lets that texture go, the ID now resolves anew in the takeover's submission (patched, no table built). Before, it kept the old texture until the ring let its serial go, then resolved anew the same way. t_pool_notes's takeover case now expects this.

Reference entry: Render targets a game moved stayed in GPU memory.

Checked

• Tests: make -C dm/gfx/tests GAME=0400486024DEC000 run. Everything passes except t_pool_retire, which failed once on its 1 s wait while the Mac was building something else; alone it passed three times out of three.
  • t_alias_takeover has three new cases. A finalized serial goes with the texture a takeover lets go (fails with DM_NVN_GHOST_TAKEOVER=0). The sweep lets go of an entry nothing resolves to after 600 submissions and keeps the live ones (fails with DM_NVN_ALIAS_SWEEP=0). Over the hold's bound, a compressible entry outlives a takeover past 16 submissions and goes past 600.
  • test_docs passes. test_env_switches fails on main already, for DM_NVN_VCONV_KEEP and four switches quoted but not read; none of this PR's switches.
• Games run: Ocarina of Time (0400486024DEC000, v131072), on my branch, which also carries nvn: preserve texture upgrades, BC render targets, and retired descriptor bounds #1731 and GPU memory guard, the system's side: stop before the Mac runs out of RAM (kern.memorystatus_level, DM_NVN_SYS_MARGIN_MB; on #1482) #1589. Not on main: this game's shaders do not convert there since 0662f9d (Ocarina of Time (0400486024DEC000 v131072, Mac): 22,296 shaders fail to compile in Metal since 0662f9d, dm_blend0's main0_out& has no address space qualifier #2272). That is also why there is no regress.py report: it is the only game I have here.
• Devices: MacBook Pro M5, 24 GB, macOS 27.2.
• Measurements: a scripted walk from one save through Castle Town, handheld, dynamic resolution on, from the shell, one run each. The host's residency report was read every 10 s and the system's memory every 2 s.

	after	before
residency set at its peak	8.0 GB	11.5 GB
footprint	12.3 GB	14.9 GB
wired, system-wide	10.8 GB	13.6 GB
300-frame windows > 17.5 ms	0 of 54	10 of 32 (seven of them 42-69 ms)
textures made in the walk	18,410	32,071

• In the app at full screen, 150 s of the same walk: worst 300-frame window 19-20 ms, against 67.7 ms before.
• With 8 GB of the Mac held wired, so the game and the rest live in 16 GB, the walk completed with two windows over 17.5 ms.
• 1,200 consecutive frames around the area change, captured one by one, show no dark frame and no one-frame flicker.
• Not measured: other areas than Castle Town, interiors, sessions longer than 34 minutes. The system still reaches its free-page floor at the area change (65 MB): this takes the stalls and about 3 GB away, not all of the pressure.

Docs

• The finding is in its game note and its docs/reference/rendering.md entry (and the ghost table's entry in nvn-objects.md), and python3 dm/tools/docs_symptoms.py was run
• A chain tool changed (none)
• No game files, keys or data extracted from a game in the change or the attachments

🤖 Generated with Claude Code

https://claude.ai/code/session_014eD437mxrMnW2UwLZhpwc7

[aCidcOde]

Data point from a long session with this PR ported (Ocarina of Time v131072, M4 Pro 24 GB, macOS 26.7.2). Our build is an older base with this PR's ghost purge and idle sweep, and with the ghost ring at 32,768.

• The purge and the sweep did their part: 1,668 finalized serials were purged with their texture, and the ring stopped re-putting (7,894 puts in total over 299,700 frames).
• The ring of 32,768 kept many finalized textures for a long time: 1,706 entries early on, about 2,200-2,460 mid-session, then 4,268 and 6,226 at the end. GPU allocated stayed at 20.6-21.1 GB from frame 53,100 to 241,200, then rose to 22.8 GB (frame 272,700) and 24.9 GB (frame 299,700). Resident memory reached 17.4 GB, and the host's memory guard (DM_MEM_CAP_GB=17) closed the game after about 83 minutes.
• The night before, the same build with the ring of 512 (before this PR) ran 204,600 frames with the table at 512, 6,723 puts in total and GPU allocated at most 20.4 GB, with no exit. These were different sessions and routes, so this is not a controlled A/B.

At about 1 MB per entry (devMlGUE's measure in #2213), a ring this large can hold several GB in a long session. Main's 034e91a keeps the ghost table by use and drains it back to 512 once entries go idle, so it looks like a better base for this PR's purge and sweep. We have ported 034e91a under them and will report after a long session.

[UserDerezzed]

With this merged, Ocarina of Time's ground in Kokiri Forest breaks after a minute or two of play. Square patches turn dark and streaked when Link comes back to a place he has already been, a patch can be seen switching from good to bad in the distance, and the patches spread as he keeps moving. Seen on an M3 Max (36 GB, macOS 27.2), v131072, in a tree with other PRs merged. DM_NVN_ALIAS_SWEEP=0 alone makes it go away, and it came back in every run with the sweep on (four).

What breaks

The patches are the game's cached shadow maps: 384x384 Depth32Float render targets (NVN format 0x34, COMPRESSIBLE) in one pool with flags 0x2a1 (VIRTUAL | COMPRESSIBLE | GPU_CACHED | CPU_NO_ACCESS), plus one square map of the same format whose size changes (466 to 1368). The game renders each one once for an area and samples it while the area is in view.

1. Out of view, nothing looks the texture up for 600 submissions, and the texture objects that made it are finalized, so the sweep lets the entry go.
2. Back in view, the game makes a new texture object over the same storage and registers it (each comeback has a new serial), and samples it without drawing into it first.
3. tex_mtl_ finds no sharing entry and makes a new Metal texture. A depth texture in a virtual pool is not filled from storage (tex_init_fill skips both), so it reads zero, and the ground under it draws fully shadowed with the shadow map's streaks.

To see what the sweep lets go and what comes back, I logged each eviction by key, and each later texture made over an evicted key with whether its first use writes it (a render-target attach) or reads it. One Kokiri session: 2,978 entries (7.6 GB) let go in the first 84 s. 400 of them were made again in that time, 280 of those read before any pass wrote them. All 400 were render targets: the shadow maps above, plus the dynamic-resolution targets (960x540, 1920x1080, 640x360, 480x270), which come back 10 to 2,221 frames later.

What didn't fix it

Keeping an idle entry while a registered texture ID still names its key (NVN has no unregistration, so a registered ID can be read at any time): in gameplay the sweep found 0 such entries. The game re-registers a new texture object each time it reuses a shadow map, so nothing but the storage links the old texture to the new one.

Why the idle rule can't tell

An entry nobody looked up for 600 submissions may be a render target the game has let go (the Castle Town case this PR is for), or one whose bytes the game expects to find again over the same storage, as on the console, where the bytes are the memory. For a texture the GPU wrote, the host's Metal texture is the only copy of those bytes, so letting it go loses them. The sweep's own comment says as much about the read path: "the IDs that read it read 0 at the next walk".

A direction for a fix

Make the eviction lossless instead of guessing. Before the sweep lets go of a GPU-written entry in storage the CPU cannot write, store its pixels into its NVN storage. The host already has the GPU swizzle for that (gsb_store / gsb_swz); it needs Depth32Float added (MTLBlitOptionDepthFromDepthStencil on the copy out). Then, when a texture is made over a key that was written back, fill it from storage, depth included. An entry whose storage was written since its last lookup (by a GPU buffer write, or another entry over it) can go without a write-back, since on the console those bytes are already gone. One that can't be stored (a combined depth / stencil format, storage with no GPU address) stays. The write-backs need a byte budget per frame: one Kokiri sweep let go of 613 entries, about 1.5 GB, which at once would be a visible hitch.

I'm working on this and can offer it as a commit for this branch, if that suits you.