[nafields]

What and why

Refs #1587, which has the design and the data.

Symptom. Ocarina of Time's Kokiri Forest load adds 3.2 GB of render targets to the residency set in about 30 s, and on a 16 GB Mac swap follows wired memory.

Cause. The game keeps its render targets as texture objects over one 305 MB virtual pool and reuses the storage by overwrite: 781 live targets over 74 MB of storage. Each one keeps a private Metal texture, and treg_tex makes one for every registered ID, dead or not.

Change (DM_NVN_RT_RECLAIM=1, off by default; nvn_mtl.m, rc_*):

• A shared texture is dead once other textures' whole-attachment passes have covered its storage range since its own last write. A partial pass covers nothing.
• Its Metal texture goes to a free list keyed on the full descriptor. It is handed over after its last use's fence, zero-filled, and trimmed after 15 idle frames.
• A reclaimed ID reads as an empty slot until a pass writes it again.
• Never reclaimed: IMAGE textures, copy and clear destinations, readback sources, pool-copy overlaps, sparse pools, textures filled from pool bytes, and history targets read before their next write.

Reference entry: Render targets dead by overwrite. The game note has a line for it, and env_switches.tsv lists the switch.

What it builds on:

• The descriptor key and the zero-fill follow from the aliasing rule measured by heap_alias in Standalone Metal probes: heap aliasing, render-target sizes, mesh index ranges, mesh pipeline archive, mesh/task sweep, task-shader hang #1583: same width, height, bpp and compression, or the bytes are scrambled.
• rt_sizes (Standalone Metal probes: heap aliasing, render-target sizes, mesh index ranges, mesh pipeline archive, mesh/task sweep, task-shader hang #1583) is why a size-class key is left as the open follow-up.
• The game runs were on a tree that also carried Retire obsolete NVN chunks after whole-pool mapping #1417 (chunk retire) and Pools that hold shader code are no longer made resident to key their shaders (Ocarina of Time: 222 pools, 334 MB the GPU never reads) #1484 (shader pools not made resident). Those are their authors' PRs and aren't part of this one. This change doesn't depend on either.

Checked

• Game: Ocarina of Time (0400486024DEC000, v131072), Kokiri Forest route to f7,500, M2 Pro 16 GB, macOS 26. Two runs with the switch on:
  • render targets in the set: 1.1–1.2 GB instead of 3.5 GB;
  • peak wired memory: 7.6–8.8 GB;
  • swap: +0.05–2.4 GB.
  • Shots at f1,500 and f2,500 match a tree without the switch in scene and mean brightness (127.4 / 96.3 against 128.9 / 96.1).
• Rebased on main, switch off (the default):
  • make -C dm/host GAME=0400486024DEC000;
  • make -C dm/host/tests GAME=0400486024DEC000 -j run;
  • check_t_alias_takeover (28, 0 failed) and check_t_image_alias (329, 0 failed);
  • test_docs.py, env_switches.py --check, docs_symptoms.py.
• Not done:
  • No unit test drives the reclaim yet.
  • scripts/regress.py check wasn't run: only Ocarina of Time is available here. The switch is off by default, so other games run as before.
  • t_nvn and t_gfx_suspend don't link on main b2d46b8 (_dm_vi_frame_queued), independent of this change.

Docs

• The finding is in docs/reference/rendering.md and the game note, and python3 dm/tools/docs_symptoms.py was run
• No chain tool changed
• No game files, keys or data extracted from a game in the change or the attachments

[nafields]

Known issue, fix in progress: with DM_NVN_RT_RECLAIM=1, Ocarina of Time takes GPU page faults at the same frames in every run (f3,677 and f4,851, both compute command buffers). A run with the switch off clears both. The suspect is the 15-frame trim: it releases a reclaimed texture, and a cached view or resource ID of it (the texture-ID registry's cached views, or a patched texture-pool slot) still names it. Runs with the trim off, and with reuse off, are under way to separate the two. The fix will be pushed to this branch, with what it was. The switch is off by default, so main's behaviour is unchanged meanwhile.

[nafields]

Update on the page faults reported above: found and fixed in #1607, which this PR now depends on. The cause wasn't in the reclaim itself. In tex_by_id's slow path (main's code), a texture ID with no texture kept its cached view: treg_tex was nil and the base view's parentTexture was nil too, so nil == nil matched. The pool tables' fast path then wrote the resource ID of a texture the trim had released and taken out of the resident set. With #1607, runs with the reclaim on to f7,750 have neither the f3,677 nor the f4,851 fault, the pace is the same, and the reclaim counts are alike. A fault at f786 remains; it has another cause and happens with the reclaim off too. This branch needs no change of its own.

[PromethOps]

Tested this PR on main. In my scene, turning the reclaim on corrupts the picture. Details below, in case they help.

Setup

• Base: main ff33639 (it includes A texture ID with no texture drops its cached view: nil == nil kept a released texture on the pool tables' fast path (GPU page faults) #1607 as 0e4480d) with 4b231ac cherry-picked. Only the game note conflicted.
• Shaders built for chain 3ee493b2c3a4.
• Ocarina of Time v131072 on an M5 Pro, from a Kokiri Forest save (the path to the Great Deku Tree, in the fog), shots at f1500 and f2400.
• No other PR in the tree. In particular, not Retire obsolete NVN chunks after whole-pool mapping #1417 or Pools that hold shader code are no longer made resident to key their shaders (Ocarina of Time: 222 pools, 334 MB the GPU never reads) #1484, which your runs had.

Results

	Picture	Residency set at f2400	Resident / GPU allocated at f1800
Switch unset	clean	5,296 allocations, 4,422 MB	8,819 / 10,636 MB
DM_NVN_RT_RECLAIM=1	a large flat green area over the top-left half of the screen, with a stepped lower edge, at f1500 and f2400	5,213 allocations, 4,286 MB	8,745 / 10,906 MB

The green shows in every reclaim-on run (two). There are no GPU errors in any run. On this base the set is only 136 MB smaller, and GPU allocations are 270 MB larger: main has already cut most of this pool's targets since you measured.

What the reclaim's own log says. The "most read while dead" targets are the ones in the 305 MB virtual pool around the scene colour target:

[768x432 f5 off 0x1800c00: 5 deaths read before a write] [384x216 f10 off 0x18c0e00] [384x216 f52 off 0x1921000] [384x216 f63 off 0x1a41600] …

That matches what I found on #1453 (#1453 (comment)):

• The engine packs live targets into the part of the 1536×864 R11G11B10F scene target at 0x1800c00 that a 768×432 dynamic-resolution render leaves untouched in NVN's block-linear layout.
• One example: a 384×216 RGBA32F history at 0x1aa1800 starts exactly at x = 768 of block row 3.
• So in this pool, live targets sit inside other targets' ranges by design.
• I see that rc_cover already ignores partial passes. So the scene target's own 768×432 pass shouldn't kill anything, and I haven't traced which full pass declares these targets dead.
• The read-while-dead targets are all in that packed region, though, and their IDs reading as empty slots would explain the flat colour.
• A full pass of a smaller target over the scene target's tail can cover a live neighbour's bytes. This can happen when the engine moves the packing as the resolution changes. A pass's coverage could then be checked against the block-linear bytes it really writes, per target layout, rather than its pool range.

Two small things

• rc_on takes getenv("DM_NVN_RT_RECLAIM") != NULL, so DM_NVN_RT_RECLAIM=0 turns the reclaim on. My first "off" run was really on. env_switches.tsv documents =1, so atoi, or sw_mode like the other switches, would match it.
• The first "most read while dead:" line printed garbage bytes before the list (most read while dead:PЕZ{). The line buffer looks uninitialized when the list is empty.

I can rerun with extra logging on this save if useful, for example the coverage decision for the target at 0x1aa1800.

[dpadr]

Same green area here with DM_NVN_RT_RECLAIM=1 (with #1607) on an M2 MacBook Air (16 GB), OoT v131072 in Kokiri Forest, handheld; clean with the switch unset.

[seedds]

Closing this one. Thank you, @nafields, and @PromethOps and @dpadr for testing it. #1607, which came out of chasing this PR's faults, is on main.

The green area that two testers saw with the switch on (M5 Pro and M2 Air) means the rule "dead once other whole-attachment passes covered its range" marks some targets dead while the game still reads them. PromethOps' finding on #1453 fits that. The game keeps live targets in the part of the full-size scene target that its current resolution leaves unused, so a full-size pass over a range doesn't mean what was there is gone. Without a liveness rule that holds for that, a reclaim of live objects can't land.

The churn part of the problem is handled on main in 7f50222 (#1831), without judging anything dead. Each render target in a GPU-only pool gets a placement heap of its own. When the host frees the texture (the sharing table's takeover, after its last use completed), the heap stays resident and the next target of about the same size goes into it. That uses your rt_sizes finding (#1583): neighbouring resolutions share Metal's allocation sizes. On an M6, while the resolution moved, the residency set grew by 27-37 MB per 300 frames with it, against 455-521 MB without it.

What this PR aimed at beyond that, the live total (about 2 GB in the village), stays open in the game note. A new PR is welcome if a sound rule for when a target is dead turns up. It would build on the recycled heaps rather than a free list of its own.