Skip to content

nvn/pool: patch tables over storage changes after an old sparse mapping - #2489

Closed
bymaxmarc wants to merge 1 commit into
seedds:mainfrom
bymaxmarc:pr/pool-table-sparse
Closed

bymaxmarc wants to merge 1 commit into
seedds:mainfrom
bymaxmarc:pr/pool-table-sparse

Conversation

@bymaxmarc

Copy link
Copy Markdown
Contributor

What and why

In bymaxmarc's two-player skate session of Nintendo Switch Sports Resort (04008E001F098000, v65536; M2 Max, 2026-11-12) the 33,024-entry texture pool table was built whole about 3 times a frame from the game's first sparse mapping (about frame 3,900): 14.7-15.4 ms of a 25 ms submit, with the dynamic resolution at its floor (854x240 a player). Two-player Avioneta (2026-11-11) showed the same pattern, 13.9 ms of a 23 ms submit at 30 and then 24 fps.

Cause: pool_gpu refused to patch a table over any storage change once nvn_sparse_gen() was not 0, that is once any sparse mapping had ever been made, and ser_sync started a new texture ID epoch at every sync after one. A sparse texture's page table changes only at a mapping, so only a mapping since a table's walk (or since the last sync) can leave an entry stale. The game maps a sparse pool a few times a minute in two-player skate; after the first mapping each of its 60-90 storage changes a frame built the table whole and sent all 17,000 texture IDs down the slow path.

The change (nvn_mtl.m, every game):

  • A table keeps the sparse generation read before its walk (pool_snap.vgen) and is built whole over a storage change only when a mapping came since; ser_sync starts a new epoch only for a mapping since the last sync (g_sparse_synced). A mapping is still taken as before, once.
  • Diagnostics, which found the cause: the pool for handles traced to no slot line splits the refusals by cause (registrations past the log, storage changes past the log, a sparse mapping, IDs past POOL_IDS_MAX) with the generations so far, and the [nvn] pool tables up to frame line gives registrations and storage changes per frame. In the skate session 213,859 of 214,245 refusals were a sparse mapping; before the split all of them read too many or sparse. The diagnostics are in this PR because the fix's test and its evidence use them; they change nothing about when a table is patched.

Reference entry: Pool tables patched instead of built whole (the sparse mapping resolved, the split), and Texture ID views revalidated only for the serials that changed; the game note.

Checked

  • Tests: make -C dm/gfx/tests GAME=010015100B514000 check_t_pool_notes: 107 checks, 0 failed, with two new cases (registrations past REG_LOG build the table whole and are counted as such; a table made after the last mapping is patched over noted storage changes with no new epoch, and a mapping after it still builds it whole). With the old sparse conditions put back on this branch, 3 of them fail. make -k -C dm/gfx/tests GAME=010015100B514000 run on this branch: every test passes (t_pool_notes 107 / 0 once rebuilt; the run had picked up a binary from the without-fix trial). dm/tools/tests test_docs, test_env_switches, test_pr_areas.
  • Games run: Nintendo Switch Sports Resort (04008E001F098000, v65536), M2 Max. Table tennis route from the playtest save with MetalFX, 300 s, the host from this change on the fork's branch: no table built whole after the first 19 (3,352 patched, 5,966 kept), 60 fps, submit p50 5.6 ms, GPU p50 8.6 ms (that route makes no sparse mapping). In an 11-minute session on 2026-11-12 with a build of the fork's branch that has this change and the fork's other fixes (not the keep-busy switch), a player played two-player skate (the practice zone, then leaving it) and two-player Avioneta with no crash: the pool tables lines counted 0.00 tables built whole a frame (about 3 before), at 30 fps (present interval 2); two-player Avioneta held mostly 2134x600 a view, while skate's dynamic resolution stayed at its 854x240 floor a view, which is the GPU clock at 30 fps, not the tables (a separate change). A second player cannot be scripted (DM_KEYS / DM_CTL drive player 1 only).
  • Not checked: regress.py check (no other game was run; Ocarina of Time and Fortune's Weave, where the patching and the ID views were measured, were not), lint_c.py tidy (no LLVM release on this Mac), the iPhone.
  • Devices and OS: M2 Max, macOS 27.2.

Docs

  • The finding is in its game note or its docs/reference/<subsystem>.md entry, and python3 dm/tools/docs_symptoms.py was run (the symptom index)
  • A chain tool changed: not applicable
  • No game files, keys or data extracted from a game in the change or the attachments

Symptom: in bymaxmarc's two-player skate session of Nintendo Switch Sports
Resort (04008E001F098000, v65536; M2 Max, 2026-11-12) the 33,024-entry
texture pool table was built whole about 3 times a frame from about frame
3,900 on, 14.7-15.4 ms of the submit thread's 25 ms, and the game's dynamic
resolution sat at its floor (854x240 a player). Two-player Avioneta
(2026-11-11) built it about 3 times a frame for 13.9 ms of a 23 ms submit,
at 30 and then 24 fps. The log counted every refusal to patch as "too many
or sparse" and classed every build as "storage", so it did not say which
limit was passed.

Cause: pool_gpu refused to patch a table over any storage change once
nvn_sparse_gen() was not 0, that is, once any sparse mapping had ever been
made, and ser_sync started a new texture ID epoch at every sync after one. A
sparse texture's page table changes only at a mapping, which bumps no
generation the ID cache or the tables follow, so only a mapping since a
table's walk (or since the last sync) can leave an entry stale. The game
maps a sparse pool a few times a minute in two-player skate (`sparse
mappings 2` by frame 4,200, 12 by frame 8,000); after its first mapping each
of its 60-90 storage changes a frame built the table whole and sent all
17,000 texture IDs down the slow path. With the refusals split by cause,
213,859 of 214,245 were `a sparse mapping`, none the registration or storage
rings or the ID limit. The Avioneta session's log predates the split; it is
likely the same mapping, not confirmed.

Fix (nvn_mtl.m, every game):
- A table keeps the sparse generation read before its walk (pool_snap.vgen)
  and is built whole over a storage change only when a mapping came since;
  ser_sync starts a new epoch only for a mapping since the last sync
  (g_sparse_synced). A mapping is still taken as before, once.
- Diagnostics: the `pool for handles traced to no slot` line splits the
  refusals by cause (registrations past the log, storage changes past the
  log, a sparse mapping, IDs past POOL_IDS_MAX) with the registration,
  storage-change and sparse-mapping generations so far; the `[nvn] pool
  tables up to frame` power line gives registrations and storage changes per
  frame. This is what found the cause; it changes nothing about when a table
  is patched.

Checked: check_t_pool_notes, 107 checks, 0 failed; two new cases:
registrations past REG_LOG since a table was made build it whole and count
as such; a table made after the last mapping is patched over noted storage
changes with no new epoch, and a mapping after it still builds it whole and
starts one (3 of these checks fail without the fix, tried on this branch).
make -C dm/gfx/tests GAME=010015100B514000 run; test_docs,
test_env_switches, test_pr_areas. In the game on an M2 Max (the host from
this change on the fork's branch, the playtest save's table tennis route
with MetalFX, 300 s): no table built whole after the first 19 (3,352
patched, 5,966 kept), 60 fps, submit p50 5.6 ms, GPU p50 8.6 ms; that route
makes no sparse mapping. In an 11-minute session on 2026-11-12 with a build
of the fork's branch that has this change and the fork's other fixes (not
the keep-busy switch), a player played two-player skate (the practice zone,
then leaving it) and two-player Avioneta with no crash: the pool tables
lines counted 0.00 tables built whole a frame (about 3 before), at 30 fps
(present interval 2); two-player Avioneta held mostly 2134x600 a view, while
skate's dynamic resolution stayed at its 854x240 floor a view, which is the
GPU clock at 30 fps, not the tables (a separate change). Not run: a scripted
two-player run (DM_KEYS / DM_CTL drive player 1 only), other games, lint_c
tidy (no LLVM release here), the iPhone.

Docs: nvn-objects.md (Pool tables patched instead of built whole: the sparse
mapping resolved, the refusal split; Texture ID views revalidated only for
the serials that changed), the game note, symptom index regenerated.
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation game: 04008E001F098000 A game labels Nov 12, 2026
@Sennbon-99

Copy link
Copy Markdown

Same change as #2408 (opened a day earlier), found on Ocarina of Time (0400486024DEC000, v131072): it maps its virtual pool once at boot, and after that every storage note rebuilt the 33,024-ID texture table whole. With the change on b149a1e4 (MacBook Pro M4 Pro, played by hand through Lake Hylia and Kakariko), whole builds went from about 20,300 in 40 minutes (59 s) to about 1,960 in 64 minutes (8 s). Both PRs keep the generation seen at the walk in pool_snap.vgen and the one of the last sync in ser_sync, so either fixes both games; linking them so they can be taken together.

@bymaxmarc

Copy link
Copy Markdown
Contributor Author

Superseded by df9ca26 (#1721): ser_sync and pool_gpu no longer treat any past sparse mapping as a reason to start a new epoch or build the table whole, and a new mapping notes only the sparse textures it overlaps, which covers the two-player skate case this fixed. Closing; #2408 is superseded the same way.

@bymaxmarc bymaxmarc closed this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 04008E001F098000 A game nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants