Repository navigation
Finalized textures in use stay in their table: no 100–490 ms frames at a streamed transition on 0400486024DEC000 - #2389
roandegraaf wants to merge 2 commits into
Conversation
… lists up to 8,192 views of retired registrations: on 0400486024DEC000 a streamed area transition no longer gives frames of 100 to 490 ms, and the pause menu no longer leaves 50 to 55 fps behind it g_ghost, the table of finalized serials a registered ID or a replayed record still names, was a ring of 512 whose oldest entry went when a new one came. With more than 512 resolved in a submission every lookup missed, resolved the texture again and evicted another, each with a deferred release and two notes: 3,669,055 insertions in 300 frames, all on the submit thread. An entry resolved in the last GHOST_IDLE (16) frames now stays, past NGHOST if need be; the ring is a queue that grows, and each submission's commit trims a bounded number of idle entries back to 512. POOL_IDS_MAX goes from 1,024 to 8,192, the limit the patch already has for the IDs it takes: past 1,024 views of retired registrations each submission built the 33,024-entry table whole (1.4 to 1.6 builds a frame of 9 ms). New reference entry (More finalized textures in use than their table held), linked from the game note. Checked: check_t_cpu_tex with a new case (100 checks), check_t_pool_notes, check_t_upgrade_alias, test_docs, host build; play on the profile by the player. Not run: regress.py (one local game), iOS.
…ms, one frame, not under 20 ms
|
Tried this on an M1 Pro (16 GB) with Ocarina of Time, cherry-picked onto 8b55ebb since that's the shader chain I have built locally (only the docs conflicted).
The honest caveat: that session never hit the burst that triggers #2213 (about 4,200 finalized textures in the whole 10 minutes, at most ~390 in any 300 frames), so I can say it doesn't break or slow anything here, but not that it fixes the slow motion. I'll keep playing on this build and report back if I manage to reach the bad case. |
@devMlGUE You're right about the limit: at ~390 finalized textures per 300 frames you never got near 512 in 16 frames, which is where the old table started thrashing. So this confirms it costs nothing on an M1 Pro, but not that it fixes the slow motion. The area-entry hitches are separate. Your base If the slow motion comes back, the line to watch is |
|
Thanks, that's a useful pointer, the area-entry hitches are what I actually notice while playing. One thing though: I can't find 370c15c on main. GitHub shows the commit, but it isn't an ancestor of main (c081686 today), and Ocarina of Time's prewarm list has the same 14,754 rows on main as on my 8b55ebb build. Is it on a branch or fork that hasn't landed yet? If you point me to where it lives I'll build with it and report how many of the entry hitches go away. And agreed on the slow motion, I'll watch the "put in total" line. If I get a run that reaches the burst I'll post the log lines here. |
The table of finalized texture serials (g_ghost in nvn_owner.m: what a replayed record or a retired registration still resolves) was a ring of 512 whose oldest entry went when a new one came, whatever its use. A retired registration's view goes at the end of each submission and is resolved again in the next, so with more than 512 of them read at once each resolution evicted an entry the next would need, and the eviction's note sent its IDs to be resolved again: the ring never settled. Ocarina of Time (GitHub issue #2213): 3,669,055 puts in 300 frames at a streamed area transition with frames of 100-490 ms on the submit thread (PR #2389), up to 43 million in a window and 10-25 fps for minutes in long sessions; reproduced without the game in the issue (600 finalized IDs a submission: 600 puts every submission). The table is now kept by use. Each entry's last lookup or put (own_resolve_ref, tex_put: the frame from the new own_hook_frame, and the submission) is kept in a list from least to most recently used. Past NGHOST (512) entries the least recently used goes only once it is idle, unused for GHOST_IDLE (16) frames or GHOST_IDLE_SUBS (1,024) submissions, so a working set larger than 512 stays while it is used; frames because a game makes many submissions a frame (about 25 here, and a window of 16 submissions left 99.6 % of the evictions coming back within a frame, PR #2389), submissions so a stretch with no present cannot keep every entry. Each put evicts up to two idle entries and each submission's commit up to GHOST_TRIM (32, under the texture ID cache's 64 notes; own_ghost_trim), so the table drains back to 512 once its textures idle. A stale view's entry leaves the list with it (own_view_drop_stale), in constant time instead of a walk of the ring; own_check_locked checks one place in the list per ghost serial, in order of use. POOL_IDS_MAX is 8,192, the IDs one pool table patch takes, and names that limit too: with 1,024, each submission behind the game's pause menu built its table whole (PR #2389's log: 1,927 of about 2,000 builds). Reference entry: More finalized textures in use than their table held; the game note links it, symptom index regenerated. Checked: t_serial_owner (176 checks: entries in use stay past 512, the least recently used goes once idle and one used again is passed over, 2,048 serials resolved every frame are all found with no put and then drain 32 a commit back to 512, idle by submissions with the frame standing still, a re-put after a stale view; four deliberate breaks of the owner each fail it), t_cpu_tex --metal (105), make -C dm/gfx/tests run, lint_c tidy on the four changed files (at the baseline), test_docs. A seeded differential driver gave main's ring and this table the same owner calls (13 seeds, up to 60,000 calls): identical lookups, drops, notes and counts below 512. regress.py check against 8a5c5e7 (M6, headless): Bastion 59.6 / 59.5 fps and Obra Dinn 59.8 / 59.2, no new red line, CHANGED on screenshots alone (embers, the rower's stroke); neither keeps more than 4 ghost entries. Ocarina of Time, the village walk on the M6 with the chain header pinned to the libraries on disk: with the table cut to 8 entries in both hosts, the ring made 4,876 puts by frame 7,800 and this table 202, growing past 8 while used and draining back; at 512 it does what main does on that walk (72 to 266 puts, 42.0 M submit instructions a frame either way). The walk falls into a slow mode after the load (frames of 60-120 ms) in 2 of 4 runs on either host with memory free, once with 73 puts: not the table's. Not run: the reporters' long sessions (the slow motion took an hour or more of play to reach), the PR's transition on this revision, the iPhone. Open: retired views are still dropped and taken again every submission. Diagnosis, the window in frames and the pool limit from PR #2389; the mechanism and the standalone reproduction from devMlGUE in issue #2213. Co-authored-by: Roan de Graaf <36153028+roandegraaf@users.noreply.github.com>
|
Thank you, landed on main as 034e91a with you as co-author. Your diagnosis, the window in frames rather than submissions, the pool list limit and the test case are all in it. What differs from this branch:
Checked on an M6 with Ocarina of Time: with the table cut to 8 entries in both builds, the ring made 4,876 puts on the village walk and the new table 202. At 512 it matches main on that walk, with no submit cost. A differential test shows it resolves exactly what the ring did below 512. Your transition was not re-run on this revision, so the reporters' long sessions in #2213 will decide the rest. |
…ture views: every submission that read such an ID made its Metal view again, about 310 of the 330–347 views Ocarina of Time (Switch 2) made a frame in Kokiri Forest The view cache kept views only for live textures. A texture finalized while one of its IDs is still registered resolves through g_ghost, and vc_set_for refused its set because nothing would drop it. tex_by_id then made a transient view that treg_finish_submission released at the submission's end, and the next submission made it again. Such a texture's set is now kept, marked as a finalized texture's. Once a frame, from tex_by_id, vc_ghost_sweep drops the sets of the serials g_ghost has let go. The ring evicts under g_smu, which can't take g_rmu. The views hold the texture only as long as g_ghost already does, plus up to a frame. DM_NVN_VC_KEEP (default 1): 0 turns it off, ab alternates it with the DM_AB phases. In-run A/B on Ocarina of Time's Kokiri Forest route (M2 Air 16 GB, 20 pairs of 240-frame phases): - views made 330–347 → 38 a frame, none from eviction; - the submit thread −1.72 M cycles a frame [−1.91, −1.49] and −5.37 M instructions [−5.65, −5.06]; - frame time within noise (GPU-bound). On main 0c23e0b (serial tables in nvn_owner.m, seedds#2389's ghost table kept by use): vc_set_for and vc_ghost_sweep ask the owner through own_ghost_has / own_ghost_has_locked, new: a look at the ghost table that is not a use, so a cached view never keeps its serial in the table. Every tex_by_id lookup still resolves the ID's texture (a use), so a finalized texture the game reads stays in the table, and its set with it; once the owner lets an idle entry go, the next sweep drops the set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ture views: every submission that read such an ID made its Metal view again, about 310 of the 330–347 views Ocarina of Time (Switch 2) made a frame in Kokiri Forest The view cache kept views only for live textures. A texture finalized while one of its IDs is still registered resolves through g_ghost, and vc_set_for refused its set because nothing would drop it. tex_by_id then made a transient view that treg_finish_submission released at the submission's end, and the next submission made it again. Such a texture's set is now kept, marked as a finalized texture's. Once a frame, from tex_by_id, vc_ghost_sweep drops the sets of the serials g_ghost has let go. The ring evicts under g_smu, which can't take g_rmu. The views hold the texture only as long as g_ghost already does, plus up to a frame. DM_NVN_VC_KEEP (default 1): 0 turns it off, ab alternates it with the DM_AB phases. In-run A/B on Ocarina of Time's Kokiri Forest route (M2 Air 16 GB, 20 pairs of 240-frame phases): - views made 330–347 → 38 a frame, none from eviction; - the submit thread −1.72 M cycles a frame [−1.91, −1.49] and −5.37 M instructions [−5.65, −5.06]; - frame time within noise (GPU-bound). On main 0c23e0b (serial tables in nvn_owner.m, seedds#2389's ghost table kept by use): vc_set_for and vc_ghost_sweep ask the owner through own_ghost_has / own_ghost_has_locked, new: a look at the ghost table that is not a use, so a cached view never keeps its serial in the table. Every tex_by_id lookup still resolves the ID's texture (a use), so a finalized texture the game reads stays in the table, and its set with it; once the owner lets an idle entry go, the next sweep drops the set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eps its views until the ghost table lets it go (PR #2482) The view cache (vc_get / vc_put) kept views only for live textures. A texture the game finalized while one of its IDs is still registered resolves through g_ghost, and vc_set_for refused its set, as nothing would drop it. So every submission that read the ID made a transient Metal view, released at the submission's end, and the next submission made it again: about 310 of the 330-347 views Ocarina of Time (Switch 2) made a frame in Kokiri Forest. Such a texture's set is now kept, marked as a finalized texture's. At the first commit of each frame, right after own_ghost_trim, vc_ghost_sweep drops the sets of serials the ghost table no longer holds (the eviction runs under g_smu, which cannot take g_rmu). The checks look at the ghost table without counting as a use (own_ghost_has), so a kept view does not keep its texture in the table. A texture buffer's 2D rows stay out of these sets: a finalized texture's rows are kept by the memory they view (g_tb2_gone), which other textures' descriptors over that memory share. DM_NVN_VC_KEEP (default 1): 0 turns it off, ab alternates it with the DM_AB phases. The view-cache log line counts the finalized textures' sets. Measured in the PR, on a line before #2389 (in-run A/B, OoT Kokiri Forest, M2 Air, 20 pairs): views made 330-347 -> 38 a frame, submit thread -1.72 M cycles a frame [-1.91, -1.49], frame time within noise (GPU-bound). Not measured again on main: no OoT libraries on disk. Checked: t_vc_ghost (new; 19 checks, 17 with DM_NVN_VC_KEEP=0), t_nvn, t_serial_owner, t_rt_lifetime, t_queue_lifetime, t_cpu_tex, t_pool_notes, t_tbuf2d, t_tbuf_align, t_readback, t_alias_takeover, t_upgrade_alias, t_pool_chunks; regress.py check on Bastion (makes no views) and Return of the Obra Dinn (4 finalized sets kept; its intermittent GPU hang hit baseline and candidate alike over 3 runs a side): fps and frame times unchanged. Co-authored-by: dpadr <34929160+dpadr@users.noreply.github.com>
…no 100–490 ms frames at a streamed transition on 0400486024DEC000 Ported to the serial tables' owner: since 1ee0252 the ghost table is nvn_owner.m's, so the PR's queue, its frame stamps and ghost_trim are there; the owner takes the frame from the replay layer through a fifth hook, own_hook_frame (nvn_mtl.m: g_cur_frame), and mtl_sub_commit calls own_ghost_trim_idle. own_check_locked checks the table against its queue instead of against NGHOST. POOL_IDS_MAX 8,192 in nvn_mtl.m as in the PR. t_serial_owner: its hook stand-in, the eviction case past GHOST_IDLE frames (in any queue order: a trim sends a serial in use to the back), and the PR's in-use case at the table level; t_cpu_tex's two changes through own_stats_locked and own_ghost_trim_idle. The reference entry names the owner. Checked: t_serial_owner 65, t_cpu_tex 100, t_pool_notes 91, t_upgrade_alias 13, t_view_cache 44, t_partial_upload 3,585, t_alias_takeover 45, test_docs, the Mac host for 0400486024DEC000. Not run: any game.
…ghost entry (2ba06d7) leaves the queue of seedds#2389, not a ring slot ghost_del_locked removes the serial's place in the queue and its frame stamp; own_check_locked checks each queue place names a ghost serial, none twice (with the place left behind, every step after a stale drop fails it). t_serial_owner: main's two ring-order cases move the test's frame GHOST_IDLE on first (a ghost serial goes once idle), the one after a stale view checks with ghost(), which resolves nothing, and its serials start at 970000, clear of the teardown's. DM_TEST_SERIAL_TWIN's row goes: 2ba06d7 fixed the twin and removed the switch. Checked: t_serial_owner 106 (also with DM_TEST_SERIAL_TWIN=1), t_srgb_alias 15, t_cpu_tex 100, t_pool_notes 91, t_upgrade_alias 13, t_view_cache 44, t_alias_takeover 45, t_codec_video, test_docs, test_env_switches, the Mac host for 0400486024DEC000.
…s into the owner beside seedds#2389's ghost queue nvn_owner.m: main's sharing table and its obj_drop (own_keeps_locked) first, seedds#2389's queue functions after them; own_check_locked checks the ghost queue, then main's sharing table (its ring-slot count is gone with the ring). The owner's hooks are main's four plus own_hook_frame. t_serial_owner: main's sharing-table cases before the teardown, which moves the test's frame GHOST_IDLE on as before. t_view_cache (seedds#2260) takes own_keeps_locked for the alias_has main removed. Checked: t_serial_owner 149 (also with DM_TEST_SERIAL_TWIN=1), t_srgb_alias 15, t_cpu_tex 100, t_pool_notes 91, t_upgrade_alias 13, t_view_cache 44, t_alias_takeover 46, t_partial_upload 3,585, all 148 test programs in dm/gfx/tests build, test_docs, test_env_switches, the Mac host for 0400486024DEC000.
…eue notes its changes through own_note Main's ad80867 moved the change generation and retirement into the owner and dropped own_hook_changed and own_hook_unheld: the queue's eviction, its put and the stale view's removal call own_note, and the owner keeps one hook of seedds#2389's beside main's two (own_hook_frame). t_serial_owner: main's retirement cases, whose ghost eviction moves the test's frame GHOST_IDLE on first; t_cpu_tex runs main's retire_after_completion and seedds#2389's ghost_keeps_used. DMSH_CHAIN regenerated (b581efc's chain change with seedds#1930's rewrite). Checked: t_serial_owner 167, t_cpu_tex 104, t_srgb_alias 15, t_pool_notes 91, t_upgrade_alias 13, t_view_cache 44, t_alias_takeover 46, t_partial_upload 3,585, all 147 test programs in dm/gfx/tests build, test_docs, test_env_switches, test_nsp_chain, the Mac host for 0400486024DEC000.
t_cpu_tex runs main's retire_across_threads and seedds#2389's ghost_keeps_used; retire_across_threads moves the frame GHOST_IDLE on before the NGHOST puts that evict its ghost entry, as handover_holders does (a ghost serial goes once idle). DMSH_CHAIN regenerated. Checked: t_cpu_tex 107 (three runs), t_serial_owner 167, t_srgb_alias 15, t_view_cache 44, all dm/gfx/tests programs build, test_docs, test_env_switches, test_game_profile, test_nsp_chain, the Mac host for 0400486024DEC000.
…dds#2389 (034e91a) replaces the batch's port PR seedds#2389 closed: main keeps finalized textures in the ghost table by use (a least-recently-used list, own_ghost_trim). The batch's port (the queue, own_ghost_trim_idle, its tests and its reference entry) goes: nvn_owner.m, nvn_owner.h, t_serial_owner.m and t_cpu_tex.m are main's, nvn_mtl.m takes main's trim and POOL_IDS_MAX comment, the reference entry and the game note's line are main's. nvn_rec.h: seedds#2341's held commit takes main's g_res_commit_mu (seedds#2379) in place of g_res_mu. DMSH_CHAIN regenerated. Checked: t_serial_owner 176, t_cpu_tex 105, t_replay_ahead, t_view_cache 44, t_dispatch_barrier (four modes), t_srgb_alias 15, all dm/gfx/tests programs build, test_docs, test_env_switches, test_game_profile, test_nsp_chain, the Mac host for 0400486024DEC000.
What and why
Symptom. On
0400486024DEC000on a Mac, a streamed area transition gave frames of 100–490 ms for about ten seconds. The GPU was at its usual 10–25 ms, with no new pipelines and no texture uploads in those frames; the[longframe]lines put the whole frame insubmit+present. In the same reports:finalized textures still referenced … put in totaldeferred-release queueSeparately, opening and closing the pause menu (the game streams behind it) left 50–55 fps for about five seconds, with
pool tables … built wholeat 1.4–1.6 a frame (9 ms each) anda new submission over too many retired IDscounting 1,927 of about 2,000 builds.Cause. Two limits sized for a few finalized textures in use at once.
g_ghost, the table of finalized serials a registered ID or a replayed record still names, was a ring of 512 whose oldest entry went when a new one came. With more than 512 resolved in a submission, every lookup missed, resolved the texture again and evicted another, each with a deferred release and two notes (ser_note).pool_snap.trans) to take them again in the next submission. The list held 1,024 IDs (POOL_IDS_MAX); past that, each submission built the 33,024-entry table whole.Change. In
nvn_mtl.m, for every game.g_ghostresolved in the lastGHOST_IDLE(16) frames stays, pastNGHOSTif need be. The ring is now a queue that grows;tex_getandtex_putstamp an entry with the frame;ghost_trimevicts only idle entries from the oldest end, and each submission's commit trims a bounded number, so the table is back at 512 once the textures idle.POOL_IDS_MAXis 8,192, the limit the patch already has for all the IDs it takes.Reference entry: More finalized textures in use than their table held, linked from the game note.
Checked
Tests:
make -C dm/gfx/tests GAME=0400486024DEC000 check_t_cpu_tex check_t_pool_notes check_t_upgrade_alias(100, 91 and 13 checks on this branch).t_cpu_texhas a new case: 2,048 finalized serials resolved every frame for 32 frames are all found with no insertion, and the table is back at 512 after they idle. Its hand-over case now lets the entry idle before the entries that evict it.test_docs.py; host build.Games run:
0400486024DEC000v131072, fullscreen play by the player through the same transition.With the higher
POOL_IDS_MAX, the player reported no slowdown after the pause menu.Devices and OS: Apple M4 Pro, macOS 27.2.
Not checked / open.
regress.pyrun: only this profile has a build on this Mac. The change runs for every NVN game: a game that never has more than 512 finalized serials in use within 16 frames sees the same evictions as before; one that does keeps those textures resident for up to 16 frames longer. No iOS run.Docs
docs/reference/nvn-objects.mdentry and the game note, andpython3 dm/tools/docs_symptoms.pywas run