Skip to content

The visibility ring grows on demand to 128 buffers, so a counting submission in flight no longer makes and drops a buffer of its own (#2535) - #2536

Closed
stan69b wants to merge 1 commit into
seedds:mainfrom
stan69b:pr/vis-ring-grow
Closed

stan69b wants to merge 1 commit into
seedds:mainfrom
stan69b:pr/vis-ring-grow

Conversation

@stan69b

@stan69b stan69b commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Fixes #2535.

What changes

VIS_RING goes from 8 (ALLOC_MAX) to 128 in nvn_mtl.m. vis_take already grows the ring lazily and reuses a buffer once its submission's fence passes, so the ring only grows to the number of counting submissions actually in flight. That's at most 8 MiB, all kept resident.

Why it matters: since fd8188c9, a frame's submissions share a command allocator, so more counting submissions are in flight than allocators. With the GPU behind, the ring of 8 was often all in flight. Each counting submission then made a 64 KiB buffer of its own and added it to the residency set, then removed it after the submission. That's two residency commits through the kernel's GPU driver per counting submission.

Measured

iPhone 17 (A19, 8 GB, iOS 27.2.1), Ocarina of Time at the launcher's 30 fps cap with dynamic resolution, from a save. The build was the owner's local build (main 547c5db6 with #1971, #2270 and #2241), with and without this change:

Before After
Kakariko Village, fps 8–12 25–26
Kakariko, frame p50 62–118 ms 37 ms
Kakariko, GPU p50 25–45 ms 20 ms
Kakariko, residency commits per 300 frames 1,700–5,000 51–89
Hyrule Field, fps 10–12 29.5
Hyrule Field, frame p50 69–78 ms 33 ms

The cause was found from a sysdiagnose spindump. The game's main thread was in nvnSyncWait → mtl_wait_at, and the Metal 4 submission threads were in IOGPUFamily. Details are in the issue.

Checked

  • t_wait_event: a new case runs 24 counting submissions in flight, none complete. Each takes a distinct ring buffer, and once they complete the next one reuses one without growing the ring. The case fails on main (2 of its 3 checks).
  • t_render_enable_queue, t_render_counter_overlap, t_counter_clock and test_docs pass.
  • The Mac host builds.
  • Docs: a reference entry in docs/reference/frame-rate.md after the shared-allocator entry, the symptom index, and a line under Ocarina of Time's "Found on this game".

Regression check

Not run yet. On this Mac only Ocarina of Time reaches counting submissions often enough to matter, and a Mac never ran the ring dry (the reference entry for fd8188c9 says the same for an M4 Pro). A scripts/regress.py check on Ocarina of Time will be added here when it has run.

…mission in flight no longer makes and drops a buffer of its own (issue seedds#2535)

Since a frame's submissions share a command allocator (fd8188c), more counting submissions are in flight than the ring of 8 held. With the GPU behind, none was free, and each one made a 64 KiB visibility buffer, added it to the residency set and removed it after the submission: two residency commits through the kernel's GPU driver per counting submission. Ocarina of Time on an iPhone 17 made about 9 a frame in Kakariko Village. The game's main thread waited in nvnSyncWait for 30-60 ms a frame, and it ran at 8-12 fps with 25-45 ms of GPU time.

VIS_RING is 128. vis_take already grows the ring lazily and reuses a buffer once its submission's fence passes, so only as many buffers are made as are in flight: at most 8 MiB, all kept resident. A pass past 128 in flight still takes a buffer of its own, as before.

Measured on the iPhone 17 (iOS 27.2.1, 30 fps cap, dynamic resolution) from the owner's save, on the owner's local build (main 547c5db with PRs seedds#1971, seedds#2270 and seedds#2241):
- Kakariko Village: residency commits per 300 frames 1,700-5,000 → 51-89; 8-12 → 25-26 fps; frame p50 62-118 → 37 ms; GPU 25-45 → 20 ms.
- Hyrule Field: 10-12 → 29.5 fps; frame p50 69-78 → 33 ms.

Checked: t_wait_event, with a new case (24 counting submissions in flight each take a distinct ring buffer, and the next one reuses one once they complete) that fails with a ring of 8; t_render_enable_queue, t_render_counter_overlap, t_counter_clock and test_docs pass; the Mac host builds.
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation game: 0400486024DEC000 A game labels Nov 13, 2026
@stan69b

stan69b commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Regression check

scripts/regress.py check --games 0400486024DEC000 (full mode), baseline 8d44aaf8 (this PR's merge base with main), candidate e2b01ddd (this PR's head). Apple M4, macOS 27.2, Xcode 27.2. Run from a tree that has Ocarina of Time's route (smoke.tsv, which main does not have yet), so the boot run took place.

game status shaders (base → cand) newly failing newly building changed MSL interface (new / known) boot fps p50 frame p99 ms peak MB prepare s new RED
Ocarina of Time 0400486024DEC000 REGRESSION (screenshots only, see below) 42848/42848 → 42848/42848 0 0 0 0 / 0 ok 59.5 → 59.5 17.82 → 17.93 2879 → 2860 3.28 → 3.68 0
  • Prebuild: 33,365 pipelines built and 0 failed on both sides (985.5 s → 981.8 s).
  • First frame: 12.1 s → 12.6 s. Highest frame number: 23,248 → 23,198. The route was reached on both.
  • The only finding is the screenshots at 350 s and 385 s (dHash 18 and 24 bits). The baseline has a single run, so its noise is 0.
    • Both runs show the title screen rendered correctly.
    • The title's background cycles through scenes. The candidate started 0.5 s later, so at the same wall-clock second it shows another scene: baseline forest and ranch, candidate desert and lake.
    • This is a timing artifact of the route's fixed screenshot times, not a picture regression.
  • No change in shaders, frame pacing, memory or red lines.

seedds added a commit that referenced this pull request Nov 13, 2026
…light, so a counting submission no longer makes and drops one of its own

Since a frame's submissions share a command allocator (PR #2090), more counting
submissions are in flight than the ring's 8 (VIS_RING = ALLOC_MAX). With the GPU
behind, each one made a 64 KiB visibility buffer, added it to the residency set
and removed it after the submission: two residency commits through the kernel's
GPU driver each. Ocarina of Time on an iPhone 17 went from 8-12 to 25-30 fps in
Kakariko Village and Hyrule Field with the ring grown (PR #2536).

The ring is sized from the bound mtl_sub_begin already keeps, SUB_FLIGHT_MAX =
ALLOC_MAX * ALLOC_SHARE (512), which POST_RING now shares, so it never runs dry;
it still grows on demand, only to the most counting submissions in flight at
once. vis_in_ring checks this submission's ring buffer in O(1) instead of
scanning the ring.

t_wait_event gets a case with 24 counting submissions in flight.

Fixes #2535

Co-authored-by: Stan Bono <3304418+stan69b@users.noreply.github.com>
@seedds

seedds commented Nov 13, 2026

Copy link
Copy Markdown
Owner

Landed in b515089, with you as co-author. Thanks for the diagnosis, the spindump and the iPhone numbers.

One change on top: the ring is now sized to the most submissions that can be in flight, not 128. mtl_sub_begin already caps that at ALLOC_MAX * ALLOC_SHARE = 512, the same limit the post ring relies on (PR #2469). Both now share SUB_FLIGHT_MAX. With that size the ring can't run dry; 128 could still run dry in a game with more counting submissions in flight. It still grows on demand, so memory stays the same in practice (32 MiB at most). vis_in_ring also became an O(1) check of this submission's ring buffer instead of a scan over the ring.

Your test, the symptom line and the game-note line are unchanged. The reference entry's Fix line was updated to match. t_wait_event, t_render_enable_queue, t_render_counter_overlap, t_counter_clock, t_perf_order and test_docs pass.

A retest on the iPhone 17 with main would confirm the numbers with the larger ring.

@seedds seedds closed this Nov 13, 2026
iDams pushed a commit to iDams/nxnative that referenced this pull request Nov 13, 2026
…light, so a counting submission no longer makes and drops one of its own

Since a frame's submissions share a command allocator (PR seedds#2090), more counting
submissions are in flight than the ring's 8 (VIS_RING = ALLOC_MAX). With the GPU
behind, each one made a 64 KiB visibility buffer, added it to the residency set
and removed it after the submission: two residency commits through the kernel's
GPU driver each. Ocarina of Time on an iPhone 17 went from 8-12 to 25-30 fps in
Kakariko Village and Hyrule Field with the ring grown (PR seedds#2536).

The ring is sized from the bound mtl_sub_begin already keeps, SUB_FLIGHT_MAX =
ALLOC_MAX * ALLOC_SHARE (512), which POST_RING now shares, so it never runs dry;
it still grows on demand, only to the most counting submissions in flight at
once. vis_in_ring checks this submission's ring buffer in O(1) instead of
scanning the ring.

t_wait_event gets a case with 24 counting submissions in flight.

Fixes seedds#2535

Co-authored-by: Stan Bono <3304418+stan69b@users.noreply.github.com>
(cherry picked from commit b515089)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

iPhone: counting submissions make and drop a visibility buffer each, 16 residency commits a frame (Ocarina of Time 8-12 fps)

2 participants