Skip to content

Guest queue scheduler: DM_NVN_CHUNK_MERGE encodes the chunks one pump offers into one command buffer (follow-up to #1245) - #1749

Open
aksmith-git wants to merge 2 commits into
seedds:mainfrom
aksmith-git:pr/chunk-merge-1245
Open

aksmith-git wants to merge 2 commits into
seedds:mainfrom
aksmith-git:pr/chunk-merge-1245

Conversation

@aksmith-git

Copy link
Copy Markdown
Contributor

What and why

Follow-up to #1245 (sn0wst0rm), on its head 5269ecc5: this branch is that PR plus one commit, e364b8c6, so the diff here shows #1245 too until it lands. Opened against main as the template asks; if it is better folded into #1245 itself, the commit cherry-picks onto 5269ecc5 as is.

Symptom. With the guest event queue ordering, Ocarina of Time (Switch 2) on an iPad Pro (M5), Docked 1536x864, runs with the GPU idle between the scheduler's chunks. The game makes about 6 submissions a frame; cut at its event signals, waits and recorded fences they become 37 host submissions, and the GPU drains between a chunk and the next while the host commits it:

iPad Pro (M5), Kokiri Forest, same scene waits ignored (before #1245) #1245 as merged
host submissions per frame 6 37
GPU idle inside the frame, p50 0.00 ms 2.5 ms
frame, p50 20.4 ms 22.9 ms
64-draw segment commits per frame 6.4-6.9 1-3

GPU time per frame is the same in both. The frame listing shows 35 chunks for 6 guest submissions, alternating queue 5 (compute) and queue 1 (graphics) about 12 times, cut by 19 signals and 26 waits that were all satisfied by the scheduler before the next chunk; the DM_NVN_GPUPROF timeline shows 1.6 ms of idle before the 760-draw colour pass (the segment counter restarts per chunk, so the pass is committed whole where it used to be fed in 64-draw segments) and about 60 smaller gaps at the other chunk boundaries.

Cause. Every chunk qe_pump offers is a host submission of its own: a command buffer, an allocator lease, a visibility buffer, a completion record, a commit and, with the encode thread, a rec_drain(). It is not a CPU wait on the previous chunk's fence (qe_read only picks the projected value over the mapped word, and sync waits are satisfied by nvn_qs_scheduled); it is the granularity.

Change. DM_NVN_CHUNK_MERGE=1 (off by default; ab alternates it with DM_AB): qe_pump keeps a chunk's command buffer open and encodes the chunks it offers next into it. mtl_chunk_end lands the chunk's passes, clears, deferred draws, uniform updates and blit encoder (what mtl_sub_commit does before it commits), the queue is sealed as after a commit, mtl_chunk_begin sets the per-chunk state of sub_begin without a new buffer, allocator, visibility buffer, completion record or segment clock. The buffer commits when the pump has no ready step left (every pump ends committed), before a present, and in submission timestamp mode when the next chunk belongs to another logical submission (post_run takes one clock per buffer; in segment mode chunks of any submission share one). The scheduler records the open buffer's fence as mtl_pending_fence() = g_fence + 1 (the convention OP_FENCE_SYNC and the PASSGPU ring use) for the chunk and for the signal's projection; every CPU wait on a fence runs its pump first under the submission lock, so no thread can wait on a fence not yet committed. Ordering is unchanged: the chunks go onto the one Metal queue in the order the scheduler offered them, and a chunk whose wait is not satisfied still waits for its producer's offer. What changes is that a satisfied wait, a signal or a recorded fence costs no commit, allocator, visibility buffer, completion record or drain.

The [nvn] timing line gains scheduler chunks per frame N, M of them merged into an open command buffer. Reference entry: docs/reference/nvn-objects.md, "Scheduler chunks committed one by one"; the switch in docs/env_switches.tsv.

Checked

Docs

  • The finding is in docs/reference/nvn-objects.md, and python3 dm/tools/docs_symptoms.py was run (the symptom index)
  • No game files, keys or data extracted from a game in the change or the attachments

sn0wst0rm and others added 2 commits November 4, 2026 21:14
Event and sync waits were ignored, allowing consumers to read stale input when another guest queue had not yet submitted its producer. This implements ready-prefix scheduling: a blocked queue yields so its producer can be submitted later, including by the same CPU thread. CPU-visible event values are published after GPU completion.

Preserve render targets, conditional rendering, visibility queries and submission-mode timestamps across event boundaries. Reuse command allocators across chunks without resetting GPU-owned data, while keeping the allocator pool bounded. Wait-only submissions retain the producer's completion fence.

Validation covers forward and bidirectional dependencies, CPU signals, queue ordering, resource retirement, allocator pressure, both timestamp modes and capture compatibility. Scheduler tests also pass with AddressSanitizer and UndefinedBehaviorSanitizer; GPU tests pass with Metal validation enabled.

Front-ends: break: rebuild host and replay tools together for the updated capture layout. Older captures are rejected; raw replay of event/sync waits remains unsupported.
… offers into one command buffer

With the event queue ordering a guest submission is cut into chunks at its event waits, signals and recorded fences, and each chunk was a host
submission of its own: a command buffer, an allocator lease, a visibility buffer, a completion record, a commit and, with the encode thread, a
drain. Ocarina of Time (Switch 2) on an iPad Pro (M5), Docked 1536x864, went from 6 to 37 submissions a frame, with the GPU draining between a
chunk and the next while the host committed it: 2.5 ms (p50) of GPU idle inside a 22.9 ms frame against none in a 20.4 ms frame with the waits
ignored, at the same GPU time; the 64-draw segment commits fell from 6.4-6.9 to 1-3 a frame, so the 760-draw colour pass was committed whole.

DM_NVN_CHUNK_MERGE=1 (off by default; ab with DM_AB): qe_pump keeps a chunk's command buffer open and encodes the chunks it offers next into it
(mtl_chunk_end / mtl_chunk_begin: the chunk's own landing and per-chunk state, without a commit or a new buffer, allocator, visibility buffer or
completion record). It commits when the pump has no ready step left, before a present, and in submission timestamp mode when the next chunk
belongs to another logical submission (one clock per buffer). The open buffer's fence is mtl_pending_fence() = g_fence + 1 for the chunk and for
the signal's projection; every CPU wait on a fence runs its pump first under the submission lock, so no thread waits on a fence not yet committed.
The order on the Metal queue is the scheduler's as before: a chunk whose wait is not satisfied still waits for its producer's offer.

The [nvn] timing line counts the scheduler's chunks per frame and how many were merged. check_t_nvn runs the event tests with a submission per
chunk and again with DM_NVN_CHUNK_MERGE=1. Reference entry in docs/reference/nvn-objects.md, the switch in docs/env_switches.tsv.
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation game: 0400C3F00006E000 A game profile labels Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400C3F00006E000 A game profile nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants