Skip to content

nvn: pre-pass indirect draw conversion and indirect dispatch/mesh clamp - #1729

Open
ksolenk38ze wants to merge 1 commit into
seedds:mainfrom
ksolenk38ze:pr/nvn-prepass-indirect-and-mesh-pad
Open

ksolenk38ze wants to merge 1 commit into
seedds:mainfrom
ksolenk38ze:pr/nvn-prepass-indirect-and-mesh-pad

Conversation

@ksolenk38ze

@ksolenk38ze ksolenk38ze commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

What and why

When an indirect draw (DrawArraysIndirect, DrawElementsIndirect, MultiDrawArraysIndirectCount, MultiDrawElementsIndirectCount) required vertex conversion (vconv), index conversion (8-bit or primitive-restart index rewrite), or geometry shader emulation inside an already-open render pass, draw_() ended the current render pass (rp_end()), dispatched the compute conversion, and reopened the pass with MTLLoadActionLoad, splitting the pass and losing tile memory state. In addition, indirect compute dispatches (DispatchComputeIndirect) and mesh draws (DrawMeshTasksIndirect) could read uninitialized or out-of-bounds workgroup dimensions when the guest indirect buffer had not yet been populated, and background task/mesh pipeline compilation in mtk_pso waited behind pp_bg_queue()'s serial utility backlog.

This change:

  • Scans the submission command list ahead of each render pass and runs compute indirect draw/index conversions (ixc_run, mdx_run) before rp_begin(), keeping the render pass intact.
  • Clamps indirect compute dispatch grids (ixc_clamp) and mesh task counts (mdx_clamp) on the GPU before indirect dispatch/draw calls (DM_NVN_IND_CLAMP).
  • Dispatches background task/mesh pipeline builds in mtk_pso on QOS_CLASS_USER_INITIATED so mesh draws without a pulling stand-in do not stall behind utility-priority background compiles.
  • Documents the mechanism in docs/reference/rendering.md (### Pre-pass indirect draw conversion and indirect dispatch clamping) and updates docs/agents/symptoms.md.

Related

Checked

  • Tests: make -C dm/gfx/tests GAME=0400486024DEC000 run, python3 dm/tools/docs_symptoms.py
  • Games run, with title ID and version: The Legend of Zelda: Ocarina of Time (0400486024DEC000, v131072, SDK 22.2.8), played through Kokiri Forest, Lost Woods, Hyrule Field, and Zora's Domain at 60 fps with foliage and indirect draws intact.
  • Devices and OS: Apple Silicon Mac, macOS 26.2

Docs

  • The finding is in its game note or its docs/reference/<subsystem>.md entry, and python3 dm/tools/docs_symptoms.py was run (the symptom index)
  • No game files, keys or data extracted from a game in the change or the attachments

@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation labels Nov 7, 2026
Batch indirect-count multi-draw masking (mdx_run) before opening a render pass
so indirect draws inside the pass do not split the active
MTL4RenderCommandEncoder mid-pass. Check OP_UPDATE_UB byte ranges during the
lookahead scan in ixc_run, wide_run, and mdx_run so inline uniform updates to
unrelated buffer regions do not prematurely stop batching, and flush deferred
inline buffer writes (ubd_land_compute) before opening the conversion compute
pass.

Also clamp GPU-written indirect compute dispatch counts (DM_NVN_DISPATCH_CLAMP,
65,535 per dimension and 4,194,304 total threadgroups) and DrawMeshTasksIndirect
workgroup counts so uninitialized or race-written indirect records cannot hang
the GPU.

Verified in The Legend of Zelda: Ocarina of Time (0400486024DEC000) in Kakariko
Village and Hyrule Field on Apple M-series (macOS).
@ksolenk38ze
ksolenk38ze force-pushed the pr/nvn-prepass-indirect-and-mesh-pad branch from 1d059e4 to c5fa499 Compare November 7, 2026 23:37
Aznracer pushed a commit to Aznracer/Mighty-Green-Kokiri-Kid that referenced this pull request Nov 8, 2026
…h reads them (each dimension to 65535, at most 2^22 threadgroups): a count from bad data runs on the GPU for ever (from upstream PR seedds#1729, that part only)
@aksmith-git

Copy link
Copy Markdown
Contributor

Data on Ocarina of Time's (0400486024DEC000 v131072) indirect task/mesh draws, since this PR touches their conversion and clamps the copied count. Kokiri Forest, Docked 1536x864, main colour pass (RG11B10 + D32F, ~800 draws, EQUAL after the depth prepass), from DM_NVN_GPUPROF with a per-pass vertex-stage stamp:

draws GPU note
rest of the colour pass 759-761 3.1-3.5 ms (iPad Pro M5) regular draws + 12 direct mesh draws
11 indirect task/mesh grass draws (+26 regular) 37 5.4-5.7 ms (iPad Pro M5), 4.7-4.9 ms (M4 Pro Mac) 6 fragment shaders; vertex phase 3.3-5.2 ms, fragment 0.45
  • Nearly the same absolute time on the M4 Pro and the M5, and it does not scale with the render size: a per-threadgroup cost of the heavy mesh bodies (4,700-8,900 lines, 16-22 words a vertex, a 2,100-3,973-word threadgroup stage array zeroed at every launch), about 20x the vertex-phase time of the light grass variants in the prepass at the same threadgroup counts.
  • Task threadgroup counts per indirect draw (the record as the CPU sees it at replay): 0-730, 700-1,300 a frame in total; each task threadgroup launches at most 32 mesh threadgroups (popcount of a 32-lane cull). No out-of-range count, so the 65,535 clamp here does not change this game's grass.
  • Encoding the 11 draws as direct draws with the record's counts (no mdx_run dispatch, no pass split) costs the same, so the indirect fetch and the split are not it.

Two questions: have you seen this pass on your Macs, and does the pre-pass conversion here change the grass's task counts or timing in your runs? I can run a build of this PR on the iPad if a particular measurement would help.

@ksolenk38ze

ksolenk38ze commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks @aksmith-git for the DM_NVN_GPUPROF breakdown! To answer your two questions:

  1. Yes, we see the same indirect task/mesh grass draws in Kokiri Forest and Hyrule Field, and as you noted their task threadgroup counts are well within bounds so the 65,535 clamp is a safety bound rather than changing valid grass counts.
  2. What mdx_run ahead of rp_begin_ex changes is render pass splitting: when DrawMeshTasksIndirect is encountered inside an already-open MTL4RenderCommandEncoder and its indirect buffer has not yet been converted in the current epoch (g_mdx_ep != g_mdx_epoch || g_mdx_sub != g_sub_serial), running mdx_run on the spot had to call rp_end() to open a compute encoder and then reopen the colour pass (MTLLoadActionLoad / MTLStoreActionStore). Running mdx_run(NULL, NULL) and ixc_run(NULL, NULL) before rp_begin_ex (and across OP_SIGNAL_EVENT barriers) keeps the colour pass in a single encoder without changing the grass task counts.

@FifteenPen

Copy link
Copy Markdown
Contributor

A data point on the indirect dispatch clamp's cost (the same dpc_clamp / DM_NVN_DISPATCH_CLAMP is in #1777), from our tree with #1729's clamp merged. Walking through Hyrule Castle Town in Ocarina of Time (47 s walks, M4 Pro): 6–9 long frames with the clamp off, against 13–24 with it on. No indirect dispatch of the game hung the GPU with it off. We turned it off for Ocarina with a profile key; it may be worth measuring the clamp's cost before it is on for every game.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants