Repository navigation
perf(pipelines): take compute pipelines from the pipeline archives instead of compiling them at the first dispatch - #1522
Conversation
…stead of compiling them at the first dispatch A compute shader's pipeline was made with newComputePipelineStateWithFunction at its first dispatch, on the submit thread: no stand-in exists for a dispatch, as the hybrid pipelines are for draws, and the pipeline archives held render bases only, so the Metal shader cache was all that stood between a dispatch and a cold compile (20-40 ms warm, 100-1,200 ms cold, a title movie network kernel 58-83 s). Ocarina of Time's effects bring about 20 new compute shaders with each area: four frames of 2.2-4.4 s in a 25-minute session, and a cold-cache route to Kokiri Forest spent 195 s of its submit thread in pipelines, 155 s of it in the title's twelve kernels. cps_make takes a compute pipeline from the archives first (the bundled one, then this Mac's files; about 0.3 ms), else compiles it through the flush period's compiler so the flush writes it into the archive, else the device as before; the prewarm's C items, the dispatch and the prefetch go through it. pso_offline.py puts every compute shader of the shader map into the bundled archive (--compute, the profile's map with --game; metal-tt's compute_pipeline_descriptors), so a packed app never compiles a compute pipeline at play: 4,117 pipelines, 0.5 GB, 8 minutes on an M3 Pro. mtl_prog_prefetch also builds a compute pipeline when the game makes the program, through the same path, and a dispatch that comes first takes a queued build over instead of waiting behind the hundreds a scene load queues (DM_NVN_PREFETCH_CPS=0 turns that part off). Render pipeline events over 100 ms on the submit thread are named in the log.
github-actions
Bot
added
nvn
# Conflicts: # dm/gfx/nvn/nvn_mtl.m
…eedds#1522) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Aax1HnchwCweUDRNWjhHx9
…s well as render bases The batched pipeline archive build keys every entry of the list, and seedds#1522 appends compute kernels to it; base_key unpacked four values and the first full 0.13.0 Ocarina build stopped with 'expected 4, got 2', leaving the app without a pipeline archive. A kernel's key is tagged so it never equals a render base's. Tests cover the keys and a batched build with a kernel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016qteK3qWJVi2jG3Gz4i1eY
|
Independent corroboration of the first-dispatch compute stalls on Ocarina of Time v131072, macOS 27.0, Apple M5 Ultra. Our downstream build was based on During the approach to the Great Deku Tree, the log recorded frames of 3.461 s, 1.865 s, 1.820 s and 1.731 s, with 3.425 s, 1.842 s, 1.805 s and 1.717 s respectively charged to pipeline acquisition. At frame 22500, aggregate pipeline time was 5431.7 ms while the named render-wait reasons accounted for only 226.1 ms. The startup manifest warmed 196 compute pipelines and grew to 246 C rows during that run. This supports the compute-path diagnosis in this PR. We independently added program-time compute prefetch with a content-keyed shared compile job and queued-demand takeover. The user reported substantially steadier play afterward. This tested a separate downstream implementation. The now-warm cache also contributes to the user result; this PR itself remains untested here. Our synthetic Metal checks covered one compile across concurrent same-key requests, a demanded kernel taking over work on a suspended background queue, an unrelated kernel completing while another compile was gated, warmup and prefetch sharing a content-keyed result, failure caching, and actual GPU output. These cases may be useful additions to this PR's tests, particularly queued takeover before the shader-library worker publishes its result. |
|
Integration heads-up: when combining this PR ( In my local combined tree I used |
…y compute shader of the map, and captured into the device's archive when the bundle lacks one (PR #1307) pso_offline.py gives the bundled archive a compute pipeline for each compute shader (the prewarm list's C rows, then the shader map's compute rows; functions with constants left out). The host's cps_new looks a compute pipeline up in the bundle, then in the device's archive files. On a miss it builds through the MTLDevice for the dispatch, as before, and compiles it again into the flush period on a background queue, so the next start finds it. plan_compute now reads the libraries through romfs.Path, as plan does. With pathlib.Path, a one-file library (b-<chain>.img, what dmsh_build.py writes) left out every compute shader as having no reflection, and the archive held none (test_pipelines_from_a_library_image). Ocarina of Time (chain f9f01e4f9994, M6): 4,125 compute pipelines in a 480 MB archive. With it, all 103 of a route's compute pipelines came from the bundle and none were built. Without it, the first start wrote 88 into the device's archive and the second took 88 from it. PR #1522 measured the cold-cache gain: 195 s of submit-thread pipelines to 0.9 s. Return of the Obra Dinn quick regress: PASS (its route dispatches no compute). Co-authored-by: Pedro Nascimento <91118+lunks@users.noreply.github.com>
|
Thanks for this. Your diagnosis and cold-cache measurements decided it. Compute pipelines in the archives landed through #1307's version (30fb213), for two reasons:
Your Ocarina numbers (195 s to 0.9 s on a cold route) are cited in the reference entry and the game note. The prefetch at program creation was left out: it had no measurement of its own beyond the archive. On Ocarina with the bundled archive, a route's 103 compute pipelines all came from it and none were built, so it has nothing left to cover there. If kernels with function constants still stall (they can't go in the archive), a prefetch limited to those, with its own before/after numbers, would be welcome as a new PR. |
…s first write stays resident while any of them holds it: releasing one took it out of the residency set (GPU page faults, issue seedds#1326) The serials of one storage share one packaged texture (g_pkg_share). At the GPU's first write over that storage, cpu_unc_drop gives each of them the package's texture itself (tex_put), so they all hold one texture that was in no shared set. The first holder released (tex_put's ghost-ring eviction, serial_put or serial_del, through obj_drop or defer_obj_release), or the package's own release (NvPkg dealloc's res_remove), took it out of the residency set while the other serials still drew with it, and a command buffer reading it faulted. pkg_hand_over enters the handed-over texture in g_alias_set and in g_handed, a list by the package's sharing key, so releasing one holder only drops a reference (alias_has). The package's release and its re-upload's spare leave a handed-over texture resident, mtl_alias_drop_pool retires it with its pool, and tex_upgrade moves its entry. It does not enter g_alias, so a new texture object over the same storage still gets a texture of its own. DM_UNC_SHARE=0 gives back the old hand-over. Ocarina of Time v131072, Mac mini M4 16 GB, macOS 27.2, on main b2d46b8 with seedds#1590, seedds#1557, seedds#1556, seedds#1575, seedds#1522, seedds#1543 and seedds#1390: page faults at 44, 29, 24 and 15.5 minutes, with the missing textures named by DM_RES_CHECK and a diagnostic of where each left the residency set, before two of them, as 260x260 terrain tiles handed over this way and released by the ghost-ring eviction. One binary: with the change, 107 minutes and no fault (86 hand-overs); with DM_UNC_SHARE=0, a fault at 37 minutes with the same kind of tiles named. t_cpu_tex --metal handover_shared: 97 checks, 0 failed; with DM_UNC_SHARE=0, 2 failed. The other GPU suites are unchanged. (cherry picked from commit 8cbc330)
…y compute shader of the map, and captured into the device's archive when the bundle lacks one (PR seedds#1307) pso_offline.py gives the bundled archive a compute pipeline for each compute shader (the prewarm list's C rows, then the shader map's compute rows; functions with constants left out). The host's cps_new looks a compute pipeline up in the bundle, then in the device's archive files. On a miss it builds through the MTLDevice for the dispatch, as before, and compiles it again into the flush period on a background queue, so the next start finds it. plan_compute now reads the libraries through romfs.Path, as plan does. With pathlib.Path, a one-file library (b-<chain>.img, what dmsh_build.py writes) left out every compute shader as having no reflection, and the archive held none (test_pipelines_from_a_library_image). Ocarina of Time (chain f9f01e4f9994, M6): 4,125 compute pipelines in a 480 MB archive. With it, all 103 of a route's compute pipelines came from the bundle and none were built. Without it, the first start wrote 88 into the device's archive and the second took 88 from it. PR seedds#1522 measured the cold-cache gain: 195 s of submit-thread pipelines to 0.9 s. Return of the Obra Dinn quick regress: PASS (its route dispatches no compute). Co-authored-by: Pedro Nascimento <91118+lunks@users.noreply.github.com>
What
cps_make(nvn_mtl.m): a compute pipeline is taken from the pipeline archives first, the bundled one and this Mac's files (MTL4Archive newComputePipelineStateWithDescriptor, about 0.3 ms), else compiled through the flush period'sMTL4Compilerso the flush writes it into the per-game archive, else through theMTLDeviceas before. The prewarm'sCitems, the dispatch and the prefetch go through it. The pipeline statistics line countscompute pipelines from the archives N / ms, compiled N / ms.pso_offline.py --compute <shader_map.tsv>(the profile's map with--game): every compute shader of the map goes into the bundled archive as acompute_pipeline_descriptorsentry of the metal-tt script, whole (there is no base to specialize). The archive key covers them, so an existing archive is rebuilt once; the render batches are reused.mtl_prog_prefetchalso builds a compute pipeline when the game makes the program (throughcps_make), and a dispatch that comes first takes a queued build over instead of waiting behind the hundreds a scene load queues (DM_NVN_PREFETCH_CPS=0turns that part off). It only helps when the program is made well before its first dispatch; the archive is the fix.Why
A compute shader's pipeline was made with
newComputePipelineStateWithFunctionat its first dispatch, on the submit thread. No stand-in exists for a dispatch, as the hybrid pipelines are for draws, and the archives held render bases only, so the Metal shader cache was all that stood between a dispatch and a cold compile: 20-40 ms warm, 100-1,200 ms cold for an effect kernel, 58-83 s for a title movie network kernel. Ocarina of Time's effects bring about 20 new compute shaders with each area.Measured
The Legend of Zelda: Ocarina of Time on an M3 Pro (18 GB). A 25-minute play session on main had four frames of 2.2-4.4 s, all on entering a new area, with
on the submit thread pipelines 6,932 msin one 300-frame window while only 20 render draws had waited (425 ms).The same scripted route (title, save load, Kokiri Forest) with the app's Metal shader cache set aside and an empty
DM_PSO_USERlist, so every pipeline is met cold:750 compute pipelines came from the archive in 263 ms in all. The offline archive grew from 4.44 to 4.97 GB; its compute batches took 8 minutes on the M3 Pro (the render batches were reused).
Checks
t_pso_archive102,t_pso_bundle58,t_pso_warm99 checks pass (make -C dm/gfx/tests GAME=0400486024DEC000 t_pso_archive t_pso_bundle t_pso_warm, run directly: thecheck_targets fail on this Mac on a path with a space, unrelated).python3 dm/tools/tests/test_pso_offline.py: 39 tests, with a new one for the compute kernels (map reading, what is left out, the script).test_docs.pyok.compute_pipeline_descriptorsand the archive lookup were checked first with a standalone program on one kernel: 0.32 ms from the archive, 31 ms through the device.scripts/regress.py check --games: not run, this Mac has only Ocarina of Time. The change reaches every NVN game on the Mac (every compute dispatch); a run on the usual five would be welcome.Docs
docs/reference/frame-rate.md: "Compute pipelines built on the submit thread at their first dispatch";docs/development.md: the archive section; the game note: "Found on this game" and the frame-rate measurements (also the display-paced present numbers, left out of the profile until the game's speed under it is checked by eye); symptom index regenerated.