Pipeline archive compaction by size tier, so the start after play recaptures what play added, not the whole archive (Ocarina of Time 713 s → 24 s) by bettterlabs · Pull Request #1880 · seedds/nxnative

bettterlabs · 2026-11-08T23:46:52Z

What and why

After a play session, the next start can spend minutes in the prepare step recompiling the whole archive. Here, 9 files of play beside a compaction's 5 made the next start capture 8,543 pipelines again: 713 s before the first frame on an M4.

Cause. Play writes a run file per timer flush (13 to 273 in a session here), and a compaction leaves a file per ARCH_PERIOD_MAX, so a few files of play bring the count over PSO_ARCH_COMPACT. The compaction then takes every run file, and its prewarm captures every base again from the original shader functions (the #342 workaround), a full compile each.

Change:

File names: a run file's name holds its pipeline count (<seq>-<count>.mtl4a).
Which files are compacted (arch_compact_pick): files are grouped by size tier, each PSO_ARCH_COMPACT times the one below (under 8, 64, 512, then up to a full period). When a tier holds more than PSO_ARCH_COMPACT files, its files and those of every smaller tier are compacted, and only those are removed. A full period's file is never compacted. A base, compute or task / mesh pipeline found in a file the compaction doesn't take is used as it is.
Older names: a file named before (or with a count that isn't all digits) counts as the largest tier below a full period, so an earlier compaction's full files stay.

Cost, from the rule alone (a test runs 200 sessions of 30 files of 6 pipelines): a typical start recaptures about what the last session added (506 pipelines on average), against the whole archive before. Now and then the tiers above fill and merge: 16,380 pipelines at sessions 91 and 191 in that run. A full file is never removed, so one whose binaries a system update makes miss stays open (about 0.06 ms per lookup that misses in it); that's derived from the code, not seen.

Related: esdras-kyo's comment on #1747 and #1840 (found independently) compact once the run files exceed the count the last compaction left (by 8, or 2x). After a session of 9 small files beside a compaction's 5, that still recompiles the whole archive (worked out from their rules, not run); under tiers those 9 files start no compaction at all.

One of two independent PRs; the other, #1879, stops the prewarm's timer flushes, so a cold start doesn't fragment the archive in the first place.

Checked

Tests (M4, macOS 27.2, on 270b6fd):
- all 124 gfx tests pass.
- t_pso_archive: 212 checks (162 on main). test_pso_archive.py: 3 tests (1 on main).
- Each part, reverted alone, fails its checks, including a compactor that picks nothing.

Game run: Ocarina of Time 0400486024DEC000 (v131072). The host was main bd979ce + eed7c42, with and without this change; this Mac's shader library is that chain, and the archive code there is the same as main's. Starts used a copy of the device archive with a 30,838-line list, stopped at the title:

	A compaction's 5 files + 9 files of play (copies, with their real counts)	Next start
before	713.42 s: the 262 new bases compiled, 8,543 captured again
with the change	23.58 s: the 262 compiled, none captured	2.35 s

After a real 17-minute session (29 files), the next start took 24.9 s, then 3.0 s.

Device: Mac mini M4, 16 GB, macOS 27.2.
Not checked: a build of current main in a game; iOS; lint-c; a large tier merge in a game; regress.py check (other games named by impact: Breath of the Wild, Pokémon Scarlet, Tears of the Kingdom).

Docs

The finding is in its game note or its docs/reference/<subsystem>.md entry, and python3 dm/tools/docs_symptoms.py was run (the symptom index)
No game files, keys or data extracted from a game in the change or the attachments

…eline count in their names, never a full period's file: on Ocarina of Time 9 files of play beside a compaction's 5 made the next start capture 8,543 archived pipelines again, 713 s before the first frame on an M4; now that start takes 23.6 s, compiling only the 262 bases new to the list Play writes a run file per timer flush (13 to 273 in a session here) and a compaction leaves a file per ARCH_PERIOD_MAX, so a few files of play brought the count over PSO_ARCH_COMPACT, and the compaction took every run file: its prewarm captured every base again from the original functions (the seedds#342 workaround), a full compile each. Now a run file's name holds its pipeline count (<seq>-<count>.mtl4a); arch_compact_pick takes the files of the highest tier (each PSO_ARCH_COMPACT times the one below: under 8, 64, 512, then up to a full period) that holds more than PSO_ARCH_COMPACT files, with every smaller tier, never a full period's file; arch_compacting leaves the bases of the other files as they are; arch_compact_finish removes only the files it took. A file named before, or with a count that is not all digits, counts as the largest tier below a full period, so an earlier compaction's full files stay and a pile of more than 8 such files is compacted once. Without a pick every run file is taken, as before. t_pso_archive (135 -> 183 checks): the tier choice from names and its boundaries, a merge leaving a ninth file in the tier above, a compaction of two full files and nine of one base, a compute pipeline in a file left out, and 200 sessions by the rule alone; test_pso_archive.py, fresh processes: the tier compaction and the start after it, and nine old names compacted once. On bd979ce with eed7c42 (this Mac's library chain), a copy of the device archive: 713.42 s before, 23.58 s with it, 2.35 s the start after. Reference entry, game note, development.md, symptom index.

…_compacting read as "is a compaction running" beside arch_capturing), and the fresh-process test of files named before the count is --oldname-* ("legacy" already names the MTLDevice path in this file)

…mesh pipelines bf420fd keeps in the pipeline archive are taken by the size tiers too (nvn_geometry.inc: a mesh pipeline found in a file the compaction does not take is not captured again), arch_harvest's rename, and the tier tests wait for an earlier test's queued archive work before picking

FifteenPen · 2026-11-09T04:01:29Z

One piece neither this PR nor #1840 has: a write that a stopped or killed run left behind stays in the archive folder for good. Metal's own serialization file (<UUID>.metallib) or the host's .mtl4a.tmp; one was 286 MB for us. On our tree, the next start removes them before opening the archives (a ^[0-9A-F]{8}-…-[0-9A-F]{12}\.metallib$ match or a .mtl4a.tmp suffix, while nothing writes the folder yet). It could sit with either PR's compaction.

…eline count in their names, never a full period's file: on Ocarina of Time 9 files of play beside a compaction's 5 made the next start capture 8,543 archived pipelines again, 713 s before the first frame on an M4; now that start takes 23.6 s, compiling only the 262 bases new to the list (PR #1880) Play writes a run file per timer flush (13 to 273 in a session) and a compaction leaves a file per ARCH_PERIOD_MAX, so a few files of play brought the count over PSO_ARCH_COMPACT, and the compaction took every run file: its prewarm captured every base again from the original functions (the #342 workaround), a full compile each. Now a run file's name holds its pipeline count (<seq>-<count>.mtl4a); arch_compact_pick takes the files of the highest tier (each PSO_ARCH_COMPACT times the one below: under 8, 64, 512, then up to a full period) that holds more than PSO_ARCH_COMPACT files, with every smaller tier, never a full period's file; a base, compute or task / mesh pipeline found in a file the compaction does not take is used as it is (arch_compact_takes); arch_compact_finish removes only the files it took. A file named before, or with a count that is not all digits, counts as the largest tier below a full period, so an earlier compaction's full files stay and a pile of more than 8 such files is compacted once. Checked on 7479adf: t_pso_archive 212 checks, test_pso_archive.py 3 tests; the other gfx tests pass except t_render_counter_overlap and t_render_enable_queue, which fail the same on main; lint-c no new finding; regress.py check OCTOPATH TRAVELER II PASS (a first run on a loaded Mac reported archive writes of 105-177 s, to p files this change does not touch; the rerun passed). Co-authored-by: bettterlabs <279109819+bettterlabs@users.noreply.github.com>

seedds · 2026-11-09T05:11:06Z

Landed as 4c3a03a on main, with you credited as co-author. The only change is a rebase over main: #1879 (4a81561) landed first, so the archive write now uses its arch_mark(1) beside your new file name. Checked on the rebased tree: t_pso_archive passes 222 checks (yours and #1879's together) and test_pso_archive.py 6 tests. Before #1879 landed, the gfx suite passed apart from two render-enable / counter tests that fail the same on main, lint-c found nothing new, and regress.py check on OCTOPATH TRAVELER II passed. A compaction in a real game on current main hasn't been run here. Thanks for the clear diagnosis and the 200-session test.

…he one file that holds the pipeline and a miss asks none, whatever the number of files: a lookup asked every file in turn (about 0.06 ms each one it missed in), which is why a few files had to make a start a compaction that compiles every base again under the #342 workaround (Ocarina of Time on an M4: 713 s after play, PR #1880; 762 s after a cold start's 80 files, issue #1747) <file>.keys (arch_write_file) holds the keys of the pipelines in the file and whether every one has a key. A key digests what the pipeline's Metal 4 descriptor sets (arch_key): its functions by library (named by its file under the shader folder, lib_from_file / lib_tag), entry name and constant values (by value: Metal has no getter, and the descriptor copies the set it is given, but the set's class describes and hashes it by value, the same in every process and in any order), the vertex layout, the colour attachments and every other field. pso4_init reads every index into one table (newest file first) and opens the files in parallel; arch_lookup, now used by the render bases, compute and task / mesh pipelines and the prebuild, asks the file the index names alone. A pipeline with no key (a library with no name) and a file with no index (written before it, or renamed) go through the files without a complete index, as before. Which files a compaction takes stays PR #1880's size-tier rule: arch_compact_takes finds a file among every run file (g_arch_run, indexed or not), and a compacted file's index is removed with it. Checked: t_pso_archive 230 checks (with the first file's index and key; the same pipeline the same key, another layout another, constant values by value in any order, none for a library with no name); test_pso_archive.py 6 tests in fresh processes (a next start asks one file per base found and none per miss, after a cold, cut-short and quit start; a compaction leaves each file with its index and removes the compacted files' indexes; renamed files lose theirs and are asked in turn). The Metal graphics tests, host tests, Mac host build and lint-c pass. ŌKAMI HD (package split again for keyed records, worktree host before the merge with PR #1880, fresh archive): a second start asking for the same pipelines took 41 of 41 from the index; some second starts compiled 2-8 again because ŌKAMI HD's prewarm paired the same pixel shaders with another vertex library than the first start (open, not from the index). regress.py check on ŌKAMI HD passed the chain; no local game has a smoke route, so it booted none; Minecraft Dungeons II and Ocarina of Time need their packages split again.

seedds · 2026-11-09T06:20:36Z

Follow-up in 70b9a68: each archive file now has an index beside it (<file>.keys), so a lookup asks only the file that holds the pipeline and a miss asks none, whatever the number of files. Your size-tier rule stays as the only compaction rule. With lookups no longer costing per file, the tiers now just keep down the number of files to open at start and the disk use. arch_compact_takes now matches a file against every run file, indexed or not, and a compacted file's index is removed with it.

…h replaces seedds#1840's compaction limit

… size, which seedds#1880's size tiers replace

…eline count in their names, never a full period's file: on Ocarina of Time 9 files of play beside a compaction's 5 made the next start capture 8,543 archived pipelines again, 713 s before the first frame on an M4; now that start takes 23.6 s, compiling only the 262 bases new to the list (PR seedds#1880) Play writes a run file per timer flush (13 to 273 in a session) and a compaction leaves a file per ARCH_PERIOD_MAX, so a few files of play brought the count over PSO_ARCH_COMPACT, and the compaction took every run file: its prewarm captured every base again from the original functions (the seedds#342 workaround), a full compile each. Now a run file's name holds its pipeline count (<seq>-<count>.mtl4a); arch_compact_pick takes the files of the highest tier (each PSO_ARCH_COMPACT times the one below: under 8, 64, 512, then up to a full period) that holds more than PSO_ARCH_COMPACT files, with every smaller tier, never a full period's file; a base, compute or task / mesh pipeline found in a file the compaction does not take is used as it is (arch_compact_takes); arch_compact_finish removes only the files it took. A file named before, or with a count that is not all digits, counts as the largest tier below a full period, so an earlier compaction's full files stay and a pile of more than 8 such files is compacted once. Checked on 242e0de: t_pso_archive 212 checks, test_pso_archive.py 3 tests; the other gfx tests pass except t_render_counter_overlap and t_render_enable_queue, which fail the same on main; lint-c no new finding; regress.py check OCTOPATH TRAVELER II PASS (a first run on a loaded Mac reported archive writes of 105-177 s, to p files this change does not touch; the rerun passed). Co-authored-by: bettterlabs <279109819+bettterlabs@users.noreply.github.com>

…he one file that holds the pipeline and a miss asks none, whatever the number of files: a lookup asked every file in turn (about 0.06 ms each one it missed in), which is why a few files had to make a start a compaction that compiles every base again under the seedds#342 workaround (Ocarina of Time on an M4: 713 s after play, PR seedds#1880; 762 s after a cold start's 80 files, issue seedds#1747) <file>.keys (arch_write_file) holds the keys of the pipelines in the file and whether every one has a key. A key digests what the pipeline's Metal 4 descriptor sets (arch_key): its functions by library (named by its file under the shader folder, lib_from_file / lib_tag), entry name and constant values (by value: Metal has no getter, and the descriptor copies the set it is given, but the set's class describes and hashes it by value, the same in every process and in any order), the vertex layout, the colour attachments and every other field. pso4_init reads every index into one table (newest file first) and opens the files in parallel; arch_lookup, now used by the render bases, compute and task / mesh pipelines and the prebuild, asks the file the index names alone. A pipeline with no key (a library with no name) and a file with no index (written before it, or renamed) go through the files without a complete index, as before. Which files a compaction takes stays PR seedds#1880's size-tier rule: arch_compact_takes finds a file among every run file (g_arch_run, indexed or not), and a compacted file's index is removed with it. Checked: t_pso_archive 230 checks (with the first file's index and key; the same pipeline the same key, another layout another, constant values by value in any order, none for a library with no name); test_pso_archive.py 6 tests in fresh processes (a next start asks one file per base found and none per miss, after a cold, cut-short and quit start; a compaction leaves each file with its index and removes the compacted files' indexes; renamed files lose theirs and are asked in turn). The Metal graphics tests, host tests, Mac host build and lint-c pass. ŌKAMI HD (package split again for keyed records, worktree host before the merge with PR seedds#1880, fresh archive): a second start asking for the same pipelines took 41 of 41 from the index; some second starts compiled 2-8 again because ŌKAMI HD's prewarm paired the same pixel shaders with another vertex library than the first start (open, not from the index). regress.py check on ŌKAMI HD passed the chain; no local game has a smoke route, so it booted none; Minecraft Dungeons II and Ocarina of Time need their packages split again.

…ion by size tier and archive indexes, seedds#1844 merged, 3D texture mirrors, native mesh pipelines in the archive, follower-scan groups) Hand-resolved: main's compaction and mesh-pipeline archive replace the tail-compaction marker and the mesh-parts draw we carried; the copy mirrors keep our `mir` flag for pkg_copy_absorb; patch_nso.py keeps fixed_resolution beside main's fs_address; the translator keeps the seedds#1585 site argument. Translator rebuilt: chain 1268cc9dae8a. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014eD437mxrMnW2UwLZhpwc7 Upstream pull requests applied in this commit (their authors' work, brought onto this tree with a three-way merge; the description is in each PR): - seedds#1585 by nafields (open): Raw global memory through a pointer read when the draw is bound: the host fills each fixed-offset pointer's pool range before the draw (chain 1ffcb6b77740) - seedds#1844 by nawangwe (closed): Shader libraries compile with -fwrapv: Ocarina of Time's mesh-shader models no longer take their ambient light from the wrong probe texels - seedds#1879 by bettterlabs (closed): Timer flushes wait during any prewarm, so the start after a cold start does not compact (Ocarina of Time 762 s → 3.3 s) - seedds#1880 by bettterlabs (closed): Pipeline archive compaction by size tier, so the start after play recaptures what play added, not the whole archive (Ocarina of Time 713 s → 24 s)

bettterlabs mentioned this pull request Nov 8, 2026

Timer flushes wait during any prewarm, so the start after a cold start does not compact (Ocarina of Time 762 s → 3.3 s) #1879

Closed

2 tasks done

github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation game: 0400486024DEC000 A game labels Nov 8, 2026

bettterlabs changed the title ~~Pipeline archive compaction by size tier: a start after play compacts what play added, not the whole archive~~ Pipeline archive compaction by size tier: a start after play compacts what play added, not the whole archive (Ocarina of Time) Nov 9, 2026

bettterlabs changed the title ~~Pipeline archive compaction by size tier: a start after play compacts what play added, not the whole archive (Ocarina of Time)~~ Fast start after play: compact the pipeline archive by size tier (Ocarina of Time 713 s → 24 s) Nov 9, 2026

bettterlabs changed the title ~~Fast start after play: compact the pipeline archive by size tier (Ocarina of Time 713 s → 24 s)~~ Faster warm starts: compact the pipeline archive by size tier (Ocarina of Time 713 s → 24 s) Nov 9, 2026

bettterlabs added 2 commits November 8, 2026 17:00

Names: arch_compact_takes (whether this compaction takes a file; arch…

2a780ed

…_compacting read as "is a compaction running" beside arch_capturing), and the fresh-process test of files named before the count is --oldname-* ("legacy" already names the MTLDevice path in this file)

bettterlabs changed the title ~~Faster warm starts: compact the pipeline archive by size tier (Ocarina of Time 713 s → 24 s)~~ Pipeline archive compaction by size tier, so the start after play recaptures what play added, not the whole archive (Ocarina of Time 713 s → 24 s) Nov 9, 2026

seedds mentioned this pull request Nov 9, 2026

Ocarina of Time (0400486024DEC000 v131072, Mac M4 Pro): every start spends about 5 minutes in the prepare step, 309 s on a warm start with nothing compiled #1747

Open

dpadr mentioned this pull request Nov 9, 2026

Ocarina of Time (0400486024DEC000 v131072, Mac): DM_UPSCALER=metalfx does nothing on main, since the profile lists no agl_ndsr_vr upscaler shaders (#1459 fixes it: -1.5 to -2.5 ms a frame on an M2 Air) #1911

Open

seedds closed this Nov 9, 2026

Know-It-All-Brother added a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026

Merge PR seedds#1880 (pipeline archive compaction by size tier), whic…

d84f708

…h replaces seedds#1840's compaction limit

Know-It-All-Brother added a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026

t_pso_archive: drop seedds#1840's check that a compaction records its…

9e0fa54

… size, which seedds#1880's size tiers replace

Know-It-All-Brother added a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026

test_pso_archive.py: drop seedds#1840's restart test of the compacted…

f3b84fd

… size, which seedds#1880's size tiers replace

Navigation Menu

Repository navigation

Pipeline archive compaction by size tier, so the start after play recaptures what play added, not the whole archive (Ocarina of Time 713 s → 24 s) - #1880

Pipeline archive compaction by size tier, so the start after play recaptures what play added, not the whole archive (Ocarina of Time 713 s → 24 s)#1880
bettterlabs wants to merge 3 commits into
seedds:mainfrom
bettterlabs:pr2-compact-tier

bettterlabs commented Nov 8, 2026 •

edited

Loading

Uh oh!

FifteenPen commented Nov 9, 2026

Uh oh!

seedds commented Nov 9, 2026

Uh oh!

seedds commented Nov 9, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

bettterlabs commented Nov 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What and why

Checked

Docs

Uh oh!

FifteenPen commented Nov 9, 2026

Uh oh!

seedds commented Nov 9, 2026

Uh oh!

seedds commented Nov 9, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bettterlabs commented Nov 8, 2026 •

edited

Loading