[Romen-Adama-Dev]

What

In one session of The Legend of Zelda: Ocarina of Time (0400486024DEC000, v131072) the log held 1,502 lines like

[RED] background render pipeline creation failed: error: cannot commit transaction no. '50643' on '…/com.apple.gpuarchiver/gpuarchiver-8e6a0d/PersistentState': MDB_MAP_FULL: Environment mapsize limit reached
[RED] background render pipeline creation failed: error: cannot write 'UNITSMAPPINGS' database: MDB_MAP_FULL: Environment mapsize limit reached
[RED] pipeline archive: compaction left the old files (the flush failed)

and Kakariko Village ran at 30-70 ms a frame with draws missing. A first start whose prewarm lists held 40,000 rows (the manifest of #1590 plus the player's own list) compiled 8,693 bases into one flush period, and Metal's store for that period's serializer (an LMDB environment of fixed map size under the app's cache folder, com.apple.gpuarchiver) filled.

Cause

arch_capture compiles a captured pipeline through the period's Metal 4 compiler, the one bound to the pipeline data set serializer. Once the serializer's store is full, every compile through that compiler fails, not only the capture, and the host handed the error on as the pipeline's: the pipeline was never built, by any compiler.

Fix

arch_capture and arch_capture_cs build a pipeline the capture refuses with the plain compiler (g_comp4), count the failure, and after ARCH_CAP_GIVE_UP (8) in a row the run captures no more, said once in red. A capture that works resets the count. A pipeline both compilers refuse reports the plain compiler's error (the capture's when it gave none). The archive is a cache, never a condition of drawing.

Reference entry A pipeline capture that fails must not fail the pipeline (with the open point: on the Mac the archiver store is never emptied; 7.1 GB in 722 folders here after two days), the game note, the symptom index.

Tests

• t_pso_archive: the failing-capture case now expects every concurrent caller to get a pipeline with no error, the failures counted, the give-up after eight in a row (the gate compiler is not asked again), and the reset. 102 → 136 checks, Metal 4 and MTLDevice modes.
• t_pso_warm (99), test_docs.py: pass.
• Reach: every game that captures pipelines into its archive (any Mac or iOS run with pso_archive); a working capture takes the same path as before. regress.py check --games cannot run on this Mac (one local game).

[seedds]

Thank you, this landed on main as eed7c42, reworked, with you credited as co-author.

Your diagnosis and the fallback are kept: a pipeline the capture refuses is now built by the plain compiler, and the archive is never a condition of drawing. What changed is why the store filled, and so what happens after a failure:

• Metal gives each serializer its own LMDB store (a gpuarchiver-* folder per flush period, 10 MB map size from the PersistentState meta page). A store fills only when one period captures too much. While a compacting prewarm ran, arch_flush waited for the prewarm's end so the recaptured bases would make one file, which is how 8,693 bases went into one period. Your log's "compaction left the old files" shows compaction was on. Periods of 3,508 bases were written whole in other logs.
• So a period is now handed over at its 2,000th capture (ARCH_PERIOD_MAX), during a compacting prewarm too. A compaction spans its periods and removes the old files only once all of them are written.
• Instead of giving up after 8 failures, a failed capture hands the full period over once, so captures continue in a fresh store. Only a failure in a period that has captured nothing stops capturing for the run.
• The reference entry is now Metal's archiver store filled by one flush period. It links the existing entry on leftover gpuarchiver-* folders. Those folders take disk space but don't fill later runs' stores, since each period makes a new one.

Tests: t_pso_archive (135 checks: the hand-over, the stop, both compilers refusing, a compaction across periods), t_pso_stall (a base both compilers refuse leaves capturing on). On the Mac, Ocarina of Time compacted 70 archive files into one with no capture failures. That start captured too few bases to reach the cap, so the cap is covered by the tests. If you can rerun your 40,000-row first start on main, the log should show files of at most 2,000 bases and no MDB_MAP_FULL.