Repository navigation
Ocarina of Time's residency set on the iPhone: the physical pool behind GPU-only targets, large targets' heap slack, the zero buffer and idle uploaded textures leave it (depends on #2270) - #2566
Open
stan69b wants to merge 76 commits into
Conversation
…k before the commit, so the commit no longer pages the guest's files in from the GPU driver's path, where the M1 iPad froze for good With the guest's memory in files (the iOS app's default when the game's pool is larger than its memory), Ocarina of Time on an M1 iPad (8 GB) froze with sound and no picture, once in Link's house and once at the title movie. A sysdiagnose spindump during the freeze: the encode thread in res_commit <- gmem_map, IOGPUFamily -> the VM -> apfs pagein, blocked by an APFS rwlock owned by a launchd background-I/O thread that stopped 4 ms after it and never ran again. Ordinary page-ins of the same files (700,000 in a run) never hung. file_wire (nvn_mtl.m): a shared buffer whose pages are file-backed (vm_region_recurse_64: external pager) is mlocked before it joins the set, and the mlocks are given back right after the commit, which holds the pages for the GPU from then on. Anonymous memory and Metal's own buffers are untouched, so nothing changes with the guest anonymous (every Mac run unless DM_GUEST_FILE=1). Counted in the 300-frame residency line. t_file_wire checks it (failed before the fix, passes after). iPad, the same route that froze twice: alive past frame 12,900, 20 fps in the house (GPU 34-42 ms), headroom 5.3 GB, commits under 2 ms per 300 frames. Mac (M4 Air 16 GB, idle) with DM_GUEST_FILE=1, a route that stalled at 7-37 s a frame before: played through, frame time p50 16.7 ms, 3.9 GB wired at most. memory.md "The residency commit froze over the guest's files"; Ocarina of Time's note. Rebased onto main 0c07813, whose residency changes are queued and applied by the commit outside g_res_mu (0c07813): res_add_at wires a buffer's file pages before it queues the change, and res_commit takes the wired ranges together with the queue, so each commit gives back after itself exactly the ranges of the buffers it made resident; a range wired for a change queued after it waits for that change's own commit.
…share bytes in a placement heap per 32 MB span, as on NVN: Ocarina of Time no longer runs out of GPU memory going outdoors on an M1 iPad With the render targets' recycled heaps (rh_texture, 7f50222) every live description in a GPU-only pool is a Metal texture of its own. On the M1 iPad (8 GB, recommended working set 5,461 MB) that reached 5,184 MB resident at the outdoor save load (544 heaps of 1,384 MB over 1,455 MB of textures), and every command buffer then failed with kIOGPUCommandBufferCallbackErrorOutOfMemory. window_texture (nvn_mtl.m): a texture whose storage lies in a CPU_NO_ACCESS pool (not sparse, not linear, one sample) is placed in a heap standing for a 32 MB span of guest addresses, at its NVN offset scaled by 9/8 and rounded to Metal's alignment, if its Metal size ends within its scaled NVN range, so textures overlap here where they overlap on NVN. A span's heap is made only once it saves memory; a texture across a span's end tries the half-offset spans; covered bytes are inherited, new ones zero-filled. Placed textures have no GPU-optimized contents: with lossless compression, another format's writes over the same bytes hung the iPad's GPU (kIOGPUCommandBufferCallbackErrorHang by Link's house). Anything that does not fit takes the usual path (image_texture, then rh_texture). On only where recommendedMaxWorkingSetSize is under 8 GB (window_wants): sharing a target pool's bytes corrupted the picture on the Mac in PR seedds#1453 (the game keeps live targets in the part of the full-size scene target its current resolution leaves unused, which only NVN's layout makes safe), so larger devices keep each target's own layout. 8 GB devices have 5.3-5.5 GB of working set, a 16 GB M4 Mac 12,124 MB; the separate heaps needed 5.2 GB at the outdoor save load, sharing 4.3 GB. DM_NVN_WINDOW_ALIAS=0/1 forces it, and a log line says which way it went. The gfx test Makefile runs every test with it off unless the test turns it on, so an 8 GB Mac runs them as a 16 GB one does. Also for reading the set (res_top: the first GPU out-of-memory error, DM_CTL restop, and once when the set passes three quarters of the working set): the set by the source line that added each object, the resident pools by raw table / binding, by who gave an address in them (hle_nvn.c passes the kind: mtl_pool_addressed_by) and by pool flags (mtl_pool_tag), the GPU-made textures by pool, and the spans' counts. Tests: t_window_alias (new; 25 checks with the spans on, direct and deferred encoding, Metal validation; 8 fail with them off, the separate allocations), including the 8 GB rule. t_image_alias, t_rt_recycle, t_alias_takeover, t_upgrade_alias, t_copy_alias, t_rt_lifetime, t_pool_chunks, t_addr_known, t_file_wire pass. memory.md "Textures in GPU-only memory took an allocation per description"; rendering.md's recycled-heap entry links it; env_switches.tsv. Rebased onto main's pool and render-target structure: the virtual pool's runs are tagged after main's one-step mtl_pool_replace (vrun_set); the new pools' tag is set after pool_new_put; the storage range for res_top is kept on the texture before own_alias_publish (nvn_owner.m); a span's heap is an NvImageHeap with window set, which main's family-heap search (own_alias_at_locked) skips and which is freed with its last texture rather than waiting in rh_wait's free list. rendering.md keeps main's Fix and Check wording and adds the window heap. Rebased onto main 0c07813: res_add_at records the adding line in g_res_line beside main's queued change (res_q_put), and sets no g_res_dirty, which the queue replaced.
…edicted sleeps, the waits a sleep ended too late for, how late the sleeps woke and the period: Ocarina of Time makes none, so its slow motion on the M1 iPad is GPU time Before, the power line counted clock waits only for its two top yielders, so a clock wait on any other thread was invisible. On an M1 iPad, Ocarina of Time ran at game speed 0.62 in one run and 0.99 in another with DM_YIELD_CLOCK=0, which pointed at 8bdfcb7's predicted sleep. dm_clock_top (hle_thread.c), printed by the power report every 300 frames (hle_nvn.c) as "clock waits up to frame <n>": the two threads with most clock-wait yields since the last report, their predicted sleeps, how many of those waits had already ended by the first poll after the sleep (slept through), how late the sleeps woke (mean / max) and the learned period. No clock wait, no line. No behaviour changes: counters only. On the M1 iPad, 12 minutes from the title through the Deku Tree (upstream main + this branch, no switches but the device's own): no line at all, so no thread of the game makes a clock wait and the predicted sleep never runs. Its slow motion there is the GPU: views that take 54-67 ms of GPU a frame cannot fit the frame step's longest hold, 3 vsyncs (game speed 0.61, then 0.53), where lighter views hold 20 fps at 34-36 ms. t_clockwait checks the clause (the thread, 40 predicted sleeps, the waits slept through, a 16.68 ms period, and nothing once no clock wait came); t_pollwait, t_preempt, t_condvar, t_condvar_old, t_sync_wait, t_system_event and t_notify_sched pass. threads.md "Clock waits sleep to their predicted end".
…ee's entrance with the guest in files and a 3.2 GB residency set; heavier views are GPU-bound "For an iPhone or iPad build" under Memory: the device (iPad Pro 11-inch 3rd generation, M1, 8 GB, iPadOS 27.0.1), the memory limit and headroom (7.97 GB of room against the 9 GB pool, so the guest in files; footprint 2.2-2.3 GB, headroom 5.87 GB at its lowest; working set 5,461 MB, residency set 3.2 GB with the GPU-only spans on), fps and GPU time (20.0 fps in every window of two 32-minute runs at the Deku Tree's entrance, GPU 34-35 ms, game speed 0.97-1.00; heavier views 54-67 ms of GPU, 11-12 fps and game speed 0.53-0.61; the title and game over screens 61 and 66-74 ms), no clock waits, no switch needed, and what is open: the heavy views' GPU time, a GPU hang on an earlier build in the draws of PR seedds#1842's M1 hangs, one kernel data abort, and PR seedds#1906 for an iOS build on 7ffa63c. Found on: "Textures in GPU-only memory took an allocation per description"; memory.md's two entries link the new section. The Open list's "no iOS run" line points at it.
…height matches the shape the host asked for gives the picture that shape instead of 16:9 A game that renders a wider or taller picture into larger window textures and crops the window to it (Ocarina of Time: a 2536x1152 crop of 2560x1440 textures at 2.2:1) was presented as a 16:9 picture. present_shape / present_fit (nvn_mtl.m) now take the crop's own ratio when it matches the screen shape mtl_present_screen set (screen_aspect any), and keep 16:9 for a crop of any other shape. t_present_crop checks a wider crop, a taller crop, a crop of another shape and a 5:4 crop filling a 5:4 canvas on the classic and Metal 4 presents: 120 / 120.
…e while the game runs, from 0.25 to 6, portrait included game.py: screen_aspect off / on / any (DM_GAME_SCREEN_ASPECT 0 / 1 / 2); on (Breath of the Wild, Animal Crossing: taller than 16:9, set at the start) is unchanged. loader.c (screen_any_apply / screen_any_write / dm_screen_view_changed): for an any profile, DM_SCREEN_ASPECT unset or auto follows the front end's view, 1.777 or 0 is 16:9, a number fixes the shape. It writes the state page: +0xa0 the ratio (0 = 16:9), +0xa4 / +0xa8 the width and height factors, +0xac a change count, +0xc4 / +0xc8 the camera's x / y multipliers (wider than 16:9 keeps the vertical angle, between 1:1 and 16:9 the horizontal one; upright takes the scale of the same screen turned landscape), tells the present the shape, and on a change pushes the display-change notifications (hle_oe.c dm_display_changed) so the game lays out again. front_mac.m reports the window's shape 300 ms after a resize settles; front_ios.m the app window's in landscape (upright stays 16:9 for now). settings.tsv: the Picture shape setting for any profiles (Match window / Match screen, or 16:9), auto by default. Front-ends: new: front_view_aspect() (front.h) answers the game view's width / height and dm_screen_view_changed(aspect) reports a settled resize; a profile with screen_aspect any (Ocarina of Time) follows them under DM_SCREEN_ASPECT auto, and renders 16:9 until a front end provides them (the loader's weak front_view_aspect returns 0).
… for matrices and float pairs in guest memory, and a write watch armed from a probe's register DM_PROBE_FP prints s0-s7 on probe hits. DM_PROBE_FINDMAT=<s>[,<n>] scans the guest heap for perspective matrices and arms the write watch on the n-th at 16:9; DM_PROBE_FINDF=<s>,<a>,<b>[,<gap>[,<n>]] scans the heap and the alias region's mapped parts for two floats and arms it on one. DM_PROBE_WATCH=@xn+<offset>[+<length>] arms the watch on that register plus the offset at the first printed probe hit, for objects whose addresses move between runs (Ocarina of Time's layouts in the alias region); DM_PROBE_XREG=N lets DM_PROBE_X2 dump xN's memory. The watch carries out CAS and single-lane ST1 itself. Found Ocarina of Time's camera copy and HUD screen size.
… its output size, window textures and camera projection follow the shape, with no bars and no stretch screen_aspect any, and four stubs as function-matched others of the frame-step rule (patches.py, found on v131072 main): - output size (main+0x1087f40, the display settings' size setter): the requested size times the shape's factors, capped at 1.5 times the pixels, each side a multiple of 8; the crop, the scene's targets and the dynamic-resolution sizes follow. It copies the camera multipliers the size was made with to state +0xb8 / +0xbc. - window crop (main+0xe9a860, setCrop): a crop beyond the 2560x1440 window textures grows them; the next acquire re-creates them. - camera copy (main+0x33a8c0 +0x2ec): the camera named "Main" hands its cached sead::PerspectiveProjection matrix here each frame; rows x / y times the multipliers, the aspect beside it, in memory and in the registers the frustum code reads after. set(), tanf and the 16/9 literals do not run in play: found with DM_PROBE_FINDMAT and the write watch. - display update (main+0x6502e0 +0x18): flag bit 17 set when the loader's change count moved, so the game recomputes its size and a resized window takes effect. Wider than 16:9 keeps the vertical angle, between 1:1 and 16:9 the horizontal one, upright the scale of the same screen turned landscape. test_oot_aspect.py runs the stubs with unicorn (12 tests). Checked on the Mac at six window shapes resized live (iPhone and iPad upright and landscape, 1700x478, 16:9). The HUD is the next commit.
…instead of a 16:9 layout stretched over the picture Every eui layout file holds a screen size of 3840x2160 (LayoutResData +0x200, file offset 0x2f8). A screen's init (main+0x773b90) builds its orthographic projection from it once and keeps it in the screen's uniform block, which the draw copies to the frame's ring each frame (main+0x2d2ba0). Two stubs, function-matched others of the frame-step rule: - hud upload (main+0x2d2ba0 +0x4c, after the memcpy): the copy's x / y rows times the HUD's multipliers and the size divided by them, each frame, so a resize takes effect at the next one; a capture's block (its own size beside the screen's) is left as it is. - layout size (main+0x583fd0, the getter): the size the HUD draws with for the game's conversions between the layout space and the screen; the screen's init still gets the file's. The HUD's multipliers are the camera's divided by the largest of them and 1: in landscape the HUD keeps the size it has at 16:9 of the same height or width; upright the whole 16:9 space fits the width, smaller, nothing cut off. test_oot_aspect.py: 6 more tests (18). Checked on the Mac at six shapes resized live, the pause menu, the "A Open" prompt and the Kokiri Forest cutscene with its location card. Left open (game note, Screen shape): eui places panes about the screen's centre and does not anchor them to its edges, so the HUD keeps its 16:9 arrangement in the middle (in wide shapes the hearts sit at the 16:9 band's edge, upright the HUD sits around mid-height), and full-screen panes (fades, the menu backdrop, 3840x2160) cover the 16:9 middle only. Moving the edge groups (60x80 nulls at the space's edges) and stretching those panes needs eui's per-frame pane transform, not found yet.
…e screen_aspect key, DM_SCREEN_ASPECT for any profiles, the probe switches, and a reference entry for a camera that caches its projection - Game note: a Screen shape section (the six stubs, the rule for wider, taller and upright, the HUD, what is open: the HUD's edge groups and full-screen panes, upright on iOS, the vertical angle upright) and its line under Found on this game. - docs/reference/rendering.md: a screen-shape stub on the projection setter changes nothing, the camera caches its matrix (find the per-frame reader with DM_PROBE_FINDMAT, DM_PROBE_FINDF and the watch armed from a register); the symptom index regenerated. - dm/games/README.md: the screen_aspect key (off / on / any). docs/env_switches.tsv: DM_SCREEN_ASPECT's any range and auto, DM_PROBE_FINDMAT, DM_PROBE_FINDF, DM_PROBE_XREG and DM_PROBE_WATCH's heap+ and @xn forms. docs/settings.md: Picture shape for any profiles. - docs/agents/game-patches.md regenerated: Ocarina of Time's screen-shape and HUD stubs are function-matched (BINDING rows in its patches.py), and Animal Crossing's row names them as the model. docs_patches.py's dry run puts a rule's other functions 4 MB apart instead of 16 MB: with 16 MB the ninth (Ocarina's layout size) landed out of a branch's reach of the patch area.
…ecord is unwired at once instead of the list being lost and written through a null pointer clang-tidy (bugprone-suspicious-realloc-usage) on 29fa5b0's g_fwire = realloc(g_fwire, ...). t_file_wire: 10 / 10.
…the GPU never reads shader code, so nothing makes those pools resident and wires their pages for the table's sake (Ocarina of Time on the iPhone: 223 pools, 317 MB) nvnMemoryPoolGetBufferAddress on a pool with NVN_MEMORY_POOL_FLAGS_SHADER_CODE (not virtual) reports kind 4, which nvn_mtl.m records for res_top but leaves out of the addressed ranges. A buffer made in such a pool still puts it in (nvnBufferInitialize reports kind 0 and 1). On the 8 GB iPhone the wired pools left too little room for the guest's file pages, which were read back 0.2-0.4 GB every 10 s while playing. t_addr_known checks it.
… that measured the same work, so a loading screen's GPU time no longer starts probes of 60 fps nvn_fstep_gpu_slowed compares the GPU time at a longer hold with a running mean of the frames at kmin; 1.3 times as much is taken for a clock that fell with the hold and starts a probe back to kmin. In Ocarina of Time on an M4 Air the only long run at hold 1 was the title and the loading screen (GPU 0.2 vsyncs, 2-60 draws). Gameplay (700-900 draws, 1.3 vsyncs at hold 1, 1.4 at hold 2) read as a GPU slowed 6 times: 32 probes of hold 1 in a 120 s walk, all failed, each failure's late frames stepping on to hold 3. nvn_fgpu keeps the draws a frame encodes beside each reference (dref_at) and at the longer hold (dcur); fstep_tick takes them from mtl_stats. The GPU counts as slowed only while those are within FSTEP_GPU_WORK (1.5) of each other; without draw counts (0) it is as before. t_fstep (81 checks, 0 failed): a loading-screen reference (30 draws) against gameplay at 900 is not slowed and makes no probe, where the same run without draw counts makes 8 failed probes; the same draws at both holds still give Breath of the Wild's one probe that holds. Measured on the M4 Air, Link's house (120 s walk): probes 32 (all failed) -> 7 (all at the title and the load), frame max 122 -> 73 ms, p95 51.1 -> 51.8 ms, but 27.2 -> 24.2 fps: the failed probes' late frames had stepped down to hold 2 now and then, and the hold-3 stretches left come from GPU time at hold 2 of p90 33 ms against 33.3. Kokiri Forest: 19.0 fps, every frame at hold 3, GPU span 42.4 ms, game speed 0.95; with DM_FPS_TARGET=30 21.0 fps at hold 2, game speed 0.92-0.94. Not run: regress.py (the owner's rule for this branch); BotW and TotK, which use the same controller, were not measured.
… its own clock, so Ocarina of Time's title movie no longer plays in slow motion under the frame step The owner saw the scene behind the title in slow motion, as the game was before the frame step. That scene is a 60 fps movie (Event/Movie/Dm_Title_*.webm, AV1 through nn::codec), and the game's player reads one picture from the decoder each game frame, never skipping: right on the console, where the title runs at 60; under the host's frame step the title runs at 20-30 fps. Measured with a new DM_MOVIE_TRACE trace of nn::codec packets and picture reads (title, 100 s, M4 Air): one read per game frame, times 16-17 ms apart, movie speed 0.50 at a 30 cap and 0.36 at the 60 cap (the title at 21.8 fps). The decoder now answers for such a player (cv_due in hle_codec_video.c): when a released buffer takes a picture, held pictures behind the movie's clock are dropped and the one nearest the clock goes in. The clock is a picture's time when put in plus host time since; it starts again at the first picture, after 200 ms with none put in (paused), after a reset and when the stream's time goes back. A dropped picture signals the submission-ready event as a packet taken does. Only streams whose pictures step 4-100 ms read as nanoseconds are clocked: the unit is the client's, and any other is left as before. A player that keeps up drops nothing. The frame step's movie rule (hold 30 for 30 fps movies, raised by movie::Decoder only) is unchanged. Test first: t_codec_video's movie clock case (written failing: at 30 reads a second no picture was skipped): 60 reads/s lose none, 30 reads/s step two pictures each read (299 of 299) and show 10 s of movie in 10 s, 20 reads/s 15 s in 15 s, a µs-timed stream drops none, a 500 ms pause gives no catch-up. "nearest the clock" rather than "last not after it": on the stream's millisecond-rounded times the latter stepped 1 and 3 pictures by turns (199 skips). t_codec_video all passed; test_env_switches and test_docs OK. In the game (dmhost_main_mclock, title, 100 s each): speed 1.00 at the 30 cap (30.0 reads/s, steps 33 / 34 ms) and at the 60 cap. Not run on the other nn::codec games (Donkey Kong Bananza, Fire Emblem Fortune's Weave, Mario Tennis Fever): no regress.py on this branch (the owner's rule for it); their players, read at their movies' rate, drop nothing by the test's 60 reads/s case. Docs: docs/reference/movies.md entry, the game note (resolved, and Found on this game), DM_MOVIE_TRACE's line in docs/env_switches.tsv, symptoms index. Rebased onto main's decoder thread (cv_worker): the clock runs in fill(), where a released buffer takes a picture, on the guest's release and on the decoder's thread alike; fill returns the pictures it dropped, and they reach ready() as a packet taken (the release, and the packet's end through cv_out.dropped). A Stop / Flush restarts the clock with the epoch. t_codec_video's paced player waits for the decoder's thread (settle) before each read, as main's checks do, and its clock is atomic since that thread reads it; with cv_due disabled it fails again (skips).
…arina of Time's title no longer probes hold 1 against the boot screens' reference 787e00d counts the GPU as slowed only while the draws at the longer hold match the reference's. Draws do not see compute: on the M1 iPad the title spends 61 ms on its movie network's 14 dispatches with 20-80 draws, while the hold-1 reference came from the boot screens (0.4 ms, no dispatch). The title read as a GPU slowed 9.9 times and probed hold 1 every 3-6 s (4 in its first 350 presents), each probe about 33 frames at hold 1, none held. nvn_fgpu now keeps the dispatches a frame encodes (+1, so none is a count) beside the draws, and both must match; mtl_dispatches() gives the count (dispatch_enc). t_fstep 94 checks, 0 failed: the title case does not probe with dispatch counts and probes 17 times without; the same draws and dispatches at both holds still give BotW's one probe that holds.
…st), and a reference entry for the shader-code pools the raw pool table wired
…ngs screen and in the in-game menu locks the orientation, and lets an iPhone be played upright DM_ROTATION, a launcher and in-game menu setting (dm/games/settings.tsv, platform:ios). The phone's Info.plist lists only the two landscapes, so an iPhone could never turn upright: the app delegate (main.m) now answers application:supportedInterfaceOrientationsForWindow: with front_ios_app_orientations (front_ios.m), which replaces that list. Under auto, the default, it returns the list itself (the phone's two landscapes, the iPad's four), so nothing changes for anyone who leaves it. landscape locks the two landscapes; portrait upright (and upside down where the device has it). The root view controller and the launcher's DmHostingController allow every orientation under auto and the locked ones otherwise (dm_rotation_vc_mask; the pure mask is DmLayout.h dm_rotation_mask). A pick calls setNeedsUpdateOfSupportedInterfaceOrientations on every view controller and, when the window's orientation is no longer allowed, requestGeometryUpdateWithPreferences, so the screen turns at once: on the settings screen as soon as it is picked (Launcher.swift onChange), in the in-game menu (dm_setting_live), and at the start for a saved pick, DM_LAUNCHER=0 or an env file. Upright, the settings screen puts the game's name and Play above the settings instead of in a third of the width. Duo harness: the fixture defines DM_GAME_SCREEN_ASPECT and the stubs add dm_screen_view_changed, dm_players_reported and front_web_touch, which front_ios.m and DmGamepad.m call since screen_aspect any and the controller applet's prompt: the harness did not compile or link without them. Its app delegate answers as main.m's. New test: the masks for each value, and the in-game menu offering Rotation. test_launcher_settings: DM_ROTATION joins the iOS-only settings (the test listed them; updated deliberately). Checked: the Duo harness on the iPhone 17 Simulator, 41 tests, 0 failures (1 skipped, iOS 27.1 only), including the rotation test of the next commit, which turns the Simulator's landscape-only iPhone upright; test_launcher_settings, test_orientation_policy, test_docs. Front-ends: new: DM_ROTATION (Rotation). An iOS host app with its own app delegate answers application:supportedInterfaceOrientationsForWindow: with front_ios_app_orientations() (DmRotation.h) for Portrait to turn a phone upright
… touch controls instead of a 16:9 picture with bars, and with a controller the whole screen A profile with screen_aspect any renders the shape the front end reports (front_view_aspect, dm_screen_view_changed). front_ios.m reported the app window's shape in landscape and 16:9 upright, so upright the game rendered 16:9 and the front end fitted it, with bars, into the part above the controls. DmLayout now carries `area`, where the picture may go: the view in the overlay layout, the part above the controls in a split. viewDidLayoutSubviews reports that area's shape (on a TV the TV's), so upright the game renders the space above the controls: 402x375 points (1.07:1) on an iPhone 17 instead of a 402x226 picture. Landscape reports the whole view, as before. Profiles without screen_aspect any keep the 16:9 picture above the controls. With a controller connected the touch controls hide (DmGamepad.m, now dm_gamepad_off, which lays the view out again when that changes): upright with no fold to split at, the picture's area is then the whole safe area (DmLayout.h dm_layout_controls_hidden); a half-folded Duo keeps its split, landscape is unchanged. Fill applies only to a landscape view, so a 16:9 picture upright with the controls hidden stays fitted instead of losing most of its width. Tests (Duo harness): the area in landscape and upright, the hidden-controls layout and Fill upright, the shape the production root reports, and a real turn of the Simulator's iPhone to Portrait and back: the shape handed to the host is the area's, and once the game presents it the picture fills the area (402x375). Checked: the Duo harness on the iPhone 17 Simulator, 41 tests, 0 failures (1 skipped, iOS 27.1 only).
…cture's (DM_SCREEN_UPRIGHT_FOV), so a phone upright no longer looks up into parts of the world the game never builds Upright, the camera rule takes the scale of the same screen turned landscape, its width getting landscape's vertical angle; a phone upright (0.46:1) then sees about 90 degrees vertically (Ocarina's fovy is 50 degrees at 16:9), and above Link's house the world is not built. loader.c now raises the camera's y multiplier (state +0xc4 / +0xc8, which the size stub copies with the size) to that of a picture of the cap's shape when it is smaller, and x with it, so the projection keeps the screen's aspect and the width narrows instead: by default the cap is a square (79 degrees). DM_SCREEN_UPRIGHT_FOV moves it (1.777: the game's own vertical angle) or turns it off (0); landscape is never capped. The HUD's multipliers are the camera's divided by the largest of them, so the HUD does not change. No stub changes: the multipliers were already the loader's (screen_camera, screen_upright_cap; the log line names them). test_oot_aspect: camera() mirrors the cap (checked against loader.c's code compiled on its own for 0, 1 and 1.777 and six shapes: equal to 1e-5). test_shapes encoded the uncapped rule for a phone upright; it now expects every upright shape to show a square's height (updated deliberately); the uncapped rule moved to test_upright_without_the_cap, and test_the_upright_cap_holds_the_vertical_angle checks the cap at 1, 0.75 and 16:9, the shape kept, and landscape untouched. Checked: test_oot_aspect (20), loader.c syntax-checked with -Wall -Wextra. Not yet looked at above Link's house or in the Kokiri Forest cutscene upright.
…nders any shape fills the whole screen with the touch controls drawn over it, as in landscape; Picture above controls keeps the split The owner asked for the picture to fill the whole screen upright with the controls over it, as in landscape, and to keep the split of the previous commit as a choice. DM_PORTRAIT_LAYOUT (launcher and in-game menu): full (the default) or split. Full screen (DmLayout.h dm_layout_full_picture) keeps the split's control arrangement in the lower half, so the controls are where the portrait layout puts them, and lays the picture over the whole view under them; the picture's area is the whole view, so the game renders the screen's shape (0.46:1 on an iPhone) and the upright cap of the previous commit holds its vertical angle to a square's. Touches above the controls go to the game's touch screen as in the split. It needs a picture that takes the screen's shape, so it applies only to a profile with screen_aspect any following the view (DM_SCREEN_ASPECT auto, front_ios.m portrait_full); other games keep their 16:9 picture above the controls, where centred on an upright screen it would sit under them, and a half-folded Duo splits at its fold. With a controller Full screen stays as it is (the controls hide). The row is shown only for a screen_aspect any profile on iOS: settings.py's shown column now takes <cond>&<cond>, as the options column already did. Tests: Duo harness, the full-screen layout (area the whole view, the controls' rectangle kept and inside the picture, landscape and a half-folded Duo unchanged) and the production root turned upright in each layout; test_launcher_settings, the row for Ocarina on iOS only. Checked: the Duo harness on the iPhone 17 Simulator, 42 tests, 0 failures (1 skipped, iOS 27.1 only); test_launcher_settings, test_docs. Front-ends: new: DM_PORTRAIT_LAYOUT (Portrait layout), and settings.tsv's shown column takes <cond>&<cond>
…after every uncapped shape 55b30f6 put the upright cap's line before the 16:9 line's else, so the else followed the cap test instead of the shape test: an iPhone run logged "screen shape 1.072 ... camera x 1.000 y 0.603" then "screen shape 16:9" although the state page held 1.072. Only the log was wrong.
…ight cap), measured on an iPhone 17 with Ocarina of Time The game note's Screen shape section follows the picture area upright (Full screen 0.46:1, capped; Picture above controls 1.07:1), records the upright cap and the HUD upright, and the six iPhone runs: every shape in proportion, the same frame rate upright and in landscape, and the one-frame present crop red line while the window textures grow. The iOS layouts reference entry gains the symptoms, the fix, the harness checks and the iPhone test; the symptom index is regenerated.
…read and the replay goes on, so the GPU no longer idles before each submission With Metal encoding on its own thread (the iOS default), mtl_sub_commit waited for the encode thread to finish the submission (rec_drain). Ocarina of Time makes about 30 submissions a frame (the queue scheduler cuts one at every event signal), each under the 64-draw segment size, so the GPU got each one only after the replay had waited for the encode thread and then translated it: on the iPhone 17 frame 3600 spanned 34.9 ms for 20.2 ms of GPU work, 13.6 ms of the gaps before a submission's first encoder, and the replay waited for the encode thread 36 times a frame (10 ms). Now the submission's records go to the encode thread with a note of its fence (RO_NOTE on g_sub_cmt) and the replay goes on. What follows on the queue is a record too; a CPU wait for the GPU waits for the fence's signal behind the commit; graphics suspend waits for the last fence. The display command buffer of a present without a snapshot, the one direct commit that reads the game's work, waits for the note of the submissions before it (sub_commit_wait). DM_NVN_SUB_DRAIN=1 brings the old wait back, ab alternates with DM_AB. t_sub_overlap holds the encode thread inside a submission and checks the submission returns, the note and a present's wait hold until the commit (3 of 8 checks fail with the old wait). With DM_NVN_ENC_THREAD=1: t_reorder_chain, t_defer_chain, t_copy_alias, t_unwritten_store, t_alias_takeover, t_rt_range, t_clear_tex, t_clear_ds, t_upgrade_alias, t_dispatch_overlap, t_perf_order, t_present_crop and check_t_present_snap pass; t_feedback's 3 segment-timestamp checks fail the same with DM_NVN_SUB_DRAIN=1 (the encode thread writes no segment timestamps, frame-rate.md), so not from this change. Left: the iPhone A/B (DM_NVN_SUB_DRAIN=ab) and the regress games.
…r hold the GPU does not explain is probed On a warm start Ocarina of Time stepped hold 1 → 2 → 3 at presents 9 and 12 (cost 1.92 and 5.57 vsyncs, GPU 0.02 and 0.01) on a progress screen while the pipeline archive was written. The game waits for its frames outside the window's acquire, so its cost at hold 3 was about the hold itself (54 ms, GPU 12.5 ms): the fit trial waited and no probe fired, and it stayed at 18 fps for about 1,470 presents. - nvn_fstep.h: the caller reports each frame's GPU time (gpu_on, gpu). A late frame with the GPU under FSTEP_GPU_IDLE (0.05 of a frame) neither counts toward the 3 late frames of a step up nor ends their run; FSTEP_RUN late frames in a row still step up - nvn_fstep.h: FSTEP_RUN frames in a row at a hold above kmin whose GPU time fits k − 1 and that waited for nothing in the host's pacing start a probe (nvn_fstep_unpaced), with their own hold-off (uhold) doubled after each failure up to 60 s and back to 1 s when kmin holds - hle_nvn.c fstep_tick: passes the GPU time; the step log names the idle-GPU step up and the new probe, and its appends clamp their length - t_fstep: idle-GPU late frames, the unpaced probe (also with the GPU clock falling at hold 3), CPU-bound games at hold 2, 2.95 and 3.2 vsyncs, a CPU-heavy scene with the GPU idle (5 of 88 checks fail without the change) - docs/reference/frame-rate.md: new entry; symptom index; Ocarina of Time's Found-on line Rebased onto main's average rule (1e3fd03): idle-GPU late frames and FSTEP_RUN late frames in a row both still need the last FSTEP_RUN frames at the hold to fall short of it on average (short_of_k), the new hold is the cheapest of the three late frames' cost, and a fixed step is held within the stub's smax. t_fstep: main's 86 checks and this commit's, 102 in all, pass.
… next vsync (60 Hz link), not on the 30 Hz link's next tick At a 30 fps cap the frame step holds at most 2 vsyncs, and the display link runs at 30 Hz, so a frame that misses its tick waits for the next one: 66.7 ms on screen. Ocarina of Time on an iPhone at the launcher's 30 cap (frames 36.5 ms, GPU 32 ms) left 33 of each ~333 ticks empty: 33 intervals of 66.7 ms in each 300 frames, the rest 33.3, 27 fps (log iphone5/log_walk.txt). nvn_fstep.h: the late frames that would step the hold up (3 in a row, or FSTEP_RUN with the GPU idle) set fine when the hold is already at kmax; it ends after FSTEP_HOLD_MAX vsyncs with no late frame, when the hold may rise again and while the game fixes its own step. hle_nvn.c: nvn_game_fps reports 60 while fine at a 30 cap, so link_sync runs the 60 Hz link, which holds each frame its two vsyncs (g_link_hold_until) and shows a late one at the next vsync: 33.3 / 50 ms, never faster than 30. The switch is logged both ways; the frame-interval line keeps its 33.3 ms target, and the on-screen line counts intervals at the hold / one vsync longer / two or more. t_fstep: 13 new checks (106, all pass); t_nvn 277 and t_thermal 21 pass.
…hold it allows and why the hold may go no higher
A log whose frame step sat at hold 2 under a 30 fps cap (iPhone, held 1 / 2 / 3 vsyncs 0 / 1800 / 0
with 80% of frames late) did not say that hold 2 was the ceiling, nor why. fstep_kmax() now gives
fstep_tick its kmax and the reason: the 30 fps cap's 30 Hz display link, where a hold of 3 would take
66.7 ms (late frames go to the next vsync of a 60 Hz link instead, B-2); fps_hold off; 120 fps; or
the games' own interval logic (1-3) at a 60 cap. The cap-change line ("frame step cap 0 → 30 fps at
present 1: hold at most 2 vsyncs, ...") and the periodic game-time line carry it, the latter with
whether late frames are shown at the next vsync now and how often that was switched on.
t_nvn 277 and t_thermal 21 pass.
…runs at frame latency 2, so a ProMotion screen shows them at 33 / 50 ms On an M1 iPad (120 Hz ProMotion) the first B-2 build switched to the 60 Hz link at frame latency 1, which calls back 16.5 ms before the target time. It took a frame whose GPU work was still running (frame 1951: GPU end 3 ms after its target), the present landed one 120 Hz refresh late, and the link skipped its next tick. At the title (40 ms frames) the acquire, which counts ticks, then held every frame 50 ms (25 -> 20 fps). In gameplay (36 ms frames) the screen showed frames on the 120 Hz grid, 33 / 42 ms (41 / 54 %), instead of 33 / 50. link_latency (nvn_mtl.m, agreed with session A, who owns the file) returns 2 while hle_nvn.c's new nvn_fstep_fine() reports the fine mode: the same 33 ms lead the 30 Hz link has at latency 1, so the input lag is the cap's own link's. link_sync already applies a latency change each frame and logs it. M1 iPad, Deku Tree spot, DM_FPS_TARGET=30, dynamic resolution on, 9,000 frames of camera turns each: before (30 Hz link): 27.9 fps, on screen 33 / 42 / 67 ms 88.6 / 3.8 / 7.5 %, 22.8 of 66.7 ms per 300, p99 66.7 B-2, latency 1: 27.2 fps, 33 / 42 / 50 ms 40.8 / 53.7 / 5.3 %, 0.2 per 300, p99 50; title held at 20 fps B-2, latency 2 (this): 27.7 fps, 33 / 42 / 50 ms 81.0 / 2.3 / 16.7 %, 0.2 per 300, p99 50; title 25 fps Game time 1.000 of real time in all three. present_rescue 101 checks, t_nvn, t_thermal pass.
…ut every 10, written down with the iPhone's and the M1 iPad's numbers docs/reference/frame-rate.md, "Late frames at a 30 fps cap on the 30 Hz display link": the 30 cap holds at most 2 vsyncs because its link runs at 30 Hz. A 36.5 ms frame comes 3.2 ms later on the tick grid each frame, so after about 10 a tick finds the queue empty and the previous frame stays 66.7 ms: 300 frames over ~333 ticks, 27 fps, 11% of the gaps at 66.7 ms, nothing discarded (iPhone empty ticks 31-33 per 300; "dropped" in the on-screen line is a gap over 1.5 holds). The frame step's paybacks (steps of 3 on 385 of 1,800 frames) do not fall on those gaps. The entry also holds the fix (B-2), the M1 iPad's three runs, and what is open: the iPad title's uneven gaps at 40 ms frames, the title movie's pictures (~210 of 300 frames get one), hold 3 under the cap (not needed), the power cost, and the iPhone, not yet measured. The game note's iPhone/iPad section and "Found on this game" link to it; the symptom index is regenerated. test_docs passes.
…ync as "N× so far", not "1 times"
…hle_nvn.c as one file compile again B-2 (effae9a5) gave nvn_mtl.m a weak default nvn_fstep_fine for the tests that have no frame step, and hle_nvn.c its strong definition. The tests that include both files in one translation unit (nvn_replay and 19 t_* tests) then stopped compiling: "redefinition of 'nvn_fstep_fine'". They rename nvn_mtl.m's weak nvn_game_fps around its include for the same reason; they now rename nvn_fstep_fine the same way. All of them build and pass again, including check_t_present_snap (8 runs) and check_t_capture_live (2 of 2 replays match).
…reed at memory pressure, and what stays of the raw pool table's pools
…s before each, and two of the owner's runs ended at the castle One driven process ran 60 rounds of the castle town and the field with a memory pressure trim before each load (2.4 hours, 152,019 idle uploaded textures freed and made again), at most 6.29 GB wired and the set at 4.43 GB. The owner's first run stopped drawing at the castle entrance's load with the sound on, the residency commit freeze's signs; the second ended at the castle gate with no report. Both are open in the game note.
…r rows in docs/env_switches.tsv
…pipelines through the Metal 4 path with the pipeline archive made offline and the hybrid pipelines Through the MTLDevice, each render pipeline first drawn in a scene state the phone had never drawn compiled on the submit thread: on an iPhone 17 in Kokiri Forest after the first boss, about 17 pipelines of about 400 ms each made 8 frames of 0.17-1.26 s (6.0 s in all), every wait "hybrid off". The archive for applegpu_g18p holds every pair's pulling base (30,690) and a pipeline per compute shader (4,125): 3,501 MB, 28.5 min of metal-tt on the Mac. From the experiment commit 7e5d12c6 (the game.tsv lines), on main's key: main has the launcher's setting (DM_PSO_MTL4) and the fix for pipeline bases missing from the archive (2f131ed, the experiment's 3af843f3), not this profile's default. The game note's "For an iPhone or iPad build" says so, and dm/games/README.md's pso_mtl4 row names the game. No shader input changes (pso_mtl4 is no shader_* / nvn_* key).
The warning that a texture which left out its largest level could not be made whole again (iPhone memory 1, tex_undrop) used RED_ONCE key 160, which main's geometry capture took in the meantime for its vertex stream in no memory pool (nvn_geometry.inc). Each one set the other's flag, so whichever fired first silenced the other for the run. The texture warning now takes key 168 (main has since taken 166 and 167 for its buffer images), and g_red_once grows to 169. Checked: no RED_ONCE key is now used by two different warnings (git grep against main), the host builds. Not run: a game (a log key).
…et one by one with their holders, and a pool of 32 MB or more says what made it resident
…U cannot reach stay out of the raw pool table, so the physical pool behind GPU-only render targets is not wired for the table's sake
…s of rows, so the zero buffer stays at 4 MB instead of growing to the largest texture zeroed and staying resident
… than its texture, not an eighth, so a large target's heap carries no memory nothing uses rh_heap made each new heap an eighth larger than the texture it was made for, so that the next steps of a dynamic resolution can reuse it. On a large target that eighth is resident memory nothing uses: Ocarina of Time's 12288x12288 Depth16 shadow target (296 MB, alive in every area) sat in a 333 MB heap, and its 4096x4096 targets (64 MB) in 72 MB heaps. The extra is now an eighth up to 4 MB (RH_SLACK): every target up to 32 MB keeps its eighth, the shadow target's heap is 300 MB. t_rt_recycle checks a 48 MB target's new heap is at most RH_SLACK larger (30 checks, 0 failed); t_image_alias and t_window_alias pass.
… are freed every 10 s, not only at memory pressure, so an area's textures leave once the game has left it A game keeps the texture objects of an area it has left. On NVN they cost nothing beyond their pool; here each is a Metal copy, and pkg_evict_idle freed the idle ones only at a memory-pressure trim. So each area load added its textures to the last one's, and on a small device they came back only once the system was short: Ocarina of Time on the iPhone 17, walking Hyrule Field into Kakariko Village, warned of memory pressure at the village load with the field's copies resident (the trim then freed 206 MB of them); on the Mac, the second Kakariko visit held 762 MB of uploaded textures against 435 MB on the first, with pools back at 1,474 MB. Where Metal's working set is under 6 GB (small_working_set, the rule for smaller textures and for staging released after each batch), pkg_sweep runs pkg_evict_idle every 10 s at a submission's start, with the same rules as at pressure (idle 30 s, never written by the GPU, not placed, sparse, a view, pinned or held). What it frees is made again at its next use, from its stand-in for the frame of a pool read, at once for a bind. DM_NVN_TEX_EVICT_IDLE=0 / 1 forces it. The system memory line counts what the sweeps freed. t_pkg_evict checks a sweep frees the idle texture without pressure, leaves the used, GPU-written and pinned ones, a read makes it again, and the next sweep waits 10 s (53 checks, 0 failed).
…ookups reach, not whole, so a small buffer bound in a large GPU-only pool no longer wires all of it Pools of BIG_MIN or more have no buffer of their own: a lookup gets a zero-copy chunk over the 16 MB stride that holds what it asked, and only what lookups reach is wired (raw global memory still takes a pool whole). BIG_MIN was 512 MB, set for Mario Kart World's 2.4 GB graphics pool. Ocarina of Time keeps its render targets in virtual pools over one 305 MB physical pool (flags 0x18c): with the raw pool table no longer naming it (kind 5, the commit before), one compute shader (f8965bffbdf51d78_b) still binds a 200 KB read-write storage buffer at an address in it every frame, and that lookup (encode_args_, NVN_RB_SBUFP) made all 305 MB resident: on the Mac run the pool showed as "not in it, looked up 4, 306 MB". From 256 MB that pool is in chunks too, and the binding wires about 32 MB of it. t_pool_chunks (151 checks), t_addr_known, t_upload_src, t_gpu_only_addr and t_file_wire pass.
…rgets, the large targets' heap slack, the zero buffer and the small devices' sweep of idle uploaded textures; the chunk threshold in nvn-objects.md; Ocarina of Time's Kakariko holders
github-actions
Bot
added
host
This was referenced Nov 13, 2026
…oot-build-vis: residency set 3.5 → 3.1 GB, wired 5.3 → 4.9 GB, same frame time; the sweep's 11 ms pass after an area load is open
Contributor
Author
|
OoT-fps combined phone build, against the baseline (session A, A-2) iPhone 17, Ocarina of Time, Kakariko Village by day, unattended drive script, 7 minutes each, back to back (2026-11-13, 22:20 and 22:36). Medians over the gameplay windows (more than 1,000 draws a frame).
Against A-1's targets:
An earlier pair that cannot be compared: the first pair (21:04-21:19) gave 11.4 fps for the baseline and 29.2 fps for the combined build. That gap is the time of day, not the change: the baseline's save loaded Kakariko at night (GPU p50 60 ms) and the combined build's by day, as B found (#2582). Every run's The combined build is the one installed on the phone. |
2 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #2270 (built on its head b8cb0bd, which adds the census and the small-device rules this changes). Merge #2270 first; this PR's own commits are the last eight.
On the iPhone 17 (8 GB), walking Hyrule Field into Kakariko Village, Ocarina of Time's residency set reached 3.9 GB, wired memory 5.7 GB of 7.67, 147 MB free, and the village load warned. This PR finds what the set holds, by holder, and cuts what a general rule can cut.
What changed
nvn_mtl.mres_top).DM_CTL restoplists the set's largest textures one by one with their holders, and a pool of 32 MB or more says what made it resident: a lookup's caller in the host image, or the raw pool table. In Kakariko this showed a 305 MB CPU_NO_ACCESS pool (flags0x18c) with nothing but virtual mappings and texture addresses in it.hle_nvn.c,mtl_pool_addressed_by, kind 5). The game keeps its render targets in GPU-only virtual pools over that physical pool. A block-linear texture there keeps its content in a Metal texture of its own, so no shader reading the table can need the pool's bytes. A buffer, a linear texture, the pool's own address and CPU-visible storage still put the pool in.BIG_MIN512 → 256 MB). Once the table let go, one compute shader (f8965bffbdf51d78_b) still bound a 200 KB storage buffer inside the 305 MB pool every frame, and the lookup wired all of it. Now only the 31 MB the lookups reach are wired.rh_heap,RH_SLACK). The 12288x12288 Depth16 shadow target (296 MB, sampled in Kakariko, so it stays) sat in a 333 MB heap; it is now in a 300 MB one. Targets up to 32 MB keep their eighth.tex_zero_fill,clear_rect_zero). The shared zero buffer stays at 4 MB; it had grown to 64 MB and stayed resident.pkg_sweep;small_working_set, a Metal working set under 6 GB). The rules are those of the memory-pressure trim (30 s unused; pinned, GPU-written, placed and sparse ones kept). Until now each area's textures stayed until the next pressure warn.DM_NVN_TEX_EVICT_IDLE=0/1forces it off or on. Macs keep waiting for pressure.No profile key, no per-game threshold. Docs: four entries in
docs/reference/memory.mdand the chunk entry innvn-objects.md; the symptom index is regenerated; the game note carries the Kakariko holders.Measurements
Mac (16 GB, the iPhone's memory settings: guest in files, window spans, smaller textures, dynamic resolution, MetalFX, encode thread, 30 fps cap), from the owner's phone save, the same route each time (Kakariko, out to Hyrule Field, back). Residency set in MB; the Mac's set carries about 560 MB of
c1banks the phone does not have.0x18cpool: 305 MB before, 31 MB after (two chunks).iPhone 17 (build
oot-build-vis, before): set 3,190 to 3,398 MB in Hyrule Field, 3,704 to 3,939 MB in Kakariko; wired 5.43 to 5.69 GB, 127 to 173 MB free; one memory-pressure warn at the village load (trim 290 MB). Holders in Kakariko (3,729 MB):0x18cpool among them)iPhone 17, after, run unattended and back to back on 2026-11-13:
iphone_drive_batch.shpicks Continue into Kakariko, then turns the camera for 7 minutes. Baselineoot-build-visagainstoot-build-visplus this PR's commits.regress.py check
scripts/regress.py check --baseline b8cb0bd6 --candidate 125d2a78 --games 0400486024DEC000, Apple M4, macOS 27.2. Only Ocarina of Time, the game the batch is about. The sweep runs only on small devices, so no Mac run takes it. Kind 5, the row-piece zero fill, the heap slack and the chunk threshold reach every NVN game. A check on the other local NVN games (Echoes of Wisdom, Super Mario 3D All-Stars) is still pending: the Mac is shared with three other sessions of this batch tonight.The two flagged screenshots (350 s and 385 s) are the title screen in both runs, whole, with its rotating backdrop on different scenes: forest at night against the desert, the ranch by day against Lake Hylia. Nothing else differs.
Tests
make -C dm/gfx/tests GAME=0400486024DEC000, all passing:t_gpu_only_addr(new)t_zero_colour(new)t_rt_recyclet_pkg_evictt_pool_chunkst_image_aliast_window_aliast_addr_knownt_zero_deptht_clear_dst_clear_text_file_wiret_upload_srcpython3 -m unittest dm/tools/tests/test_docs.pypasses.