Skip to content

Enable compiler optimizations and LTO for macOS host, libvpx and app launcher - #556

Closed
relay4 wants to merge 3 commits into
seedds:mainfrom
relay4:pr/compiler-optimizations
Closed

relay4 wants to merge 3 commits into
seedds:mainfrom
relay4:pr/compiler-optimizations

Conversation

@relay4

@relay4 relay4 commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

What and why

Enable speed optimization and full link-time optimization for the macOS host, libvpx and app launcher. The host previously used -O1 without LTO, while libvpx and the launcher's C code used -O2.

  • Build host C/Objective-C and libvpx with -O3 -flto=full. NATIVE_OPT controls the host flags; VPX_OPT inherits them and can be overridden independently.
  • Add Makefile dependencies so changes to the default flags rebuild the host and libvpx objects.
  • Build the launcher's C code with -O3 -flto=full, and Swift with -O -whole-module-optimization -lto=llvm-full.
  • Document the compiler policy and comparison-build overrides in Host compiler settings.

These flags optimize host code; games continue executing their existing ARM64 instructions natively. Offline Metal shaders retain their existing -O1 policy.

Checked

  • Builds and tests: optimized Wonder host and libvpx build; C/Swift launcher build; libvpx objects verified as LLVM bitcode. Eight CPU graphics tests pass with -O3 -flto=full; 47 targeted Python tests pass. Python lint reports zero new findings against the baseline.
  • Game run: Super Mario Bros. Wonder (010015100B514000), version 1.2.1. Compared the opening Flower Kingdom dialogue with a frozen save, scripted Blue Toad selection, matching shaders and matching draw/pass counts. This covers a stationary rendered scene, not a full gameplay playthrough.
  • Device and OS: Apple M1 Pro, macOS 27.2; Apple Clang 21.0.0.
  • Regression coverage: attempted scripts/regress.py check --games for Wonder, Tears of the Kingdom, Cuphead, Hades and Balatro, covering Nintendo/Unity NVN, Vulkan and OpenGL paths. The harness stopped because only Wonder had a local Mac build; the other four games remain unchecked. The full GPU suite and iOS runtime were not validated.

Host measurements

Both comparisons isolate host flags: -O1 versus -O3 -flto=full, with identical game code and shaders. The live-game comparison also holds the libvpx archive at the same -O2 build.

Live game, capped at 60 FPS: four AB/BA pairs, 1,200 measured frames per run (4,800 per build), handheld mode. Values below are medians of run means.

Metric per frame Before After Reduction
Submit CPU time 3.439 ms 3.337 ms 2.99%
Whole-process CPU time 12.446 ms 12.228 ms 1.75%
Submit instructions 11.218 M 10.628 M 5.27%

Instruction reduction was consistent, but CPU timings varied with recorded clock estimates; this run does not establish a repeatable timing gain. Both builds stayed near the 60 FPS cap, with GPU p95 below 8 ms.

Headless frame replay: eight AB/BA pairs, 10 discarded warm-up iterations and 40 measured iterations per process (320 measured replays per build). Metal encoding and uniform copies run on the measured thread at the same QoS. Values below are geometric means of run means.

Metric per replay Before After Reduction
Submit CPU time 13.174 ms 13.021 ms 1.16%
Submit instructions 148.251 M 146.056 M 1.48%
Submit CPU cycles 42.362 M 41.879 M 1.14%

All eight pairs favored the optimized build. The paired run-level bootstrap 95% interval for CPU-time reduction is 0.72–1.59%. Clock estimates stayed near 3.22 GHz. All 800 replays, including warm-ups, matched the captured image bit for bit: 435 draws, 59 passes, no RED diagnostics.

The CPU region excludes state restoration, explicit GPU completion waits and image verification; measured internal waits were zero. However, restoring captured memory marks texture storage dirty and causes extra uploads inside the measured region. This is a repeatable improvement for the replay workload, not an estimate of a whole-game speedup. Its absolute costs and instruction counts are not directly comparable to the live-game run. Neither comparison demonstrates an FPS improvement.

libvpx measurements

Compared native -O2 objects against -O3/full-LTO bitcode with the same benchmark caller: 60 synthetic 1280×720 motion frames per codec, eight timed decodes after a validation/warm-up decode, four process samples per setting in balanced order.

Codec Decoder threads Before (decoded frames/s) After (decoded frames/s) Throughput change
VP8 1 1311.40 1321.74 +0.79%
VP9 1 1013.97 1022.35 +0.83%
VP8 4 1315.00 1337.67 +1.72%
VP9 4 2224.40 2247.55 +1.04%

Decoded-plane checksums match between builds. These small fixture-specific differences do not establish faster in-game movies. Launcher performance was not measured.

Docs

  • Updated docs/development.md with the host, codec and launcher compiler settings and their scope.
  • No shader-chain tools changed; chain regeneration is not applicable.
  • No game files, keys or data extracted from a game in the change or the attachments.

@relay4
relay4 force-pushed the pr/compiler-optimizations branch 2 times, most recently from 48eeda6 to 3f053c0 Compare October 30, 2026 11:53
@Xipzer

Xipzer commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

A finding after merging this with current main: -flto=full breaks the big-stack entry from #721. Breath of the Wild 1.9.0 crashes on the Mac with SIGBUS at dm_bigstack_enter+0x70 right after oe::SetFocusHandlingMode in nnMain, every run. -O3 without LTO runs clean, and so does -O1. dm_bigstack_top is only called from the entry's inline assembly; marking it __attribute__((used)) did not fix it, so I dropped LTO and kept -O3. On this Mac, -O3 + LTO against -O1 had cut submit+present from 4.80 to 4.55 ms on BotW before #721.

@relay4
relay4 force-pushed the pr/compiler-optimizations branch from 3f053c0 to 81ae214 Compare November 2, 2026 10:23
@relay4
relay4 force-pushed the pr/compiler-optimizations branch from 81ae214 to 0179010 Compare November 2, 2026 10:29
@relay4

relay4 commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

@Xipzer thank you for reporting this. I think this should be fixed in the newest commit.

@github-actions github-actions Bot added host The Horizon host: SVCs, services, threads, memory (dm/host) nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers ios The iPhone and iPad app (dm/ios) mac The Mac front end and app (dm/mac, front_mac.m) docs Documentation shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders labels Nov 3, 2026
@moretti

moretti commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

A data point for this: on Donkey Kong Bananza, building the host with just -O2 instead of -O1 greatly improves performance in its heaviest scenes.

These are the second world's dialog close-ups, with 21,000+ draws a frame, on a Mac M5 Pro in fullscreen, measured with the host profiler (#1041). The frame-rate controller patch from #1765 and the encode thread were on in both runs.

-O1 -O2
fps in the close-ups 55.6 58.6
seconds under 57 fps 25 of 46 2 of 34
frame time, p50 17.8 ms 16.8 ms
host CPU a frame 13.1 ms 11.7 ms
host CPU per 1,000 draws 0.588 ms 0.526 ms

No [RED] lines appeared in the run. The submit thread's hot spots were encode_args_ and draw_, so the per-draw path gains the most from optimisation.

@relay4

relay4 commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

@moretti thank you for measuring the performance.

Possibly someone can test if this also helps with OOT performance, since that is also a Switch 2 title like DKB. If so, that person could mention this PR in the OOT performance discussion #1789.

seedds added a commit that referenced this pull request Nov 10, 2026
…g Bananza's heaviest close-ups go from 55.6 to 58.6 fps (PR #556)

The host was the only native code still at -O1 (libvpx, dav1d and the launcher build at -O2; the iOS host at Xcode Release's -Os). Its per-draw path (encode_args_, draw_) runs some 21,000 times a frame in Donkey Kong Bananza's second-world dialogue close-ups; there, on an M5 Pro, moretti measured host CPU 13.1 -> 11.7 ms a frame (0.588 -> 0.526 ms per 1,000 draws), 55.6 -> 58.6 fps, and 2 of 34 s under 57 fps instead of 25 of 46. On Mario Wonder's opening scene, held at the 60 fps cap, relay4 measured 5% fewer submit instructions (at -O3 + LTO).

Not -O3: no measurement shows a gain over -O2. Not -flto: it separates hle_bigstack.c's inline-assembly entry from its return path (Xipzer: SIGBUS in dm_bigstack_enter on Breath of the Wild 1.9.0, every run). libvpx, the launcher and the Metal pipeline keep their levels: no measured symptom. The Makefile is now a prerequisite of dmhost, so a change to CFLAGS rebuilds the host.

Checked: the host builds with no new warnings; the host unit tests, test_lint_c and test_docs pass. regress.py check on OCTOPATH TRAVELER II (M4 Pro, macOS 27.2): fps 59.8 -> 59.8, p99 18.27 -> 18.21 ms, screenshots within baseline noise, no new RED, first frame 21.0 -> 18.5 s; its prebuild 20.5 -> 59.4 s was taken while other sessions' Metal compiles ran (the harness marked the Mac busy) and is Apple's compiler, not host code; not re-measured idle. Mario Kart World did not run (its shader package and map come from different splits, on main too).

Co-authored-by: relay4 <255982243+relay4@users.noreply.github.com>
@seedds

seedds commented Nov 10, 2026

Copy link
Copy Markdown
Owner

Landed the host part only, as c4ffb9d: the Mac host now builds at -O2, and the Makefile is a prerequisite of dmhost so a flag change rebuilds it. relay4 is credited as co-author.

Thanks to @moretti for the Donkey Kong Bananza numbers (55.6 → 58.6 fps, host CPU 13.1 → 11.7 ms a frame in the close-ups), which are what this lands on, and to @Xipzer for finding the LTO crash in dm_bigstack_enter.

Left out:

  • -O3: nothing measured shows a gain over -O2. If a heavy scene shows one, it's a one-line follow-up.
  • -flto: it separates the big-stack entry's inline assembly from its return path. The .alt_entry fix was never retested on Breath of the Wild, and LTO has no measured gain over -O2 alone.
  • libvpx, the launcher and the Metal pipeline: no measured symptom (the Metal change would also have forced shader rebuilds).

Checked: OCTOPATH TRAVELER II through regress.py check on an M4 Pro: same fps, frame times and picture as main, no new red lines.

@seedds seedds closed this Nov 10, 2026
@relay4

relay4 commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for merging the -O2 part.

I still think there's potential in -O3 and LTO for the host code, though I haven't measured these separately yet.

Fasen24-AI pushed a commit to Fasen24-AI/swordboiMacNative that referenced this pull request Nov 11, 2026
…g Bananza's heaviest close-ups go from 55.6 to 58.6 fps (PR seedds#556)

The host was the only native code still at -O1 (libvpx, dav1d and the launcher build at -O2; the iOS host at Xcode Release's -Os). Its per-draw path (encode_args_, draw_) runs some 21,000 times a frame in Donkey Kong Bananza's second-world dialogue close-ups; there, on an M5 Pro, moretti measured host CPU 13.1 -> 11.7 ms a frame (0.588 -> 0.526 ms per 1,000 draws), 55.6 -> 58.6 fps, and 2 of 34 s under 57 fps instead of 25 of 46. On Mario Wonder's opening scene, held at the 60 fps cap, relay4 measured 5% fewer submit instructions (at -O3 + LTO).

Not -O3: no measurement shows a gain over -O2. Not -flto: it separates hle_bigstack.c's inline-assembly entry from its return path (Xipzer: SIGBUS in dm_bigstack_enter on Breath of the Wild 1.9.0, every run). libvpx, the launcher and the Metal pipeline keep their levels: no measured symptom. The Makefile is now a prerequisite of dmhost, so a change to CFLAGS rebuilds the host.

Checked: the host builds with no new warnings; the host unit tests, test_lint_c and test_docs pass. regress.py check on OCTOPATH TRAVELER II (M4 Pro, macOS 27.2): fps 59.8 -> 59.8, p99 18.27 -> 18.21 ms, screenshots within baseline noise, no new RED, first frame 21.0 -> 18.5 s; its prebuild 20.5 -> 59.4 s was taken while other sessions' Metal compiles ran (the harness marked the Mac busy) and is Apple's compiler, not host code; not re-measured idle. Mario Kart World did not run (its shader package and map come from different splits, on main too).

Co-authored-by: relay4 <255982243+relay4@users.noreply.github.com>
(cherry picked from commit c4ffb9d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation host The Horizon host: SVCs, services, threads, memory (dm/host) ios The iPhone and iPad app (dm/ios) mac The Mac front end and app (dm/mac, front_mac.m) nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants