Skip to content

A game's own upscaler drawn by MetalFX, the player's choice in the launcher: Donkey Kong Bananza's NDSR network 3.5-5 ms → about 1 ms a frame on an M1 - #1086

Closed
AdyaGMD wants to merge 1 commit into
seedds:mainfrom
AdyaGMD:metalfx-upscaler
Closed

AdyaGMD wants to merge 1 commit into
seedds:mainfrom
AdyaGMD:metalfx-upscaler

Conversation

@AdyaGMD

@AdyaGMD AdyaGMD commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

What and why

Donkey Kong Bananza upscales its frame with agl's NDSR network (agl_ndsr.pck): one compute dispatch between the render-resolution passes and the output-resolution ones, which then draw the HUD over it. On an M1 in handheld mode (960x540 → 1920x1080) it is the largest single item of a heavy frame, 3.5–5 ms (DM_NVN_GPUPROF), because its HMMA products are emulated and its shared-memory matrix loads move their row addresses between lanes with shuffles (41 % of its time). MetalFX's spatial scaler does the same job, same sizes, in about 1 ms.

A new launcher option: Game's upscaler. The menu before the game starts gets a Game's upscaler row with Game's own (the default) and MetalFX. It appears only for games whose profile names their upscaler shaders (the new profile key metalfx_upscaler; today only Bananza, its 24 NDSR compute variants), so other games' menus do not change. It is a choice and not a default because the picture is then Apple's scaler rather than the game's trained network. It is not the same as the existing Upscaling row: that one scales the finished picture, HUD included, to the screen; this one replaces the game's own step inside the frame. Both can be on. docs/settings.md explains it for players; the variable is DM_UPSCALER=game|metalfx.

How it works (up_mfx_encode in nvn_mtl.m). A dispatch of a named shader is replaced by MetalFX's Metal 4 spatial scaler in HDR mode, from the input rectangle to the output rectangle the shader would use (its push constants c[24]: origin and inclusive last texel of each). Bananza's colour is R11G11B10F, which MetalFX does not take, so the input rectangle is drawn into an RGBA16F scaler texture and the result drawn into the output rectangle (copy_t2t_draw); RGBA16F textures with whole rectangles go straight through. MetalFX is encoded through rec_cb_block (new in nvn_rec.h), so on the encoding thread (iOS's default) it gets the real command buffer. The game's shader still runs for an output not larger than the input, an indirect dispatch, an output texture that is not a render target, or a size MetalFX refuses.

Credit: the rectangle decoding, HDR mode and the encoding-thread handling follow @givethesourceplox's NDSR replacement in #1073 (credited as co-author). This PR differs from that one in being a player setting in the launcher, covering every NDSR variant and size ratio, and encoding on the Metal 4 queue instead of a Metal 3 queue synchronized by events.

Checked

  • Tests: make -C dm/gfx/tests GAME=040074A01BF12000 check_t_upscaler_mfx: a named shader's 960x540 → 1920x1080 dispatch is drawn by MetalFX; a cropped R11G11B10F input with a highlight above 1.0 into an offset output rectangle, every pixel inside and outside checked; an output no larger than the input and an unnamed shader keep the shader; nothing changes without the switch; also on the encoding thread. dm/tools/tests/test_game_profile.py (the key's values) and test_settings.py (the row shows for Bananza with "Game's own" as default, not for NieR:Automata); test_docs.py.

  • Games run, with title ID and version: Donkey Kong Bananza 040074A01BF12000 v458752, handheld, the same heavy area, on a build with this change, the border-colour change (Pixel shaders without the border-colour correction in their main library: it spilled registers in every pixel where it never ran #1032) and mesh draw runs (a separate PR):

    Game's upscaler GPU ms a frame frame ms fps
    Game's own (NDSR) 17.37 19.44 51.4
    MetalFX 15.32 16.6 60

    No visual difference noticed between the two in play.

    The launcher's new row (shown only for Bananza):

    The launcher's Game's upscaler row with Game's own and MetalFX

    The same heavy area with the frame-rate counter, the game's own upscaler (NDSR):

    Donkey Kong Bananza with the game's own upscaler

    With MetalFX:

    Donkey Kong Bananza with MetalFX Memory was flat over a long MetalFX run (resident +15 MB between frames 8,100 and 11,100; one texture usage upgrade in the whole run), after an earlier build that upgraded the output texture every frame was fixed by keeping the shader for a non-render-target output.

  • Devices and OS: MacBook Pro (13-inch, M1, 8-core GPU), macOS 26.6.2, Xcode 27.2.

  • scripts/regress.py: not run. The shader chain does not change; the host change only acts for a game with metalfx_upscaler and DM_UPSCALER=metalfx.

  • Not checked: the iPhone. One open note (in the reference entry): a standalone MetalFX test on the M1 intermittently produced invalid pixels at small sizes, and one handheld run showed one wrong HDR pixel; not seen in play since.

  • Known Metal-side quirk: MetalFX's Metal 4 scaler stops under Metal validation (MTL_DEBUG_LAYER=1) on a barrier-stage setting it does not expose (_outputTextureBarrierStages not set), so that test runs without validation; Donkey Kong Bananza on iPhone: border corrections out of the main pixel libraries, NDSR replacements, pacing and fence fixes #1073 reports the same.

Docs

  • The finding is in its docs/reference/rendering.md entry ("A game's own upscaler drawn by MetalFX"), the game note and docs/settings.md; python3 dm/tools/docs_symptoms.py was run
  • A chain tool changed (no)
  • No game files, keys or data extracted from a game in the change or the attachments

🤖 Generated with Claude Code

…uncher: Donkey Kong Bananza's NDSR network 3.5-5 ms → about 1 ms a frame on an M1

Donkey Kong Bananza upscales its frame with agl's NDSR network, one compute dispatch between the render-resolution passes and the output-resolution ones. On an M1 in handheld mode
(960x540 → 1920x1080) it takes 3.5-5 ms of the heavy areas' frame: its HMMA products are emulated and its shared-memory matrix loads move their row addresses between lanes with
shuffles. MetalFX's spatial scaler does the same job in about 1 ms.

The profile key metalfx_upscaler names a game's upscaler shaders (Bananza: the 24 NDSR compute variants). For such a game the launcher shows Game's upscaler (DM_UPSCALER: Game's own,
the default, or MetalFX). With MetalFX, dispatch gives those shaders' dispatches to up_mfx_encode: MetalFX's Metal 4 spatial scaler in HDR mode from the input rectangle to the output
rectangle (the push constants c[24]: origin and inclusive last texel of each), through RGBA16F scaler textures when the game's formats or rectangles need it (Bananza's R11G11B10F),
straight otherwise. MetalFX is encoded through rec_cb_block, so on the encoding thread it gets the real command buffer. An output not larger than the input, an indirect dispatch, an
output texture that is not a render target, or a size MetalFX refuses runs the game's shader.

The rectangle decoding, HDR mode and the encoding-thread handling follow the NDSR replacement in the iPhone port's PR seedds#1073.

Co-authored-by: givethesourceplox <272928635+givethesourceplox@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) tools Tools and scripts (dm/tools, scripts) docs Documentation game: 040074A01BF12000 Donkey Kong Bananza labels Nov 3, 2026
seedds added a commit that referenced this pull request Nov 3, 2026
…uncher: Donkey Kong Bananza's NDSR network 3.5-5 ms → about 1 ms a frame on an M1 (#1086)

The launcher's new Game's upscaler row (DM_UPSCALER=game|metalfx) is shown for games whose
shader map holds agl's NDSR network: game.py takes the compute shaders found in agl_ndsr.pck
(Bananza's 24, Splatoon Raiders' 20) into DM_GAME_UPSCALER_KEYS, with no profile key. With
MetalFX, up_mfx_encode replaces such a dispatch with MetalFX's Metal 4 spatial scaler in HDR
mode between the rectangles of its push constants, through RGBA16F where the game's formats
or rectangles need it; anything else keeps the game's shader.

MetalFX encodes on the real command buffer through rec_cb_block (nvn_rec.h), on the encoding
thread when it runs. NVN2's DLSS (dlss_eval) now goes the same way: on the encoding thread it
stopped the host at its first evaluation (DmRecCB has no stand-in for -useResidencySet:);
check_t_dlss_mfx covers it with the thread off and on.

Co-authored-by: Adya <114347520+AdyaGMD@users.noreply.github.com>
Co-authored-by: givethesourceplox <272928635+givethesourceplox@users.noreply.github.com>
@seedds

seedds commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Landed in 85468b6, with you and givethesourceplox as co-authors. Thank you for this. The MetalFX path, the RGBA16F conversion, the fallbacks and rec_cb_block are yours as written. Changes on top:

  • No profile key. The game's upscaler shaders now come from the shader map: upscaler_keys in dm/games/game.py takes the compute shaders found in agl_ndsr.pck. For Bananza that gives exactly your 24 keys. Splatoon Raiders ships the same network (20 shaders) and now gets the row too; the swap hasn't been run on it. The header macros are now DM_GAME_UPSCALER_KEYS / DM_GAME_UPSCALER_N.
  • DLSS through rec_cb_block as well. dlss_eval encoded MetalFX on the recording's stand-in command buffer. With DM_NVN_ENC_THREAD=1 it stopped the host (DmRecCB has no stand-in for -useResidencySet:). It now encodes through your block, with the scaler's settings made inside it. The new test check_t_dlss_mfx runs it with the thread off and on.
  • check_t_upscaler_mfx also runs with the encoding thread on.
  • Docs: env_switches.tsv order (its test failed), a settings.tsv trailing tab, apostrophes and the make -C command in the reference entry, and a link to the open LDSM/MOVM item in "Ampere HMMA through simdgroup_matrix". The network's own cost on the default path is still there. I also dropped the "sharpens edges" line from settings.md, since it wasn't measured.

Not checked: the iPhone, and Raiders with MetalFX.

@seedds seedds closed this Nov 3, 2026
seedds added a commit that referenced this pull request Nov 3, 2026
…ing them: lint-c's ratchet passes again (seven nullability findings from #1086, #1084 and #1073, whose own runs were cancelled)
seedds added a commit that referenced this pull request Nov 3, 2026
…e's first upscaler key (DM_GAME_UPSCALER_KEYS), not Donkey Kong Bananza's 7629d857a0a93833, and a profile without upscalers swaps none; make run failed it for every other game (#1086)
Know-It-All-Brother pushed a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…uncher: Donkey Kong Bananza's NDSR network 3.5-5 ms → about 1 ms a frame on an M1 (seedds#1086)

The launcher's new Game's upscaler row (DM_UPSCALER=game|metalfx) is shown for games whose
shader map holds agl's NDSR network: game.py takes the compute shaders found in agl_ndsr.pck
(Bananza's 24, Splatoon Raiders' 20) into DM_GAME_UPSCALER_KEYS, with no profile key. With
MetalFX, up_mfx_encode replaces such a dispatch with MetalFX's Metal 4 spatial scaler in HDR
mode between the rectangles of its push constants, through RGBA16F where the game's formats
or rectangles need it; anything else keeps the game's shader.

MetalFX encodes on the real command buffer through rec_cb_block (nvn_rec.h), on the encoding
thread when it runs. NVN2's DLSS (dlss_eval) now goes the same way: on the encoding thread it
stopped the host at its first evaluation (DmRecCB has no stand-in for -useResidencySet:);
check_t_dlss_mfx covers it with the thread off and on.

Co-authored-by: Adya <114347520+AdyaGMD@users.noreply.github.com>
Co-authored-by: givethesourceplox <272928635+givethesourceplox@users.noreply.github.com>
Know-It-All-Brother pushed a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…ing them: lint-c's ratchet passes again (seven nullability findings from seedds#1086, seedds#1084 and seedds#1073, whose own runs were cancelled)
Know-It-All-Brother pushed a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…e's first upscaler key (DM_GAME_UPSCALER_KEYS), not Donkey Kong Bananza's 7629d857a0a93833, and a profile without upscalers swaps none; make run failed it for every other game (seedds#1086)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 040074A01BF12000 Donkey Kong Bananza nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) tools Tools and scripts (dm/tools, scripts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants