Skip to content

Ocarina of Time's chained upscaler (agl_ndsr_vr) through MetalFX - #1951

Open
AdyaGMD wants to merge 5 commits into
seedds:mainfrom
AdyaGMD:oot-ndsr-vr-metalfx
Open

AdyaGMD wants to merge 5 commits into
seedds:mainfrom
AdyaGMD:oot-ndsr-vr-metalfx

Conversation

@AdyaGMD

@AdyaGMD AdyaGMD commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

What and why

Ocarina of Time (0400486024DEC000) upscales with agl's chained upscaler, agl_ndsr_vr.pck / agl_ndsr_vr_light.pck. That's four compute dispatches a frame where Donkey Kong Bananza's agl_ndsr.pck uses one:

  1. The first reads the frame's colour (1440x810 R11G11B10F in handheld mode) and writes a quarter-size image and two storage buffers.
  2. The second and third pack that into two R32Uint images.
  3. The network reads the last of them and writes the 1920x1080 output.

DM_UPSCALER=metalfx (the launcher's Game's upscaler) only named agl_ndsr.pck's shaders, so it did nothing in this game. On an M1 the chain took about 3.4–4.4 ms of every frame.

This change:

  • game.py finds the chain's shaders in the shader map (DM_GAME_UPSCALER_CHAIN_KEYS).
  • up_chain_dispatch draws the network with MetalFX's spatial scaler from the colour the first dispatch read, using the network's own rectangles (up_mfx_encode_from), and leaves out the three stages before it.
  • The network is the dispatch with one texture and one image where the image is larger than the texture. The first stage has the same shape but writes a smaller image.
  • A network variant whose code has no colour curve is linear, so it goes through MetalFX in HDR mode. A variant with an sRGB curve runs the chain.
  • The stages are left out only while the previous frame's network was drawn by MetalFX and this frame's colour is known. If MetalFX refuses, no frame is lost: the chain runs whole from the next frame on, and the left-out stages of that frame are logged in red.

The option is the existing launcher setting Game's upscaler (DM_UPSCALER), so players choose it. It changes the look of the upscaled picture: MetalFX's spatial scaler instead of the game's network.

The game note (dm/games/0400486024DEC000/README.md) describes the chain.

Checked

  • Ocarina of Time 0400486024DEC000 v131072, Mac app, M1 (8-core GPU), macOS 26, handheld mode, 30 fps target. Played through the opening and some gameplay.
    • Log: upscaler chain fa3033d05c0a41d6_b: the network, no colour curve: MetalFX in HDR mode, then drawn by MetalFX from the chain's colour, 1440x810 … → 1920x1080. The three earlier stages are left out. No red lines.
    • The second submission's compute fell from about 4.4 ms to 0.5 ms a frame.
    • Gameplay went from 24–25 fps to 27–30 fps, measured together with the scene-pass change (separate PR).
  • Tests: dm/tools/tests/test_game_profile.py covers the new chain keys (test_chain_from_shader_map). It and the GPU tests were not run locally; CI runs the tool tests.
  • Not checked: other games using agl_ndsr_vr (none known besides this one), and iPhone.
  • No shader-chain change.

Docs

  • The finding is in the game note (dm/games/0400486024DEC000/README.md)
  • No game files, keys or data extracted from a game in the change or the attachments

This touches dm/games/game.py and test_game_profile.py next to the movie-enhancement PR. Whichever lands second needs a small rebase.

🤖 Generated with Claude Code

AdyaGMD and others added 2 commits November 9, 2026 09:35
… only agl_ndsr.pck's shaders, none of which this game runs; its agl_ndsr_vr chain (three stages packing the frame's colour, then the network to the output) is now drawn by the spatial scaler from the chain's first input, with the network's rectangles, and the stages before it are left out while the previous frame's network was drawn; a network variant with an sRGB curve, or a draw up_mfx_encode refuses, runs the chain

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…than its texture: the chain's first stage also reads one texture and writes one image (a quarter-size one), so it was taken for the network, the frame's colour was never recorded and MetalFX never drew; and the colour is taken only from a texture that is not an integer one (the packed intermediates are R32Uint). In Ocarina of Time on an M1 the network fa3033d05c0a41d6 is now drawn by MetalFX (1440x810 to 1920x1080) and the three stages before it are left out: the second submission's compute fell from about 4.4 ms to 0.5 ms a frame, gameplay from 24-25 to 27-30 fps at the 30 fps target

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) tools Tools and scripts (dm/tools, scripts) docs Documentation game: 0400486024DEC000 A game labels Nov 9, 2026
AdyaGMD and others added 2 commits November 9, 2026 11:40
…-upscaler declaration next to main's nn::ndve movie-engine one

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aler early return stays before main's threadgroup-spill setup in dispatch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bquilty25

Copy link
Copy Markdown

60 FPS Validation on Apple Silicon:

Tested this change over an extended 24,000+ frame (~7 minute) gameplay session in The Legend of Zelda: Ocarina of Time (0400486024DEC000 v131072) with a 60 FPS target (DM_FPS_TARGET=60 and DM_UPSCALER=metalfx):

  • Spatial Scaler GPU Timing: Present command buffer GPU time consistently measures p50 = 0.318–0.512 ms, p95 = 0.623–0.912 ms (down from ~4.4 ms compute with the full chain).
  • Frame Pacing: Sustained 59.8–60.0 FPS with 0 dropped frames across Kokiri Forest and area transitions.
  • Stability: The 3 intermediate dispatches are cleanly elided without visual degradation or missing passes.

Note on CI Failure (lint-c/clang-tidy):
Clang-tidy is failing on line 15036 of nvn_mtl.m:

if (!cs->up_color && DM_GAME_UPSCALER_CHAIN_N && up_mfx_on() && up_chain_named(cs))

Because DM_GAME_UPSCALER_CHAIN_N expands to a constant macro integer, clang flags it as [-Wconstant-logical-operand]. Changing it to DM_GAME_UPSCALER_CHAIN_N > 0 resolves the warning and should turn the CI check green.

…on-null stamp and directory, so clang-tidy's nullability check sees a non-null path; its count in this file went from 2 to 3 on main's latest code, and the ratchet failed every pull request on that base

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ivan-leschinsky pushed a commit to ivan-leschinsky/nxnative that referenced this pull request Nov 9, 2026
Ported from seedds#1951 (open upstream) onto this branch: its own changes from its merge base with main, conflicts resolved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U35V38c6UYLbzyyPx6Up7P
ivan-leschinsky pushed a commit to ivan-leschinsky/nxnative that referenced this pull request Nov 9, 2026
…MetalFX or the skip leaves sharpened (DM_SHARPEN, contrast-adaptive, 0 to 1)

DM_UPSCALER=off leaves the game's upscaler network out: up_mfx_encode_from draws its input rectangle bilinearly into its output rectangle instead of encoding MetalFX, for the single network (Donkey Kong Bananza, Splatoon Raiders) and for Ocarina of Time's chained agl_ndsr_vr alike (up_chain_dispatch, from the chain's colour, the stages before it left out as with MetalFX). A network that reads sRGB-encoded colour keeps running, as the bilinear draw would not decode it. DM_SHARPEN draws the result through copy.metal's new fsh (AMD CAS on the four neighbours) on its way into the game's image: after MetalFX the copy out of the scaler's texture becomes the sharpening draw (the straight path is not taken then), after the skip a scratch RGBA16F picture of the output size sits between the two draws. With the game's own network it does nothing. Both are in the launcher (Game's upscaler: Skipped; Sharpening), docs/settings.md and docs/env_switches.tsv.

From EmmanueleC's swordboiMacNative (5f08715, Fixed resolution ... skipped upscaler + DM_SHARPEN): the skip and the sharpening only, rewritten for this branch's upscaler path (main's plus seedds#1951's chain); its fixed resolution is left out, as main's render scale and dynamic-resolution stub already hold Ocarina of Time at a fixed size.

Co-Authored-By: EmmanueleC <emmanuelec@affarinoro.net>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U35V38c6UYLbzyyPx6Up7P
@fheitzer

Copy link
Copy Markdown

Measured on an M1 Max (32-core GPU, applegpu_g13s, 32 GB, macOS 27.0.1), Ocarina of Time v131072, main f02d047 plus #1970 and #1901, handheld (where the game runs agl_ndsr_vr_light), 60 fps target, scripts/bench.py.

On main without this PR, DM_UPSCALER=metalfx does nothing on this game (46.1–46.6 fps either way, GPU 19.3 ms): the upscaler keys come only from agl_ndsr.pck.

With it, the game's upscaler goes through MetalFX (3 runs each, alternating):

fps frame p50 / p95 GPU span p50 frames shown 2 vsyncs
combined, DM_UPSCALER=game 49.7 (standing part of kokiri-deck) 20.2 / 22.8 ms 23.6 ms 20–21%
combined, DM_UPSCALER=metalfx 53.3 (standing part) 18.5 / 21.2 ms 20.4 ms 12–13%

On a 2-minute standing route (same view at any frame rate): main 47.15 fps → combined with metalfx 52.9 fps, frame p99 23.2 → 21.3 ms. On the full kokiri-deck window the metalfx runs reach 53.1 fps, but Link's path diverges at the higher frame rate (he ends on a ledge instead of in the pond), so I quote the standing numbers. The per-encoder profile before it showed the network as a 2.46 ms compute block per frame on this GPU. The picture looks right in the screenshots (colours, no washed-out image). It conflicts with main in two places (main's ndve_engine_dispatch declaration next to up_chain_dispatch, and main's threadgroup-spill setup in dispatch); keeping both, with the chain's early return before the spill setup as your merge commit says, builds and runs.

dpadr pushed a commit to dpadr/nxnative that referenced this pull request Nov 11, 2026
…ale of the game's is drawn by MetalFX, or when MetalFX scales straight between the game's textures

Without the upscale the hook (up_mfx_encode_from, seedds#1951's chained-upscaler path included) is never reached and nothing was generated, silently:
devMlGUE's report on seedds#2279. Compile-checked on main ccd834e + seedds#1951; not run.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) tools Tools and scripts (dm/tools, scripts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants