Skip to content

nvn/upscaler: MetalFX takes an upscaler with square-root colour - #2366

Open
roandegraaf wants to merge 4 commits into
seedds:mainfrom
roandegraaf:upscaler-sqrt
Open

roandegraaf wants to merge 4 commits into
seedds:mainfrom
roandegraaf:upscaler-sqrt

Conversation

@roandegraaf

@roandegraaf roandegraaf commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Rebased on main at 034e91a (2026-11-12). Conflicts only in the game note and the profile keys table in dm/games/README.md (main's newer draw_barriers row kept, upscaler_sqrt after it); the code applied as it was. #2388 is stacked on this PR.

Overlaps #1951 and #1459 (same upscaler, other approaches), and touches the same code as #2340:

What and why

Problem. DM_UPSCALER=metalfx replaced no dispatch on 0400486024DEC000 in play (GPU 15.35 against 15.39 ms a frame). Two reasons:

  • In play the title anti-aliases and upscales with four dispatches of agl_ndsr_vr.pck and agl_ndsr_vr_light.pck, which the shader map's agl_ndsr.pck pattern does not name. Only the last (in the scene measured fa3033d05c0a41d6, texture slot 0 at 1536x864 to image slot 1 at 2048x1152) is the upscale and has the shape up_mfx_encode takes.
  • Its colour has no sRGB curve for up_mfx_color to find: the dispatch before it stores each channel's square root (packed as RGB9E5), and the step writes v * |v|, linear. Read from the translated source.

Change, first commit.

  • Profile key upscaler_sqrt (comma-separated shader keys): the named shaders join DM_GAME_UPSCALER_KEYS and are listed in DM_GAME_UPSCALER_SQRT_KEYS.
  • A third colour shape, UP_COLOR_SQRT: the draw of the input rectangle into the scaler's RGBA16F texture squares the colour, MetalFX runs in HDR mode, the result is drawn into the game's image.
  • DM_NVN_SKIP_CS=<compute key>[,…] (diagnostic): drops those shaders' dispatches, to measure one inside an encoder that holds several.

Change, second commit.

  • Game's upscaler sharpness (DM_UPSCALER_SHARPNESS, 0.0–1.0; in the launcher under Game's upscaler while it is MetalFX, renamed in tools/settings: the upscaling settings say where they act #2388; profile key upscaler_sharpness, 1.0 without one). MetalFX's spatial scaler has no strength setting (its descriptor takes sizes, formats and a colour mode only) and is crisper than agl's networks. The draw of its result into the game's image blends in the scaler's own input, sampled bilinearly, by 1 − sharpness. Not for sRGB-encoded input. At 1.0, every other game's value, the picture is as before.
  • 0400486024DEC000: upscaler_sharpness 0.7, and all eight variants of agl_ndsr_vr_light.pck whose translated code ends in v * |v| are named. The pack's seven others multiply by 12.92 after the square (an sRGB encode) and keep their shader, as do agl_ndsr_vr.pck's fifteen.
  • A per-game key for the shape because it was read by a person; nothing in the code marks it. DM_UPSCALER still defaults to game.

Reference entries: An upscaler with square-root colour, MetalFX's picture is crisper than a game's own upscaler; the numbers are in the game note's Frame rate section.

Checked

  • Tests: make -C dm/gfx/tests GAME=0400486024DEC000 check_t_upscaler_mfx and GAME=040074A01BF12000 (97 checks each, 0 failed after the rebase: a square-root case and a blended one, encode thread off and on); test_game_profile.py, test_settings.py, test_docs.py, test_env_switches.py, test_nsp_chain.py; host build.

  • Games run: 0400486024DEC000 v131072, a GPU-bound village scene from a save, DM_FPS_HOLD=off, 100 s runs.

  • Devices and OS: Apple M4 Pro (20-core GPU), macOS 27.2.

  • Measurements, two runs a side in turn (sharpness 1.0):

    compute per frame GPU per frame
    headless, game's upscaler 3.91, 3.95 ms 14.84, 15.61 ms
    headless, MetalFX 2.80, 2.77 ms 13.64, 13.48 ms
    windowed, game's upscaler 3.90, 3.89 ms 15.59, 15.13 ms
    windowed, MetalFX 2.76, 2.79 ms 14.63, 14.46 ms

    With the dispatch dropped altogether (DM_NVN_SKIP_CS) compute is 2.78–2.80 ms, so the step is 1.12 ms and MetalFX with its two copies about 0.0–0.1 ms. With the final build at the profile's 0.7: compute 2.77 ms (one run), the same.

  • Which variants the game uses (pass dumps, the game's own upscaler): four areas docked and handheld mode fa3033d05c0a41d6; render scale 0.5 d99fbeeb53d38847; with the dynamic resolution on four more on the way to its floor. No encoding variant, none of agl_ndsr_vr.pck and none of agl_ndsr.pck in any of eight situations; the title uses none. Each of the six was then drawn by MetalFX with no red line (nine input sizes, 1536x864 down to 763x429, in the dynamic-resolution run), the end shots' mean colours within 1 of 255 of the game's upscaler in the same mode. Compute per frame, one run a side: 4.01 → 2.58 ms at render scale 0.5, 3.25 → 2.27 ms handheld.

  • Picture: the end shots' mean colours agree with the game's upscaler within 1 of 255. Sharpness was chosen by eye in 2x crops of the same spot against the game's network, among seven variants, one run each: a 30 % blend closest, 60 % softer than the network; the input blurred before the scaler close at half strength and too soft at full; MetalFX's perceptual mode on the square roots hardly different from HDR mode on the squared input.

Not checked / open.

  • The game runs were made on a working tree with this change on a slightly older main (chain b8856126d238), not on this branch's build; on this branch only the tests and the host build ran.
  • Two of the eight named variants (8df148d58b7cbe8e, c8b20adc9d4978e0) were read from their translated code only; no run dispatched them.
  • No regress.py run: only this profile has a build on this Mac. Other games reach the new code through copy_t2t_draw (its pipeline key is now format × 4, its parameter block 48 bytes, one more function in the copy library) and take the new blend only below sharpness 1.0.
  • Stills of one night scene; not judged in motion, nor with a render scale, the dynamic resolution or in a cutscene. No iOS run.

Front ends

Front-ends: new: the settings list has DM_UPSCALER_SHARPNESS (range 0.0–1.0, shown while DM_UPSCALER is metalfx, default from the game profile).

Docs

  • The findings are in the game note and docs/reference/rendering.md, docs/settings.md has the setting, and python3 dm/tools/docs_symptoms.py was run
  • A chain tool changed: no
  • No game files, keys or data extracted from a game in the change or the attachments

@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) tools Tools and scripts (dm/tools, scripts) docs Documentation game: 0400486024DEC000 A game labels Nov 11, 2026
@roandegraaf roandegraaf changed the title MetalFX takes an upscaler with square-root colour: 1.12 ms of GPU a frame to about 0.1 on 0400486024DEC000 MetalFX takes an upscaler with square-root colour, with an Upscaler sharpness setting (0400486024DEC000: 1.12 ms of GPU a frame to about 0.1) Nov 11, 2026
@roandegraaf roandegraaf changed the title MetalFX takes an upscaler with square-root colour, with an Upscaler sharpness setting (0400486024DEC000: 1.12 ms of GPU a frame to about 0.1) MetalFX takes an upscaler with square-root colour, with a Game's upscaler sharpness setting (0400486024DEC000: 1.12 ms of GPU a frame to about 0.1) Nov 11, 2026
dpadr pushed a commit to dpadr/nxnative that referenced this pull request Nov 11, 2026
… roandegraaf) into main for the frame-generation restack

Co-Authored-By: Claude <noreply@anthropic.com>
dpadr pushed a commit to dpadr/nxnative that referenced this pull request Nov 11, 2026
…pscaler sharpness blend (seedds#2366's upscaler_sharpness)

The real frame's upscale blends a bilinear picture of the scaler's input into MetalFX's result; the generated scene was upscaled bare, so with a sharpness
under 1 (Ocarina of Time: 0.7) generated frames would be crisper than real ones. fg_comp blends the generated scene before its upscale in by the same share.
Compile-checked; not run.

Co-Authored-By: Claude <noreply@anthropic.com>
dpadr pushed a commit to dpadr/nxnative that referenced this pull request Nov 11, 2026
… hook is its square-root upscaler's MetalFX draw, with [RED] lines when no upscale is drawn by MetalFX

The draft generated no frame on main (devMlGUE on seedds#2279): its hook is the game's upscale drawn by MetalFX, and main does not draw Ocarina of
Time's upscaler (agl_ndsr_vr_light, square-root colour) with MetalFX. seedds#2366 (roandegraaf) does; this merge carries it. The generated frame takes seedds#2366's
sharpness blend as the real frame does. Compile-checked only (make -C dm/host GAME=0400486024DEC000 dmhost; test_game_profile, test_settings, test_dmsh_abi);
untested on main: no game run.

Co-Authored-By: Claude <noreply@anthropic.com>
DM_UPSCALER=metalfx takes an upscaler whose colour is a square root: the profile key upscaler_sqrt names it, its input is squared into MetalFX's (0400486024DEC000: 1.12 ms of GPU a frame to about 0.1).

The upscale step of agl_ndsr_vr_light.pck samples colour the dispatch before it stored as square roots and writes v * |v|, so its code has no sRGB curve for the classifier and the option replaced nothing. DM_NVN_SKIP_CS (diagnostic) drops named compute dispatches, to measure one inside an encoder.
Upscaler sharpness (DM_UPSCALER_SHARPNESS, the profile's upscaler_sharpness): MetalFX's picture blended with a bilinear one of the same input, 0.7 for 0400486024DEC000, whose eight square-root upscale variants are named.

MetalFX's spatial scaler has no strength setting and is crisper than agl's networks. The draw of its result into the game's image blends in the scaler's input, sampled bilinearly, by 1 - sharpness; 1.0 (every other game) is the picture as before.

Front-ends: new: the settings list has DM_UPSCALER_SHARPNESS (range 0.0-1.0, shown while DM_UPSCALER is metalfx, default from the game profile)
0400486024DEC000's notes: six of the eight named upscale variants seen in play, by input size, and drawn by MetalFX; no other variant dispatched in eight situations.
The sharpness setting is named after its parent in the launcher: Game's upscaler sharpness.
@roandegraaf roandegraaf changed the title MetalFX takes an upscaler with square-root colour, with a Game's upscaler sharpness setting (0400486024DEC000: 1.12 ms of GPU a frame to about 0.1) nvn/upscaler: MetalFX takes an upscaler with square-root colour Nov 12, 2026
Kayyluhh added a commit to Kayyluhh/nxnative that referenced this pull request Nov 14, 2026
…ur, with a Game's upscaler sharpness setting (0400486024DEC000)
Kayyluhh added a commit to Kayyluhh/nxnative that referenced this pull request Nov 14, 2026
…d-texture view sets take views the cache by Metal texture (seedds#2260) finds; seedds#2366's upscaler scales behind mfx_order; t_sbw keeps seedds#1320's and main's cases
Kayyluhh added a commit to Kayyluhh/nxnative that referenced this pull request Nov 14, 2026
…y the Metal texture (seedds#2542) in place of seedds#2260's; the game's upscaler takes the game's textures straight (seedds#2559) except square-root input and a blended output (seedds#2366)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) tools Tools and scripts (dm/tools, scripts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant