Scratch: frame generation for The Legend of Zelda: Ocarina of Time (0400486024DEC000) from scene depth and the game's camera (DM_FRAMEGEN)

dm/games/0400486024DEC000/README.md

@@ -31,6 +31,7 @@ One [NUP container](../../../docs/development.md#the-packing-script-in-detail) o
 Measured on an M6 Mac with 16 GB, headless, on one walk with camera turns (`walk.sh` in the work folder).
 
 - Current on an M5 Pro MacBook Pro (16-inch, 20-core GPU, 24 GB, macOS 27.2, 2026-11-09), `kokiri-deck` from a deck autosave with the full library of chain `dedc9391c30b`, at full resolution (Dynamic resolution off), two runs a variant: in a window a locked 60 fps (59.8–60.0, frame p99 16.8–19.4 ms, GPU p50 10.2 ms). In fullscreen on its ProMotion screen it held 40 fps (39.6–40.1) at the same GPU time, with every present a 120 Hz refresh late and every other link tick skipped: [The 60 Hz link at 40 fps in fullscreen on a ProMotion screen](../../../docs/reference/frame-rate.md#the-60-hz-link-at-40-fps-in-fullscreen-on-a-promotion-screen). With the 120 Hz link there, 58.8–59.6 fps, frame p99 25.3 ms; with the screen held at its fixed 60 Hz mode, 60.0 fps, frame p99 16.9 ms, which fullscreen now switches to. MetalFX present upscaling to the 3456-wide fullscreen cost no measurable GPU time (59.4–59.7 fps). Resident about 10 GB, GPU allocated 11.2–11.4 GB; the prepare step took 7.3 s on a first start.
+- Current: `DM_UPSCALER=metalfx` draws the upscale step with MetalFX (M4 Pro, 20-core GPU, 2026-11-11, Kakariko Village from a save, chain `b8856126d238`, `DM_FPS_HOLD=off`). In play the game anti-aliases and upscales with four dispatches of `agl_ndsr_vr.pck` and `agl_ndsr_vr_light.pck` (`f8965bffbdf51d78`, `d17eefaff4f0bc23`, `a335c1db6270c355`, `fa3033d05c0a41d6`), one encoder of 1.25 ms; only the last is the upscale (texture slot 0 at 1536x864 to image slot 1 at 2048x1152). With its dispatches dropped (`DM_NVN_SKIP_CS`, the picture stale) the frame's compute fell 3.91-3.93 → 2.78-2.80 ms: 1.12 ms. Its colour, from its translated code: the dispatch before it stores each channel's square root, packed as RGB9E5 bits in a single-channel image; the step samples that, adds its network's residual and writes `v * |v|`, linear, with no sRGB curve, so the host's classifier kept the shader. The profile names it ([`upscaler_sqrt`](game.tsv)): [An upscaler with square-root colour](../../../docs/reference/rendering.md#an-upscaler-with-square-root-colour). Two runs a side in turn: headless, compute 3.91 and 3.95 → 2.80 and 2.77 ms, the frame's GPU work 14.84 and 15.61 → 13.64 and 13.48 ms (median sum of encoder times over 37-38 profiled frames); in a window, compute 3.90 and 3.89 → 2.76 and 2.79 ms, summed GPU p50 15.59 and 15.13 → 14.63 and 14.46 ms, 60.0 fps on both. The end shots' mean colours agree (15.9 / 25.3 / 29.4 against 15.8 / 24.8 / 28.4 of 255; two shots of the game's own differ as much, by the moving grass); MetalFX's picture is crisper, the network's softer, which the player preferred: the profile's `upscaler_sharpness 0.7` blends 30 % of a bilinear picture in, the closest of seven variants in 2x crops ([MetalFX's picture is crisper than a game's own upscaler](../../../docs/reference/rendering.md#metalfxs-picture-is-crisper-than-a-games-own-upscaler)), at the same GPU time. The profile names the pack's eight variants that end as this one does; which one runs goes by the input size: `fa3033d05c0a41d6` at full size docked (also in the castle town, a dungeon and Kokiri Forest) and handheld (1440x810 → 1920x1080), `d99fbeeb53d38847` at render scale 0.5 (781x439), and with the dynamic resolution on `aad7ea7e5c02f41d`, `b106afba1a18faa9`, `04135bb812a0d66e` and `7c511cc23ff85d14` on the way down to its floor (nine sizes from 1536x864 to 763x429 in one run). All six were drawn by MetalFX with no red line, the end shots' mean colours within 1 of 255 of the game's upscaler in the same mode; `8df148d58b7cbe8e` and `c8b20adc9d4978e0` were not seen. None of the pack's seven encoding variants, of `agl_ndsr_vr.pck`'s fifteen or of `agl_ndsr.pck`'s was dispatched in those eight situations (pass dumps; the title uses none). What MetalFX saves grows with the upscale: compute 4.01 → 2.58 ms at render scale 0.5 and 3.25 → 2.27 ms handheld (one run a side). The three anti-aliasing dispatches still run. Open: not judged in motion, nor with a render scale, the dynamic resolution or a cutscene.
 - Current: the title and menu, a movie behind them, are a locked 60 fps (p50 16.67 ms, GPU span 15.6 ms).
 - Game speed: the game steps its time by a frame ratio, its base rate (60, or 30) / its int frame rate, whatever the frame took (`main+0x6e6cc4` on v131072), so frames longer than a vsync played in slow motion (seen by the user on a slower Mac; [issue 1375](https://github.com/seedds/nxnative/issues/1375)'s M1 Max ran the title at about 28 fps). With `fps_target 60` the ratio is the host's frame step × base / 60 ([A fixed frame step plays in slow motion on a slow Mac](../../../docs/reference/frame-rate.md#a-fixed-frame-step-plays-in-slow-motion-on-a-slow-mac)), a stub placed by its function's fingerprint ([`patches.py`](patches.py)). Mac runs on the M6 (2026-11-08, a test host on the libraries of chain `3ee493b2c3a4`): the user's play sessions held 60 fps with 2- and 3-vsync holds while areas loaded, game time 1.000 of real time by the host's count. That count is the stub's own; no run on a Mac slow enough to show the slow motion before and after.
 - Sizes since the host answers graphics scale tier 0 docked ([The graphics scale does not follow the operation mode](../../../docs/reference/services.md#the-graphics-scale-does-not-follow-the-operation-mode)): the game's output is 2560x1440 and its top internal resolution 1920x1080. Every size and time in this note was taken with tier 1, at 0.8 of that (2048x1152 and 1536x864, and the dynamic resolution's levels with them), and has not been taken again. With tier 0 on `main` (two contributors' runs, only that answer changed): M4 Pro (20-core GPU), docked, fullscreen, GPU p50 15.0 ms against 12.9-14.0 ms, 59.0 fps against 55.8-59.8; M5 Pro (20-core GPU, 48 GB), Dynamic resolution off, 24 minutes of play, GPU p50 12.7 ms, 58.4 fps. Open: a slower Mac, and the routes of [`bench.tsv`](bench.tsv), with tier 0.
@@ -126,6 +127,8 @@ Its 37 kernels that correct samples, NDSR's network among them, gave about 0.25
 
 ## Found on this game
 
+- [An upscaler with square-root colour](../../../docs/reference/rendering.md#an-upscaler-with-square-root-colour): `DM_UPSCALER=metalfx` replaced no dispatch; with the profile's `upscaler_sqrt` the upscale step's 1.12 ms of GPU goes to MetalFX (Kakariko Village, M4 Pro).
+- [MetalFX's picture is crisper than a game's own upscaler](../../../docs/reference/rendering.md#metalfxs-picture-is-crisper-than-a-games-own-upscaler): the launcher's Game's upscaler sharpness, 70 % here by the player's eye.
 - [A shader build's Python finishes one shader at a time](../../../docs/reference/shaders.md#a-shader-builds-python-finishes-one-shader-at-a-time): resolved, the whole Mac map's cold build took 58 minutes on an M3 Max with 4 to 5 of its 16 cores busy; 30 minutes with worker processes.
 - [Reads past a uniform binding's size](../../../docs/reference/shaders.md#reads-past-a-uniform-bindings-size): open ([GitHub issue #2218](https://github.com/seedds/nxnative/issues/2218)), long white streaks from the horses at Lon Lon Ranch and from Link's slingshot, and flickering parts of the slingshot and of the ReDeads' faces, reported with the uniform read bound on. The bound now takes the size the driver gives the GPU, rounded up to 256 bytes; the game binds sizes that are not multiples of 256. Not reproduced here before or after (the slingshot at Link's house, the ReDeads of adult Castle Town), so the reporters' retest decides.
 - [The blend helper's output reference on iOS](../../../docs/reference/shaders.md#the-blend-helpers-output-reference-on-ios): resolved, 22,296 of 42,848 Mac shaders failed to compile on macOS 26 ([GitHub issue #2272](https://github.com/seedds/nxnative/issues/2272)); the Mac compile now names the host's minimum macOS.

dm/games/0400486024DEC000/game.tsv

@@ -48,3 +48,10 @@ draw_barriers	game
 # As Tears of the Kingdom: the terrain's height, blend and hole maps are written by the CPU into textures after they are created. Without these keys they stay zero: holes in the ground and the wrong materials
 tex_flush_upload	on
 tex_dirty_upload	on
+# The upscale step of agl_ndsr_vr_light.pck, the last of the four dispatches the game anti-aliases and upscales with in play: it samples colour the dispatch before it stored as square
+# roots (RGB9E5) and writes v * |v|, linear, so its code has none of the sRGB constants the host reads a colour shape from. Named here, DM_UPSCALER=metalfx draws it with MetalFX
+# (fa3033d05c0a41d6 in Kakariko Village: 1.12 ms of GPU a frame on an Apple M4 Pro, 2026-11-11). The eight variants of the pack whose translated code ends that way; its seven others
+# encode their result (x 12.92 after the square) and keep their shader, as do agl_ndsr_vr.pck's fifteen, which take linear colour and have not been seen in play
+upscaler_sqrt	04135bb812a0d66e,7c511cc23ff85d14,8df148d58b7cbe8e,aad7ea7e5c02f41d,b106afba1a18faa9,c8b20adc9d4978e0,d99fbeeb53d38847,fa3033d05c0a41d6
+# The player compared MetalFX's picture with the network's in 2x crops and found it too crisp; with 30 % of a bilinear picture blended in it came closest (2026-11-11)
+upscaler_sharpness	0.7

dm/games/README.md

@@ -253,6 +253,8 @@ One row per profile; the notes are each profile's `README.md`.
 | `tex_dirty_upload` | `on` / `off` (the default), needs `tex_flush_upload`: a registered texture whose pages the CPU writes after `nvnTextureInitialize` without any NVN flush is uploaded as well (its pages are write-protected at registration, checked on first use; `nvn_ft_written` in `hle_nvn.c`). Tears of the Kingdom fills its ASTC terrain material arrays this way; without it the cave is magenta. FINAL FANTASY TACTICS - The Ivalice Chronicles loads its UI textures straight into CPU-uncached pools, which need no flush, after creating them; without both keys its screen stays black. The Great Ace Attorney Chronicles copies every texture into a CPU-cached pool after creating it and never flushes (its `main` has no cache maintenance at all): black too. The Legend of Zelda: Ocarina of Time writes its terrain tiles' height, blend and hole maps this way, and without the keys has holes in the ground |
 | `tex_dyn_slots` | `on` / `off` (the default): a shader whose buffer-texture handle the translator could not trace to one slot gets Ryujinx's fallback arrays (`<stage>_<tb|s>_cb0_40000000`: 32 buffer textures and samplers indexed by the handle's texture / sampler ID; any other sampled texture indexes the whole pool without a key, [Texture handles from game memory index the pools](../../docs/reference/shaders.md#texture-handles-from-game-memory-index-the-pools)). With the key the replay puts slot indices in that shader's c2 texture and sampler tables (texture slot r → r, sampler slot r → r << 20) and binds slot k at element k, so the shader's own arithmetic picks the binding (`NVN_RB_TEXD` / `NVN_RB_SAMPD` in `nvn_mtl.m`). Right only where the handle comes from c2's tables: Octopath Traveler 0's 61 such vertex shaders read separate texture slots 0-10 and samplers 0-4 there and nothing else (FINAL FANTASY RESONANCE's 2,077 the texture table words 0x168-0x1c0 and sampler table words 0x568-0x590). Off, such shaders do not load. Not part of the shader chain version |
 | `draw_barriers` | `full` (the default) / `game`: how draw passes are ordered. `full`: each waits for all earlier GPU work. `game`: by the game's own barriers (NVN asks for one before results of rendering are read), so a pass's vertex work overlaps the previous pass's fragments. Ocarina of Time's GPU time 25 → 18 ms, about 180 full barriers a frame halved. Only vertex-stage reads of the previous pass's output without a barrier change order. `DM_NVN_BARRIER` overrides it |
+| `upscaler_sqrt` | comma-separated 16-digit keys of upscaler compute shaders, outside `agl_ndsr.pck`, that sample square-root-encoded colour and write it squared (linear): `DM_UPSCALER=metalfx` squares their input into MetalFX's and runs it in HDR mode ([An upscaler with square-root colour](../../docs/reference/rendering.md#an-upscaler-with-square-root-colour)). Read the shape from the shader's translated code before naming one |
+| `upscaler_sharpness` | `0.0` to `1.0` (the default): with `DM_UPSCALER=metalfx`, the share of MetalFX's own picture in the game's image, the rest the scaler's input scaled bilinearly; the launcher's **Game's upscaler sharpness** starts from it (`DM_UPSCALER_SHARPNESS`). Lower it for a game whose own network is softer than MetalFX, by eye against the game's picture |
 | `vconv_range` | `on` / `off` (the default): vertex attributes the replay converts on the CPU (integer shader inputs fed float-type formats and the like, `vconv_bind` in `nvn_mtl.m`) are converted only over the vertices each draw reads (first / count, or the smallest and largest index of a direct indexed draw, kept while the index pages stay unwritten) instead of the whole bound range. Age of Calamity binds whole mesh pools: in a battle that was millions of vertices a frame. Indirect draws, per-instance streams and indices with a GPU write queued on them keep the whole range. `DM_NVN_VCONV_CHECK=1` scans the indices again at every reuse and reports differences; `DM_NVN_VCONV_RANGE=0` / `1` overrides it |
 | `shader_key_skip` | `<offset>+<length>` (none by default): control-section bytes every shader key hashes as zeros, in `shader_split.py` (`key_ctl`) and the host (`hle_nvn.c` `keys_hash`), for an engine that shares one control section among shaders that differ only there. Batman: Arkham Knight: `0x770+0x78`, GLSLC's compile information (+0x778..+0x791 and +0x7d0..+0x7e7); without it the game asks for shaders the map does not have, each the control section of one record with the code area of another. It must stay clear of `+0x700..+0x75f`, which the tools read; it is a shader chain input, as every `shader_*` key |
 | `shader_groups`, `shader_note` | `shader_split.py` source groups and the map header note (Engage: `engine` for `data.unity3d` and `Resources/`, `assets` for the Addressables bundles). Both are shader inputs, the note too: changing either one, even by a word, builds every shader of the game again. Keep them as they are when a provisional profile moves into `dm/games` |

dm/games/game.py

@@ -108,11 +108,19 @@ def by_build(main_id):   # the profile whose build rows register this main build
 def upscaler_keys(g):
     """the game's own upscaler shaders, from its shader map (shader_split.py's source column): sorted 16-digit keys, none without a map"""
     p = g.table('shader_map.tsv')
-    if not p.exists(): return []
+    if not p.exists(): return upscaler_sqrt_keys(g)
     keys = set()
     for line in p.read_text(encoding='utf-8').splitlines():
         c = line.split('\t')
         if len(c) >= 5 and c[2] == 'compute' and UPSCALER_SOURCE.search(c[4]) and re.fullmatch(r'[0-9a-f]{16}', c[3]): keys.add(c[3])
+    return sorted(keys | set(upscaler_sqrt_keys(g)))
+
+
+def upscaler_sqrt_keys(g):
+    """upscaler shaders outside agl_ndsr.pck whose colour is no sRGB curve but a square root: the profile's upscaler_sqrt (comma-separated 16-digit keys), read from each shader's translated code"""
+    keys = [k for k in g.get('upscaler_sqrt', '').split(',') if k]
+    for k in keys:
+        if not re.fullmatch(r'[0-9a-f]{16}', k): sys.exit(f'{g.id}: upscaler_sqrt must be comma-separated 16-digit shader keys: {k}')
     return sorted(keys)
 
 
@@ -203,6 +211,8 @@ def header_one(g):
     if vcr not in ('on', 'off'): sys.exit(f'{g.id}: vconv_range must be on or off: {vcr}')
     mml = g.get('mem_map_limit', 'off')   # MapPhysicalMemory refused beyond the application pool, as on Horizon (hle_svc.c): on / off
     if mml not in ('on', 'off'): sys.exit(f'{g.id}: mem_map_limit must be on or off: {mml}')
+    ush = g.get('upscaler_sharpness', '1.0')   # DM_UPSCALER=metalfx: the share of MetalFX's own picture, the rest a bilinear one (nvn_mtl.m up_mfx_sharpness; DM_UPSCALER_SHARPNESS overrides): 0.0-1.0
+    if not re.fullmatch(r'[01]\.\d{1,2}', ush) or float(ush) > 1: sys.exit(f'{g.id}: upscaler_sharpness must be 0.0 to 1.0: {ush}')
     ups = upscaler_keys(g)   # the game's own upscaler shaders, which DM_UPSCALER=metalfx (the launcher's Game's upscaler) replaces with MetalFX's spatial scaler (nvn_mtl.m up_mfx_encode)
     dbr = g.get('draw_barriers', 'full')   # how draw passes are ordered (nvn_mtl.m barrier_game): full = each waits for all earlier GPU work, game = by the game's own barriers, so vertex work overlaps the previous pass's fragments
     if dbr not in ('full', 'game'): sys.exit(f'{g.id}: draw_barriers must be full or game: {dbr}')
@@ -260,7 +270,8 @@ def header_one(g):
          f'#define DM_GAME_TEX_DIRTY_UPLOAD {int(tdu == "on")}',
          f'#define DM_GAME_TEX_DYN_SLOTS {int(tds == "on")}',
          f'#define DM_GAME_DRAW_BARRIERS_GAME {int(dbr == "game")}',
-         f'#define DM_GAME_UPSCALER_KEYS {q(",".join(ups))}', f'#define DM_GAME_UPSCALER_N {len(ups)}',
+         f'#define DM_GAME_UPSCALER_KEYS {q(",".join(ups))}', f'#define DM_GAME_UPSCALER_N {len(ups)}', f'#define DM_GAME_UPSCALER_SQRT_KEYS {q(",".join(upscaler_sqrt_keys(g)))}',
+         f'#define DM_GAME_UPSCALER_SHARPNESS {ush}',
          f'#define DM_GAME_VCONV_RANGE {int(vcr == "on")}',
          *([f'#define DM_GAME_NVN_API_MAJOR {api.split(".")[0]}', f'#define DM_GAME_NVN_API_MINOR {api.split(".")[1]}'] if g.get('device_api', '') else []),   # without the key: by the SDK version (hle_nvn.c nvn_api)
          '']

dm/games/settings.tsv

@@ -20,6 +20,7 @@ DM_RENDER_SCALE	range	0.5:1.0:0.05	1.0	fixups:RENDER_SCALE	Game	Render scale		le
 DM_OPMODE	choice	1=Docked,0=Handheld	DM_GAME_OPMODE	always	Game	Console mode		level=basic
 DM_PRESENT_UPSCALE	choice	off=Standard,metalfx=MetalFX	off	always	Game	Upscaling	Uses more GPU time and can increase heat.
 DM_UPSCALER	choice	game=Game's own,metalfx=MetalFX	game	DM_GAME_UPSCALER_N!=0	Game	Game's upscaler
+DM_UPSCALER_SHARPNESS	range	0.0:1.0:0.05	DM_GAME_UPSCALER_SHARPNESS	DM_GAME_UPSCALER_N!=0	Game	Game's upscaler sharpness		needs=DM_UPSCALER=metalfx
 DM_LANG	choice	languages	build:DM_LANG	languages>1	Game	Game language		level=basic
 DM_FULLSCREEN	choice	0=Window,1=Fullscreen	0	platform:mac	Game	Display mode		level=basic
 DM_PICTURE_FIT	choice	fit=Fit,fill=Fill	fit	platform:ios	Game	Picture	Fill crops the game's 16:9 picture to cover the screen. Health bars, subtitles or maps near the edges can be cut off. A full-screen picture without cropping needs a patch for each game.

dm/gfx/msl/copy.metal

@@ -6,6 +6,13 @@ vertex V vs(uint id [[vertex_id]]) { float2 uv = float2((id << 1) & 2, id & 2);
 // p[0] = source origin (texels) + source texels per destination texel, p[1] = destination origin + source level size
 fragment float4 fs(V in [[stage_in]], texture2d<float> t [[texture(0)]], sampler s [[sampler(0)]], constant float4 *p [[buffer(0)]]) {
   float2 st = p[0].xy + (in.p.xy - p[1].xy) * p[0].zw; return t.sample(s, st / p[1].zw, level(0)); }
+// The copies around MetalFX's spatial scaler (up_mfx_encode). p[2].x = blend with t1, a smooth picture of the same rectangle (0 = none); p[2].y = 1: the colour squared, sign kept
+// (square-root-encoded colour into linear); p[2].zw = t1's coordinate per source texel of the rectangle
+fragment float4 fsup(V in [[stage_in]], texture2d<float> t [[texture(0)]], texture2d<float> t1 [[texture(1)]], sampler s [[sampler(0)]], constant float4 *p [[buffer(0)]]) {
+  float2 st = p[0].xy + (in.p.xy - p[1].xy) * p[0].zw; float4 c = t.sample(s, st / p[1].zw, level(0));
+  if (p[2].x > 0) c = mix(c, t1.sample(s, (st - p[0].xy) * p[2].zw, level(0)), p[2].x);
+  if (p[2].y > 0) c.rgb = c.rgb * abs(c.rgb);
+  return c; }
 struct D { float d [[depth(any)]]; };
 fragment D fsd(V in [[stage_in]], depth2d<float> t [[texture(0)]], sampler s [[sampler(0)]], constant float4 *p [[buffer(0)]]) {
   float2 st = p[0].xy + (in.p.xy - p[1].xy) * p[0].zw; D o; o.d = t.read(uint2(st)); return o; }