[dinadur]

Builds on #1459 (its two commits are the base of this branch; only the last commit, c64c6e8, is this change). Once #1459 merges, this PR shows the one commit.

What and why

With #1459, DM_UPSCALER=metalfx draws The Legend of Zelda: Ocarina of Time's agl_ndsr_vr.pck / agl_ndsr_vr_light.pck upscale, classifying every variant as the no-curve linear shape. In play the picture came out washed out: pale, low contrast and foggy across the whole scene, with the UI drawn after it correct (reported in #1459 (comment)).

Cause: these networks keep their picture square-root encoded. The variants that write the picture compute c * |c| of the sampled input plus the network's residual: in the translated MSL of fa3033d05c0a41d6, the output is (sample + residual) * abs(sample + residual) (63 such products), and the input is RGB9E5 holding the encoded colour. Drawn as the linear shape, MetalFX passed the encoded colour through, so the output was sqrt of the colour. The square is a plain HMUL2 with |Rb|, with no MUFU and no constant, so the curve and power checks cannot see it.

Change:

• up_mfx_has_sq counts HMUL2 in register form (0x232 in the low 12 bits) with both sources one register (not RZ) and |Rb| (bit 62), from 8, as the curve constants are counted. Ocarina of Time's four picture-writing variants (fa3033d05c0a41d6, aad7ea7e5c02f41d, b106afba1a18faa9, 04135bb812a0d66e) have 32 to 36 each in the raw code. The passes beside them (f8965bffbdf51d78, d17eefaff4f0bc23, a335c1db6270c355) have none.
• No curve, no power and squares make the new shape UP_COLOR_SQ. It is drawn through the scaler's RGBA16F input, never straight, with the input squared on the way in (copy_t2t_draw with a host-only COPY_T2T_SQUARE flag, which the guest's copy flags are masked against, and copy.metal's fssq). MetalFX then runs in HDR mode, as for linear colour.
• Bananza's and Splatoon Raiders' variants have curve constants and never reach this branch, so their shapes are unchanged.
• Docs: the "Square root" item in docs/reference/rendering.md's An upscaler's colour read from its code.

Checked

• Tests: make -C dm/gfx/tests GAME=0400486024DEC000 t_upscaler_mfx, run as check_t_upscaler_mfx runs it (DM_NVN_ENC_THREAD=0 and 1 with DM_UPSCALER=metalfx: 107 checks; without: 21; and MTL_DEBUG_LAYER=1), all passing. New checks:
  • the classifier on synthetic code: eight squares → UP_COLOR_SQ;
  • seven squares, x * x without |x|, two registers, and RZ → linear;
  • squares beside a power → kept;
  • a square-root input, full and as a packed R11G11B10F subrectangle, comes out squared (0.5, 0.25, 0.75 → 0.25, 0.0625, 0.5625, alpha kept), in its destination rectangle only.
• Games run: The Legend of Zelda: Ocarina of Time, 0400486024DEC000, update v131072, Mac app, both MetalFX options on. Played in Kokiri Forest and inside the Great Deku Tree.
  • The colours now match the game's own upscaler instead of being washed out.
  • The log reads upscaler fa3033d05c0a41d6_b: square-root-encoded input, linear output … squared before MetalFX in HDR mode and first dispatch of fa3033d05c0a41d6_b drawn by MetalFX, 1536x864 … (format 93) → 2048x1152 … (format 92), through RGBA16F, squared from its square-root encoding.
  • The other three picture-writing variants and two more (d99fbeeb53d38847, 7c511cc23ff85d14) were classified the same way. f8965bffbdf51d78, d17eefaff4f0bc23 and a335c1db6270c355 stayed linear.
  • Frame time: the first ~2 minutes after loading the Deku Tree had long frames while the game prepared the new area, then it settled at the game's dynamic resolution of 1195x672 to 1536x864. The added square is one full-screen draw that was already there (the RGBA16F input copy) with a different fragment shader.
• Devices and OS: Mac Studio, M5 Ultra, 256 GB, macOS 27.2.
• Not checked: scripts/regress.py check --games; Donkey Kong Bananza and Splatoon Raiders in play (not owned; their classification cannot change, see above); iOS; Mario Kart World's variants.

Docs

• The finding is in its docs/reference/rendering.md entry, and python3 dm/tools/docs_symptoms.py was run (the symptom index; no change)
• A chain tool changed: not the case
• No game files, keys or data extracted from a game in the change or the attachments

🤖 Generated with Claude Code

[dinadur]

Closing this as superseded: @PromethOps found the same cause independently and fixed it in #1459 (d715366). That fix also covers the FMUL form in agl_ndsr_vr.pck, which this PR only matched as HMUL2, and it measures the result against the game's own upscaler. Thanks for picking it up so quickly.