diff --git a/dm/games/01007EF00011E000/validated.tsv b/dm/games/01007EF00011E000/validated.tsv index 7ef329bebe..bb434ffae0 100644 --- a/dm/games/01007EF00011E000/validated.tsv +++ b/dm/games/01007EF00011E000/validated.tsv @@ -1,3 +1,3 @@ # The chain this game was last green on in scripts/regress.py check / baseline --record (one row per mode, the latest of each). Written by the harness, committed with the pull request. # mode chain= commit=[+dirty] date= machine= macos=. No game data. -full chain=e65a4ebd3322 commit=4a1f683018+dirty date=2026-11-01 machine=Mac16,7 macos=27.2 +full chain=0ad559cdbb28 commit=ca080711d3 date=2026-11-04 machine=Mac16,7 macos=26.7.1 diff --git a/dm/games/0400486024DEC000/README.md b/dm/games/0400486024DEC000/README.md index 6ca51f13fc..3d0fe85a0f 100644 --- a/dm/games/0400486024DEC000/README.md +++ b/dm/games/0400486024DEC000/README.md @@ -63,6 +63,15 @@ Run headless, `DM_CTL` at a control file. The work folder has `boot.sh `. From a boot at the title (about 80 s): `to_race.sh` presses L+R, then A nine times with waits (Single Player, Grand Prix, Mario, the kart, the Mushroom Cup, 50cc, "Start Grand Prix?"). The opening and the race's start then take about a minute; `A~40` drives. `measure.sh` does all of this, waits for the race (over 1,500 draws a frame, no movie thread), and writes the timing and the `restop` residency lines to `run/.res`. Control presses run in order, so `A~40` holds up later lines. @@ -91,6 +104,7 @@ Update 1.8.0 (v917504, SDK 21.4.0) is not on this Mac. Pull request #1116 ported - **Mesh draws:** `DrawMeshTasks3D` and the indirect forms are not implemented; no run reached them. - `649e36162cad9e79`, a foliage shader with an 8x8 local size, keeps a first-invocation race: its setup loops over a uniform's count, so [the host cannot make it as stores](../../../docs/reference/shaders.md#a-store-only-a-dispatchs-first-invocation-makes). Its list's count read 0 in every one of 7,350 traced title frames, so the race had nothing to act on there; whether that list should be empty is not known. - The engine class screen offers 50, 100 and 150cc in this build. +- **HDR:** only the title was run with HDR on. The window's primaries are taken to be BT.2020, as on Donkey Kong Bananza, but are not measured here. On an EDR screen the picture looked slightly brighter than SDR (by eye, one MacBook Pro). Fullscreen is not timed. ## Found on this game @@ -105,6 +119,7 @@ Update 1.8.0 (v917504, SDK 21.4.0) is not on this Mac. Pull request #1116 ported - [Comparisons with .FTZ](../../../docs/reference/shaders.md#comparisons-with-ftz) - [Variable rate shading ignored](../../../docs/reference/rendering.md#variable-rate-shading-ignored) - [Image views of the type a shader declares](../../../docs/reference/rendering.md#image-views-of-the-type-a-shader-declares) (the dark box around each kart's shadow, issue #1648) +- [The status query and the present for an HDR window](../../../docs/reference/rendering.md#the-status-query-and-the-present-for-an-hdr-window): the same HDR window as Donkey Kong Bananza's - [A DualShock 4 that connects after the first stays on Core Haptics](../../../docs/reference/services.md#a-dualshock-4-that-connects-after-the-first-stays-on-core-haptics) - [Identical controllers that GameController lists as one](../../../docs/reference/services.md#identical-controllers-that-gamecontroller-lists-as-one) (two Pro Controllers over USB, split-screen races) - [Scaled shared memory addresses](../../../docs/reference/shaders.md#scaled-shared-memory-addresses) diff --git a/dm/games/settings.py b/dm/games/settings.py index bff40091da..8c86e224b2 100644 --- a/dm/games/settings.py +++ b/dm/games/settings.py @@ -59,6 +59,11 @@ def fixups_keys(path): # '# KEYvalue' header rows of a rewriter table UNKNOWN_LANGUAGES = 'This game\'s files do not list its languages, so some of these may not be in the game. It may then show another one.' +def switch2(patched): # the sdk module names the Switch 2's build spec ("spec: Ounce"). dm_sdk_ounce (dm/host/loader.c) searches the loader's range (read-only data, data, bss) for it; this searches the whole file, a superset that gives the same answer on the local modules + path = Path(patched) / 'sdk.patched.bin' + return path.is_file() and b'spec: Ounce' in path.read_bytes() + + def build(gid, patched, control): platform = os.environ.get('SETTINGS_PLATFORM') or 'ios' if platform not in ('ios', 'mac'): sys.exit(f'SETTINGS_PLATFORM must be ios or mac: {platform}') @@ -75,6 +80,7 @@ def macro(name, n): def shown(cond, n): if cond == 'always': return True if cond == 'languages>1': return len(langs) > 1 + if cond == 'switch2': return switch2(patched) if cond.startswith('fixups:'): return cond[7:] in keys if cond.startswith('gfxapi:'): return os.environ.get('GFXAPI', '') == cond[7:] if cond.startswith('platform:'): return platform == cond[9:] diff --git a/dm/games/settings.tsv b/dm/games/settings.tsv index 75f5e62243..07493d5ff4 100644 --- a/dm/games/settings.tsv +++ b/dm/games/settings.tsv @@ -8,7 +8,7 @@ # default a value (a toggle's: 1 on, 0 off); DM_GAME_: the host's own default from the game profile (game_profile.h, game.py); build:: the IPA build's setting (build_ipa.sh) # | selects the platform's default before resolving a profile macro or build variable # shown always; DM_GAME_!= or ==; fixups:: main's rewriter table has that key (a stub in the game's build, patch_nso.py); languages>1 (more than one to pick, so also when the control data lists none); gfxapi:: the packer's GFXAPI; -# platform:: the app's platform (SETTINGS_PLATFORM, ios when unset), for a switch only that platform's front end reads +# platform:: the app's platform (SETTINGS_PLATFORM, ios when unset), for a switch only that platform's front end reads; switch2: a Switch 2 game (its sdk module names the Ounce spec) # section Game, Controls or Performance: a heading on the single settings screen. Rows appear in table order within each section # Reset to tested defaults clears all launcher choices; values fixed by an env file stay # label, note the setting's name and the line under it (may be empty; only for a setting with a cost: latency, memory, heat) @@ -23,6 +23,9 @@ DM_UPSCALER choice game=Game's own,metalfx=MetalFX game DM_GAME_UPSCALER_N!=0 Ga DM_LANG choice languages build:DM_LANG languages>1 Game Game language level=basic DM_FULLSCREEN choice 0=Window,1=Fullscreen 0 platform:mac Game Display mode level=basic DM_PICTURE_FIT choice fit=Fit,fill=Fill fit platform:ios Game Picture Fill crops the game's 16:9 picture to cover the screen. Health bars, subtitles or maps near the edges can be cut off. A full-screen picture without cropping needs a patch for each game. +DM_HDR toggle - 0 switch2 Game HDR output Takes effect on a screen with HDR headroom; an SDR screen keeps SDR. level=basic +DM_HDR_PAPER_WHITE choice 120=120 nits,160=160 nits,203=203 nits (standard),250=250 nits,300=300 nits,400=400 nits 203 switch2 Game HDR paper white level=basic;needs=DM_HDR=1 +DM_HDR_PEAK choice auto=Automatic?platform:mac,400=400 nits,600=600 nits,1000=1000 nits (standard),1600=1600 nits 1000 switch2 Game HDR peak level=basic;needs=DM_HDR=1 DM_SHOW_FPS toggle - build:DM_SHOW_FPS platform:ios Game Frame-rate counter DM_TOUCH_HAPTICS toggle - 1 platform:ios Controls Touch feedback DM_RUMBLE toggle - 1 always Controls Game rumble level=basic diff --git a/dm/gfx/hle_vi.c b/dm/gfx/hle_vi.c index eb043504af..10c4b7b5d8 100644 --- a/dm/gfx/hle_vi.c +++ b/dm/gfx/hle_vi.c @@ -1,7 +1,9 @@ // nn::vi (display layer): on a real device, the vi service is used to open the display, create a layer and get the native window handle; the handle is passed to NVN's window builder. // Host: returns host static objects; when hooking up Metal, the native window corresponds to the front end's CAMetalLayer (front_metal_layer()). #include "../host/hle.h" +#include #include +#include static struct { uint32_t magic; } g_display = {0x50534944}, g_layer = {0x5259414c}; static struct { uint32_t magic; void *layer; } g_native = {0x4e49574e, 0}; @@ -119,13 +121,105 @@ HLE_FORCE("_ZN2nn2vi8FinalizeEv", "display service shutdown; the host has no vi // ---------- Switch 2 SDKs: HDR, VRR and extended swap intervals ---------- // The SDK's versions check the display against its own default display object and forward the layer to vi2, whose layer object the host never made (Donkey Kong Bananza: a call -// through null from SetLayerNextHdrContentGamutMetadata, main+0x2036f4). The host's output is SDR at a fixed refresh rate: no HDR output, no VRR, and the layer metadata has -// nothing to apply to. HdrOutputFormat 0 is SDR; paper white is the BT.2408 reference, 203 nits, read only when HDR is on -HLE_FORCE("_ZN2nn2vi21QueryDisplayHdrStatusEPbPNS0_15HdrOutputFormatEPtPKNS0_7DisplayE", "display HDR status: SDR output (the host presents SDR)", Result, h_vi_HdrStatus, +// through null from SetLayerNextHdrContentGamutMetadata, main+0x2036f4). No VRR on the host, and SDR unless DM_HDR=1. Paper white is the BT.2408 reference, 203 nits (DM_HDR_PAPER_WHITE overrides it while HDR is on) +// DM_HDR=1 is the HDR mode (Switch 2 games; the HDR output setting of the Mac and iOS launchers sets it): while the window's screen can show EDR (front_hdr_screen), the display reports HDR on, so the +// game takes its HDR path (its window becomes HDR10: RGB10A2 with PQ values in BT.2020). On a screen without EDR it reports SDR, and the game keeps its own SDR picture. +// The HDR layer takes the PQ window as it is (dm_hdr_layer_float below); Core Animation maps it into the screen's headroom by the layer's HDR10 metadata (front_mac.m, front_ios.m) +// The format is 1 with HDR on and 0 with it off: the SDK's own answer is on = (format == 1), and a format above 1 is an error, 0x272 (Donkey Kong Bananza v458752, +// nn::vi::detail::DefaultDisplay::QueryHdrStatus, sdk+0x55bd60; found in pull request #1249). The peak (DM_HDR_PEAK, default 1000) is the extended luminance range the game declares (its metadata call, below). It and the paper white are the launcher's HDR peak and HDR paper white rows: what a console's HDR calibration would set +// Donkey Kong Bananza boots its SDR window (0x25), queries the status, then makes an RGB10A2 window (0x3d) of PQ code values with SDR white at 203 nits. +// Neither it nor Mario Kart World imports the SDK's HDR status event (CreateHdrStatusEvent and the rest), so a game learns of another screen only by asking again: the queries are counted in the log +// A decimal number, clamped to 16 bits (the answers are 16-bit) +static uint32_t vi_env_u32(const char *name, uint32_t def) { const char *e = getenv(name); if (!e || !*e) return def; unsigned long v = strtoul(e, NULL, 10); return v > 65535 ? 65535 : (uint32_t)v; } +// The HDR mode: 0 off, 1 HDR while the window's screen can show EDR, 2 HDR on any screen. DM_HDR is read the way the front ends read it (atoi: 2, or any other non-zero +// number for 1) when first asked; the front end changes it while the game runs (dm_hdr_set_mode: the iOS in-game menu), and the game, which asks for its status every frame, +// switches its window on its next query, as when the window moves to another screen. For Switch 2 games only (dm_sdk_ounce): a Switch 1 game answers SDR +static _Atomic int g_vi_hdr_mode = -1; // -1 until DM_HDR is read +static _Atomic int g_vi_hdr_live; // the mode was changed while the game runs: its SDR answers are logged too +int dm_hdr_mode(void) { + int m = atomic_load(&g_vi_hdr_mode); + if (m >= 0) return m; + const char *e = getenv("DM_HDR"); int v = e && *e ? atoi(e) : 0, unread = -1; + atomic_compare_exchange_strong(&g_vi_hdr_mode, &unread, v == 2 ? 2 : v != 0); // a mode set meanwhile by the front end wins + return atomic_load(&g_vi_hdr_mode); +} +void dm_hdr_set_mode(int m) { + m = m == 2 ? 2 : m != 0; + if (atomic_exchange(&g_vi_hdr_mode, m) != m) { atomic_store(&g_vi_hdr_live, 1); LOG("[nvn2-hdr] HDR output %s while the game runs", m == 2 ? "on any screen" : m ? "on" : "off"); } +} +static int vi_hdr_on(void) { return dm_hdr_mode() && dm_sdk_ounce(); } +// Whether the window's screen can show EDR (its potential headroom is above 1): the front end reads it before the guest starts and whenever the window's screen changes (front_mac.m, front_ios.m). +// No front end (a windowless run): taken as yes, so DM_HDR alone decides +__attribute__((weak)) int front_hdr_screen(void) { return 1; } +// The game's declared extended luminance range in nits, for the HDR layer's HDR10 metadata (front_mac.m and front_ios.m): Core Animation maps the picture into the screen's headroom by it; none without a front end +__attribute__((weak)) void front_hdr_content_range(unsigned lo, unsigned hi) { (void)lo; (void)hi; } +// The window's screen's current EDR headroom (front_mac.m: NSScreen.maximumExtendedDynamicRangeColorComponentValue, read four times a second beside the potential headroom): +// how many times the SDR white the screen can show right now, which falls as the brightness slider rises and depends on the screen's mode. 1.0 without a front end +// (a windowless run) and on the iPhone, whose front end does not read it: the automatic peak is then the paper white +__attribute__((weak)) float front_hdr_headroom(void) { return 1.0f; } +// The paper white in nits, read once (the present and the front ends take it from here too; nvn_mtl.m present_paper_white): a number from 1 to 65535, else 203 +static float vi_paper_white_v; +static void vi_paper_white_read(void) { const char *e = getenv("DM_HDR_PAPER_WHITE"); double v = e && *e ? strtod(e, NULL) : 203.0; vi_paper_white_v = (v >= 1.0 && v <= 65535.0) ? (float)v : 203.0f; } +float dm_hdr_paper_white(void) { static pthread_once_t once = PTHREAD_ONCE_INIT; pthread_once(&once, vi_paper_white_read); return vi_paper_white_v; } // pthread_once: every guest thread sees the value written before +// The peak the status query answers, in nits. DM_HDR_PEAK= is that number (16-bit, default 1000), read at every query as before. DM_HDR_PEAK=auto (the HDR peak row's Automatic) +// is the paper white × the screen's current headroom: what the screen can show above SDR white right now, so the range the game declares follows the brightness slider and the +// screen's reference modes, and the game tone-maps its highlights into it instead of Core Animation compressing a fixed range. Ocarina of Time declares min(peak, 3.5 × paper white) +// (DM_HDR_PEAK=400 → 0..400, 1600 → 0..710, DM_HDR_PAPER_WHITE=300 → 0..1000), so the automatic peak changes its picture only while the headroom is below 3.5. +// Rounded to 10 nits, so the slow drift of the headroom (auto-brightness) does not change the answer at every read; never below the paper white (a headroom under 1.0, or not yet +// granted, still leaves SDR white its own level) and at most 65535 (the answer is 16-bit). *automatic says which it was, for the log +static uint32_t vi_hdr_peak(int *automatic) { + const char *e = getenv("DM_HDR_PEAK"); + *automatic = e && !strcmp(e, "auto"); + if (!*automatic) return vi_env_u32("DM_HDR_PEAK", 1000); + float pw = dm_hdr_paper_white(), h = front_hdr_headroom(); + double v = isfinite(h) && h > 1.0f ? (double)pw * h : (double)pw; // NaN or under 1: the paper white + v = round(v / 10.0) * 10.0; + if (v < pw) v = ceil(pw); + return v > 65535.0 ? 65535u : (uint32_t)v; +} +// The HDR layer of both front ends (front_mac.m, front_ios.m): RGB10A2 in BT.2100 PQ, which takes the game's HDR10 window as it is, unless DM_HDR_LAYER=float: the RGBA16Float extended linear +// BT.2020 layer it replaced, which showed black blocks over flat areas on a MacBook Air M5 (docs/reference/rendering.md, the status query and the present for an HDR window) +int dm_hdr_layer_float(void) { const char *e = getenv("DM_HDR_LAYER"); return e && !strcmp(e, "float"); } +// Set once the game's status query has answered HDR on: its RGB10A2 window then holds PQ code values (present_mode, nvn_mtl.m); an RGB10A2 window of a game that never asked for HDR is not decoded. +// It stays set when the window later moves to a screen without EDR: the game keeps its HDR window until it asks again, and the present still decodes it +static _Atomic int g_vi_hdr_answered; +int vi_hdr_answered(void) { return atomic_load(&g_vi_hdr_answered); } +HLE_FORCE("_ZN2nn2vi21QueryDisplayHdrStatusEPbPNS0_15HdrOutputFormatEPtPKNS0_7DisplayE", "display HDR status: SDR output, or HDR on with DM_HDR=1 on a screen that can show EDR", Result, h_vi_HdrStatus, uint8_t *on, uint32_t *fmt, uint16_t *peak, const void *disp) { - (void)disp; if (on) *on = 0; if (fmt) *fmt = 0; if (peak) *peak = 0; return RES_OK; } -HLE_FORCE("_ZN2nn2vi25QueryDisplayHdrPaperWhiteEPtPKNS0_7DisplayE", "display HDR paper white: 203 nits (BT.2408); HDR is off", Result, h_vi_PaperWhite, uint16_t *nits, const void *disp) { - (void)disp; if (nits) *nits = 203; return RES_OK; } + (void)disp; + int mode = dm_sdk_ounce() ? dm_hdr_mode() : 0; + if (!mode && !atomic_load(&g_vi_hdr_live)) { if (on) *on = 0; if (fmt) *fmt = 0; if (peak) *peak = 0; return RES_OK; } // off from the start: SDR, not logged + // DM_HDR=2 answers HDR whatever the screen: on a screen without headroom Core Animation maps the HDR picture down instead of the game drawing its own SDR one (on Mario Kart World, more saturated than its SDR) + int forced = mode == 2, hdr = mode && (forced || front_hdr_screen() != 0); + int automatic; uint32_t p = vi_hdr_peak(&automatic); + // Logged when the answer changes, and at the 1st, 2nd, 10th, 100th and 1000th query: whether the game asks again (and so can follow the window to another screen) + static atomic_int queries, last = -1; + int n = atomic_fetch_add(&queries, 1) + 1, was = atomic_exchange(&last, hdr); + if (was != hdr || n == 1 || n == 2 || n == 10 || n == 100 || n == 1000) + LOG("[nvn2-hdr] vi::QueryDisplayHdrStatus (query %d): %s", n, hdr ? (forced && !front_hdr_screen() ? "HDR on, format 1 (DM_HDR=2: on a screen without EDR)" : "HDR on, format 1") + : mode ? "SDR (DM_HDR is on, but the window's screen cannot show EDR)" : "SDR (HDR output turned off while the game runs)"); + if (hdr) atomic_store(&g_vi_hdr_answered, 1); + // The automatic peak (DM_HDR_PEAK=auto) is logged when the answer differs from the last one logged and a second has passed since that line: the settled value of a brightness + // ramp is logged within a second of the ramp's end (the game asks every frame), and the answers between are counted rather than written + static _Atomic uint32_t peak_last, peak_logged; static _Atomic int64_t peak_logged_at; static atomic_int peak_changes; + if (hdr && automatic) { + if (atomic_exchange(&peak_last, p) != p) atomic_fetch_add(&peak_changes, 1); + if (p != atomic_load(&peak_logged)) { + struct timespec ts; clock_gettime(CLOCK_MONOTONIC, &ts); + int64_t now = (int64_t)ts.tv_sec * 1000 + ts.tv_nsec / 1000000, at = atomic_load(&peak_logged_at); + if ((at == 0 || now - at >= 1000) && atomic_compare_exchange_strong(&peak_logged_at, &at, now)) { // one thread logs when two ask at once + uint32_t was_p = atomic_exchange(&peak_logged, p); int c = atomic_exchange(&peak_changes, 0); + if (!was_p) LOG("[nvn2-hdr] vi::QueryDisplayHdrStatus: peak %u nits (automatic: paper white %.0f × the screen's headroom %.2f)", p, dm_hdr_paper_white(), front_hdr_headroom()); + else LOG("[nvn2-hdr] vi::QueryDisplayHdrStatus: peak now %u nits (automatic: the screen's headroom %.2f; was %u, %d answer%s since that line)", + p, front_hdr_headroom(), was_p, c, c == 1 ? "" : "s"); + } + } + } + if (on) *on = (uint8_t)hdr; if (fmt) *fmt = (uint32_t)hdr; if (peak) *peak = hdr ? (uint16_t)p : 0; return RES_OK; } +HLE_FORCE("_ZN2nn2vi25QueryDisplayHdrPaperWhiteEPtPKNS0_7DisplayE", "display HDR paper white: 203 nits (BT.2408) unless DM_HDR_PAPER_WHITE", Result, h_vi_PaperWhite, uint16_t *nits, const void *disp) { + (void)disp; uint16_t v = vi_hdr_on() ? (uint16_t)lroundf(dm_hdr_paper_white()) : 203; // dm_hdr_paper_white is 1 to 65535, so the cast is in range; 203 while HDR is off, as before + static atomic_int n; if (vi_hdr_on() && atomic_fetch_add(&n, 1) < 3) LOG("[nvn2-hdr] vi::QueryDisplayHdrPaperWhite: %u nits", v); + if (nits) *nits = v; return RES_OK; } HLE_FORCE("_ZN2nn2vi24QueryDisplayVrrSupportedEPbPKNS0_7DisplayE", "display VRR: not supported (fixed refresh rate)", Result, h_vi_VrrSupported, uint8_t *on, const void *disp) { (void)disp; if (on) *on = 0; return RES_OK; } HLE_FORCE("_ZN2nn2vi23SetDisplayVrrCapabilityEPNS0_7DisplayEb", "display VRR capability: the host has no VRR", Result, h_vi_SetVrr, void *disp, unsigned on) { @@ -151,14 +245,22 @@ HLE_FORCE("_ZN2nn2vi28SetLayerExtendedSwapIntervalEPNS0_5LayerEii", "layer exten void *layer, int a, int b) { (void)layer; static atomic_int once; if (!atomic_exchange(&once, 1)) LOG("vi::SetLayerExtendedSwapInterval %d %d: no VRR on the host, the NVN swap interval stays", a, b); return RES_OK; } HLE_FORCE("_ZN2nn2vi30ClearLayerExtendedSwapIntervalEPNS0_5LayerE", "layer extended swap interval cleared", Result, h_vi_ExtSwapClear, void *layer) { (void)layer; return RES_OK; } -HLE_FORCE("_ZN2nn2vi35SetLayerNextHdrContentGamutMetadataEPNS0_5LayerENS0_15HdrContentGamutE", "layer HDR metadata: SDR output, nothing to apply", Result, h_vi_HdrGamut, void *layer, int g) { - (void)layer; (void)g; return RES_OK; } -HLE_FORCE("_ZN2nn2vi39SetLayerNextHdrToSdrToneMappingMetadataEPNS0_5LayerENS0_14HdrToneMappingE", "layer HDR metadata: SDR output, nothing to apply", Result, h_vi_HdrToneMap, void *layer, int m) { - (void)layer; (void)m; return RES_OK; } -HLE_FORCE("_ZN2nn2vi52SetLayerNextHdrExtendedContentLuminanceRangeMetadataEPNS0_5LayerEtt", "layer HDR metadata: SDR output, nothing to apply", Result, h_vi_HdrExtLum, - void *layer, uint32_t lo, uint32_t hi) { (void)layer; (void)lo; (void)hi; return RES_OK; } -HLE_FORCE("_ZN2nn2vi52SetLayerNextHdrStandardContentLuminanceRangeMetadataEPNS0_5LayerEtt", "layer HDR metadata: SDR output, nothing to apply", Result, h_vi_HdrStdLum, - void *layer, uint32_t lo, uint32_t hi) { (void)layer; (void)lo; (void)hi; return RES_OK; } +// The SDK's HDR metadata calls for the layer (the gamut the game declares, its tone mapping to SDR, its luminance ranges): the host sends none of them to the screen but the extended range, which becomes the layer's HDR10 metadata. +// The first three of each are logged: the game's own statement of the gamut and luminance it targets, as it passed them (the luminance ranges are in nits: 0..203 and 0..1000 on Donkey Kong Bananza and Mario Kart World) +#define VI_META_LOG(...) do { static atomic_int n_; if (vi_hdr_on() && atomic_fetch_add(&n_, 1) < 3) LOG(__VA_ARGS__); } while (0) // logged only while HDR is on +HLE_FORCE("_ZN2nn2vi35SetLayerNextHdrContentGamutMetadataEPNS0_5LayerENS0_15HdrContentGamutE", "layer HDR metadata: dropped (the host sends none); the gamut is logged", Result, h_vi_HdrGamut, void *layer, int g) { + (void)layer; VI_META_LOG("[nvn2-hdr] vi::SetLayerNextHdrContentGamutMetadata: gamut %d (dropped)", g); return RES_OK; } +HLE_FORCE("_ZN2nn2vi39SetLayerNextHdrToSdrToneMappingMetadataEPNS0_5LayerENS0_14HdrToneMappingE", "layer HDR metadata: dropped (the host sends none); the tone mapping is logged", Result, h_vi_HdrToneMap, void *layer, int m) { + (void)layer; VI_META_LOG("[nvn2-hdr] vi::SetLayerNextHdrToSdrToneMappingMetadata: tone mapping %d (dropped)", m); return RES_OK; } +HLE_FORCE("_ZN2nn2vi52SetLayerNextHdrExtendedContentLuminanceRangeMetadataEPNS0_5LayerEtt", "layer HDR metadata: dropped (the host sends none); the luminance range is logged", Result, h_vi_HdrExtLum, + void *layer, uint32_t lo, uint32_t hi) { // passed on to the HDR layer's metadata (front_hdr_content_range) + (void)layer; VI_META_LOG("[nvn2-hdr] vi::SetLayerNextHdrExtendedContentLuminanceRangeMetadata: %u..%u", lo & 0xffff, hi & 0xffff); + // Each change after that is logged too, the first 20 (Mario Kart World declares 0..1000 nits while it makes its HDR window, then 0..1801 at its title) + static _Atomic uint32_t last; static atomic_int changes; uint32_t r = (lo & 0xffff) | (hi & 0xffff) << 16, was = atomic_exchange(&last, r); + if (vi_hdr_on() && was && was != r && atomic_fetch_add(&changes, 1) < 20) LOG("[nvn2-hdr] vi::SetLayerNextHdrExtendedContentLuminanceRangeMetadata: now %u..%u (was %u..%u)", r & 0xffff, r >> 16, was & 0xffff, was >> 16); + if (vi_hdr_on()) front_hdr_content_range(lo & 0xffff, hi & 0xffff); return RES_OK; } +HLE_FORCE("_ZN2nn2vi52SetLayerNextHdrStandardContentLuminanceRangeMetadataEPNS0_5LayerEtt", "layer HDR metadata: dropped (the host sends none); the luminance range is logged", Result, h_vi_HdrStdLum, + void *layer, uint32_t lo, uint32_t hi) { (void)layer; VI_META_LOG("[nvn2-hdr] vi::SetLayerNextHdrStandardContentLuminanceRangeMetadata: %u..%u (dropped)", lo & 0xffff, hi & 0xffff); return RES_OK; } // The rest of SDK 22's layer HDR metadata calls (the Bezier tone curve and the clears): the SDK forwards them through a table its graphics system fills, which is null on the host, so // unanswered they call through null (Mario Kart World 1.8.0 sets the curve every frame, by pull request #1116) HLE_FORCE("_ZN2nn2vi39SetLayerNextHdrAdjustableBezierMetadataEPNS0_5LayerEfff", "layer HDR metadata: SDR output, nothing to apply", Result, h_vi_HdrBezier, diff --git a/dm/gfx/msl/present.metal b/dm/gfx/msl/present.metal index 5bfdfeac47..18ce4abfb4 100644 --- a/dm/gfx/msl/present.metal +++ b/dm/gfx/msl/present.metal @@ -3,5 +3,60 @@ using namespace metal; struct V { float4 p [[position]]; float2 uv; }; vertex V vs(uint id [[vertex_id]]) { float2 uv = float2((id << 1) & 2, id & 2); V o; o.p = float4(uv * float2(2, -2) + float2(-1, 1), 0, 1); o.uv = uv; return o; } -struct Crop { float4 rect; float4 edge; }; -fragment float4 fs(V in [[stage_in]], texture2d t [[texture(0)]], sampler s [[sampler(0)]], constant Crop& c [[buffer(0)]]) { return t.sample(s, clamp(in.uv * c.rect.zw + c.rect.xy, c.edge.xy, c.edge.zw)); } +// mode.x (nvn_mtl.m present_mode) says how the source is encoded: 0 as stored (a float source: linear BT.709); 1 a gamma-coded 8-bit BT.709 (sRGB) source for the extended-range BT.2020 layer (fs_linear converts it); +// 2 an HDR10 source (the game's RGB10A2 window: SMPTE ST 2084 code values in BT.2020 primaries), where mode.y is the paper white in nits; 3 a gamma-coded BT.709 (sRGB) source for the PQ layer (RGB10A2 in BT.2100 PQ, DM_HDR): fs encodes it as HDR10 at the paper white; +// 4 a float source (linear BT.709) for the PQ layer, encoded the same way (untested: no local game presents a float window). mode.z and mode.w are unused. +struct Crop { float4 rect; float4 edge; float4 mode; }; +// Linear BT.709 and BT.2020 (ITU-R BT.2087; each row sums to 1, so white stays white). Metal's float3x3 takes its columns. +constant float3x3 kRec709To2020 = float3x3(float3(0.627404, 0.069097, 0.016391), float3(0.329283, 0.919540, 0.088013), float3(0.043313, 0.011362, 0.895595)); +constant float3x3 kRec2020To709 = float3x3(float3(1.660491, -0.124550, -0.018151), float3(-0.587641, 1.132900, -0.100579), float3(-0.072850, -0.008349, 1.118730)); +static inline float3 present_srgb_to_linear(float3 c) { + float3 lo = c * (1.0 / 12.92); + float3 hi = pow((c + 0.055) * (1.0 / 1.055), float3(2.4)); + return select(hi, lo, c <= 0.04045); +} +static inline float3 present_linear_to_srgb(float3 l) { + float3 lo = l * 12.92; + float3 hi = 1.055 * pow(l, float3(1.0 / 2.4)) - 0.055; + return select(hi, lo, l <= 0.0031308); +} +// SMPTE ST 2084 EOTF: code value e to absolute luminance (10000 nits = 1.0), divided by the paper white so that 1.0 is the display's SDR white and values above it are headroom +static inline float3 present_pq_to_linear(float3 e, float paper) { + const float m1 = 0.1593017578125, m2 = 78.84375, c1 = 0.8359375, c2 = 18.8515625, c3 = 18.6875; + float3 ep = pow(max(e, float3(0.0)), float3(1.0 / m2)); + float3 num = max(ep - c1, float3(0.0)); + float3 den = max(c2 - c3 * ep, float3(1e-6)); + return 10000.0 * pow(num / den, float3(1.0 / m1)) / paper; +} +// SMPTE ST 2084 inverse EOTF: linear light where 1.0 is the paper white, to code values (10000 nits = 1.0). pow() gets a positive base (the library is built with fast math) +static inline float3 present_linear_to_pq(float3 l, float paper) { + const float m1 = 0.1593017578125, m2 = 78.84375, c1 = 0.8359375, c2 = 18.8515625, c3 = 18.6875; + float3 ym = pow(max(l * (paper / 10000.0), float3(1e-30)), float3(m1)); + return pow((c1 + c2 * ym) / (1.0 + c3 * ym), float3(m2)); +} +// A highlight brought to SDR white as an SDR screen shows it: the largest channel goes to 1.0 and the others scale with it, so the hue holds (a screenshot of an HDR frame) +static inline float3 present_to_sdr_white(float3 c0) { + float3 c = clamp(c0, -65504.0, 65504.0); // finite: an infinite channel would give inf * 0 below + float m = max(max(c.r, c.g), c.b); + return m > 1.0 ? c * (1.0 / m) : c; +} +// Every target but the float HDR layer. The PQ HDR layer (RGB10A2 in BT.2100 PQ): the game's HDR10 window as it is (mode 0), an SDR or float source encoded as HDR10 at the paper white +// (modes 3, 4). The SDR drawable (gamma-coded 8-bit, BT.709; a screenshot): an HDR10 source at its paper white, its highlights brought to SDR white with their hue +// (present_to_sdr_white), then converted to BT.709; a colour outside BT.709 is clipped +fragment float4 fs(V in [[stage_in]], texture2d t [[texture(0)]], sampler s [[sampler(0)]], constant Crop& c [[buffer(0)]]) { + float4 v = t.sample(s, clamp(in.uv * c.rect.zw + c.rect.xy, c.edge.xy, c.edge.zw)); + if (c.mode.x > 3.5) v.rgb = present_linear_to_pq(kRec709To2020 * clamp(v.rgb, 0.0, 65504.0), c.mode.y); // the PQ layer: a float source (linear BT.709) as HDR10 at the paper white + else if (c.mode.x > 2.5) v.rgb = present_linear_to_pq(kRec709To2020 * present_srgb_to_linear(v.rgb), c.mode.y); // the PQ layer: an SDR source as HDR10 at the paper white + else if (c.mode.x > 1.5) v.rgb = present_linear_to_srgb(saturate(kRec2020To709 * present_to_sdr_white(present_pq_to_linear(v.rgb, c.mode.y)))); + return v; +} +// The float HDR layer (RGBA16Float, extended linear BT.2020, DM_HDR_LAYER=float; the PQ layer replaced it): linear light, where 1.0 is the paper white. The layer carries HDR10 metadata (front_mac.m, front_ios.m), so +// Core Animation maps the values above 1.0 into the screen's headroom: they are written as they are +fragment float4 fs_linear(V in [[stage_in]], texture2d t [[texture(0)]], sampler s [[sampler(0)]], constant Crop& c [[buffer(0)]]) { + float4 v = t.sample(s, clamp(in.uv * c.rect.zw + c.rect.xy, c.edge.xy, c.edge.zw)); + if (c.mode.x > 1.5) v.rgb = present_pq_to_linear(v.rgb, c.mode.y); // HDR10 is already BT.2020 + else if (c.mode.x > 0.5) v.rgb = kRec709To2020 * present_srgb_to_linear(v.rgb); // a BT.709 (sRGB) source: decoded, converted to BT.2020 + else v.rgb = kRec709To2020 * clamp(v.rgb, -65504.0, 65504.0); // a float source as stored: linear BT.709, converted to BT.2020 (untested: no local game presents a float window); finite first, so the matrix cannot make a NaN + v.rgb = clamp(v.rgb, -65504.0, 65504.0); // finite in the half-float layer + return v; +} diff --git a/dm/gfx/nvn/nvn_mtl.m b/dm/gfx/nvn/nvn_mtl.m index 1ed62fe698..a0107fd50f 100644 --- a/dm/gfx/nvn/nvn_mtl.m +++ b/dm/gfx/nvn/nvn_mtl.m @@ -590,7 +590,7 @@ void mtl_init(void) { g_present_args = [g_dev newArgumentTableWithDescriptor:pad error:&ate]; if (!g_present_args) MDIE("failed to create the present argument table: %s", ate.localizedDescription.UTF8String); for (int i = 0; i < 3; i++) { - g_present_uv[i] = [g_dev newBufferWithLength:sizeof(float) * 8 options:MTLResourceStorageModeShared]; + g_present_uv[i] = [g_dev newBufferWithLength:sizeof(float) * 12 options:MTLResourceStorageModeShared]; if (!g_present_uv[i]) MDIE("failed to create present UV buffer[%d]", i); res_add(g_present_uv[i]); } @@ -13794,7 +13794,7 @@ static void shot(id src, const char *path, int screen, const int cro read = out; } if (read.pixelFormat != MTLPixelFormatRGBA8Unorm && read.pixelFormat != MTLPixelFormatRGBA8Unorm_sRGB && - read.pixelFormat != MTLPixelFormatBGRA8Unorm && read.pixelFormat != MTLPixelFormatBGRA8Unorm_sRGB) { MLOG("[RED] screenshot does not support format %lu", (unsigned long)read.pixelFormat); return; } + read.pixelFormat != MTLPixelFormatBGRA8Unorm && read.pixelFormat != MTLPixelFormatBGRA8Unorm_sRGB && read.pixelFormat != MTLPixelFormatRGB10A2Unorm) { MLOG("[RED] screenshot does not support format %lu", (unsigned long)read.pixelFormat); return; } size_t w = read.width, h = read.height, row = w * 4; id buf = [g_dev newBufferWithLength:row * h options:MTLResourceStorageModeShared]; id cb = q_cb(); @@ -13805,10 +13805,21 @@ static void shot(id src, const char *path, int screen, const int cro if (screen && read == src && atomic_load_explicit(&g_origin_ll, memory_order_relaxed)) { // lower-left origin: the window image is stored bottom-up (see g_origin_ll) uint8_t *p = buf.contents, *tmp = malloc(row); if (tmp) { for (size_t y = 0; y < h / 2; y++) { memcpy(tmp, p + y * row, row); memcpy(p + y * row, p + (h - 1 - y) * row, row); memcpy(p + (h - 1 - y) * row, tmp, row); } free(tmp); } } + // An RGB10A2 frame (a 10-bit window: the HDR window with DM_HDR=1, or any other 10-bit window, which used to be refused): the PNG holds each channel's top 8 bits as the code values, not decoded; path.raw keeps the 32-bit words (R bits 0-9, G 10-19, B 20-29) for analysis + uint8_t *pix = buf.contents, *c8 = NULL; + if (read.pixelFormat == MTLPixelFormatRGB10A2Unorm) { + const uint32_t *s = buf.contents; + c8 = malloc(w * h * 4); + if (!c8) { MLOG("[RED] screenshot %s: no memory for the 10-bit conversion", path); return; } + { for (size_t i = 0; i < w * h; i++) { uint32_t v = s[i]; c8[4 * i] = (v & 0x3ff) >> 2; c8[4 * i + 1] = ((v >> 10) & 0x3ff) >> 2; c8[4 * i + 2] = ((v >> 20) & 0x3ff) >> 2; c8[4 * i + 3] = 255; } pix = c8; } + char rawp[1024]; snprintf(rawp, sizeof rawp, "%s.raw", path); + FILE *rf = fopen(rawp, "wb"); + if (rf) { fprintf(rf, "rgb10a2 %zu %zu\n", w, h); fwrite(buf.contents, 4, w * h, rf); fclose(rf); } + } int bgra = read.pixelFormat == MTLPixelFormatBGRA8Unorm || read.pixelFormat == MTLPixelFormatBGRA8Unorm_sRGB; CGColorSpaceRef cs = CGColorSpaceCreateWithName(kCGColorSpaceSRGB); CGBitmapInfo bi = bgra ? (kCGImageAlphaNoneSkipFirst | kCGBitmapByteOrder32Little) : (CGBitmapInfo)kCGImageAlphaNoneSkipLast; - CGContextRef ctx = CGBitmapContextCreate(buf.contents, w, h, 8, row, cs, bi); + CGContextRef ctx = CGBitmapContextCreate(pix, w, h, 8, row, cs, bi); CGImageRef img = ctx ? CGBitmapContextCreateImage(ctx) : NULL; CFURLRef url = CFURLCreateFromFileSystemRepresentation(NULL, (const UInt8 *)path, strlen(path), false); CGImageDestinationRef dst = url ? CGImageDestinationCreateWithURL(url, (__bridge CFStringRef)UTTypePNG.identifier, 1, NULL) : NULL; @@ -13816,7 +13827,7 @@ static void shot(id src, const char *path, int screen, const int cro int ok = 0; if (!ctx || !img || !dst) MLOG("[RED] screenshot %s failed: %s", path, !ctx ? "cannot create the bitmap context" : !img ? "cannot get the image" : "cannot write (directory missing or not writable)"); else { CGImageDestinationAddImage(dst, img, NULL); ok = CGImageDestinationFinalize(dst); } - if (dst) CFRelease(dst); if (url) CFRelease(url); CGImageRelease(img); CGContextRelease(ctx); CGColorSpaceRelease(cs); + if (dst) CFRelease(dst); if (url) CFRelease(url); CGImageRelease(img); CGContextRelease(ctx); CGColorSpaceRelease(cs); free(c8); MLOG("screenshot %zux%zu format %lu → %s %s", w, h, (unsigned long)read.pixelFormat, path, ok ? "ok" : "failed"); if (g_defer_ab >= 0 && getenv("DM_NVN_DEFER_AB") && !strcmp(getenv("DM_NVN_DEFER_AB"), "1") && ok) { static int shots; if (++shots == 2) { g_defer_ab_on = 1; MLOG("DM_NVN_DEFER_AB: second screenshot written; deferred pass merging on"); } } if (g_udc_ab >= 0 && getenv("DM_NVN_UDC_AB") && !strcmp(getenv("DM_NVN_UDC_AB"), "1") && ok) { @@ -14040,7 +14051,9 @@ static void present_pipeline(MTLPixelFormat format) { id lib = host_lib("present", &e); if (!lib) MDIE("present shader compile failed %s", e.localizedDescription.UTF8String); MTLRenderPipelineDescriptor *pd = [MTLRenderPipelineDescriptor new]; - pd.vertexFunction = [lib newFunctionWithName:@"vs"]; pd.fragmentFunction = [lib newFunctionWithName:@"fs"]; + // The float HDR layer (RGBA16Float, extended linear BT.2020, DM_HDR_LAYER=float) takes the picture in linear light: fs_linear converts gamma-coded 8-bit BT.709 sources to BT.2020 and PQ-decodes HDR10 sources. + // Every other target, the PQ HDR layer (RGB10A2) included, is drawn by fs (present_encode and present_encode_m4 set the mode) + pd.vertexFunction = [lib newFunctionWithName:@"vs"]; pd.fragmentFunction = [lib newFunctionWithName:format == MTLPixelFormatRGBA16Float ? @"fs_linear" : @"fs"]; pd.colorAttachments[0].pixelFormat = format; g_blit_pso = [g_dev newRenderPipelineStateWithDescriptor:pd error:&e]; if (!g_blit_pso) MDIE("present pipeline failed %s", e.localizedDescription.UTF8String); @@ -14097,6 +14110,24 @@ static int present_crop_rect(id src, const int crop[4], NSUInteger r } r[0] = crop[0]; r[1] = crop[1]; r[2] = crop[2]; r[3] = crop[3]; return 1; } +// The Crop mode (present.metal): how the source is encoded. An RGB10A2 source is the game's HDR10 window (SMPTE ST 2084 code values in BT.2020 primaries) once its status query has answered HDR on; an 8-bit gamma source (BT.709, sRGB) drawn into the extended-range BT.2020 layer is decoded and converted to BT.2020 +__attribute__((weak)) int vi_hdr_answered(void) { return 0; } // hle_vi.c; the graphics unit tests have no vi service +static float present_mode(MTLPixelFormat src, MTLPixelFormat dst) { + // An RGB10A2 window is PQ code values only once the game's status query has answered HDR on (vi_hdr_answered); any other RGB10A2 window is read as stored + // Not answered: an SDR 10-bit window, gamma-coded like the 8-bit ones, so it is decoded to linear light for an HDR layer + int pq_win = src == MTLPixelFormatRGB10A2Unorm && vi_hdr_answered(); + // The PQ HDR layer (RGB10A2 in BT.2100 PQ, front_mac.m and front_ios.m): the game's HDR10 window goes in as it is (0); a float source is linear BT.709, encoded as HDR10 (4); + // any other is gamma-coded BT.709, decoded and encoded as HDR10 (3), its white at the paper white either way + if (dst == MTLPixelFormatRGB10A2Unorm) + return pq_win ? 0.0f : src == MTLPixelFormatRGBA16Float || src == MTLPixelFormatRGBA32Float || src == MTLPixelFormatRG11B10Float || src == MTLPixelFormatRGB9E5Float ? 4.0f : 3.0f; + if (src == MTLPixelFormatRGB10A2Unorm) return pq_win ? 2.0f : (dst == MTLPixelFormatRGBA16Float ? 1.0f : 0.0f); + if (dst != MTLPixelFormatRGBA16Float) return 0.0f; + // Gamma-coded 8-bit sources. The sRGB-typed ones are sampled through a non-sRGB view (present_raw_src), so the shader reads gamma-coded values for them too + return src == MTLPixelFormatRGBA8Unorm || src == MTLPixelFormatBGRA8Unorm || src == MTLPixelFormatRGBA8Unorm_sRGB || src == MTLPixelFormatBGRA8Unorm_sRGB ? 1.0f : 0.0f; +} +// The paper white in nits: the luminance of the picture's SDR white, the value nn::vi answers the game (hle_vi.c dm_hdr_paper_white: DM_HDR_PAPER_WHITE, default 203); 203 in the graphics unit tests, which have no vi service +__attribute__((weak)) float dm_hdr_paper_white(void) { return 203.0f; } +static float present_paper_white(void) { return dm_hdr_paper_white(); } static int present_encode(id cb, id src, id dst, const int crop[4]) { NSUInteger r[4]; if (!present_crop_rect(src, crop, r)) return 0; NSUInteger cx = r[0], cy = r[1], cw = r[2], ch = r[3]; @@ -14118,8 +14149,8 @@ static int present_encode(id cb, id src, id en = [cb renderCommandEncoderWithDescriptor:rp]; [en setViewport:vp]; [en setRenderPipelineState:g_blit_pso]; [en setFragmentTexture:present_raw_src(src) atIndex:0]; [en setFragmentSamplerState:g_blit_samp atIndex:0]; - float uv[8] = {(float)cx / src.width, (float)cy / src.height, (float)cw / src.width, (float)ch / src.height, - (cx + 0.5f) / src.width, (cy + 0.5f) / src.height, (cx + cw - 0.5f) / src.width, (cy + ch - 0.5f) / src.height}; + float uv[12] = {(float)cx / src.width, (float)cy / src.height, (float)cw / src.width, (float)ch / src.height, + (cx + 0.5f) / src.width, (cy + 0.5f) / src.height, (cx + cw - 0.5f) / src.width, (cy + ch - 0.5f) / src.height, present_mode(src.pixelFormat, dst.pixelFormat), present_paper_white(), 0, 0}; if (atomic_load_explicit(&g_origin_ll, memory_order_relaxed)) { uv[1] += uv[3]; uv[3] = -uv[3]; } // lower-left origin: the window image is stored bottom-up [en setFragmentBytes:uv length:sizeof uv atIndex:0]; [en drawPrimitives:MTLPrimitiveTypeTriangle vertexStart:0 vertexCount:3]; @@ -14369,6 +14400,8 @@ static int present_encode_m4(id cb, id src, id cb, id src, id src, const int crop[4], id out, void (^done)(int)) { id al = [g_dev newCommandAllocator]; id cb = [g_dev newCommandBuffer]; - id uvb = [g_dev newBufferWithLength:sizeof(float) * 8 options:MTLResourceStorageModeShared]; + id uvb = [g_dev newBufferWithLength:sizeof(float) * 12 options:MTLResourceStorageModeShared]; if (!al || !cb || !uvb) { MLOG("[RED] offscreen present: no command allocator or buffer"); return 0; } id raw = present_raw_src(src); fx_state fx = {0}; @@ -14418,6 +14451,25 @@ static void present_gpu_note(double s, int fx) { n[fx] = 0; } static double g_link_it_tin; // Enqueue time of the frame the display-link callback is presenting (read and written only on the display-link thread) +// DM_PRESENT_SHOWN=1: how long the system takes to put a frame on screen, from the drawable's presentedTime (the time it really appeared, CACurrentMediaTime base). +// Every 300 frames one line: from the present call to on screen, and from the display-link tick the frame was meant for to on screen (display-link pacing only), p50 / p90 / max, +// and the frames the system dropped; with the layer's format and whether it has EDR metadata (DM_HDR), to compare HDR with SDR +static int present_shown_on(void) { static int v = -1; if (v < 0) { const char *e = getenv("DM_PRESENT_SHOWN"); v = e && atoi(e) != 0; } return v; } +static int present_shown_cmp(const void *a, const void *b) { double x = *(const double *)a, y = *(const double *)b; return (x > y) - (x < y); } +static void present_shown_note(double shown, double call, double target, CAMetalLayer *layer) { // the drawable's presented handler (any thread) + static pthread_mutex_t mu = PTHREAD_MUTEX_INITIALIZER; static double c[300], t[300]; static int n, nt, dropped; + pthread_mutex_lock(&mu); + if (!(shown > 0)) dropped++; + else { c[n++] = (shown - call) * 1000.0; if (target > 0) t[nt++] = (shown - target) * 1000.0; } + if (n == 300) { + qsort(c, 300, sizeof *c, present_shown_cmp); if (nt) qsort(t, (size_t)nt, sizeof *t, present_shown_cmp); + char tgt[96] = "no display-link target"; if (nt) snprintf(tgt, sizeof tgt, "p50 %.2f / p90 %.2f / max %.2f ms", t[nt / 2], t[nt * 9 / 10], t[nt - 1]); + MLOG("present shown (DM_PRESENT_SHOWN), 300 frames: present call to on screen p50 %.2f / p90 %.2f / max %.2f ms; display-link target to on screen %s; dropped %d; layer format %lu, %s", + c[150], c[270], c[299], tgt, dropped, (unsigned long)layer.pixelFormat, layer.EDRMetadata ? "with EDR metadata" : "no EDR metadata"); + n = nt = dropped = 0; + } + pthread_mutex_unlock(&mu); +} static int present_now(const tex_t *t, double period, uint64_t frame, uint64_t ticket, const int crop[4], id supplied, double target_ts) { if (g_present_stall_ns) { struct timespec ts = {0, (long)g_present_stall_ns}; nanosleep(&ts, NULL); } g_present_n++; @@ -14530,10 +14582,12 @@ static int present_now(const tex_t *t, double period, uint64_t frame, uint64_t t #if TARGET_OS_SIMULATOR // The iOS simulator SDK has no presentation-time callback and no minimum display duration (used only for picture acceptance, display-link ticks are not counted) (void)period; [dr present]; #else + double shown_call = CACurrentMediaTime(), shown_target = target_ts; // DM_PRESENT_SHOWN: when the frame was handed to the system, and the display-link tick it was meant for uint64_t pn = ticket; // Record the present time by the game's present sequence number (ticket number): hle's present statistics are taken by "the game's nth present"; in the display link path the actual present count != the game's present count (old frames get merged and dropped) [dr addPresentedHandler:^(id d) { // Actual presentation time (the time this frame is really put on screen, CACurrentMediaTime base) CFTimeInterval tt = d.presentedTime; g_pres[pn % PRING] = (pres_ent){pn, tt > 0 ? tt : -1}; + if (present_shown_on()) present_shown_note(tt, shown_call, shown_target, layer); }]; // Each frame is displayed for at least one present period: when the GPU completion time jitters around a 120 Hz refresh boundary, the paired "25 ms + 8.33 ms" stutter no longer appears if (target_ts > 0) [dr present]; diff --git a/dm/gfx/tests/Makefile b/dm/gfx/tests/Makefile index 733f0705a0..b389f86583 100644 --- a/dm/gfx/tests/Makefile +++ b/dm/gfx/tests/Makefile @@ -28,10 +28,10 @@ MTL_T_DEP=../nvn/hle_nvn.c ../../host/crash.c ../../host/hle_bigstack.c t_host_s MTL_T_SRC=../nvn/hle_nvn.c ../../host/crash.c ../../host/hle_bigstack.c t_host_stubs.c $(filter-out ../nvn/nvn_mtl.m,$(GFX)) # The run list: every t_*.c and t_*.m but those in NOT_RUN. `make run` builds them all (in parallel with -j), then runs them one after another, each through its check_ # target: ./t_x by default, with the environment its RUN_ENV gives it (check_t_x: RUN_ENV = MTL_DEBUG_LAYER=1, beside its build rule), or a check_t_x recipe of its own when it -# runs several ways. A new test needs its source and its build rule; no list to add it to. NOT_RUN: helpers linked into other tests (t_host_stubs, t_present_rescue_focus, t_gpu_lost_front), -# a fixture test_image_atomic_bounds.py builds itself (t_atomic_return), t_geometry_primitive_id (the GPU half of test_dmsh_geometry.py, which gives it the directory of +# runs several ways. A new test needs its source and its build rule; no list to add it to. NOT_RUN: helpers linked into other tests (t_host_stubs, t_present_rescue_focus, t_gpu_lost_front, +# t_present_hdr_front), a fixture test_image_atomic_bounds.py builds itself (t_atomic_return), t_geometry_primitive_id (the GPU half of test_dmsh_geometry.py, which gives it the directory of # its translated fixtures), and t_gpuprof, which CI's CPU subset runs (.github/workflows/tests.yml) -NOT_RUN = t_host_stubs t_present_rescue_focus t_gpu_lost_front t_atomic_return t_geometry_primitive_id t_gpuprof +NOT_RUN = t_host_stubs t_present_rescue_focus t_gpu_lost_front t_present_hdr_front t_atomic_return t_geometry_primitive_id t_gpuprof RUN_TESTS = $(filter-out $(NOT_RUN),$(basename $(sort $(wildcard t_*.c t_*.m)))) check_t_%: t_% $(RUN_ENV) ./$< @@ -190,6 +190,9 @@ t_present_rescue: t_present_rescue.m t_present_rescue_focus.c $(MTL_T_DEP) .PHONY: check_t_present_rescue check_t_present_rescue: t_present_rescue MTL_DEBUG_LAYER=1 ./t_present_rescue --metal +t_present_hdr: t_present_hdr.m t_present_hdr_front.c ../hle_vi.c $(MTL_T_DEP) + clang $(CFLAGS) -fobjc-arc -o $@ t_present_hdr.m t_present_hdr_front.c ../hle_vi.c $(MTL_T_SRC) $(FW) +check_t_present_hdr: RUN_ENV = MTL_DEBUG_LAYER=1 t_present_alloc: t_present_alloc.m $(MTL_T_DEP) clang $(CFLAGS) -fobjc-arc -o $@ t_present_alloc.m $(MTL_T_SRC) $(FW) .PHONY: check_t_present_alloc diff --git a/dm/gfx/tests/t_present_hdr.m b/dm/gfx/tests/t_present_hdr.m new file mode 100644 index 0000000000..738e02b9b7 --- /dev/null +++ b/dm/gfx/tests/t_present_hdr.m @@ -0,0 +1,220 @@ +// HDR output (DM_HDR=1): the status query's answer and the present's decode, on the GPU with closed-form expectations (no game, no window). +// The vi answer: HDR on only while the window's screen can show EDR, format 1 with it (the SDK's on = format == 1), the decode latched once answered. The present: PQ code values +// (an RGB10A2 window) written as they are into the PQ layer (RGB10A2 in BT.2100 PQ) and SDR and float sources encoded for it at the paper white; for the float layer, +// decoded with 1.0 at the 203-nit paper white and written as they are above it (Core Animation maps them by the layer's HDR10 metadata), an 8-bit sRGB +// source converted to BT.2020, a screenshot of an HDR frame brought to SDR white with its hue, and a 10-bit window of a game +// that never asked for HDR read as stored. hle_vi.c is linked beside this file; t_present_hdr_front.c stands in for the front end +#include "../nvn/nvn_mtl.m" +#include +#include +void *front_metal_layer(void) { return NULL; } +module_t *module_of(const void *p) { (void)p; return NULL; } +module_t *g_mod; int g_nmod; +void svc_thread_dump(void) {} +uint64_t now_ns(void) { return bp_now(); } +extern int t_hdr_screen; extern unsigned t_hdr_lo, t_hdr_hi; extern float t_hdr_headroom; +int vi_hdr_answered(void); int dm_hdr_mode(void); void dm_hdr_set_mode(int m); +static int checks, failed; +#define CHECK(name, ok) do { int pass_ = !!(ok); checks++; failed += !pass_; printf("%s: %s\n", name, pass_ ? "PASS" : "FAIL"); } while (0) + +// hle_vi.c's HLE entries are static: found by their mangled names in the table the loader reads (hle.h HLE_FORCE) +typedef struct { const char *name; void *fn; const char *why; } force_ent; +static void *hle_fn(const char *name) { + unsigned long n = 0; const force_ent *e = (const force_ent *)(const void *)getsectiondata(&_mh_execute_header, "__DATA", "swforce", &n); + for (unsigned long i = 0; e && i < n / sizeof *e; i++) if (!strcmp(e[i].name, name)) return e[i].fn; + return NULL; +} +typedef uint32_t (*status_fn)(uint8_t *, uint32_t *, uint16_t *, const void *); +typedef uint32_t (*paper_fn)(uint16_t *, const void *); +typedef uint32_t (*range_fn)(void *, uint32_t, uint32_t); + +// SMPTE ST 2084: nits to a 10-bit code value, and a 10-bit code value back to nits (what the shader's decode must give) +static unsigned pq_code(double nits) { + const double m1 = 2610.0 / 16384, m2 = 2523.0 / 32, c1 = 3424.0 / 4096, c2 = 2413.0 / 128, c3 = 2392.0 / 128; + double y = pow(nits / 10000.0, m1); + return (unsigned)lround(pow((c1 + c2 * y) / (1 + c3 * y), m2) * 1023.0); +} +static double pq_nits(unsigned code) { + const double m1 = 2610.0 / 16384, m2 = 2523.0 / 32, c1 = 3424.0 / 4096, c2 = 2413.0 / 128, c3 = 2392.0 / 128; + double e = pow(code / 1023.0, 1.0 / m2); + return 10000.0 * pow(fmax(e - c1, 0) / (c2 - c3 * e), 1.0 / m1); +} +static double srgb_dec(double c) { return c <= 0.04045 ? c / 12.92 : pow((c + 0.055) / 1.055, 2.4); } +static double srgb_enc(double l) { return l <= 0.0031308 ? l * 12.92 : 1.055 * pow(l, 1.0 / 2.4) - 0.055; } +#define N 8 +// An N x 1 RGB10A2 source of code values (r, g, b per pixel) +static id src10(const unsigned c[N][3]) { + MTLTextureDescriptor *d = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatRGB10A2Unorm width:N height:1 mipmapped:NO]; + d.storageMode = MTLStorageModeShared; d.usage = MTLTextureUsageShaderRead; + id t = [g_dev newTextureWithDescriptor:d]; uint32_t w[N]; + for (int i = 0; i < N; i++) w[i] = c[i][0] | c[i][1] << 10 | c[i][2] << 20 | 3u << 30; + [t replaceRegion:MTLRegionMake2D(0, 0, N, 1) mipmapLevel:0 withBytes:w bytesPerRow:sizeof w]; + return t; +} +static id src8(const uint8_t c[N][3]) { + MTLTextureDescriptor *d = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatRGBA8Unorm width:N height:1 mipmapped:NO]; + d.storageMode = MTLStorageModeShared; d.usage = MTLTextureUsageShaderRead; + id t = [g_dev newTextureWithDescriptor:d]; uint8_t b[N * 4]; + for (int i = 0; i < N; i++) { b[4 * i] = c[i][0]; b[4 * i + 1] = c[i][1]; b[4 * i + 2] = c[i][2]; b[4 * i + 3] = 255; } + [t replaceRegion:MTLRegionMake2D(0, 0, N, 1) mipmapLevel:0 withBytes:b bytesPerRow:sizeof b]; + return t; +} +// An N x 1 RGBA16Float source of linear values (a float window) +static id src16f(const float c[N][3]) { + MTLTextureDescriptor *d = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatRGBA16Float width:N height:1 mipmapped:NO]; + d.storageMode = MTLStorageModeShared; d.usage = MTLTextureUsageShaderRead; + id t = [g_dev newTextureWithDescriptor:d]; __fp16 h[N * 4]; + for (int i = 0; i < N; i++) { for (int k = 0; k < 3; k++) h[4 * i + k] = (__fp16)c[i][k]; h[4 * i + 3] = 1; } + [t replaceRegion:MTLRegionMake2D(0, 0, N, 1) mipmapLevel:0 withBytes:h bytesPerRow:sizeof h]; + return t; +} +// The present of src into an N x 1 target of the given format; out gets RGB as floats (16-bit float targets), 0-1023 (RGB10A2, the PQ layer) or 0-255 (8-bit targets) +static int run(id src, MTLPixelFormat fmt, double out[N][3]) { + MTLTextureDescriptor *d = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:fmt width:N height:1 mipmapped:NO]; + d.storageMode = MTLStorageModeShared; d.usage = MTLTextureUsageRenderTarget; + id dst = [g_dev newTextureWithDescriptor:d]; id cb = [g_q commandBuffer]; + if (!present_encode(cb, src, dst, NULL)) return 0; + [cb commit]; [cb waitUntilCompleted]; + if (cb.status != MTLCommandBufferStatusCompleted) return 0; + if (fmt == MTLPixelFormatRGBA16Float) { + __fp16 h[N * 4]; [dst getBytes:h bytesPerRow:sizeof h fromRegion:MTLRegionMake2D(0, 0, N, 1) mipmapLevel:0]; + for (int i = 0; i < N; i++) for (int k = 0; k < 3; k++) out[i][k] = h[4 * i + k]; + } else if (fmt == MTLPixelFormatRGB10A2Unorm) { // the PQ layer: 10-bit code values + uint32_t w[N]; [dst getBytes:w bytesPerRow:sizeof w fromRegion:MTLRegionMake2D(0, 0, N, 1) mipmapLevel:0]; + for (int i = 0; i < N; i++) for (int k = 0; k < 3; k++) out[i][k] = w[i] >> (10 * k) & 0x3ff; + } else { + uint8_t b[N * 4]; [dst getBytes:b bytesPerRow:sizeof b fromRegion:MTLRegionMake2D(0, 0, N, 1) mipmapLevel:0]; + for (int i = 0; i < N; i++) for (int k = 0; k < 3; k++) out[i][k] = b[4 * i + k]; + } + return 1; +} +static int near(double a, double b, double rel, double abs_) { return fabs(a - b) <= fmax(abs_, rel * fabs(b)); } + +int main(void) { @autoreleasepool { + setenv("DM_HDR", "1", 1); unsetenv("DM_HDR_PAPER_WHITE"); unsetenv("DM_HDR_PEAK"); + mtl_init(); + mtl_present_layer(N, 1); // a layer of the fixtures' N x 1 shape, so each source texel lands on its own target texel: left to the display's 16:9, the picture is a bar in the middle (present_shape) + status_fn status = (status_fn)hle_fn("_ZN2nn2vi21QueryDisplayHdrStatusEPbPNS0_15HdrOutputFormatEPtPKNS0_7DisplayE"); + paper_fn paper = (paper_fn)hle_fn("_ZN2nn2vi25QueryDisplayHdrPaperWhiteEPtPKNS0_7DisplayE"); + range_fn range = (range_fn)hle_fn("_ZN2nn2vi52SetLayerNextHdrExtendedContentLuminanceRangeMetadataEPNS0_5LayerEtt"); + CHECK("the vi HLEs are registered", status && paper && range); + if (!status || !paper || !range) return 1; + double o[N][3]; + + // A 10-bit window of a game whose status query has not answered HDR: read as stored, not as PQ + { + unsigned c[N][3]; for (int i = 0; i < N; i++) c[i][0] = c[i][1] = c[i][2] = 512; + CHECK("not answered: an RGB10A2 window into an 8-bit target is its stored value (512 → 128)", run(src10(c), MTLPixelFormatRGBA8Unorm, o) && fabs(o[0][0] - 128) <= 1 && fabs(o[N - 1][2] - 128) <= 1); + CHECK("not answered: into the HDR layer it is decoded as gamma-coded SDR, not PQ", run(src10(c), MTLPixelFormatRGBA16Float, o) && near(o[0][1], srgb_dec(512 / 1023.0), 0.01, 0.002)); + } + + // The status query: SDR on a screen without EDR, HDR (format 1, peak 1000) on one with it; the decode stays on once answered + { + uint8_t on = 9; uint32_t fmt = 9; uint16_t peak = 9; + t_hdr_screen = 0; status(&on, &fmt, &peak, NULL); + CHECK("a screen without EDR: the status answers SDR (on 0, format 0, peak 0)", on == 0 && fmt == 0 && peak == 0); + CHECK("a screen without EDR: the window is not taken as PQ", vi_hdr_answered() == 0); + t_hdr_screen = 1; status(&on, &fmt, &peak, NULL); + CHECK("a screen with EDR: HDR on, format 1, peak 1000", on == 1 && fmt == 1 && peak == 1000); + CHECK("a screen with EDR: the window is taken as PQ", vi_hdr_answered() == 1); + t_hdr_screen = 0; status(&on, &fmt, &peak, NULL); + CHECK("the window moves to a screen without EDR: SDR is answered, and the HDR window already made is still decoded", on == 0 && fmt == 0 && vi_hdr_answered() == 1); + t_hdr_screen = 1; + uint16_t pw = 0; paper(&pw, NULL); + CHECK("the paper white is 203 nits", pw == 203); + range(NULL, 0, 1000); + CHECK("the declared extended range reaches the front end (0..1000)", t_hdr_lo == 0 && t_hdr_hi == 1000); + } + + // HDR output changed while the game runs (the iOS in-game menu, dm_hdr_set_mode): the next status query follows, DM_HDR=1 having been read at start + { + uint8_t on = 9; uint32_t fmt = 9; uint16_t peak = 9; + int m0 = dm_hdr_mode(); + dm_hdr_set_mode(0); status(&on, &fmt, &peak, NULL); + CHECK("turned off while the game runs: the next query answers SDR (on 0, format 0, peak 0)", m0 == 1 && dm_hdr_mode() == 0 && on == 0 && fmt == 0 && peak == 0); + dm_hdr_set_mode(1); status(&on, &fmt, &peak, NULL); + CHECK("turned on again: HDR on, format 1, peak 1000", on == 1 && fmt == 1 && peak == 1000); + t_hdr_screen = 0; dm_hdr_set_mode(2); status(&on, &fmt, &peak, NULL); + CHECK("mode 2 (DM_HDR=2): HDR on a screen without EDR too", on == 1 && fmt == 1); + dm_hdr_set_mode(1); t_hdr_screen = 1; + } + + // The peak (the HDR peak row): a number as it is; DM_HDR_PEAK=auto, the paper white (203) times the screen's current headroom, rounded to 10 nits, never below the paper white + // and at most 16 bits; SDR still answers peak 0. DM_HDR_PEAK is read at every query, so the cases can follow one another + { + uint8_t on = 9; uint32_t fmt = 9; uint16_t peak = 9; + setenv("DM_HDR_PEAK", "1600", 1); status(&on, &fmt, &peak, NULL); + CHECK("DM_HDR_PEAK=1600: peak 1600", on == 1 && peak == 1600); + setenv("DM_HDR_PEAK", "auto", 1); + t_hdr_headroom = 3.5f; status(&on, &fmt, &peak, NULL); + CHECK("automatic, headroom 3.5: peak 710 (203 × 3.5 = 710.5, to 10 nits)", on == 1 && fmt == 1 && peak == 710); + t_hdr_headroom = 16.0f; status(&on, &fmt, &peak, NULL); + CHECK("automatic, headroom 16 (the XDR panel's potential headroom): peak 3250", peak == 3250); + t_hdr_headroom = 1.0f; status(&on, &fmt, &peak, NULL); + uint16_t p1 = peak; t_hdr_headroom = 0.5f; status(&on, &fmt, &peak, NULL); + uint16_t p05 = peak; t_hdr_headroom = NAN; status(&on, &fmt, &peak, NULL); + CHECK("automatic, headroom 1, 0.5 or not a number: the paper white, rounded up (203 → 203)", p1 == 203 && p05 == 203 && peak == 203); + t_hdr_headroom = 1000.0f; status(&on, &fmt, &peak, NULL); + CHECK("automatic, a headroom past 16 bits (203 × 1000): peak 65535", peak == 65535); + t_hdr_headroom = 2.0f; t_hdr_screen = 0; status(&on, &fmt, &peak, NULL); + CHECK("automatic on a screen without EDR: SDR, peak 0", on == 0 && peak == 0); + t_hdr_screen = 1; t_hdr_headroom = 1.0f; unsetenv("DM_HDR_PEAK"); + status(&on, &fmt, &peak, NULL); + CHECK("unset again: peak 1000", peak == 1000); + } + + const double nits[N] = {0, 0.1, 10, 80, 203, 500, 1000, 10000}; + unsigned grey[N][3]; double want[N]; + for (int i = 0; i < N; i++) { grey[i][0] = grey[i][1] = grey[i][2] = pq_code(nits[i]); want[i] = pq_nits(grey[i][0]) / 203.0; } + + // The HDR layer holds linear light, 1.0 at 203 nits, highlights as they are: Core Animation maps them into the screen's headroom by the layer's HDR10 metadata + { + int ok = run(src10(grey), MTLPixelFormatRGBA16Float, o), bad = 0; + for (int i = 0; ok && i < N; i++) bad += !near(o[i][0], want[i], 0.002, 1e-5) || o[i][0] != o[i][1] || o[i][1] != o[i][2]; + CHECK("PQ 0, 0.1, 10, 80, 203, 500, 1000 and 10000 nits decode to nits / 203, grey stays grey", ok && !bad); + CHECK("203 nits is 1.0, SDR white", ok && near(o[4][0], 1.0, 0.002, 0)); + CHECK("10000 nits is written as it is (about 49), not limited by the host", ok && o[7][0] > 45.0); + } + { + unsigned c[N][3]; for (int i = 0; i < N; i++) { c[i][0] = pq_code(1000); c[i][1] = pq_code(500); c[i][2] = pq_code(100); } + int ok = run(src10(c), MTLPixelFormatRGBA16Float, o); + CHECK("a coloured highlight keeps each channel's own luminance (1000, 500 and 100 nits)", ok && near(o[0][0], pq_nits(c[0][0]) / 203.0, 0.002, 0) && near(o[0][1], pq_nits(c[0][1]) / 203.0, 0.002, 0) && near(o[0][2], pq_nits(c[0][2]) / 203.0, 0.002, 0)); + } + // An 8-bit sRGB source in the HDR layer (the game's SDR window before it switches): decoded and converted to BT.2020, its white at 1.0 + { + uint8_t c[N][3] = {{255, 255, 255}, {128, 128, 128}, {255, 0, 0}, {0, 255, 0}, {0, 0, 255}, {0, 0, 0}, {64, 64, 64}, {255, 255, 255}}; + int ok = run(src8(c), MTLPixelFormatRGBA16Float, o); + CHECK("SDR white is 1.0 in the HDR layer, the paper white the HDR window has", ok && near(o[0][0], 1.0, 0.002, 0) && near(o[0][2], 1.0, 0.002, 0)); + CHECK("SDR mid grey 128 decodes to its linear value", ok && near(o[1][1], srgb_dec(128 / 255.0), 0.003, 1e-4)); + CHECK("SDR red is BT.709 red in BT.2020 (0.6274, 0.0691, 0.0164)", ok && near(o[2][0], 0.627404, 0.003, 0) && near(o[2][1], 0.069097, 0.01, 1e-4) && near(o[2][2], 0.016391, 0.02, 1e-4)); + } + // The PQ layer (RGB10A2 in BT.2100 PQ, the Mac's default HDR layer): the game's HDR10 window goes in as it is, an 8-bit sRGB source is encoded as HDR10 with its white at the paper white + { + int ok = run(src10(grey), MTLPixelFormatRGB10A2Unorm, o), bad = 0; + for (int i = 0; ok && i < N; i++) for (int k = 0; k < 3; k++) bad += fabs(o[i][k] - grey[i][k]) > 0.5; + CHECK("PQ layer: the HDR10 window's code values are written as they are", ok && !bad); + uint8_t c[N][3] = {{255, 255, 255}, {128, 128, 128}, {255, 0, 0}, {0, 255, 0}, {0, 0, 255}, {0, 0, 0}, {64, 64, 64}, {255, 255, 255}}; + ok = run(src8(c), MTLPixelFormatRGB10A2Unorm, o); + CHECK("PQ layer: SDR white is the paper white (203 nits), black is code 0", ok && fabs(o[0][0] - pq_code(203)) <= 1.5 && fabs(o[0][2] - pq_code(203)) <= 1.5 && o[5][0] <= 1); + CHECK("PQ layer: SDR mid grey 128 is its linear value at the paper white", ok && fabs(o[1][1] - pq_code(srgb_dec(128 / 255.0) * 203)) <= 1.5); + float f[N][3] = {{1, 1, 1}, {0.5f, 0.5f, 0.5f}, {4, 4, 4}, {0, 0, 0}, {-1, -1, -1}, {1, 1, 1}, {1, 1, 1}, {1, 1, 1}}; + ok = run(src16f(f), MTLPixelFormatRGB10A2Unorm, o); + CHECK("PQ layer: a float source is linear light, 1.0 at the paper white, 4.0 at four times it, not decoded as sRGB", ok && fabs(o[0][1] - pq_code(203)) <= 1.5 && fabs(o[1][1] - pq_code(101.5)) <= 1.5 && + fabs(o[2][1] - pq_code(812)) <= 1.5); + CHECK("PQ layer: a negative float value is black", ok && o[3][1] <= 1 && o[4][1] <= 1); + } + // A screenshot of an HDR frame (an 8-bit target): as an SDR screen shows it, highlights to SDR white with their hue, then BT.709 + { + unsigned c[N][3]; for (int i = 0; i < N; i++) c[i][0] = c[i][1] = c[i][2] = pq_code(nits[i]); + c[7][0] = pq_code(1000); c[7][1] = pq_code(500); c[7][2] = pq_code(100); + int ok = run(src10(c), MTLPixelFormatRGBA8Unorm, o), bad = 0; + for (int i = 0; ok && i < 7; i++) bad += fabs(o[i][0] - 255.0 * srgb_enc(fmin(want[i], 1.0))) > 1.0; + CHECK("screenshot: grey to 203 nits as SDR, brighter grey at SDR white (255)", ok && !bad && o[4][0] == 255 && o[6][0] == 255); + double r = pq_nits(c[7][0]), v[3] = {1.0, pq_nits(c[7][1]) / r, pq_nits(c[7][2]) / r}, e[3]; + static const double m[3][3] = {{1.660491, -0.587641, -0.072850}, {-0.124550, 1.132900, -0.008349}, {-0.018151, -0.100579, 1.118730}}; + for (int k = 0; k < 3; k++) { double x = m[k][0] * v[0] + m[k][1] * v[1] + m[k][2] * v[2]; e[k] = 255.0 * srgb_enc(fmin(fmax(x, 0), 1)); } + CHECK("screenshot: a coloured highlight keeps its hue (1000, 500 and 100 nits as 1, 0.5 and 0.1 of SDR white, then BT.709)", ok && fabs(o[7][0] - e[0]) <= 1.5 && fabs(o[7][1] - e[1]) <= 1.5 && fabs(o[7][2] - e[2]) <= 1.5); + } + printf("%d checks, %d failed\n", checks, failed); + return failed != 0; +} } diff --git a/dm/gfx/tests/t_present_hdr_front.c b/dm/gfx/tests/t_present_hdr_front.c new file mode 100644 index 0000000000..8f00957db1 --- /dev/null +++ b/dm/gfx/tests/t_present_hdr_front.c @@ -0,0 +1,11 @@ +// What t_present_hdr takes of a front end and the loader: strong definitions in a file of their own, which replace the weak defaults in hle_vi.c +#include +#include +int t_hdr_screen = 1; // whether the window's screen can show EDR (front_hdr_screen) +unsigned t_hdr_lo, t_hdr_hi; // the last range the game declared (front_hdr_content_range) +int front_hdr_screen(void) { return t_hdr_screen; } +float t_hdr_headroom = 1.0f; // the screen's current EDR headroom (front_hdr_headroom: the automatic peak, DM_HDR_PEAK=auto) +float front_hdr_headroom(void) { return t_hdr_headroom; } +void front_hdr_content_range(unsigned lo, unsigned hi) { t_hdr_lo = lo; t_hdr_hi = hi; } +int dm_sdk_ounce(void) { return 1; } // a Switch 2 game (loader.c reads the sdk module's spec) +int dm_opmode(void) { return 0; } // handheld (hle_vi.c's display buffer answer) diff --git a/dm/host/front.h b/dm/host/front.h index d36e61c645..80e8f2aae5 100644 --- a/dm/host/front.h +++ b/dm/host/front.h @@ -20,6 +20,14 @@ int front_screen_max_fps(void); int front_display_pixels(int *w, int *h); // Whether the window's current screen mode supports variable refresh (Mac front_mac.m; none on iOS). Any thread int front_screen_variable_refresh(void); +// HDR output (DM_HDR=1; front_mac.m, front_ios.m): whether the window's screen can show EDR at all (its potential headroom is above 1: hle_vi.c answers HDR only then), and the +// game's declared extended luminance range in nits, for the layer's HDR10 metadata. hle_vi.c has weak defaults for runs without a front end +int front_hdr_screen(void); +void front_hdr_content_range(unsigned lo, unsigned hi); +float dm_hdr_paper_white(void); // hle_vi.c: the paper white in nits (DM_HDR_PAPER_WHITE, default 203), the luminance the picture's SDR white has +int dm_hdr_mode(void); // hle_vi.c: the HDR mode, 0 off, 1 by the screen, 2 on any screen (DM_HDR at start) +void dm_hdr_set_mode(int m); // hle_vi.c: change it while the game runs (the iOS in-game menu); the game switches its window on its next status query +int dm_hdr_layer_float(void); // hle_vi.c: DM_HDR_LAYER=float, the RGBA16Float extended linear BT.2020 layer; else RGB10A2 in BT.2100 PQ // The frame rate the display link paces the game at (60, 30, 120; nvn_mtl.m link_sync): in fullscreen, a screen whose refresh rate is not a multiple of it is switched, while the game runs, // to a mode of the same size whose rate is, and back when the window leaves fullscreen or that screen (front_mac.m; none on iOS). Called from the present thread void front_screen_match(int fps); diff --git a/dm/host/front_mac.m b/dm/host/front_mac.m index 7cd59bb10e..b1b8e2edd0 100644 --- a/dm/host/front_mac.m +++ b/dm/host/front_mac.m @@ -11,6 +11,7 @@ // DM_PAD_MERGE=1 merges every controller into player 1 instead, DM_KBD_PLAYER=own makes the keyboard player 1 on its own, with the controllers from player 2 on. #import #import +#include #import #import #include "dm.h" @@ -31,6 +32,40 @@ #include static CAMetalLayer *g_layer; +// Whether the window's screen can show EDR at all (its potential headroom is above 1; hle_vi.c answers HDR only then): -1 until the first read, taken as yes. +// Read four times a second on the main thread while the HDR layer is on, so it follows the window to another screen (an external SDR monitor beside the built-in XDR screen) +static _Atomic int g_hdr_screen = -1; +int front_hdr_screen(void) { return atomic_load(&g_hdr_screen) != 0; } +// The screen's current EDR headroom (maximumExtendedDynamicRangeColorComponentValue), read with the potential one: hle_vi.c's automatic peak (DM_HDR_PEAK=auto) is the paper white times it. +// 1.0 until the first read and without the HDR layer. The system grants the headroom while the window is in front, whatever range the game declares, so the automatic peak does not +// hold itself down: on the XDR panel it read 1.2 when the layer was made and 16 by the game's first query, then followed the brightness slider both ways (16 at low brightness, +// 3.2 = 1600 / 500 nits at full). In a window behind others it stays at 1.2 with any declared range. Polled, as the system offers no notification for it; the game asks every frame, +// so a change reaches it within a quarter second +static _Atomic float g_hdr_headroom = 1.0f; +float front_hdr_headroom(void) { return atomic_load(&g_hdr_headroom); } +static void front_hdr_screen_read(NSScreen *s) { + atomic_store(&g_hdr_headroom, (float)s.maximumExtendedDynamicRangeColorComponentValue); + double pot = s.maximumPotentialExtendedDynamicRangeColorComponentValue; + int can = pot > 1.0; + if (atomic_exchange(&g_hdr_screen, can) != can) LOG("front end: the window's screen %s EDR (%s, potential headroom %.2f)", can ? "can show" : "cannot show", s.localizedName.UTF8String, pot); +} +// The HDR layer's HDR10 metadata (CAEDRMetadata): Core Animation maps the layer's range into the screen's current headroom by it. The range is the one the game declares +// (0 to 1000 nits until it says, hle_vi.c front_hdr_content_range). opticalOutputScale (the paper white: 1.0 in the layer) applies to the float layer only (DM_HDR_LAYER=float): +// for a normalized format such as the PQ layer's RGB10A2, Core Animation takes it as 10,000 (CAEDRMetadata.h), so PQ codes are absolute nits. Chosen over a roll-off of the host's own: on +// Mario Kart World the two looked alike, reached the screen in the same time, and the system's follows brightness and the screen's modes as the rest of macOS does +static int g_hdr_meta; // the layer has the metadata (DM_HDR, and CAEDRMetadata is available) +static _Atomic uint32_t g_hdr_range = 1000u << 16; // the declared range: low nits in bits 0-15, high in 16-31 +static void front_hdr_metadata(void) { // main thread + uint32_t r = atomic_load(&g_hdr_range); + g_layer.EDRMetadata = [CAEDRMetadata HDR10MetadataWithMinLuminance:(float)(r & 0xffff) maxLuminance:(float)(r >> 16) opticalOutputScale:dm_hdr_paper_white()]; +} +void front_hdr_content_range(unsigned lo, unsigned hi) { + if (!g_hdr_meta || hi <= lo || hi > 65535) return; + uint32_t r = lo | hi << 16; + if (atomic_exchange(&g_hdr_range, r) == r) return; + LOG("front end: HDR10 metadata: the game's range %u..%u nits", lo, hi); + dispatch_async(dispatch_get_main_queue(), ^{ front_hdr_metadata(); }); +} static uint64_t g_btn; static int g_w, g_a, g_s, g_d; static int g_rl, g_ru, g_rr, g_rd; // arrow keys pressed with Shift held: the right stick (each goes back up as it went down, whatever Shift does meanwhile) @@ -1449,6 +1484,24 @@ void front_run(void (*guest)(void)) { g_layer = [CAMetalLayer layer]; g_layer.device = MTLCreateSystemDefaultDevice(); g_layer.pixelFormat = MTLPixelFormatBGRA8Unorm; + // DM_HDR=1 (the HDR mode): an HDR layer whatever the window's screen, with HDR10 metadata: Core Animation maps the picture into the screen's headroom, and a screen without + // headroom shows the SDR range. The layer is RGB10A2 in BT.2100 PQ (hle_vi.c dm_hdr_layer_float): the game's HDR10 window goes in as it is, an SDR source is encoded as HDR10 at the + // paper white (nvn_mtl.m present_mode). DM_HDR_LAYER=float keeps the RGBA16Float extended linear BT.2020 layer (present.metal fs_linear), which showed black blocks with coloured + // squares in them over flat areas (Mario Kart World's water on the course map), now and then and after the window came back to the front, on a MacBook Air M5 (macOS 27.0.1) + if (getenv("DM_HDR") && atoi(getenv("DM_HDR"))) { + int pq = !dm_hdr_layer_float(); + g_layer.pixelFormat = pq ? MTLPixelFormatRGB10A2Unorm : MTLPixelFormatRGBA16Float; + CGColorSpaceRef ecs = CGColorSpaceCreateWithName(pq ? kCGColorSpaceITUR_2100_PQ : kCGColorSpaceExtendedLinearITUR_2020); + g_layer.colorspace = ecs; CGColorSpaceRelease(ecs); + g_layer.wantsExtendedDynamicRangeContent = YES; + if (CAEDRMetadata.available) { g_hdr_meta = 1; front_hdr_metadata(); } + else LOG("[RED] front end: no EDR metadata on this Mac (CAEDRMetadata): values above the screen's headroom clip"); + if (atoi(getenv("DM_HDR")) == 2) win.title = [win.title stringByAppendingString:@" (DM_HDR=2: HDR on any screen)"]; // told apart from the setting's DM_HDR=1 + LOG("front end: HDR output on (DM_HDR): the window's layer is %s%s; the screen's headroom %.2f now, at most %.2f", pq ? "RGB10A2 in BT.2100 PQ" : "RGBA16Float in extended linear BT.2020 (DM_HDR_LAYER=float)", g_hdr_meta ? ", with HDR10 metadata" : "", + (win.screen ?: NSScreen.mainScreen).maximumExtendedDynamicRangeColorComponentValue, (win.screen ?: NSScreen.mainScreen).maximumPotentialExtendedDynamicRangeColorComponentValue); + front_hdr_screen_read(win.screen ?: NSScreen.mainScreen); + [NSTimer scheduledTimerWithTimeInterval:0.25 repeats:YES block:^(NSTimer *t) { (void)t; front_hdr_screen_read(win.screen ?: NSScreen.mainScreen); }]; + } g_layer.contentsScale = win.backingScaleFactor; // Apple's HUD over the game (frame rate, frame and GPU time, memory): front_hud_env turns it on before the first device. The layer's developerHUDProperties // stay unset: mode=default forces this layer's HUD on, so the Metal HUD menu's Hide HUD left it on screen diff --git a/dm/ios/Sources/front_ios.m b/dm/ios/Sources/front_ios.m index 8ebee3cad5..1636836943 100644 --- a/dm/ios/Sources/front_ios.m +++ b/dm/ios/Sources/front_ios.m @@ -5,6 +5,7 @@ #import #import #import +#include #import #import #include @@ -60,6 +61,70 @@ - (void)layoutSubviews { // render at the pixels of the screen the view is on l.drawableSize = CGSizeMake(self.bounds.size.width * l.contentsScale, self.bounds.size.height * l.contentsScale); } @end +// DM_HDR=1 (the HDR mode; the Mac's layer is made in front_mac.m the same way, and the in-game menu turns it on and off while the game runs): an HDR layer, whatever the screen the game is shown on, with HDR10 metadata: Core Animation +// maps the picture into the screen's headroom. It is RGB10A2 in BT.2100 PQ (hle_vi.c dm_hdr_layer_float): the game's HDR10 window goes in as it is, an SDR source is encoded as +// HDR10 at the paper white (nvn_mtl.m present_mode); DM_HDR_LAYER=float keeps the RGBA16Float extended linear BT.2020 layer. It is set before the game draws. A TV the picture moves to (tv_attach) keeps the layer +static UIScreen *g_edr_screen; // the screen the layer was made for: the fallback while the layer is not on a window +static CAMetalLayer *g_edr_layer; +// Whether the game view's screen can show EDR at all (potentialEDRHeadroom above 1; hle_vi.c answers HDR only then): -1 until the first read, taken as yes. Read four +// times a second on the main thread, so it follows the picture to a TV (tv_attach) +static _Atomic int g_hdr_screen = -1; +int front_hdr_screen(void) { return atomic_load(&g_hdr_screen) != 0; } +static void front_ios_hdr_screen_read(void) { + UIScreen *s = ((UIView *)g_edr_layer.delegate).window.windowScene.screen ?: g_edr_screen; // the game view's own screen (a TV after tv_attach) + if (!s) return; // no screen yet: unknown + double pot = s.potentialEDRHeadroom; + int can = pot > 1.0; + if (atomic_exchange(&g_hdr_screen, can) != can) LOG("front end: the game's screen %s EDR (potential headroom %.2f)", can ? "can show" : "cannot show", pot); +} +// The layer's HDR10 metadata (CAEDRMetadata), as on the Mac (front_mac.m front_hdr_metadata): the range is the one the game declares; opticalOutputScale, the paper white, applies to the float layer only +static int g_hdr_layer_on; // the layer is the HDR one (main thread) +static _Atomic uint32_t g_hdr_range = 1000u << 16; // the declared range: low nits in bits 0-15, high in 16-31 +static void front_ios_hdr_metadata(void) { // main thread + if (!g_hdr_layer_on || !CAEDRMetadata.available) return; + uint32_t r = atomic_load(&g_hdr_range); + g_edr_layer.EDRMetadata = [CAEDRMetadata HDR10MetadataWithMinLuminance:(float)(r & 0xffff) maxLuminance:(float)(r >> 16) opticalOutputScale:dm_hdr_paper_white()]; +} +void front_hdr_content_range(unsigned lo, unsigned hi) { // kept while the layer is SDR too, for the metadata once it is HDR (the in-game menu) + if (hi <= lo || hi > 65535) return; + uint32_t r = lo | hi << 16; + if (atomic_exchange(&g_hdr_range, r) == r) return; + LOG("front end: HDR10 metadata: the game's range %u..%u nits", lo, hi); + dispatch_async(dispatch_get_main_queue(), ^{ front_ios_hdr_metadata(); }); +} +// The game view's layer, HDR or SDR (main thread): at start by DM_HDR, and from the in-game menu while the game runs (dm_setting_live). The SDR layer is the one DmGameView makes: +// BGRA8Unorm with no colour space and no EDR. A drawable already taken keeps its format; the present draws each frame by its drawable's format (nvn_mtl.m present_mode), so the frames +// around the change are drawn for the layer they reach: the game's HDR window into the SDR layer as its screenshot is, its SDR window into the HDR layer at the paper white +static void front_ios_hdr_layer_set(int on) { + CAMetalLayer *l = g_edr_layer; if (!l || on == g_hdr_layer_on) return; + g_hdr_layer_on = on; + if (!on) { + l.EDRMetadata = nil; l.wantsExtendedDynamicRangeContent = NO; l.colorspace = NULL; l.pixelFormat = MTLPixelFormatBGRA8Unorm; + LOG("front end: HDR output off: the layer is BGRA8Unorm (SDR)"); return; + } + static int timer; // the screen's EDR read, four times a second from the first time HDR is on + if (!timer++) { front_ios_hdr_screen_read(); [NSTimer scheduledTimerWithTimeInterval:0.25 repeats:YES block:^(NSTimer *t) { (void)t; front_ios_hdr_screen_read(); }]; } + int pq = !dm_hdr_layer_float(); + l.pixelFormat = pq ? MTLPixelFormatRGB10A2Unorm : MTLPixelFormatRGBA16Float; + CGColorSpaceRef cs = CGColorSpaceCreateWithName(pq ? kCGColorSpaceITUR_2100_PQ : kCGColorSpaceExtendedLinearITUR_2020); + l.colorspace = cs; CGColorSpaceRelease(cs); + l.wantsExtendedDynamicRangeContent = YES; + if (CAEDRMetadata.available) front_ios_hdr_metadata(); + else LOG("[RED] front end: no EDR metadata on this device (CAEDRMetadata): values above the screen's headroom clip"); + UIScreen *s = ((UIView *)l.delegate).window.windowScene.screen ?: g_edr_screen; + LOG("front end: HDR output on (DM_HDR): the layer is %s%s; the screen's potential headroom %.2f", pq ? "RGB10A2 in BT.2100 PQ" : "RGBA16Float in extended linear BT.2020 (DM_HDR_LAYER=float)", + CAEDRMetadata.available ? ", with HDR10 metadata" : "", s.potentialEDRHeadroom); // 0 when the view has no screen yet +} +static void front_ios_hdr_layer(CAMetalLayer *l, UIScreen *screen) { // at start, before the game's first drawable + g_edr_screen = screen; g_edr_layer = l; + if (dm_hdr_mode()) front_ios_hdr_layer_set(1); +} +// HDR output from the in-game menu (dm_setting_live): the layer first when it turns on, so the HDR window the game makes on its next status query reaches an HDR layer; the answer +// first when it turns off, so the game goes back to its SDR window +static void front_ios_hdr_live(int on) { + if (on) { front_ios_hdr_layer_set(1); dm_hdr_set_mode(1); } + else { dm_hdr_set_mode(0); front_ios_hdr_layer_set(0); } +} static void tv_attach(void); @interface DmRootView : UIView @@ -166,6 +231,7 @@ - (void)startGame { self.view.window.traitOverrides.preferredContentSizeCategory = UIContentSizeCategorySmall; // UIKit draws the in-game menu (GameMenu.swift) in this window: it keeps one compact text size over the game, whatever the system's text size self.game = [[DmGameView alloc] initWithFrame:self.view.bounds]; [self.view addSubview:self.game]; + front_ios_hdr_layer((CAMetalLayer *)self.game.layer, self.view.window.windowScene.screen); // the layer's format before the game's first drawable (DM_HDR) g_layer = (CAMetalLayer *)self.game.layer; if (getenv("DM_METAL_HUD") && atoi(getenv("DM_METAL_HUD"))) { g_layer.developerHUDProperties = @{@"mode": @"default"}; LOG("front end: Metal performance HUD on (DM_METAL_HUD)"); } // Apple's HUD over the game: frame rate, frame and GPU time, memory dm_haptics_apple_init(); @@ -285,6 +351,7 @@ bool dm_setting_live(const char *env, const char *value) { dm_set_opmode(m); dm_gamepad_opmode_changed(); LOG("front end: console mode %s (in-game menu)", m ? "docked" : "handheld"); return true; } + if (!strcmp(env, "DM_HDR")) { if (!dm_sdk_ounce()) return false; if (value) front_ios_hdr_live(atoi(value) != 0); return true; } // Switch 2 games: the status query follows at once if (!strcmp(env, "DM_SHOW_FPS")) { if (value) [g_root showFPS:atoi(value) != 0]; return true; } if (!strcmp(env, "DM_TOUCH_HAPTICS")) { if (value) dm_gamepad_touch_feedback(atoi(value) != 0); return true; } if (!strcmp(env, "DM_RUMBLE")) { if (value) { dm_haptics_set_enabled(atoi(value) != 0); LOG("front end: game rumble %s (in-game menu)", atoi(value) ? "on" : "off"); } return true; } diff --git a/dm/tools/tests/test_settings.py b/dm/tools/tests/test_settings.py index 62fd72dfb7..7c177aeeb5 100644 --- a/dm/tools/tests/test_settings.py +++ b/dm/tools/tests/test_settings.py @@ -1,6 +1,6 @@ -"""Offer the resolution control only when the rewritten build supports it, and the game's upscaler choice only to games whose shader map holds their own upscaler (agl's NDSR network: game.py upscaler_keys).""" +"""Offer the resolution control only when the rewritten build supports it, the game's upscaler choice only to games whose shader map holds their own upscaler (agl's NDSR network: game.py upscaler_keys), and the HDR output, with its paper white and peak, only to Switch 2 games (their sdk module names the Ounce spec), on the Mac and iOS (the automatic peak on the Mac only).""" from pathlib import Path -import sys,tempfile,unittest +import os,sys,tempfile,unittest from unittest.mock import patch sys.path.insert(0,str(Path(__file__).resolve().parents[2]/'games')) import settings @@ -39,4 +39,31 @@ def test_only_for_games_naming_their_upscaler(self): self.assertEqual(bool(up),shown,gid) if up:self.assertEqual((up[0]['default'],up[0]['options']),('game',[['game',"Game's own"],['metalfx','MetalFX']])) +def hdr_rows(gid,platform,sdk,env='DM_HDR'): # the rows for env settings.py offers for a game whose sdk module is sdk (bytes; None: no sdk module) + with tempfile.TemporaryDirectory() as tmp,patch.object(settings.nacp,'languages',return_value=['en-US']),patch.dict(os.environ,{'SETTINGS_PLATFORM':platform}): + (Path(tmp)/'main.fixups.tsv').write_text('') + if sdk is not None:(Path(tmp)/'sdk.patched.bin').write_bytes(sdk) + return [s for s in settings.build(gid,tmp,'unused')['settings'] if s['env']==env] + +class HdrSetting(unittest.TestCase): + def test_switch2_games_on_mac_and_ios(self): # general, not per game: shown where the sdk module names the Switch 2 spec, on either platform + ounce,nx=b'FS_ACCESS: { sdk_version: 20.5.19, spec: Ounce }',b'FS_ACCESS: { sdk_version: 12.0.0, spec: NX }' + for platform in ('mac','ios'): + for gid,sdk,shown in [('040074A01BF12000',ounce,True),('040074A01BF12000',nx,False),('010056B015FE8000',nx,False),('040074A01BF12000',None,False)]: # Donkey Kong Bananza (Switch 2); the same game on a Switch 1 SDK; NieR:Automata (Switch 1); no sdk module + hdr=hdr_rows(gid,platform,sdk) + self.assertEqual(bool(hdr),shown,(platform,gid,sdk)) + if hdr:self.assertEqual((hdr[0]['kind'],hdr[0]['default']),('toggle','0')) + + def test_paper_white_and_peak_are_sub_settings_of_hdr_output(self): # the console's HDR calibration: shown with HDR output, drawn only while it is on; defaults are the host's own (203, 1000), so an untouched row changes nothing + ounce,nx=b'FS_ACCESS: { sdk_version: 20.5.19, spec: Ounce }',b'FS_ACCESS: { sdk_version: 12.0.0, spec: NX }' + for platform in ('mac','ios'): + for env,default in [('DM_HDR_PAPER_WHITE','203'),('DM_HDR_PEAK','1000')]: + self.assertEqual(hdr_rows('040074A01BF12000',platform,nx,env),[],(platform,env)) + row,=hdr_rows('040074A01BF12000',platform,ounce,env) + self.assertEqual((row['kind'],row['default'],row['needs']),('choice',default,'DM_HDR=1'),(platform,env)) + self.assertIn(default,[o[0] for o in row['options']],(platform,env)) + for platform,auto in [('mac',True),('ios',False)]: # Automatic reads the screen's current headroom, which only the Mac front end reads (front_mac.m front_hdr_headroom) + row,=hdr_rows('040074A01BF12000',platform,ounce,'DM_HDR_PEAK') + self.assertEqual(['auto','Automatic'] in row['options'],auto,platform) + if __name__=='__main__':unittest.main() diff --git a/docs/agents/symptoms.md b/docs/agents/symptoms.md index 417722f5be..33ce7eae93 100644 --- a/docs/agents/symptoms.md +++ b/docs/agents/symptoms.md @@ -674,6 +674,7 @@ One line per symptom in the [subsystem references](../reference/): what you see - one-draw passes on one target, hundreds of times a frame, with the same attachments each time (a `DM_NVN_GPUPROF` frame: `load` and `store` of the whole target in every line); a low frame rate that follows the target's size. → [Setting the same targets again continues the pass](../reference/rendering.md#setting-the-same-targets-again-continues-the-pass) - a pass on a very large target costs about the same however little it draws; `DM_NVN_GPUPROF` shows time proportional to the target's area. → [A render area for passes on very large targets](../reference/rendering.md#a-render-area-for-passes-on-very-large-targets) - `[RED] NVN unimplemented function called: nvnCommandBufferShadingRateImagePalette`, or `nvnCommandBufferBindShadingRateImage` or `nvnCommandBufferSetShadingRatePaletteMode`, every frame. → [Variable rate shading ignored](../reference/rendering.md#variable-rate-shading-ignored) +- no HDR output on a Switch 2 game that asks for its HDR status: its window stays RGBA8 and the picture is SDR. → [The status query and the present for an HDR window](../reference/rendering.md#the-status-query-and-the-present-for-an-hdr-window) - angle-dependent character smearing, or vertex-only pipeline failures with `invalid depth/stencil pipeline formats 16843012/260`, although the pass began with valid `260/260` formats. → [Ten attachment records for eight colors and depth-stencil](../reference/rendering.md#ten-attachment-records-for-eight-colors-and-depth-stencil) - Metal aborts at pipeline creation: `depthAttachmentPixelFormat (MTLPixelFormatInvalid) and stencilAttachmentPixelFormat (MTLPixelFormatDepth32Float_Stencil8) must match`. → [Combined depth-stencil formats stay paired in passes and pipeline keys](../reference/rendering.md#combined-depth-stencil-formats-stay-paired-in-passes-and-pipeline-keys) - undefined values blended into attachments the fragment shader leaves alone. → [No writes to colour outputs the shader does not write](../reference/rendering.md#no-writes-to-colour-outputs-the-shader-does-not-write) diff --git a/docs/env_switches.tsv b/docs/env_switches.tsv index 02fdbbe7c0..a9853f03e5 100644 --- a/docs/env_switches.tsv +++ b/docs/env_switches.tsv @@ -68,6 +68,10 @@ DM_GUEST_FILE_CHUNK switch `=` (whole MB, K / M / G; default 1G, at least DM_GUEST_FILE_DIR path Folder for `DM_GUEST_FILE`'s files (default the user's temporary folder; the iOS app sets it to `NSTemporaryDirectory()`) DM_GUEST_FILE_MIN_FREE switch `=` (whole MB, K / M / G; default 5G, `0` never): with `DM_GUEST_FILE=1`, the game is stopped (exit code 87) when the guest files' disk has less free space than that DM_GUEST_SPAWN_TEST diagnostic `=1` runs a self-test before `nnMain`: a host-started guest thread does SDK malloc / free and locks an `nn::os` mutex, and the result is logged; `=2` also keeps a resident thread +DM_HDR player `1` turns on HDR output for Switch 2 games (the Mac and iOS settings row; any non-zero value; `0` or unset is SDR). While the window's screen can show EDR, the status query answers HDR, so the game makes its RGB10A2 PQ window; on a screen without EDR it answers SDR (`2` answers HDR there too: the system maps the HDR picture down instead of the game drawing its own SDR). The layer is RGB10A2 in BT.2100 PQ either way (`DM_HDR_LAYER`). The iOS in-game menu turns it on and off while the game runs (`hle_vi.c` `dm_hdr_set_mode`). Measured on Donkey Kong Bananza, and on Mario Kart World to its title. +DM_HDR_LAYER diagnostic `float` (Mac and iOS): the HDR layer is RGBA16Float in extended linear BT.2020, the present decoding PQ to linear light, instead of RGB10A2 in BT.2100 PQ with the game's HDR10 window as it is (the default; the float layer showed black blocks over flat areas on an M5). Kept until that is understood and the PQ layer has run on an iPhone +DM_HDR_PAPER_WHITE player `=` the paper white (the Mac and iOS settings row HDR paper white): the answer to `vi::QueryDisplayHdrPaperWhite` while HDR is on, the level an SDR source's white is encoded at in the PQ layer, and, with `DM_HDR_LAYER=float`, the level a PQ window's SDR white decodes to and 1.0 of the layer's HDR10 metadata (1 to 65535, read once; default 203, the BT.2408 reference). Ocarina of Time caps its declared extended range at 3.5 times it (300 → 0..1000 with the default peak) +DM_HDR_PEAK player `=` the display's peak that `DM_HDR` reports (the Mac and iOS settings row HDR peak; decimal, 16-bit; default 1000), read at every query. `auto` (the row's Automatic, offered on the Mac) answers the paper white times the window's screen's current EDR headroom (`front_mac.m` `front_hdr_headroom`), rounded to 10 nits, never below the paper white: the range the game declares then follows the brightness slider. Without a front end that reads the headroom (iOS, windowless runs) `auto` is the paper white. A game declares at most the peak: Ocarina of Time min(peak, 3.5 × paper white) (400 → 0..400, 1600 → 0..710) DM_HID_LOG diagnostic `1` logs the HID button usage number of each button pressed on a raw HID gamepad (`pad_hid_mac.h`), to write a `DM_HID_MAP` DM_HID_MAP switch `=` such as `A=1,B=2,X=4,Y=5`: which HID button usage presses each Switch button on a raw HID gamepad (names A B X Y LS RS L R ZL ZR PLUS MINUS LEFT UP RIGHT DOWN; default 8BitDo's D-input numbering by position, `pad_hid.h`); Mac only DM_HID_PADS switch `0` stops reading raw HID gamepads that GameController does not list, such as an 8BitDo in its own mode (`pad_hid_mac.h`); Mac only @@ -244,6 +248,7 @@ DM_PRESENT_HZ switch `=`: the frequency of presentation pacing (default 60); DM_PRESENT_INTERVAL switch `display` presents one frame per display-link tick whatever interval the game sets; `game` follows the game's present interval (default: the profile's `present_interval`) DM_PRESENT_LATENCY switch `=1` or `2`: overrides the display link's preferred frame latency for every screen and window mode; larger positive values use 2 (default 2 in variable-refresh fullscreen, 1 otherwise) DM_PRESENT_LINK switch Present pacing: `0` acquires the drawable itself, `1` (the default) presents on a 60 Hz display link, `2` uses a 120 Hz display link with a frame every other tick +DM_PRESENT_SHOWN diagnostic `1` logs, every 300 frames, how long the system took to put frames on screen (the drawable's presented time): from the present call and from the display-link tick it was meant for, p50, p90 and max, with the frames it dropped and whether the layer has EDR metadata DM_PRESENT_SNAP player `=`: present snapshots, a ring of n host textures (2 to 4; `1` = 2) the presents are copied to so the game's swapchain texture is free at once; `0` off (default: the profile's `present_snap`) DM_PRESENT_UPSCALE player `metalfx` scales a picture smaller than its rectangle on the screen through MetalFX's spatial scaler; `off` (the default) scales it bilinearly DM_PREV_PROGRAM switch What `oe::GetPreviousProgramIndex` answers (default -1, started from HOME); `oe::ExecuteProgram` sets it for the program it starts diff --git a/docs/reference/rendering.md b/docs/reference/rendering.md index 32005ad770..bc0cb3cb63 100644 --- a/docs/reference/rendering.md +++ b/docs/reference/rendering.md @@ -696,6 +696,22 @@ The replay on Metal (`dm/gfx/nvn/nvn_mtl.m`) and the Vulkan and OpenGL layers. E - **Fix:** `hle_nvn.c` accepts the three calls and records nothing. Shading every pixel is the most any rate asks for, so the picture is the same or sharper, and the only cost is GPU time where the game meant to save it. This drops a feature rather than emulating it, and the choice is a platform limit. - **Found on:** [Mario Kart World](../../dm/games/0400C3F00006E000/README.md), every frame; [Splatoon Raiders](../../dm/games/0400380020056000/README.md), the palette. +## HDR output + +### The status query and the present for an HDR window + +- **Symptom:** no HDR output on a Switch 2 game that asks for its HDR status: its window stays RGBA8 and the picture is SDR. +- **Cause:** `vi::QueryDisplayHdrStatus` answered SDR, so the game never made its RGB10A2 PQ window, and the present had no decode for it. +- **Fix:** with `DM_HDR=1` the status answers HDR on (Switch 2 SDKs only) while the window's screen can show EDR: its potential headroom is above 1 (`maximumPotentialExtendedDynamicRangeColorComponentValue` on the Mac, `potentialEDRHeadroom` on the iPhone). The front end reads it four times a second, so the answer follows the window to another screen. Moving the window between screens drops some frames whether or not the game rebuilds its window ([Mario Kart World](../../dm/games/0400C3F00006E000/README.md#hdr-output)). On a screen without EDR the status answers SDR and the game draws its own SDR picture, as a Switch 2 does on a TV without HDR10; `DM_HDR=2` answers HDR there too, and Core Animation maps the HDR picture down instead. The format is 1 with HDR on: the SDK's `DefaultDisplay::QueryHdrStatus` sets on to (format == 1), and a format above 1 is an error. The game asks every frame, so it can switch windows when the answer changes. That also lets the iOS in-game menu turn HDR output on and off while the game runs (`dm_hdr_set_mode`, `front_ios.m` `front_ios_hdr_live`): the layer turns HDR before the answer does and SDR after it, and the frames around the change are drawn for the layer they reach. + - The layer is RGB10A2 in BT.2100 PQ (`kCGColorSpaceITUR_2100_PQ`) on any screen, on the Mac and the iPhone (`hle_vi.c` `dm_hdr_layer_float`): the game's HDR10 window is written as it is, and `present.metal` `fs` encodes a gamma-coded BT.709 source (the game's SDR window before it switches) as HDR10 with its white at `DM_HDR_PAPER_WHITE` (default 203; it and `DM_HDR_PEAK` are the HDR paper white and HDR peak launcher rows, standing for what a console's HDR calibration sets; `present_mode` 3), and a float source as linear BT.709 the same way (mode 4; no local game presents one). An RGB10A2 window is taken as PQ only after the status has answered HDR on; any other RGB10A2 window is treated as gamma-coded SDR. + - The first layer was RGBA16Float in extended linear BT.2020 (`fs_linear`: PQ decoded to linear light, 1.0 at the paper white; kept as `DM_HDR_LAYER=float`). On a MacBook Air M5 (macOS 27.0.1, built-in screen, potential headroom 2) it showed black blocks with coloured squares in them over flat areas, Mario Kart World's water on its course map, for a frame now and then and for a moment after the window came back to the front. The frames the game drew had none: `DM_NVN_MAGSCAN` found no non-finite pixel in any pass, 100 frames saved through `DM_CTL` `shot` while the blocks showed were clean, and present snapshots (`DM_PRESENT_SNAP=2`) did not remove them. Guarding the decode's `pow` against a zero base (the present library is built with fast math) and clipping colours to Display P3 did not either. With the PQ layer they are gone and the colours are the same by eye; why the extended-range layer shows them is open (Core Animation, or the system's handling of that format on this Mac). + - The layer carries HDR10 metadata (`CAEDRMetadata`: the range is the one the game declares), so Core Animation maps the highlights into the screen's current headroom, following brightness and the screen's modes. The present writes them as they are. The metadata's `opticalOutputScale` (the paper white) counts only for the float layer, whose 1.0 it says the luminance of: for a normalized format such as RGB10A2, Core Animation takes it as 10,000 (`CAEDRMetadata.h`), so the PQ layer's codes are absolute nits. The SDR white is at the paper white in both. Apple's Metal HUD (`DM_METAL_HUD`) is drawn into the layer with SDR values unconverted: on the PQ layer its white is a code of 1.0, 10,000 nits, so it shows at the screen's peak. A screenshot of an HDR frame brings each highlight's largest channel to SDR white and keeps its hue. + - The status query's peak is `DM_HDR_PEAK` (1000), or with `DM_HDR_PEAK=auto` (the HDR peak row's Automatic, Mac only) the paper white times the screen's current headroom (`maximumExtendedDynamicRangeColorComponentValue`, read with the potential headroom four times a second; rounded to 10 nits, never below the paper white), logged when it changes, at most once a second. A game declares its extended range within the peak and its own cap: [Ocarina of Time](../../dm/games/0400486024DEC000/README.md#hdr-output) declares min(peak, 3.5 × paper white) (peak 400 → 0..400, peak 1600 → 0..710, paper white 300 → 0..1000), so on the XDR panel Automatic changes its picture only while the headroom is under 3.5, at high brightness. The system grants the headroom while the window is in front, whatever range the game declares, so the automatic peak does not hold itself down. On the XDR panel the headroom read 1.2 when the layer was made and 16 by the game's first query, then followed the brightness slider both ways: 3250, 2470, 650, 1360, 2300 nits, with the game's range going 0..710 → 0..650 → 0..710 during play. At full brightness the headroom is 3.2 (1600 / 500 nits), so on that panel Automatic changes Ocarina of Time's range by at most 710 → 650 nits. A window behind others reads 1.2 with any declared range. + - Before this, the present rolled the highlights off into the headroom itself. On [Mario Kart World](../../dm/games/0400C3F00006E000/README.md#hdr-output) the two looked alike by eye and reached the screen in the same time in a window, so the system's mapping replaced it. +- **Check:** `make -C dm/gfx/tests GAME= run` runs `t_present_hdr`: the status answer on a screen with and without EDR, PQ codes from 0 to 10,000 nits, SDR sources, screenshots with a highlight's hue, the PQ layer (the HDR10 window as it is, SDR and float sources at the paper white), and a 10-bit window of a game that never asked for HDR. `DM_PRESENT_SHOWN=1` logs how long frames take to reach the screen. +- **Open:** fullscreen (where the system may show the layer without compositing it) is not timed. On an iPhone 17 Pro (iOS 27.2, headroom 8), Donkey Kong Bananza shows HDR on the PQ layer; there, with HDR on only, the picture jumps back a frame now and then while the frame rate cannot hold 60 or 30 (with HDR off it holds a frame instead), cause not found. Whether the float layer's blocks are in its contents or in the compositor: a readback of the float layer while they show would tell. The window's primaries are settled in the Donkey Kong Bananza note. +- **Found on:** [Donkey Kong Bananza](../../dm/games/040074A01BF12000/README.md#hdr-output); [Mario Kart World](../../dm/games/0400C3F00006E000/README.md#hdr-output), the same window, to the title. [Ocarina of Time](../../dm/games/0400486024DEC000/README.md#hdr-output), the same window, to the opening movie. + ## Blend, depth and stencil ### Ten attachment records for eight colors and depth-stencil diff --git a/docs/reference/services.md b/docs/reference/services.md index cec24bfe5f..acd1e7649b 100644 --- a/docs/reference/services.md +++ b/docs/reference/services.md @@ -576,7 +576,7 @@ Most service calls can be found before they crash, before the first boot or the - **Symptom:** `CRASH sig=11 addr=0 pc=0` with the link register just after a call to a `vi::SetLayerNextHdr…` or `vi::ClearLayerNextHdr…` function. - **Cause:** the SDK forwards a layer's HDR metadata through a table its graphics system fills, which is null on the host. -- **Fix:** every layer HDR metadata call of SDK 22's free functions succeeds with nothing to apply (`dm/gfx/hle_vi.c`): the content gamut, the HDR to SDR tone mapping, the standard and extended content luminance ranges, the adjustable Bezier tone curve, and the clears of the gamut, the tone mapping and the curve. The host presents SDR. +- **Fix:** every layer HDR metadata call of SDK 22's free functions succeeds with nothing to apply (`dm/gfx/hle_vi.c`): the content gamut, the HDR to SDR tone mapping, the standard and extended content luminance ranges, the adjustable Bezier tone curve, and the clears of the gamut, the tone mapping and the curve. The host sends no metadata: it presents SDR, or with `DM_HDR=1` decodes the game's PQ window itself ([The status query and the present for an HDR window](rendering.md#the-status-query-and-the-present-for-an-hdr-window)). - **Found on:** [Donkey Kong Bananza](../../dm/games/040074A01BF12000/README.md) (the set calls); the curve and the clears: [Mario Kart World](../../dm/games/0400C3F00006E000/README.md) 1.8.0 sets the curve every frame, from pull request #1116 (that build is not run here). ### Layer frame numbers diff --git a/docs/settings.md b/docs/settings.md index f03a53f22c..1fb8484397 100644 --- a/docs/settings.md +++ b/docs/settings.md @@ -3,7 +3,7 @@ The screen before the game starts shows the settings the game supports. A setting that only matters while another is on appears under it and disappears when that one is turned off. Tap **Play** to start; the app remembers your choices for the next start. All settings are on one scrollable screen, under **Game**, **Controls** and **Performance**. Tap a menu or move a slider there; no settings page needs opening. **Upscaling** is in **Game**, beside **Console mode**. A Mac app shows the same settings in a [window of its own](#the-mac-settings-window). - [Tested defaults](#tested-defaults) -- [Game](#game): [Frame rate](#frame-rate), [Render scale](#render-scale), [Console mode](#console-mode), [Upscaling](#upscaling), [Game's upscaler](#games-upscaler), [Game language](#game-language), [Picture](#picture), [Frame-rate counter](#frame-rate-counter) +- [Game](#game): [Frame rate](#frame-rate), [Render scale](#render-scale), [Console mode](#console-mode), [Upscaling](#upscaling), [Game's upscaler](#games-upscaler), [Game language](#game-language), [Picture](#picture), [HDR output](#hdr-output), [HDR paper white](#hdr-paper-white), [HDR peak](#hdr-peak), [Frame-rate counter](#frame-rate-counter) - [Controls](#controls): [Touch feedback](#touch-feedback), [Game rumble](#game-rumble), [Rumble strength](#rumble-strength), [Rumble cutoff](#rumble-cutoff), [Press a button to join](#press-a-button-to-join) - [Performance](#performance): [Frame buffering](#frame-buffering), [Metal 4 pipelines](#metal-4-pipelines), [Pipeline prebuild](#pipeline-prebuild), [Encode thread](#encode-thread), [Hold frames when late](#hold-frames-when-late), [Performance HUD](#performance-hud), [Reset to tested defaults](#reset-to-tested-defaults) - [The Mac settings window](#the-mac-settings-window): [Display mode](#display-mode) @@ -79,6 +79,18 @@ The game's own languages. The app was packed with one of them; pick another here iPhone and iPad. **Fit**, the default, shows the game's picture (16:9, or the taller one of [Picture shape](#picture-shape)) as large as it fits, with black bars where the screen has another shape. **Fill** enlarges it until it covers the whole screen and crops what lies outside; nothing is stretched. An iPhone loses some of the top and bottom, an iPad some of each side, so health bars, subtitles or maps near the edges can be cut off. A picture that fills the screen without cropping needs a patch for each game that renders a wider view. Fill applies in landscape; in portrait the picture stays fitted above the controls, and on a TV it is always fitted. +### HDR output + +Shown for Switch 2 games, on the Mac and the iPhone (measured on Donkey Kong Bananza and, to its title screen, Mario Kart World; the other Switch 2 titles are untested). **On** starts the game with its own HDR output: on a screen with HDR headroom, whites stay at the screen's normal white and the picture's highlights can go brighter above it; on a screen that cannot show HDR (most external monitors) the game is told so and shows its own SDR picture, and moving the window between screens switches it (Mario Kart World). **Off**, the default, keeps the picture as SDR. On the iPhone and iPad the in-game menu turns it on and off while the game runs (the game changes its picture within a frame or two); on the Mac it applies from the next start. + +### HDR paper white + +Shown under **HDR output** while it is on. How bright the game draws its normal white, and so everything that is not a highlight. **203 nits (standard)**, the default, is the reference level HDR video uses. A higher value brightens the whole picture, as a console's HDR brightness calibration does. It also lifts the highlights: Ocarina of Time lets its brightest highlights reach 3.5 times the paper white (710 nits at 203), within the limit [HDR peak](#hdr-peak) sets. A lower value suits a dim room. + +### HDR peak + +Shown under **HDR output** while it is on. The brightest level the game is told the screen can show, which it keeps its highlights within. **1000 nits (standard)** is the default. **Automatic**, on the Mac, follows the screen: the paper white times how many times brighter than white the screen can go at the moment (its headroom). That headroom shrinks as the brightness slider goes up, so with Automatic the game fits its highlights into what the screen can show at the current brightness, instead of macOS compressing a fixed range. The game itself may stop lower: Ocarina of Time never goes past 3.5 times the paper white, so at 203 nits a peak above 710 changes nothing there, and Automatic changes its picture only at high screen brightness. On a MacBook Pro's XDR screen that is only at full brightness, where it lowers Ocarina of Time's highlights from 710 to 650 nits. + ### Picture shape Games whose profile supports it (Breath of the Wild, Animal Crossing: New Horizons). **iPhone Duo screen (1.43:1)** renders a taller picture of the Duo's own shape: the sides of 16:9 with more above and below, and the HUD at the screen's edges. **16:9** is the Switch's picture. The taller picture has about a quarter more pixels, so it runs a little slower and warmer. @@ -241,7 +253,7 @@ iCloud Drive takes a moment to carry a file from one device to the other. Quit t ## In-game menu -On the iPhone and iPad the gear at the top centre of the touch controls opens a menu while the game runs; it stays when a controller hides the other controls. It holds **Console mode**, **Picture** and the **Frame-rate counter**, then **Controller layout** (**2D**, **3D**, and **Tap**, no controls and every touch to the game, in handheld mode), **Touch feedback** and **Game rumble**: the settings that take effect at once. A choice opens to show its options, the current one ticked. The menu's text keeps one size whatever the system text size, so it stays short over the game. A change is remembered as on the settings screen, which shows it at the next start; a setting `Documents/env.txt` fixes is shown, not changed. The other settings need the game to start again, so they are only on the settings screen: close the app fully and open it again. +On the iPhone and iPad the gear at the top centre of the touch controls opens a menu while the game runs; it stays when a controller hides the other controls. It holds **Console mode**, **Picture**, **HDR output** (Switch 2 games) and the **Frame-rate counter**, then **Controller layout** (**2D**, **3D**, and **Tap**, no controls and every touch to the game, in handheld mode), **Touch feedback** and **Game rumble**: the settings that take effect at once. A choice opens to show its options, the current one ticked. The menu's text keeps one size whatever the system text size, so it stays short over the game. A change is remembered as on the settings screen, which shows it at the next start; a setting `Documents/env.txt` fixes is shown, not changed. The other settings need the game to start again, so they are only on the settings screen: close the app fully and open it again. Console mode changes as on a Switch put in or taken out of its dock. While a TV shows the game it stays **Docked**, and the mode you pick applies when the TV is disconnected. **Docked** removes the Tap layout, as a docked Switch reads no touch screen.