diff --git a/dm/games/0400C3F00006E000/README.md b/dm/games/0400C3F00006E000/README.md index b796303386..60c7bb296d 100644 --- a/dm/games/0400C3F00006E000/README.md +++ b/dm/games/0400C3F00006E000/README.md @@ -87,6 +87,7 @@ With the HDR output setting (`DM_HDR=1`), on the base game to the title screen o - Its declared extended range is 0 to 1000 nits while it makes the HDR window, then 0 to 1801 at the title. - Brighter than SDR, by the game's own grade: its title logo, the same image in both modes, is about 1.8 times brighter in linear light and more saturated in the decoded screenshot. That fits its SDR colours taken as Display P3 and drawn near 360 nits. - The host's own highlight roll-off against Core Animation's mapping by the layer's HDR10 metadata, which replaced it ([the reference entry](../../../docs/reference/rendering.md#the-status-query-and-the-present-for-an-hdr-window)): side by side on the built-in screen they looked alike. In a window on the built-in screen of a MacBook Pro (M4 Pro, macOS 26.7.1), 80 s to the title each, `DM_PRESENT_SHOWN=1`. From the present call to on screen, p50 per 300 frames: SDR 49.58 to 49.68 ms, the host's roll-off 49.53 to 49.61, Core Animation 49.52 to 49.59. Every frame reached the display-link tick it was meant for (p50 within 0.03 ms), and each dropped 0 to 3 of 300 frames. The metadata adds no frame here. +- On a MacBook Air M5 (macOS 27.0.1, built-in screen), in play: with the RGBA16Float extended-range layer, black blocks with coloured squares in them flashed over the course map's water, now and then and after ⌘-Tab back to the game; the game's frames were clean. The RGB10A2 PQ layer has none, colours the same by eye ([the reference entry](../../../docs/reference/rendering.md#the-status-query-and-the-present-for-an-hdr-window)). ## Driving it on the Mac diff --git a/dm/gfx/msl/present.metal b/dm/gfx/msl/present.metal index 179cb1ba93..0119e55fa4 100644 --- a/dm/gfx/msl/present.metal +++ b/dm/gfx/msl/present.metal @@ -4,7 +4,7 @@ using namespace metal; struct V { float4 p [[position]]; float2 uv; }; vertex V vs(uint id [[vertex_id]]) { float2 uv = float2((id << 1) & 2, id & 2); V o; o.p = float4(uv * float2(2, -2) + float2(-1, 1), 0, 1); o.uv = uv; return o; } // mode.x (nvn_mtl.m present_mode) says how the source is encoded: 0 as stored (a float source: linear BT.709); 1 a gamma-coded 8-bit BT.709 (sRGB) source for the extended-range BT.2020 layer (fs_linear converts it); -// 2 an HDR10 source (the game's RGB10A2 window: SMPTE ST 2084 code values in BT.2020 primaries), where mode.y is the paper white in nits. mode.z and mode.w are unused. +// 2 an HDR10 source (the game's RGB10A2 window: SMPTE ST 2084 code values in BT.2020 primaries), where mode.y is the paper white in nits; 3 a gamma-coded 8-bit BT.709 (sRGB) source for the PQ layer (BT.2100 PQ, DM_HDR): fs encodes it as HDR10 at the paper white. mode.z and mode.w are unused. struct Crop { float4 rect; float4 edge; float4 mode; }; // Linear BT.709 and BT.2020 (ITU-R BT.2087; each row sums to 1, so white stays white). Metal's float3x3 takes its columns. constant float3x3 kRec709To2020 = float3x3(float3(0.627404, 0.069097, 0.016391), float3(0.329283, 0.919540, 0.088013), float3(0.043313, 0.011362, 0.895595)); @@ -27,6 +27,12 @@ static inline float3 present_pq_to_linear(float3 e, float paper) { float3 den = max(c2 - c3 * ep, float3(1e-6)); return 10000.0 * pow(num / den, float3(1.0 / m1)) / paper; } +// SMPTE ST 2084 inverse EOTF: linear light where 1.0 is the paper white, to code values (10000 nits = 1.0). pow() gets a positive base (the library is built with fast math) +static inline float3 present_linear_to_pq(float3 l, float paper) { + const float m1 = 0.1593017578125, m2 = 78.84375, c1 = 0.8359375, c2 = 18.8515625, c3 = 18.6875; + float3 ym = pow(max(l * (paper / 10000.0), float3(1e-30)), float3(m1)); + return pow((c1 + c2 * ym) / (1.0 + c3 * ym), float3(m2)); +} // A highlight brought to SDR white as an SDR screen shows it: the largest channel goes to 1.0 and the others scale with it, so the hue holds (a screenshot of an HDR frame) static inline float3 present_to_sdr_white(float3 c0) { float3 c = clamp(c0, -65504.0, 65504.0); // finite: an infinite channel would give inf * 0 below @@ -37,7 +43,8 @@ static inline float3 present_to_sdr_white(float3 c0) { // then converted to BT.709; a colour outside BT.709 is clipped fragment float4 fs(V in [[stage_in]], texture2d t [[texture(0)]], sampler s [[sampler(0)]], constant Crop& c [[buffer(0)]]) { float4 v = t.sample(s, clamp(in.uv * c.rect.zw + c.rect.xy, c.edge.xy, c.edge.zw)); - if (c.mode.x > 1.5) v.rgb = present_linear_to_srgb(saturate(kRec2020To709 * present_to_sdr_white(present_pq_to_linear(v.rgb, c.mode.y)))); + if (c.mode.x > 2.5) v.rgb = present_linear_to_pq(kRec709To2020 * present_srgb_to_linear(v.rgb), c.mode.y); // the PQ layer: an SDR source as HDR10 at the paper white + else if (c.mode.x > 1.5) v.rgb = present_linear_to_srgb(saturate(kRec2020To709 * present_to_sdr_white(present_pq_to_linear(v.rgb, c.mode.y)))); return v; } // The HDR layer (RGBA16Float, extended linear BT.2020, DM_HDR=1): linear light, where 1.0 is the paper white. The layer carries HDR10 metadata (front_mac.m, front_ios.m), so diff --git a/dm/gfx/nvn/nvn_mtl.m b/dm/gfx/nvn/nvn_mtl.m index 23a2f2590b..90eef56405 100644 --- a/dm/gfx/nvn/nvn_mtl.m +++ b/dm/gfx/nvn/nvn_mtl.m @@ -12800,6 +12800,8 @@ static int present_crop_rect(id src, const int crop[4], NSUInteger r static float present_mode(MTLPixelFormat src, MTLPixelFormat dst) { // An RGB10A2 window is PQ code values only once the game's status query has answered HDR on (vi_hdr_answered); any other RGB10A2 window is read as stored // Not answered: an SDR 10-bit window, gamma-coded like the 8-bit ones, so it is decoded to linear light for an extended-range layer + // An RGB10A2 drawable is the Mac's HDR layer in BT.2100 PQ (front_mac.m): the game's HDR10 window goes in as it is; any other source is gamma-coded BT.709 and is encoded as HDR10 (mode 3) + if (dst == MTLPixelFormatRGB10A2Unorm) return src == MTLPixelFormatRGB10A2Unorm && vi_hdr_answered() ? 0.0f : 3.0f; if (src == MTLPixelFormatRGB10A2Unorm) return vi_hdr_answered() ? 2.0f : (dst == MTLPixelFormatRGBA16Float ? 1.0f : 0.0f); if (dst != MTLPixelFormatRGBA16Float) return 0.0f; // Gamma-coded 8-bit sources. The sRGB-typed ones are sampled through a non-sRGB view (present_raw_src), so the shader reads gamma-coded values for them too diff --git a/dm/gfx/tests/t_present_hdr.m b/dm/gfx/tests/t_present_hdr.m index d30f67c3b4..be2d502213 100644 --- a/dm/gfx/tests/t_present_hdr.m +++ b/dm/gfx/tests/t_present_hdr.m @@ -58,7 +58,7 @@ static double pq_nits(unsigned code) { [t replaceRegion:MTLRegionMake2D(0, 0, N, 1) mipmapLevel:0 withBytes:b bytesPerRow:sizeof b]; return t; } -// The present of src into an N x 1 target of the given format; out gets RGB as floats (16-bit float targets) or 0-255 (8-bit targets) +// The present of src into an N x 1 target of the given format; out gets RGB as floats (16-bit float targets), 0-1023 (RGB10A2, the PQ layer) or 0-255 (8-bit targets) static int run(id src, MTLPixelFormat fmt, double out[N][3]) { MTLTextureDescriptor *d = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:fmt width:N height:1 mipmapped:NO]; d.storageMode = MTLStorageModeShared; d.usage = MTLTextureUsageRenderTarget; @@ -69,6 +69,9 @@ static int run(id src, MTLPixelFormat fmt, double out[N][3]) { if (fmt == MTLPixelFormatRGBA16Float) { __fp16 h[N * 4]; [dst getBytes:h bytesPerRow:sizeof h fromRegion:MTLRegionMake2D(0, 0, N, 1) mipmapLevel:0]; for (int i = 0; i < N; i++) for (int k = 0; k < 3; k++) out[i][k] = h[4 * i + k]; + } else if (fmt == MTLPixelFormatRGB10A2Unorm) { // the PQ layer: 10-bit code values + uint32_t w[N]; [dst getBytes:w bytesPerRow:sizeof w fromRegion:MTLRegionMake2D(0, 0, N, 1) mipmapLevel:0]; + for (int i = 0; i < N; i++) for (int k = 0; k < 3; k++) out[i][k] = w[i] >> (10 * k) & 0x3ff; } else { uint8_t b[N * 4]; [dst getBytes:b bytesPerRow:sizeof b fromRegion:MTLRegionMake2D(0, 0, N, 1) mipmapLevel:0]; for (int i = 0; i < N; i++) for (int k = 0; k < 3; k++) out[i][k] = b[4 * i + k]; @@ -137,6 +140,16 @@ int main(void) { @autoreleasepool { CHECK("SDR mid grey 128 decodes to its linear value", ok && near(o[1][1], srgb_dec(128 / 255.0), 0.003, 1e-4)); CHECK("SDR red is BT.709 red in BT.2020 (0.6274, 0.0691, 0.0164)", ok && near(o[2][0], 0.627404, 0.003, 0) && near(o[2][1], 0.069097, 0.01, 1e-4) && near(o[2][2], 0.016391, 0.02, 1e-4)); } + // The PQ layer (RGB10A2 in BT.2100 PQ, the Mac's default HDR layer): the game's HDR10 window goes in as it is, an 8-bit sRGB source is encoded as HDR10 with its white at the paper white + { + int ok = run(src10(grey), MTLPixelFormatRGB10A2Unorm, o), bad = 0; + for (int i = 0; ok && i < N; i++) for (int k = 0; k < 3; k++) bad += fabs(o[i][k] - grey[i][k]) > 0.5; + CHECK("PQ layer: the HDR10 window's code values are written as they are", ok && !bad); + uint8_t c[N][3] = {{255, 255, 255}, {128, 128, 128}, {255, 0, 0}, {0, 255, 0}, {0, 0, 255}, {0, 0, 0}, {64, 64, 64}, {255, 255, 255}}; + ok = run(src8(c), MTLPixelFormatRGB10A2Unorm, o); + CHECK("PQ layer: SDR white is the paper white (203 nits), black is code 0", ok && fabs(o[0][0] - pq_code(203)) <= 1.5 && fabs(o[0][2] - pq_code(203)) <= 1.5 && o[5][0] <= 1); + CHECK("PQ layer: SDR mid grey 128 is its linear value at the paper white", ok && fabs(o[1][1] - pq_code(srgb_dec(128 / 255.0) * 203)) <= 1.5); + } // A screenshot of an HDR frame (an 8-bit target): as an SDR screen shows it, highlights to SDR white with their hue, then BT.709 { unsigned c[N][3]; for (int i = 0; i < N; i++) c[i][0] = c[i][1] = c[i][2] = pq_code(nits[i]); diff --git a/dm/host/front_mac.m b/dm/host/front_mac.m index c07126f0b9..2728615490 100644 --- a/dm/host/front_mac.m +++ b/dm/host/front_mac.m @@ -1364,14 +1364,19 @@ void front_run(void (*guest)(void)) { // SDR content is converted from BT.709 in present.metal), in linear light (present.metal fs_linear), with HDR10 metadata: Core Animation maps the values above 1.0 // into the screen's headroom, and a screen without headroom shows the SDR range if (getenv("DM_HDR") && atoi(getenv("DM_HDR"))) { - g_layer.pixelFormat = MTLPixelFormatRGBA16Float; - CGColorSpaceRef ecs = CGColorSpaceCreateWithName(kCGColorSpaceExtendedLinearITUR_2020); + // The layer takes the game's HDR10 window as it is: RGB10A2 in BT.2100 PQ (present_mode: an SDR source is encoded as HDR10). An RGBA16Float extended linear BT.2020 layer + // (DM_HDR_LAYER=float) showed black blocks with coloured squares in them over flat areas (Mario Kart World's water on the course map), now and then and after the window came back + // to the front, on a MacBook Air M5 (macOS 27.0.1, potential headroom 2): the frames the game drew and the host's own copies of them were clean, so the blocks appear after the present + int pq = !(getenv("DM_HDR_LAYER") && !strcmp(getenv("DM_HDR_LAYER"), "float")); + g_layer.pixelFormat = pq ? MTLPixelFormatRGB10A2Unorm : MTLPixelFormatRGBA16Float; + CGColorSpaceRef ecs = CGColorSpaceCreateWithName(pq ? kCGColorSpaceITUR_2100_PQ : kCGColorSpaceExtendedLinearITUR_2020); + g_layer.colorspace = ecs; CGColorSpaceRelease(ecs); g_layer.wantsExtendedDynamicRangeContent = YES; if (CAEDRMetadata.available) { g_hdr_meta = 1; front_hdr_metadata(); } else LOG("[RED] front end: no EDR metadata on this Mac (CAEDRMetadata): values above the screen's headroom clip"); if (atoi(getenv("DM_HDR")) == 2) win.title = [win.title stringByAppendingString:@" (DM_HDR=2: HDR on any screen)"]; // told apart from the setting's DM_HDR=1 - LOG("front end: HDR output on (DM_HDR): the window's layer is RGBA16Float in extended linear BT.2020%s; the screen's headroom %.2f now, at most %.2f", g_hdr_meta ? ", with HDR10 metadata" : "", + LOG("front end: HDR output on (DM_HDR): the window's layer is %s%s; the screen's headroom %.2f now, at most %.2f", pq ? "RGB10A2 in BT.2100 PQ" : "RGBA16Float in extended linear BT.2020 (DM_HDR_LAYER=float)", g_hdr_meta ? ", with HDR10 metadata" : "", (win.screen ?: NSScreen.mainScreen).maximumExtendedDynamicRangeColorComponentValue, (win.screen ?: NSScreen.mainScreen).maximumPotentialExtendedDynamicRangeColorComponentValue); front_hdr_screen_read(win.screen ?: NSScreen.mainScreen); [NSTimer scheduledTimerWithTimeInterval:0.25 repeats:YES block:^(NSTimer *t) { (void)t; front_hdr_screen_read(win.screen ?: NSScreen.mainScreen); }]; diff --git a/docs/env_switches.tsv b/docs/env_switches.tsv index f49196460f..f7c59734eb 100644 --- a/docs/env_switches.tsv +++ b/docs/env_switches.tsv @@ -64,6 +64,7 @@ DM_GPU_CAPTURE_DIR path Folder for `DM_GPU_CAPTURE`'s traces (default `gpu_captu DM_GPU_CAPTURE_KEEP diagnostic Before each GPU capture, prunes the capture folder's `dm_*.gputrace` entries so that the newest `KEEP - 1` stay (default 2) DM_GUEST_SPAWN_TEST diagnostic `=1` runs a self-test before `nnMain`: a host-started guest thread does SDK malloc / free and locks an `nn::os` mutex, and the result is logged; `=2` also keeps a resident thread DM_HDR player `1` turns on HDR output for Switch 2 games (the Mac and iOS settings row; any non-zero value; `0` or unset is SDR). While the window's screen can show EDR, the status query answers HDR, so the game makes its RGB10A2 PQ window; on a screen without EDR it answers SDR (`2` answers HDR there too: the system maps the HDR picture down instead of the game drawing its own SDR). The layer is extended-range BT.2020 either way. Measured on Donkey Kong Bananza, and on Mario Kart World to its title. +DM_HDR_LAYER diagnostic `float` (Mac): the HDR layer is RGBA16Float in extended linear BT.2020, the present decoding PQ to linear light, instead of RGB10A2 in BT.2100 PQ with the game's HDR10 window as it is (the default; the float layer showed black blocks over flat areas on an M5) DM_HDR_PAPER_WHITE diagnostic `=` the paper white: the answer to `vi::QueryDisplayHdrPaperWhite` while HDR is on, the level a PQ window's SDR white decodes to in the HDR layer, and 1.0 of its HDR10 metadata (1 to 65535, read once; default 203, the BT.2408 reference). No launcher setting sets it: it is the SDR white level a console's HDR calibration would set. Untested at other values DM_HDR_PEAK diagnostic `=` the display's peak that `DM_HDR` reports (decimal, 16-bit; default 1000, the extended luminance range the games declare). No launcher setting sets it: it is the peak a console's HDR calibration would measure. Untested at other values DM_HID_LOG diagnostic `1` logs the HID button usage number of each button pressed on a raw HID gamepad (`pad_hid_mac.h`), to write a `DM_HID_MAP` diff --git a/docs/reference/rendering.md b/docs/reference/rendering.md index 400897b5e1..5b95259cf1 100644 --- a/docs/reference/rendering.md +++ b/docs/reference/rendering.md @@ -628,7 +628,8 @@ The replay on Metal (`dm/gfx/nvn/nvn_mtl.m`) and the Vulkan and OpenGL layers. E - **Symptom:** no HDR output on a Switch 2 game that asks for its HDR status: its window stays RGBA8 and the picture is SDR. - **Cause:** `vi::QueryDisplayHdrStatus` answered SDR, so the game never made its RGB10A2 PQ window, and the present had no decode for it. - **Fix:** with `DM_HDR=1` the status answers HDR on (Switch 2 SDKs only) while the window's screen can show EDR: its potential headroom is above 1 (`maximumPotentialExtendedDynamicRangeColorComponentValue` on the Mac, `potentialEDRHeadroom` on the iPhone). The front end reads it four times a second, so the answer follows the window to another screen. Moving the window between screens drops some frames whether or not the game rebuilds its window ([Mario Kart World](../../dm/games/0400C3F00006E000/README.md#hdr-output)). On a screen without EDR the status answers SDR and the game draws its own SDR picture, as a Switch 2 does on a TV without HDR10; `DM_HDR=2` answers HDR there too, and Core Animation maps the HDR picture down instead. The format is 1 with HDR on: the SDK's `DefaultDisplay::QueryHdrStatus` sets on to (format == 1), and a format above 1 is an error. The game asks every frame, so it can switch windows when the answer changes. - - The layer is RGBA16Float in extended linear BT.2020 on any screen. `present.metal` `fs_linear` decodes PQ to linear light (1.0 at `DM_HDR_PAPER_WHITE`, default 203; it and `DM_HDR_PEAK` have no launcher setting, standing for what a console's HDR calibration sets) and maps BT.709 sources into BT.2020, their white at 1.0 too. An RGB10A2 window is decoded as PQ only after the status has answered HDR on; any other RGB10A2 window is treated as gamma-coded SDR. + - On the Mac the layer is RGB10A2 in BT.2100 PQ (`kCGColorSpaceITUR_2100_PQ`) on any screen: the game's HDR10 window is written as it is, and `present.metal` `fs` encodes a gamma-coded BT.709 source (the game's SDR window before it switches) as HDR10 with its white at `DM_HDR_PAPER_WHITE` (default 203; it and `DM_HDR_PEAK` have no launcher setting, standing for what a console's HDR calibration sets; `present_mode` 3). An RGB10A2 window is taken as PQ only after the status has answered HDR on; any other RGB10A2 window is treated as gamma-coded SDR. + - The first layer was RGBA16Float in extended linear BT.2020 (`fs_linear`: PQ decoded to linear light, 1.0 at the paper white; still the iPhone's, and the Mac's with `DM_HDR_LAYER=float`). On a MacBook Air M5 (macOS 27.0.1, built-in screen, potential headroom 2) it showed black blocks with coloured squares in them over flat areas, Mario Kart World's water on its course map, for a frame now and then and for a moment after the window came back to the front. The frames the game drew had none: `DM_NVN_MAGSCAN` found no non-finite pixel in any pass, 100 frames saved through `DM_CTL` `shot` while the blocks showed were clean, and present snapshots (`DM_PRESENT_SNAP=2`) did not remove them. Guarding the decode's `pow` against a zero base (the present library is built with fast math) and clipping colours to Display P3 did not either. With the PQ layer they are gone and the colours are the same by eye; why the extended-range layer shows them is open (Core Animation, or the system's handling of that format on this Mac). - The layer carries HDR10 metadata (`CAEDRMetadata`: 1.0 is the paper white, the range is the one the game declares), so Core Animation maps the values above 1.0 into the screen's current headroom, following brightness and the screen's modes. The present writes them as they are. A screenshot of an HDR frame brings each highlight's largest channel to SDR white and keeps its hue. - Before this, the present rolled the highlights off into the headroom itself. On [Mario Kart World](../../dm/games/0400C3F00006E000/README.md#hdr-output) the two looked alike by eye and reached the screen in the same time in a window, so the system's mapping replaced it. - **Check:** `make -C dm/gfx/tests GAME= run` runs `t_present_hdr`: the status answer on a screen with and without EDR, PQ codes from 0 to 10,000 nits, SDR sources, screenshots with a highlight's hue, and a 10-bit window of a game that never asked for HDR. `DM_PRESENT_SHOWN=1` logs how long frames take to reach the screen.