Skip to content

Add an opt-in packed HDR conversion measurement tool - #1168

Closed
oliverames wants to merge 2 commits into
seedds:mainfrom
oliverames:diagnostic-metal-packed-conversion
Closed

oliverames wants to merge 2 commits into
seedds:mainfrom
oliverames:diagnostic-metal-packed-conversion

Conversation

@oliverames

@oliverames oliverames commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

What and why

Add an opt-in RG11B10Float measurement tool for investigating differences between shader replay predictions and GPU attachment output. It submits 95 exact FP32 bit patterns, reads the packed attachment words back, and reports nearest-even and lower-representable candidates without requiring either result. It measures the combined fragment-output, attachment-store and blit path, not isolated rounding hardware or game sampling.

Run python3 dm/gfx/tests/metal_conversion_probe.py --output-dir /path/to/new-run on a Mac with Xcode and Metal. The directory must be new. Each completed report includes raw words, binary artifacts, run ID, OS/GPU/compiler/SDK information and source/input hashes. Failed attempts record failure rather than leave a prior successful report. No game data, keys, profile or renderer is used. The standalone Makefile target is excluded from make run; the four deterministic tests verify tool behavior, not GPU rounding policy. An iPhone wrapper remains outstanding.

@seedds, this is part of the Switch 2 games running on iPhone investigation for DKB on iPhone 17 Pro Max. Conditional replay had small packed-color differences, but actual game constants and sampling precision remain unmeasured. The tool may help other titles using packed HDR targets. It does not establish A19 Pro conversion behavior, a shader mismatch, or a pacing/aliasing/fur-shimmer improvement. Draft pending hosted CI and further platform integration/verification.

Superseded #1154 is closed and no longer appears in this diff; the five diagnostic files are unchanged. Updated to main a70f071 with the maintainer's equivalent mesh-stage fix. This change is also folded into oliverames#9, a stacked PR; this PR stays open as it is.

Checked

  • Tests: four deterministic tests, 11 documentation tests, the full Python ratchet (zero new or stale findings), targeted clang-tidy (zero findings) and whitespace checks pass. After the merge with main, the four tool tests and 11 documentation tests pass again. Parent and independent reviewer decoded every raw word and channel; source hashes match the tested files. Hosted CI is pending.
  • Games run, with title ID and version: not stated. The tool uses no game data, keys, profile or renderer.
  • Devices and OS: Apple M2 Pro (OS version not stated). Result: 95 complete cases. All 285 channel observations matched the lower representable candidate; 113 differed from nearest-even. This is a device/run observation, not a Metal contract or a DKB defect. Input SHA-256: 90d94428e0a6ccb1b4e36a0e99f5238496daa04cb099c938ef4b8ad56151bbfb; packed-output SHA-256: 59465088ebe292331077338b2a437f2e6a519a5f3ee8aef466ed2523a50a77ae. Final run ID: b77c67d9-cd15-41e2-8726-e6f9c6318b23. No iPhone run (the wrapper is outstanding).

Front ends

None: no front end has to act on this.

Docs

  • The finding is in its game note or its docs/reference/<subsystem>.md entry, and python3 dm/tools/docs_symptoms.py was run (the symptom index). The diff changes dm/gfx/README.md only; no reference entry or game note is in it, and the description does not say the symptom index was run.
  • A chain tool changed: dmsh_chain.py --header was run and dm/gfx/nvn/dmsh_chain.h is part of the change. Not applicable: no chain tool changed (the five files are the probe, its tests, the Makefile and dm/gfx/README.md).
  • No game files, keys or data extracted from a game in the change or the attachments

Rebased onto main bd979ce; conflicts resolved in dm/gfx/tests/Makefile.

@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders labels Nov 4, 2026
@oliverames

Copy link
Copy Markdown
Contributor Author

@seedds, the reusable packed-HDR measurement for Switch 2 games running on iPhone is now at 49979db, reconciled with your landed mesh-stage fix. The 95-case M2 Pro run and independently checked raw words reproduce the earlier conversion observation, without asserting that rounding mode is universal. #1153 now has a separately reviewed iPhone measurement mode and a signed build, but no physical iPhone 17 Pro Max conversion result yet. This stays draft; DKB constants, sampling precision and fur-shimmer causation remain unmeasured. The probe may help packed-target investigations in other titles.

@oliverames

Copy link
Copy Markdown
Contributor Author

@seedds Fresh physical-device evidence for the Switch 2 games running on iPhone work: the RG11B10Float diagnostic ran on Newt, iPhone 17 Pro Max / Apple A19 Pro GPU, iOS 27.2. All 95 ordered input words and packed output words exactly match the earlier M2 Pro measurement. Using the reference analysis for this controlled input set, all 285 channel observations match the lower representable candidate; 113 differ from nearest-even. Packed-output SHA-256: 59465088ebe292331077338b2a437f2e6a519a5f3ee8aef466ed2523a50a77ae.

Probe source: 49979db6841ab92f23c783d3f6ca25ea3bdb1c0f; wrapper source is the measurement mode in #1153 (edd9817d). Device report run ID: 8b17f739-4eb1-4dee-970e-8d1818e13a16, complete status, input version original-95-v1, input SHA-256 90d94428e0a6ccb1b4e36a0e99f5238496daa04cb099c938ef4b8ad56151bbfb. Reproduce using the documented standalone probe or #1153's --measurement metal_conversion --run with your signing/device settings.

This measures fragment output stored to an RG11B10Float attachment and blitted to a buffer. It does not establish a universal rounding rule, DKB sampling arithmetic, an emulator bug, or reduced fur shimmer. It provides a device-specific reference potentially useful for other titles using this format. Actual DKB constants and sustained gameplay remain separate work. Build hashes and matching run identity establish our artifact chain, but the device report does not independently attest the installed executable bytes.

@oliverames

Copy link
Copy Markdown
Contributor Author

Head 2e0fded fixes the Tool tests failure this PR caused: dm/gfx/tests/test_metal_conversion_probe.py imported its sibling metal_conversion_probe without putting its directory on sys.path, and the CI runs each test file from the repository root (ModuleNotFoundError, run 41779422648). It now follows the pattern in test_dmsh_ampere_alu.py; python3 -m unittest dm/gfx/tests/test_metal_conversion_probe.py fails before the change and runs 4 tests OK after it.

The other failures are not from this diff:

  • The earlier Sanitizers failure was Assertion failed: (signaled(device_stopped, 3)) in t_audio_device.c:191 under TSan, a timing assertion in audio code this PR does not touch; it passed on the other PRs' runs.
  • The earlier clang-tidy findings (t_cpu_storage.m, t_pool_chunks.m, t_sparse_tiles.m, hle_fs.c, 6 extra reallocs in nvn_mtl.m) came from main at that time and are not in this PR's diff. Run 41839124677 on 2e0fded fails only on four pairs that also fail on main: run 41819072419 at 0b8e0dc (dm/gfx/vk/hle_vk.m, dm/host/hle_jit.c, dm/host/hle_oe.c, dm/host/web_apple.m).
  • ruff and pyright (run 41839124618) reports the same 5 Object of type "None" is not subscriptable findings (394 findings, 389 in the baseline) as main at aafe01d, run 41837854038.

@oliverames
oliverames force-pushed the diagnostic-metal-packed-conversion branch from 56cd971 to 3b32464 Compare November 7, 2026 19:36
@oliverames

Copy link
Copy Markdown
Contributor Author

Folded into one HDR pull request, oliverames#9, which carries this probe together with the per-model peak default and the launcher rows. It is stacked on #1176, so it can only merge after that. Closing this one to keep the HDR work in a single place.

@oliverames oliverames closed this Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant