Repository navigation
iOS: run depth/stencil regression checks on physical iPhones - #1153
oliverames wants to merge 1 commit into
Conversation
|
Commit 54e15b37 adds Both new choices passed project/header generation using the iPhoneOS SDK. Ruff and pyright report zero findings for the runner, all 11 documentation tests pass, and the missing-source check passes. The initial sandboxed generation failed on access to Xcode's compiler cache; generation succeeded with cache access. The local runner previously exercised these choices on iPhone 17 Pro Max: 31 window-copy checks and 28 push-capture checks passed. Those results apply to their recorded fixture hashes, not the newer 32-check window-copy completion case. No new phone run occurred for this commit. The PR remains draft, and neither fixture establishes sustained gameplay, visible-screen output or a fur-shimmer improvement. |
|
@seedds, new head edd9817d adds a separate raw-measurement mode for the Switch 2 games running on iPhone investigation. It uses #1168's standalone conversion probe, embeds the same 95 exact inputs, and records raw output without treating rounding as a regression pass. The iOS 26 app builds and signs, source/input hashes are checked, report-contract tests and Python/docs checks pass, and independent review found no actionable issues. It has not been installed or measured on iPhone 17 Pro Max. Existing fixture behavior remains separate; this stays draft pending physical measurement and hosted CI. |
|
@seedds Flagging from my open-PR index of my 32 open PRs here: this one stays draft pending hosted CI and the remaining device verification. The description above is still the full picture, nothing new from me. Happy to rebase, split or retitle whenever review suits. |
|
@seedds Marking this ready for review now, so it stops waiting in draft on me. The remaining item from the description is still open: hosted CI and the remaining device verification. It can carry on alongside a review instead of ahead of it. The description stays the source of truth; happy to rebase, split or retitle whenever review suits. |
9f02a07 to
92b6d73
Compare
…(dm/gfx/tests/t_clear_ds.m) on a connected iPhone and reads its result back The GPU tests under dm/gfx/tests run on the Mac only, so a depth/stencil fault that shows on an iPhone's GPU alone had no test to run there. This tool builds a small signed app around t_clear_ds.m, compiled unchanged (a wrapper file renames its main and returns its two counters), with the support sources of the tests' Makefile (MTL_T_SRC), the profile header and the host's Metal shaders made for the iPhone SDK. With --run it installs the app, starts it and copies results.json and the test's log back from the app's Documents folder. --generate-only stops after the Xcode project and --build-only after the signed app. No game files or keys are read. clear_ds is the only fixture offered. What the tool decides is in four small functions. accept_report takes results.json only when it is a JSON object with this build's run ID and complete set: the same bundle ID keeps its Documents folder across installs, so an earlier run's report is still there until the new app writes its own, and a copy that does not parse is polled again rather than ending the run. report_passed wants a zero exit code, at least one check, no failed check and the app's own passed field, with the counts as integers. fixture_path and app_path name the test source and the built app. Checked on a Mac at this commit, on main fd8188c: dm/ios/tests/test_gfx_phone_runner.py (11 tests: the report cases above, the two names, and that support_sources still matches the tests' Makefile), which CI's tool tests job picks up with the other dm/ios/tests files; dm/tools/tests/test_docs.py (6 tests); the Python lint ratchet against origin/main (323 findings, all in the baseline, none new); --help; and --generate-only followed by an unsigned xcodebuild of the generated project for generic/platform=iOS (CODE_SIGNING_ALLOWED=NO, Xcode 27.3, profile 040074A01BF12000), which compiled and linked. Not checked at this commit: nothing was signed, installed or run on a phone, so --build-only's signing steps and all of --run are as they were on 2026-11-04, when the first form of this tool ran clear_ds on an iPhone (the pull request's description has what that run showed). The report handling has changed since then and has only the test above behind it. docs/development.md (Continuous integration) says how to run it and what a pass means.
92b6d73 to
2dba6cd
Compare
|
Closing this. It only runs on a physical iPhone, and the checks it would add are better made on the Mac, where you work. Thanks! |
What and why
The GPU tests under
dm/gfx/testsonly run on a Mac, so a depth/stencil fault that shows on an iPhone's GPU alone has no test to run there. This addsdm/ios/tools/test_gfx_on_iphone.py, which builds a small signed app aroundt_clear_ds.m, compiled unchanged, and with--runinstalls it on a connected iPhone, starts it and reads the result back.--generate-onlystops after the Xcode project and--build-onlyafter the signed app. It reads no game files or keys.It came from testing Donkey Kong Bananza on an iPhone 17 Pro Max. The fixture covers the shared depth/stencil clear behaviour, so it can help check renderer changes other titles use. It doesn't change how any game renders.
I've cut this PR down to that one change. Earlier versions also carried a
window_copyfixture, apush_capturefixture and a--measurement metal_conversionmode. Those are gone from this branch:window_copycame from #1140, which was closed,push_captureonly exists in #1160, and the measurement probe belongs to separate HDR work. The old head (92b6d73) is kept on my fork asarchive/iphone-gfx-runner-extras-20261110in case any of it is wanted later.The branch is now one commit on main
fd8188c9(head2dba6cd1).clear_dsis the only fixture offered.While trimming I moved what the tool decides into four small functions so it can be tested without a phone:
results.jsonis accepted only when it's a JSON object with this build's run ID andcompleteset. The same bundle ID keeps its Documents folder across installs, so an earlier run's report is still there until the new app writes its own. A copy that doesn't parse is polled again instead of ending the run.passedfield. Before, the tool trustedpassedalone.Checked
python3 dm/ios/tests/test_gfx_phone_runner.py(11 tests, new: which report is accepted, what counts as a pass, the names of what it builds, and that the support source list still matches the tests' Makefile). CI's tool tests job picks it up with the otherdm/ios/testsfiles.python3 dm/tools/tests/test_docs.py(6 tests). The Python lint ratchet against main: 323 findings, all in the baseline, none new.--generate-onlyon mainfd8188c9, then an unsignedxcodebuildof the generated project forgeneric/platform=iOS(Xcode 27.3, profile 040074A01BF12000). It compiled and linked.--build-only's signing steps and all of--runare untested at this commit. The only phone run was on 2026-11-04 with the first version of the tool, on an iPhone 17 Pro Max (A19 Pro GPU, iOS 27.2): all 11 cases and 59 checks passed withMetal API Validation Enabledin the collected log, and the full--runpath built, signed, installed, launched and pulled the matching report (Debug app, upstream renderer at1d4b8e82). The report handling has changed since then and only has the unit test behind it.regress.pyreport: the change adds a tool and a test and doesn't touch the host, the shader chain or any game.Front ends
None: nothing here is for a front end to act on.
Docs
docs/development.md(Continuous integration) says how to run the tool, what a pass means and what the test covers. There's no game finding or reference entry in this change, sodocs_symptoms.pywasn't needed.