Skip to content

Preserve GPU-write page identities across hash collisions - #1762

Closed
thehobbit85 wants to merge 1 commit into
seedds:mainfrom
thehobbit85:codex/gpu-write-pages-1505
Closed

thehobbit85 wants to merge 1 commit into
seedds:mainfrom
thehobbit85:codex/gpu-write-pages-1505

Conversation

@thehobbit85

Copy link
Copy Markdown
Contributor

Fixes #1505.

Two distinct running GPU-write pages could collide in a ledger slot and replace its page identity with a wildcard. Any untouched page with that hash then looked GPU-written, refusing a safe partial uniform shadow and ending the render pass.

Keep each page identity and newest fence within a bounded 16-slot probe range in both existing ledgers. Insertions search past completed holes before reusing them, and lookups search the same range. No live entry is evicted; a full range raises the existing conservative global fence until the unlisted write completes. The two ledgers remain 2 MiB with no allocations. This is a collision-resolution change, not a table-capacity increase.

The final synthetic regression has 51 checks. It covers distinct collisions, genuine overlap, wrapping and the last probe, completion and monotonic fences, expired-slot reuse, full-capacity overflow, a later tracked hazard surviving global-fence completion, actual uniform-shadow bytes, and refusal reason counters. Exact main production code fails 23 checks; the candidate passes all 51 in both encoding modes with Metal validation enabled.

Validation on Apple M2 Max, macOS 26.6, Xcode 27.2 (27B52), native ARM64 Python 3.14.6:

  • make -C dm/host GAME=0400486024DEC000 -j1: passed.
  • make -C dm/gfx/tests GAME=0400486024DEC000 -j1 -k run: all run targets attempted. Only check_t_present_crop fails with SIGSEGV 11; an external binary using exact main production code reproduces SIGSEGV 11. Every other target passes, including counter-overlap 42 checks and render-enable continuation 38 checks in both encoding modes.
  • Python test workflow: 1,852 tests in 165 modules, with two unrelated failing modules. test_env_switches reports the same generated table under candidate and HEAD reads; test_xcode_workflow lacks its synthetic RomFS.img in six subcases. The Python inputs are unchanged from HEAD; the parent independently observed these baseline failures and PR Fix CI fixtures and lint ratchets #1746 addresses them.
  • Documentation: regenerated symptom index, 11 checks pass; whitespace check passes. Historical OoT capacity-mitigation measurements moved from the reference into the game note and explicitly separated from this change.
  • Python lint: 376 findings versus 371 in its ratchet, five excess findings in unchanged test_patches.py. All Python and lint configuration inputs equal HEAD. C compilation database generated; clang-tidy is unavailable on this host and no tool was installed.

Impacted games selected for runtime regression:

Game Why selected Available validation
Ocarina of Time (0400486024DEC000) Original source-only collision report, shared uniform shadows No Mac build available
Splatoon Raiders (0400380020056000) Running inline updates, partial shadow holes and high write counts No Mac build available
Breath of the Wild (01007EF00011E000) Partial uniform updates; committed usage manifest reaches affected functions No Mac build available
Tears of the Kingdom (0100F2C0115B6000) Partial uniform updates and the shared NVN write tracking; manifest reaches affected functions No Mac build available
Fire Emblem Engage (0100A6301214E000) Documented partial-shadow/pass-ending path No Mac build available

The committed-range impact report was generated with scripts/regress.py impact a00f03bb..HEAD. Its broad rule reaches every NVN profile; profiles without usage manifests remain unknown. Game selection preflight reports all five builds missing, so no gameplay, FPS or iPhone result is claimed. More than 16 colliding live pages can still cause conservative unrelated refusals until the overflow fence completes. The issue's provided patch was evidence only; no extra mode, larger ledger, spill allocation or game rule was added.

Related PR overlap reviewed: #1320 changes wide intervals, #1162 caches conflict checks, #1367 enables another deferred-uniform path, and #1435's dynamic pending-write list is already on main. None resolves page-identity collisions.

Tested base: a00f03bb3f7275c3f897df5834e2d6da91783992. Immediately before publication, current main 62f6a52c4fa9b42b73e5c37ef9a88794c8d15ae5 and all PR states were inspected. Its Unity/vblank changes do not alter the page ledgers or implement this fix.

Use bounded probing in the existing page ledgers, retain newest fences across completed-slot holes, and conservatively protect overflow. Avoid refusing unrelated partial uniform shadows solely because live pages hash to the same slot.

Fixes seedds#1505. Synthetic tests demonstrate the ledger and shadow behavior; current-main gameplay and performance are unmeasured.
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation game: 0400486024DEC000 A game labels Nov 8, 2026
@seedds

seedds commented Nov 8, 2026

Copy link
Copy Markdown
Owner

Thank you for this. The probing design is sound and the synthetic tests are thorough. Before landing it I measured whether main still produces the false hazard, and on Ocarina of Time it doesn't, so I'm closing this without merging.

What I measured. Main at 028a263, plus a temporary counter in ubp_get that splits gw_live_in(g_gwx) refusals into four causes (global fence, wide write, exact page, wildcard only) and logs the live slots and live wildcards in g_gwx every 300 frames. Ocarina of Time v131072 on an M6 Mac, our village walk from a save (230 s, camera turns, window on).

Frames 3600–6900 (11 windows of 300) Result
earlier submission running refusals 0 (every other refusal reason was also 0)
Partly covered blocks read from memory about 10,200 made / 22,800 cached per window
Live slots in g_gwx 0 to 14,150
Live wildcard slots at most 1

Why so few collisions. At 14,150 live pages in 65,536 slots, uniformly random keys would give about 1,300 colliding slots. gw_slot is a Fibonacci (multiplicative) hash, and it spreads runs of consecutive pages across the table without collisions. Running GPU writes are mostly contiguous ranges, so collisions almost never happen. The 291 refusals per 300 frames recorded after the 65,536-slot change no longer happen on current main.

With no false refusals left to remove, the PR would only add its costs: 16 slot comparisons per page on every submission's commit, and an overflow fallback that refuses every page instead of the colliding ones. If a game ever shows wildcard-only refusals, this bounded-probing design is what we'd use, and we'd come back to this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Contribution: retain GPU-write page identities across hash collisions (current-main patch)

2 participants