Skip to content

Extract conservative raw-load grouping and high-product shader lowering - #1891

Open
bmjask wants to merge 1 commit into
seedds:mainfrom
bmjask:codex/shader-conservative-load-lowering
Open

bmjask wants to merge 1 commit into
seedds:mainfrom
bmjask:codex/shader-conservative-load-lowering

Conversation

@bmjask

@bmjask bmjask commented Nov 9, 2026 •

Copy link
Copy Markdown

Extract two conservative shader-lowering experiments from #1663 without replacing current main's translator or undoing its newer vector loads and scalar extended-multiply overloads.

  • Opt-in DMSH_RAW_LOAD_QUADS=1: combine four proven adjacent reads through a strictly recognized legacy raw-memory helper in read-only ordinary Ampere compute. One lookup is shared only when the whole 16-byte span resolves to the same winning range. Boundaries, overlaps, wrap and unknown helper forms retain scalar behavior.
  • Simplify the unsigned high half of multiplication by a constant power of two to its exact shift identity before upstream adds scalar overloads. Signed, non-power-of-two, ambiguous and unknown forms stay unchanged.

This is a deliberately narrower extraction than the integration branch's pointer-site cache and mesh/compute grouping implementation. Those newer helper-ABI changes are not included. No image-quality settings are reduced; no translator binary or lockfile is replaced.

Historical A/B evidence — not this extraction's performance result

Campaign chat, 2026-11-08 02:42–02:46 UTC: the broader ordinary-compute grouped-load implementation reduced median GPU time 11.77 → 11.09 ms (5.8%), with 5.5% reported in reverse test order, matching GPU clocks and no shader compilation during measurement. Those numbers belong to the combined branch's pointer-site implementation. They do not establish a speedup for this legacy-helper extraction against current main. New measurements are pending.

Current checks

  • 19,584 scalar/grouped GPU word comparisons against a CPU oracle, zero failures, under Metal validation. Covers range boundaries, overlap priority, 32-bit carry, hint paths, and 8,192/16,384-entry tables.
  • 22 Python lowering tests pass (nine wide-load, nine high-product, four dynamic-helper). The high-product tests include 32,128 arithmetic cases and verify its order before upstream's overload pass.
  • 17 chain tests, eight cache-fingerprint tests and 11 documentation tests pass. Header regenerated for chain dc2cecb15d7f; the opt-in flag changes the chain identity too.
  • Python lint attempted: six findings in unchanged check_thread_cuts.py, test_astral_fps.py, and unity_bucket.py prevent a clean overall ratchet result. No findings reported in the changed files. Baseline was not relaxed.

Regression status: incomplete, not a pass

regress.py impact origin/main..HEAD was run. Five-game check selected Ocarina (Ampere/raw loads), Breath of the Wild and Tears of the Kingdom (other NVN rendering), Pokémon Scarlet (different NVN engine), and Hades (other shader coverage). The harness stopped before shader builds/game runs:

[RED] regress.py: no Mac build of 01007EF00011E000, 0100F2C0115B6000, 0100A3D008C5C000, 0100535012974000 under /Users/Shared/Games (found: 0400486024DEC000)

Outstanding validation: full Ocarina shader rebuild, host rebuild, gameplay/image comparison, matched A/B measurements and other-game validation. The user's current playable app was not rebuilt or replaced. This PR is for review of the extraction, not a ready-to-merge performance claim.

Front-ends: rebuild matching shader libraries and host for this chain; enabling DMSH_RAW_LOAD_QUADS requires a header generated with the same option. Do not mix these shaders with an existing app's older chain.

Reproducible evidence

Tested extraction: 917379a2d0672358a9caab0b90498a596ef44785. Test source at that commit; game note at that commit. Run on an Apple-silicon Mac with the repository's build dependencies configured:

make -C dm/gfx/tests GAME=0400486024DEC000 check_t_raw_quad
python3 -m unittest discover -s dm/gfx/tests -p test_dmsh_mulhi.py
python3 -m unittest discover -s dm/gfx/tests -p test_dmsh_raw_quads_dynamic.py

Saved local test-output excerpt (not a new test run or a CI result):

raw quad GPU: 19584 word comparisons, 0 failures

Historical A/B figures above describe the older combined branch. They are not fresh performance validation of this extracted commit. The remaining gameplay/regression limitations above still apply.

Rebase validation — 2026-11-09

Current head: 918c2182251d72ddec02ec34f4b187758db5cd9c, one commit above upstream 9983d659e6101867215e4afe5dd6287ad8e0d4a3.

Focused Python shader/chain/docs checks passed 45 tests. The raw-load GPU oracle passed 19,584 word comparisons. Regenerated DMSH_CHAIN is 26bc96c08027. Full graphics suite and full game shader/runtime regression on this shader extraction remain pending.

Mac fixtures were tested before the final upstream iOS-linker-only commit; verified that dm/ios/tools/gen_build.py is the sole subsequent file difference. Impact analysis was also run. No new gameplay A/B performance claim: historical figures above still describe the combined branch. Full current-chain game rebuild and gameplay regression remain pending; only Ocarina assets are locally available, and other titles/iOS are untested. The installed app and codex/ocarina-playability were preserved.

@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders game: 0400486024DEC000 A game labels Nov 9, 2026
@bmjask
bmjask marked this pull request as ready for review November 9, 2026 01:14
@bmjask
bmjask force-pushed the codex/shader-conservative-load-lowering branch from 917379a to 918c218 Compare November 9, 2026 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants