Skip to content

Experiment: defer redundant pull-only pipeline archives - #4

Closed
oliverames wants to merge 1 commit into
codex/experiment-basefrom
codex/experiment-lazy-pipeline-archives
Closed

oliverames wants to merge 1 commit into
codex/experiment-basefrom
codex/experiment-lazy-pipeline-archives

Conversation

@oliverames

Copy link
Copy Markdown
Owner

This is a fork-only experiment targeting codex/experiment-base. Keep it isolated from main and the existing upstream PR heads. The switch is off by default.

What and why

Pipeline initialization opens every device archive before it knows whether a usable bundled archive supplies the needed base. DM_PSO_LAZY_DEVICE_ARCHIVES=1 defers only device p-files when a usable bundle exists. Mixed/layout archives and compaction remain eager. Every existing sequence is reserved before capture, and a real miss opens deferred files exactly once into an immutable snapshot. Draw and prebuild fallback are preserved.

Checked

  • Real Mac Metal bundle tests: 67 checks in each mode; archive tests: 74 in each mode.
  • Fresh-process seed/replay/compaction verification in both modes, with and without function constants.
  • Covered draws leave deferred files closed; a real subset-bundle miss hits device fallback; capture preserves old files and uses the next sequence. Documentation/environment checks pass.
  • No game startup-time or memory savings are claimed. No Newt archive performance comparison or gameplay route was run. The synthetic archive tests exercise the actual production lookup/capture paths; real-game benefit remains to be measured.

Docs

  • Opt-in, fallback and measurement limits documented in the shader reference.
  • No game files, keys or extracted game data.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-29T01:37:06.811459Z a0ff9e1 PR opened
🔒 Security Review ✅ Completed 2026-10-29T01:40:03.721046Z a0ff9e1 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

oliverames pushed a commit that referenced this pull request Nov 2, 2026
The walk from the load back to the bounds check noticed copies and masks but no
other write: cmp w18,#4; b.hi; add w18,w18,#1; ldrsw x18,[x4,x18,lsl #2] read
entries 0 to 4 while the code uses 1 to 5, so one case's target was missing with
no sign of it. A site with a write of the index after its check, or a call with
a caller-saved index, is now refused, as jump_adr_offsets and half_count already
refuse a write. A refusal goes on to the other recognizers and ends in the jump
table assertion. Writes before the check do not count, which keeps the sites
check_before_copy takes.

A write that keeps the value, such as sxtw x0,w0, is refused too; no module on
the drive has one.

Checked: test_jump_copies.py (the new cases fail on the previous rewriter). Of the
345 unique modules on the drive, old and new recognition are the same on 281, and
60 more mains were scanned on the previous code with no site this refuses. That
includes Smash 13.0.5, EA SPORTS FC 27 and Well Dweller. Four could not be checked:
New Super Mario Bros. U Deluxe's main, sdk and rtld fail alike (no profile), and
Smash 1.0.0's main stops at a table the game's note already lists. No regress.py
run: nothing changes in the modules checked.
@oliverames

Copy link
Copy Markdown
Owner Author

Closing: experiment evidence and testing are tracked in the upstream nxnative PRs, so the fork-only experiment copy comes down. Keeping the branch here for reference.

@oliverames oliverames closed this Nov 5, 2026
oliverames pushed a commit that referenced this pull request Nov 7, 2026
…d jump_adr_offsets took the nearest cmp of the index, whatever followed it (issue seedds#977, PR seedds#1515)

guarded_count takes a compare only when, between it and the table load, the first branch on an unsigned condition comes with the compare's flags still in place: b.hi out of the dispatch reads N + 1 entries, b.hs / b.cs N (read as N + 1 before), b.ls / b.lo into it the same counts when the instruction before its target ends the path that failed. A compare with no such branch only sets the flags and the walk goes on to the check before it; with none the site is refused. Before, `cmp w18,#4; b.hi exit; cmp w18,#1; ldrsw ...` read 2 entries of a table of 5 with no error.

Checked (by the author): 165 sites of 27 modules of 6 games, all cmp; b.hi, rewritten byte-identically; GuardedCheck, 8 cases, 7 failing before. Checked here, as seedds#977 asks before a stricter rule goes in: 35 modules of 12 games on the T7 (Cuphead, Three Houses, Minecraft Dungeons II, Octopath Traveler 0 and II, Okami HD, Sonic Superstars, Ocarina of Time, DK Bananza, Mario Tennis Fever, Pokopia, Splatoon Raiders) rewrite byte-identically before and after, and so do Super Smash Bros. Ultimate 13.0.5, Mario Kart 8 Deluxe and Super Mario Odyssey's main, sdk and subsdk0 (against origin/main, with the page-zero rule of the commit before); test_jump_copies 75 OK, lint_python 0 new, test_docs. No regress.py run: no module that rewrote before changes. Modules with b.hs or a branch into the dispatch are still tested only synthetically.

Co-authored-by: Atomsk-debug <135618857+Atomsk-debug@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant