Skip to content

Roadmap: keeping every game green as the project grows #313

Open

Description

@v0lumehi

Building on @jthrilly's analysis in #272: one translator and one host serve every profile, but each of us can only test the games we own, and CI has no game files and no Metal 4 (#112). So a change that is well tested on its own games can still break others. Examples: c3a0880 broke SMT V and Yoshi (#173, #218, #272); my prepare step hangs on Odyssey (#294, #276); my offline pipeline archive fails on TotK (#283).

I'm pausing my own feature work (Arkham Knight, 4K) to help put a foundation in place first. Nothing here needs a feature freeze. Every step below is a small PR that stands on its own, keeps the current behaviour, and is proven before it lands.

What the numbers say (upstream/main, last ~460 commits)

  • Merge conflicts sit in a few files. docs/development.md (49 PRs), the generated docs/agents/symptoms.md (47), the games table (41), dm/gfx/tests/Makefile (36) and its .gitignore (28), plus 80 commits that edited the game count line in AGENTS.md. 28 of 58 open PRs are conflicting.
  • The host only accepts libraries built by its exact chain version. DMSH_CHAIN was bumped 17 times in 15 days. Every bump rebuilds every game and moves every game onto an unvalidated translator. A cold rebuild of TotK takes 45 min for the Mac, 42 min for iOS and 7 min for the pipeline archive. The driver segment layout is written down twice: nvn_mtl.m and dmsh_build.py.
  • nvn_mtl.m has 10,617 lines and 1 MB (~290k tokens) (10,961 on Oct 27, 11,178 on Oct 28, 11,944 on Oct 31). It holds about 780 g_* globals and 125 getenv calls; 40 tests include it, and it was touched in 144 commits and 75 PRs. An agent has to load all of it for a small change.
  • Only 14 of 63 gfx tests run in CI. No test works at the frame level.
  • There are 229 DM_* variables, 66 of them undocumented. The 39 profile keys, by contrast, are all documented.

What stays

These are the right decisions and remain:

  • one translator for every game;
  • profiles with every fixed address gated by build ID;
  • the per-service host;
  • the record format with its static asserts;
  • fail-soft sidecars and the [RED] discipline;
  • the content-addressed Metal compile cache;
  • the docs culture, including test_docs.py.

The problem is coupling and centralisation, not these decisions.

Steps (each one a PR; the order puts safety before refactoring)

Conflicts out of the way

A safety net that knows every game

A reproducible setup

Code quality checks in CI (free on GitHub's runners for public repos; each one is a ratchet, so nothing goes red today, only regressions do)

Decouple the chain

Rules that enforce themselves

Extract, one module at a time, each proven by identical object code or replay digests

A clear platform scope

Triage, so issues and PRs find each other

When this was written, 61 open issues had no label, 24 had no reply, none of the 74 open PRs linked the issue it fixes, some were already covered by main (#315, #316, both closed since), and 33 of the 74 conflicted with it. On Oct 27: 78 open issues, none labelled, 33 without a reply; 121 open PRs, 60 of them conflicting. On Oct 31: 112 open issues, one of them labelled, 64 without a reply, and 183 open PRs.

What only the repository owner can switch on

Each of these is optional and takes one click:

  • Discussions, for design questions like this one;
  • branch protection with a merge queue;
  • Copilot code review on all PRs, via a ruleset (Settings → Rulesets → "Automatically request Copilot code review"). It needs a Copilot plan on your account, and GitHub offers Copilot Pro at no cost to maintainers of popular open source repositories: if you qualify, Copilot settings (avatar → Copilot settings) shows "Get access to GitHub Copilot" (docs). A second, independent review on every PR would help with the volume of PRs coming in;
  • CodeQL's default setup (see the item above);
  • a weekly release tag.

I'm happy to set these up and to do the work above.

@JMRBDev, your Wonder data in #272 is exactly what the translator diff would report automatically. Anyone who owns games we don't have: running regress.py on them (it landed with #328) is the most useful thing you can do.

Status (Oct 31): the boxes are ticked once the PRs land. Landed since Oct 28: #448 (96820ed, frame-exact replay in 710559c), #425 (a345f5b, without the clang-format report), #439 (68db795), the fix for #454 (713ecaa, from #605) and regress.py capture and replay (8325eef). #339 is closed, since Arkham Knight landed separately in d8bc921. Open now: #322, #400, #407, #408, #422, #446, #457, #462, #472, #479, #486. Next, in this order: the nvn_mtl.m split (#479, re-cut on 2006d5e, conflicting again with today's main), the coupling measure, then shlib as the first module, checked with regress.py replay. Also open: several frames per capture (step 2 of #402), the Breath of the Wild replay noise from #473, and two more timed rebuilds, one with nothing changed and one with a real chain change, to complete the picture of where a rebuild spends its time. Graphical front ends have their own issue: #388.

Activity

  1. v0lumehi commented on Oct 26, 2026

    @v0lumehi
    ContributorAuthor

    @seedds, a few quick yes/no questions so I don't step on your toes:

    1. Are you okay with me working through this list as small PRs, in the style you already merge?
    2. Would you turn on Discussions? Support questions, game requests and design questions like the platform scope (Add iOS 16.1/17 compatibility and tvOS support for Pokémon Legends: Z-A #310, Intel Mac: build the shader translator for x86_64 and document the pipeline #189) would get a home of their own, the issue tracker would stay focused on bugs and work, and people could answer each other's questions without you having to.
    3. The Metal tests (49 of 63 gfx tests, plus the translator's GPU tests) can't run on GitHub's runners (Run-time shader capture without Metal 4 (virtual Macs / CI): Balatro can't pack on GitHub's macos-26 runner #112). Could you run them on one of your Macs, for example as a self-hosted runner that only runs main (never PR code) or as a scheduled job? Or how would you like to solve that?

    No pressure on any of these; the PRs work without them.

  2. seedds commented on Oct 26, 2026

    @seedds
    Owner

    Great idea. Please continue this work

  3. v0lumehi commented on Oct 26, 2026

    @v0lumehi
    ContributorAuthor

    On it, will keep you updated

  4. v0lumehi commented on Oct 27, 2026

    @v0lumehi
    ContributorAuthor

    Progress update. Thanks @seedds for taking in so much in one day, and for the fixes on top. The review notes on #325, #329 and #333 caught real problems.

    Landed (boxes ticked above):

    Rebased on main 75065df, conflict-free again: #306, #307, #303, #327, #339, #340, #352.

    New: #391, the first step of "Decouple the chain". dmsh_abi.h is the one definition of the shader library ABI: the driver segment layout and the 19 binding kinds, which the host and dmsh_build.py each kept their own copy of. A test guards it with a ratchet. The host objects are identical, and all 19,592 Breath of the Wild shaders are unchanged (regress.py chain: PASS). It bumps the chain once. The next step (loading libraries by ABI compatibility) is what stops later tool changes from forcing a rebuild of every game. It ends with two questions for you about things that look off between the two sides.

    Split off: graphical front ends have their own issue now, #388. Steps already open there: #306 (NX2Apple first in the issue chooser), #387 (README, docs/frontends.md), and the scope question on #374.

    Next here: the docs de-conflict (the game count out of AGENTS.md, symptoms.md regenerated in CI), then ABI-compatible library loading on top of #391.

  5. 19 remaining items

  6. v0lumehi commented on Oct 28, 2026

    @v0lumehi
    ContributorAuthor

    @seedds the roadmap PRs are all rebased on today's main, green and mergeable. I'm pausing here until you've had a chance to go through them; no new roadmap PRs from me in the meantime (the coupling measure and the first module, shlib, wait for #479).

    Ready for review, in a suggested order (smallest and most independent first):

    1. Process: A pre-commit hook runs CI's fast checks before each commit #407 (pre-commit hook with CI's fast checks), Area labels on pull requests from the paths they change #408 (area labels from changed paths), Agents fill in the PR template and the issue forms, which gh and the API skip #457 (agents fill in the PR template and issue forms).
    2. Docs de-conflict, as a stack: Merge docs/reference and the symptom index with git's union driver #400 (union merge for docs/reference and the symptom index) → The symptom index is regenerated on main by CI, so a reference entry no longer needs it in the PR #472 (the index regenerated on main by CI).
    3. Platform scope, as a stack: README states what nxnative runs on, in one place, and the bug form asks for it #446 (Requirements in the README and the bug form) → Bug reports from a device README's Requirements excludes get a label and one comment pointing there #462 (one comment and a label for devices outside it).
    4. Checks as ratchets: Architecture checks: no title IDs in the host, build IDs only in its per-build tables #422 (no title IDs in the host core), CI: line coverage of the host, CPU texture and tool tests, with diff coverage per PR #439 (coverage with diff coverage per PR), Lint C and Objective-C in CI: a clang-tidy ratchet and a clang-format report #425 (clang-tidy; its baseline is 244 findings on today's main).
    5. Bug reports: Record player input as DM_KEYS steps (DM_REC), for bug reports and smoke routes #448 (DM_REC records input as DM_KEYS steps).
    6. Split nvn_mtl.m into topic parts with a script that re-cuts it on any main (mechanical, same machine code) #479, the nvn_mtl.m split, best at a quiet moment: it conflicts with any open PR that edits nvn_mtl.m. If you'd rather cut it yourself, take only its first commit and run python3 dm/tools/split_nvn_mtl.py on your main.

    Also rebased and green, outside the roadmap: #339 (Unreal Engine 3) with #340 on top, #303 (DM_NVN_POOLPEAK) and #76 (Breath of the Wild above 60 fps and 1080p).

    The checklist above is current. Thanks!

  7. relay4 commented on Oct 30, 2026

    @relay4
    Contributor

    Following up on the sanitizer work in #412, I’d suggest extending the checks in two ways. I tested these additions locally.

    For CI, using Apple Clang from the existing Xcode toolchain:

    • Add -fsanitize=implicit-conversion,local-bounds to the existing ASan/UBSan builds. These catch lossy integer conversions, sign changes and local-array bounds violations that -fsanitize=undefined does not include.
    • Add a separate -fsanitize=undefined,integer,implicit-conversion,local-bounds target for buffer/count arithmetic: heap sizes, audio renderer command/work buffers, HID reports, keyboard/font buffers and CPU texture layout/deswizzle.

    The broader integer checks need selective coverage: unsigned wrapping is intentional in hashes, ring counters and codec bitstreams. I’d document narrow exceptions for deliberate operations.

    These targets passed locally with Apple Clang, including the host ASan/UBSan and TSan suites.

    For occasional local audits, Homebrew Clang provides additional options:

    • LeakSanitizer (-fsanitize=leak) checks allocation ownership and cleanup.
    • RealtimeSanitizer (-fsanitize=realtime, with audio callbacks marked [[clang::nonblocking]]) detects allocation, locking and other unsafe operations on the real-time callback path.
    • TypeSanitizer (-fsanitize=type) checks strict-aliasing violations.

    All three ran on my arm64 Mac with Homebrew Clang 23.1.4. The audit found a concrete RTSan issue (however I'm unsure how severe that is): the audio callback’s periodic logging takes stdio/internal locks and can allocate. It also found RomFS image leaks in the filesystem test’s simulated process restarts.

    The Homebrew checks need some caution: TySan produced false positives for valid accesses to array members of static structs, and standalone LSan hit an internal allocator assertion during CoreAudio startup, reproducible in a minimal program without host code. I’d therefore keep these as local audit tools initially, with findings reduced and verified before changing production code.

    GCC's static -fanalyzer (Homebrew GCC 16.2.0)

    1. New, but minor: an unused game-environment string leaks once per process

    dm/host/hle_os.c:126–129, game_env_init.

    GCC reports -Wanalyzer-malloc-leak. The initializer always calls strdup(e).
    If the input is empty or contains no accepted assignments, no pointer into that
    buffer is stored in g_env_kv, and the duplicate is never freed. Nonempty
    accepted assignments intentionally retain their backing storage for the
    process lifetime. Initialization is protected by pthread_once, so this is
    not a leak that grows per frame or call.

    Fix: avoid duplicating input with no assignments, or free the duplicate when
    no entries were accepted. This is a low-priority cleanup, not a demonstrated
    gameplay failure.

    1. Confirmed defensive issue

    dm/host/hle_fs.c:985, h_fs_OpenDirectory.

    After opendir succeeds, the result of calloc is immediately dereferenced
    without checking for failure. GCC reports
    -Wanalyzer-possible-null-dereference. A real allocation failure therefore
    crashes the host instead of being handled as an allocation/resource error.

    Fix: check the allocation, close the already opened DIR on failure and return
    an appropriate failure result. Similar unchecked allocations exist in thread
    creation/startup and shader diagnostic code; these form a broader allocation-
    failure handling category.

    GCC's static analyzer could be added in addition to other static analysis methods, since it's complementary.

  8. relay4 commented on Oct 31, 2026

    @relay4
    Contributor

    Consider cppcheck and flawfinder alongside clang-tidy

    As an addition to the roadmap's existing static-analysis work, I'd suggest evaluating cppcheck and flawfinder alongside clang-tidy and Clang's static analyzer.

    I tried cppcheck 2.22.0 on the first-party C code and flawfinder 2.0.20 on the C/Objective-C sources and headers. The audit led to three confirmed issues:

    1. Audio command logging can read beyond a stack buffer
      Cppcheck flagged an uninitialized histogram string in hle_audren.c:capture(). A malformed command list can trigger an out-of-bounds read during logging, reproduced under ASan. Histogram formatting can also exceed its buffer after truncation.
      Fix: validate the complete list before consuming histogram/sink discovery, initialize the string and bound formatting.

    2. Save-sync metadata writes can leak a file descriptor
      Reviewing a cppcheck report exposed an error path in save_sync.c:write_base(): if fprintf() fails, short-circuit evaluation skips fclose(). Cleanup can also overwrite the original error.
      Fix: always close the stream, preserve the first error and replace the previous metadata only after writing and closing succeed.

    3. Watchdog sampling fails for paths containing apostrophes
      Flawfinder flagged system() in crash.c. DM_WATCH_SAMPLE is interpolated into a single-quoted shell command, so a path such as /tmp/John's Mac/hang.txt breaks parsing.
      Fix: launch /usr/bin/sample with posix_spawn() and a separate argument vector, retaining output redirection and error handling.

    Both tools produced substantial noise, so I'd start with non-blocking local or scheduled audits. After configuring Apple API models and triaging false positives, selected cppcheck checks could become a ratchet like clang-tidy's. Flawfinder is better treated as a source of manual-review hints. The findings themselves can be fixed separately, each with a regression test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions