You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Symptom: package identification stops with ValueError: subsection not found in pfs0_names, or LibHac reports ResultFsPathNotFound opening the last file of an NSP.
Cause: older NSZ writers omitted the final NUL in the PFS0 string table. The final name is bounded by the table length, but the Python scanner required a terminator and LibHac reads names as NUL-terminated strings.
Fix: the scanner accepts the table boundary as the end of the final name. nx_extract supplies LibHac with a virtual header containing the missing NUL followed by the original payload storage. File offsets remain relative to that payload, and the source package and encrypted NCA bytes are unchanged. Truncated tables and out-of-table name offsets still fail. Synthetic package tests cover both readers and payload preservation.
Found on: Moving Out 2 (010065D018172000), base 1.1.254 NSP and update 1.3.315 NSZ. Package reading was checked, not gameplay.
A NUP with complete standalone updates stops at "no base game"
Symptom:no base game, only update packages: pack the base game and its update into one NUP, even when the update carries the complete game's files (issue #1195).
Cause:pack_nup.scan required a base record. NUP v1 permits a container without one when every update has the standalone_update flag (package flags bit 0, SPEC §3.1.2). The reference packer writes one for a dump with no base/ folder and, since its commit 92089de3, for an empty base/ too; before that an empty base/ gave an empty base record, which got through the check. The issue was filed before that commit, so its NUP came from a dump with no base/ folder or from another packer; the file itself was not seen.
Fix: no base record and a base record with no files are one case: scan gives None for the base, and only the update versions present are offered. Both packing entry points select the newest update by default; requesting version 0 fails. The selected update must have exefs/main, exefs/main.npdm and its own RomFS; it cannot borrow another update's RomFS. Control data the update lacks comes from the record that holds application metadata: the empty base record, else the first package (SPEC §5.1). A base with files still supplies the RomFS for an update without one. Program-ID, DLC, content-size and SHA-256 checks apply as before. A version that cannot be packed stops at a [RED] line: the disk check reads its RomFS folder before extraction, so cli() catches pack_nup.Error.
Not known from the file: whether the update is the whole game. The flag does not promise it (the spec says such an update needs a matching base elsewhere), and an update holding only changed files would pack and then fail at boot, after a full shader build. The scripts print that the update is taken as the whole game. Measured: both NUPs at hand have an update RomFS that holds every path of its base (Age of Imprisonment: 2,065 base files, 2,153 in the update, none missing; Yoshi and the Mysterious Book: 147 and 147, none missing).
Found on:Mario Kart World. Reproduced with its base dump repackaged by the reference tool as a standalone-update fixture; this checks packaging, not the reported 1.8.0's contents or gameplay. Synthetic tests cover both entry points, both forms of no base, multiple versions, control data, DLC and incomplete updates.
A NUP or a dump of a game with no profile stops before the survey
Symptom:[RED] a NUP packs only a game with a profile, and <title ID> has none: the survey that makes a provisional profile reads NSPs, or the same line for a dump folder (issue #1265). A folder holding exefs/ and romfs/ directly, or only updates/<version>/, was not taken for a dump at all: the script asks for prod.keys, then stops with no NSP with a readable title ID.
Cause: the survey and the build ID lookup (same_build) ran nx_extract --manifest, which reads NSPs. pack_dump.is_dump asked for base/exefs/main.npdm.
Fix: each source leaves the same files for the survey: survey_nup reads the newest version from the container (its ExeFS, a RomFS manifest in nx_extract's format: path, size, first 32 bytes, Nup.head inflating only those; the NROs whole and the control data), and survey_dump from the folder, whose name names the game, as a dump has no control data. same_build reads main from the container or the folder. A dump may have no base/ when an update's folder holds code and a RomFS, and exefs/ and romfs/ directly in the folder, as nup extract writes them, count as base/: version 0, so a profile that registers another version packs them from a derived profile.
Found on: Star Fox (0400E88025B1A000, update v131072; no profile in the repository then), The Legend of Zelda: Ocarina of Time (0400486024DEC000, issue #1265) and Fire Emblem Fortune's Weave, whose dump holds updates/131072/ alone.
A damaged file in a NUP or a dump has no hash to fail
Symptom: the game reloads one asset for ever: a background loader takes about a core and the process several (Ocarina of Time: 412% CPU on the Mac, OptimizeChangeBanc and ParallelDecompThread busy all session, one model and its texture opened about 22 times a second). Or, from the NSP script: vN: K file(s) whose bytes do not start as their names promise (a .zs file that is not a zstd stream), so the game files are damaged, then the files.
Cause: the game files themselves. An NSP's files are checked against the NCA's hashes at extraction, so a damaged one stops the build (--allow-damaged). A NUP or a dump folder carries no such hashes: whatever bytes it holds are packed. On one NUP of Ocarina of Time three .zs textures began with 12,288 zero bytes, so no zstd frame: the game could not decompress them, and its loader asked for them again on every pass (PR #1530). The container's own SHA-256 of each file matched, so the bytes were like this when the NUP was packed. Another NUP of the same version holds all three as zstd streams, and none of its 102,185 .zs files fails the check.
Fix: the NSP script reads the first bytes of every file whose name promises a format (romfs_image.FORMAT_HEADS: .zs starts with a zstd frame or a skippable frame) once per extracted image (check_formats, after extraction; the result is kept in RomFS.img.formats beside the image until the image changes: Ocarina of Time 16 s from a USB SSD) and stops naming the damaged files: pack again from a good copy of the game. --allow-damaged lists them and packs them as they are. The host serves a game's files as they are: substituting other files for damaged ones would hide the damage and change what the game draws.
A provisional profile of a Switch 2 game shows no SDK and a Switch 1 memory pool
Symptom: the survey of a Switch 2 game says SDK: ? and no SDK version string found, and its provisional profile has app_pool 0xCD500000.
Cause: the SDK version pattern did not take the variant the Switch 2's SDKs name (NintendoSdk_nnSdk-Generic_21_4_4), and guess had one pool, a 4 GB Switch's.
Fix: the pattern takes the variant, and an sdk built for the Switch 2 (spec: Ounce, as dm_sdk_ounce reads it) gets the 9 GB pool every Switch 2 profile has. Open: only the pool changes; a game with system resource keeps the 5 GB block (Star Fox's profile has 21 GB with a 16 GB alias region), so a provisional profile may still not fit and needs the bring-up.
Found on: Star Fox and Kirby Air Riders (040077201BE50000; no profile), read from their NUP and extraction; first reported with Fire Emblem Fortune's Weave.
Build tools
Homebrew .NET is on PATH but apphosts cannot find the runtime
Symptom: the first extraction stops with nx_extract exit code 131 and .NET location: Not found, even though dotnet is on PATH. The shader translator can fail the same way.
Cause: standalone .NET apphosts search runtime roots and registered installations, not PATH (.NET runtime discovery). Homebrew's formula installs the runtime under libexec and its bin/dotnet can be a wrapper that sets DOTNET_ROOT only for its own child (Homebrew caveat). Resolving that wrapper's symlink gives bin, not the runtime root.
Fix:scripts/build_ipa_from_nsp.pyconfigure_dotnet derives the root from the Microsoft.NETCore.App entry in dotnet --list-runtimes, using the build's environment, before any extraction or survey. It exports DOTNET_ROOT both to that environment and to os.environ, for tools launched with an explicit environment and those that inherit it. Caller-supplied roots, including architecture-specific overrides, retain precedence. A failed or malformed probe leaves the environment unchanged rather than guessing the wrapper's directory.
Found on:Balatro, first extraction with Homebrew .NET (PR #284). Applies to every game's build on that installation setup.
The shader step stops with an empty xlate_source
Symptom: the extraction and captures succeed, then the shader step stops with translation chain tools do not match the lock file: xlate_source: lock … / actual and nothing after actual, which reads like a stale translator.
Cause: the translator (dmsh_xlate_bin/dmsh_xlate) is a universal framework-dependent apphost. When it finds no .NET 10 runtime for the slice it starts as, it exits 131 with an empty stdout. A Python under Rosetta (an Intel Homebrew's) starts it as x86_64, and it finds only the arm64 .NET; extraction is not affected, because nx_extract is built for the SDK's own architecture only. An Intel .NET on Apple silicon, or a DOTNET_ROOT that names one, fails the same way. tools_now (then in dmsh_chain.py) kept only the stdout of dmsh_xlate --source-sha, without its exit code or stderr, so the apphost's own message was lost.
Fix: the NSP script checks the environment before extracting anything: check_native_python stops under Rosetta (sysctl.proc_translated), and check_prereqs starts the translator once with --source-sha and stops with its first message line (check_translator). It also checks glslang and, for an iPhone build, the iOS SDK (check_ios_sdk, before the Metal check, whose message would blame the Metal toolchain). own_compilers.py no longer asks a Mac-only shader build (DMSH_TARGETS=mac) for the iOS Metal compiler. tool_lock.py records --source-sha's failure instead of its empty stdout (xlate_embedded: does not run (<first message line>)), so dmsh_build.py run directly and --update-lock stop with the same message as the NSP script, without the lock and rebuild lines. The tool checks used to be in dmsh_chain.py, which is part of the chain version and of every shader's cache key, so that fix rebuilt every library; they now live in tool_lock.py, outside both, and a change to them rebuilds nothing.
An Intel Mac cannot load the bundled translator's dll
Symptom: on an Intel Mac, the NSP script stops at its translator check with [RED] the shader translator does not run (Unhandled exception. System.IO.FileLoadException: Could not load file or assembly '…/dmsh_xlate.dll'. The assembly architecture is not compatible with the current process architecture.), although the .NET 10 runtime for x86_64 is installed.
Cause: the launcher dmsh_xlate is universal, but dmsh_xlate.dll beside it was arm64-only (PE32+, machine 0xAA64). A .NET build for a runtime ID (-r osx-arm64) sets an executable's PlatformTarget to that architecture unless the project names one, and the repository had no single command for rebuilding the translator, so the bundled dll flipped between AnyCPU and arm64 from one rebuild to the next. The Ryujinx libraries stayed AnyCPU, because a library does not take its architecture from the runtime ID.
Fix:dmsh_xlate.csproj pins <PlatformTarget>AnyCPU</PlatformTarget>, so a build with any -r, or none, gives an AnyCPU dll (PE32, machine 0x14C, IL only). dm/tools/tests/test_xlate_anycpu.py reads the PE/CLI headers of every dll in dmsh_xlate_bin and the launcher's Mach-O header, and fails when a dll is tied to an architecture or the launcher lacks x86_64 or arm64; it needs no .NET. The rebuilt translator writes the same SPIR-V and reflection files as the arm64-only one (9,556 shaders compared byte for byte).
Found on: an Intel MacBook Pro building any game (GitHub issue #803, which bisected the flips). Applies to every game.
The Vulkan shader step fails with Argument list too long: 'xcrun'
Symptom:vk_shaders.py stops with OSError: [Errno 7] Argument list too long: 'xcrun' after the Metal compiles, in the metallib link.
Cause:metallib() passed every .air file as an absolute path in one command. The NSP script runs it with TMPDIR set to the work folder, so each path holds that folder's name; a game with thousands of generated modules (Super Mario Galaxy: 3,083 captured shaders, more with the dual-source variants) under a long NSP folder name exceeds macOS's 1 MB ARG_MAX.
Fix:metallib() runs from the files' common folder with relative paths and an absolute output path, so the command length no longer depends on the work folder's name.
Found on:Super Mario Galaxy, with its dump in a folder named after both games of the cartridge. Applies to every Vulkan game's shader step.
The split shader tables differ from a game profile's
Symptom: the shader split stops with the split shader tables differ from game profile <id>'s, on the game version the profile registers.
Cause: the check compares the split's data rows with the profile's byte for byte, so it stops for either of two reasons: other game files, or a change to shader_split.py that reads the same files differently. A profile's tables can only be split again where its game files are, so continuous integration cannot catch the second reason.
Fix: the error shows each differing table's row counts and its first row on each side. For a splitter change, check that the new rows are right, then copy the tables from shader/pkg.partial/tables in the build folder into the profile. A change to the splitter's output also needs the other profiles it reaches split again.
Found on:Splatoon Raiders (issue #1263): NVN2 task and mesh shaders moved one row from vertex to mesh. Donkey Kong Bananza, Pokemon Pokopia and Hyrule Warriors: Age of Imprisonment were checked and hold no such shader; Fire Emblem Fortune's Weave was not checked, as no dump was at hand.
IPA archives
A streamed file changes size and the IPA fails its CRC check
Symptom: packing succeeds, but unzip -t or Python's ZipFile.testzip() reports a bad CRC for a large deflated file.
Cause:pack_ipa.py wrote the uncompressed size collected before reading. Its streaming check compared the output size only for stored files, so a compressible file that grew or shrank still published an archive with the old size.
Fix: count uncompressed bytes as they are read and compare them with the listed size before completing either streamed method. A mismatch stops the pack and removes the partial archive, preserving the previous IPA and its reuse index. Regression tests cover growth and shrinkage with both deflate and store.
Found on: synthetic IPA inputs while reviewing PR #279; no game files required.
RomFS images
RomFS folder transfers fail or leave stale game files
Symptom:devicectl copying a large RomFS folder fails with POSIX error 24 (EMFILE) or spends most of its time on small files; deleting an old folder during iOS startup hits the launch watchdog. A push after changing extracted files can also keep sending old assets if an image is reused solely because it exists.
Cause: a folder transfer performs hundreds of thousands of file operations. The image cache must track its source tree, and magic plus total length cannot establish that an image's index is valid. Accepting that weak check before deleting the working folder loses the fallback when mounting later rejects the index.
Fix:dm/tools/romfs_image.py packs one DMROMFS1 file: a 32-byte header, file bytes, then the index and name pool. Each directory has a sorted, contiguous run of child entries; the host binary-searches each path component and reads file ranges with pread through one descriptor per mount. Overlay folders keep precedence and directory listings merge both sources. The shared dm/host/romfs_image.h loader checks the full index before either mounting or iOS folder cleanup: length, entry and name bounds, file ranges, a rooted breadth-first directory tree, and sorted, unique child names. It supports an empty root too. Invalid images fall back to the folders on iOS; an explicit invalid DM_ROMFS image is refused with a [RED] line. The index check detects interrupted sequential transfers and structural damage, not same-length corruption inside file contents. The packer publishes by rename only after the source stays stable. Its reuse stamp tracks source names and file metadata including ctime, the packer and image identity; the push helper always checks it, and --repack forces a rebuild. Usage: Quick reinstalls.
Found on:Tears of the Kingdom, whose 304,000-file transfer and startup deletion failures are reported in PR #369. ANIMAL WELL provides the empty-image boot case. Synthetic packer/host tests cover actual file reads, case aliases, overlay merging, corrupt indexes and changed sources; the macOS Foundation test executes iOS's locator to check that rejected images preserve its folder.
A reboot packs a RomFS image again
Symptom: the first build after restarting the Mac spends minutes in unpack ([1/5] unpack v131072: 5%, 1.2/~24.4 GB) although the game, the packer and the image are as they were: 8 minutes for a 24.4 GB image from a NUP on a network share.
Cause: the stamp beside an image (RomFS.img.inputs.json) held st_dev in the image's identity, and macOS numbers a volume when it mounts it. Across one reboot the Data volume went from 16777234 to 16777232 while inode, size and both times stayed, so reusable() refused every game's image at once.
Fix:file_identity leaves st_dev out, as the IPA and app packers' stamps already did. Inode, size, modification and change times still catch an image replaced or written over. Every image is packed once more after this change, as after any change to the packer, since its bytes are in each stamp's digest.
Found on: Ocarina of Time (Switch 2, 0400486024DEC000, packed from a NUP; PR #1299); the stamps of Star Fox and Mario Kart World show the same two numbers. dm/tools/tests/test_romfs_image.py remounts an image under another device number.
A stale DM_ROMFS path crashes the game soon after nnMain
Symptom: a direct dmhost run logs fs mount rom: → <path> (read-only) for a path that does not exist, then crashes in the game's code a fraction of a second after nnMain (Echoes of Wisdom: CRASH sig=11 addr=0x88 0.3 s in). The host now stops at the mount instead with FATAL the game files <path> (rom:) are missing: neither a folder nor a RomFS image.
Cause: an old run command still named the extraction's RomFS folder after a rebuild replaced it with RomFS.img (RomFS folder transfers). The read-only mount accepted any path, so every read failed and the game used the missing data. The iOS app and the Mac app launcher check their game files before they start the host; direct runs had no such check.
Fix:mount_ex (dm/host/hle_fs.c) stops on a read-only, non-save mount whose path is neither a folder nor a regular file, with the same stat that tells an image from a folder. Add-on content and system data check their paths before mounting, so only the rom mount (DM_ROMFS, or $DM_ROOT/romfs when it is unset) reaches it. dm/host/tests/t_fs.c checks a missing path and a fifo in a forked child.
Separate RomFS deployment loses its image or folder fallback
Symptom: a successful --romfs separate --no-keep-extracted build prints a push command pointing at deleted game files. On iOS, a complete image for another build can also prevent startup despite a valid Documents/romfs folder.
Cause: separate mode omitted the extraction-retention check, so normal post-build cleanup deleted the image needed for transfer. The startup identity rejection returned before the existing folder lookup.
Fix: reject separate mode without --keep-extracted before extraction or building. Reject a mismatching image while continuing to the validated folder fallback; move a folder aside for deletion only after accepting the image. Matching images and the default bundled route retain their behavior. The identity covers image size, header and index, not same-length file-content changes.
Found on: synthetic CLI and Foundation startup tests while integrating PR #493, after PR #478; no game or device is needed to reproduce either failure.
A hand-copied RomFS.img is not found on iOS
Symptom: a --romfs separate app whose image was copied into its Documents folder by hand (Finder, the Files app, a LiveContainer guest) logs boot: game files not found (Documents/romfs.img, Documents/romfs or <App>/romfs), and a build with romfs.id shows "There is no game file image on this device.", though the complete image is there. Renaming it to romfs.img makes the game start.
Cause: the NSP script writes the image as extract/<version>/RomFS.img, and push_romfs.sh copies it to Documents/romfs.img. A hand copy keeps the source's name. The Mac's file system ignores letter case, so the mismatch never shows there, but iOS's does not, and main.m looked up only the exact romfs.img.
Fix:main.m (romfs_document_images) lists Documents for romfs.img in any letter case, romfs.img itself first, and takes the first that is complete and passes the romfs.id check; a rejected or incomplete one falls through to the next spelling, then to the folders. The identity report, the backup exclusion and the log lines use the names found. Folder names (Documents/romfs) are still matched exactly.
Found on: South Park: The Fractured But Whole 1.05 (01008F2005154000, no profile here) on an iPhone 15 Pro Max, issue #582. The Foundation startup test runs the locator on a case-sensitive APFS disk image.
Building creates a full RomFS tree before packing its image
Symptom: reducing the iPhone transfer to one image still creates thousands of resource files on the Mac, and build scanners require the expanded folder.
Cause: folder extraction followed by a separate image pack duplicates the work; ordinary filesystem APIs cannot address files inside an image.
Fix: the NSP pipeline passes --romfs-image to nx_extract. LibHac layers the update over the base and checks its integrity while the writer streams file bytes into a temporary DMROMFS1 image, appends the sorted directory index and publishes by rename. ExeFS remains expanded; RomFS and each DLC remain one physical file. dm/tools/romfs.py gives the scanners read-only virtual paths, bounded seeks and archive mappings (Python 3.12+), without changing the process's standard filesystem APIs. The IPA packer maps the physical image to Payload/DmIOS.app/romfs.img; iOS and the Mac launcher validate and mount it in place. Neither installation nor startup expands the resource files. Documents overrides and legacy folder inputs remain supported. A source image is uncompressed; the IPA can deflate it, so file-count reduction does not imply a smaller installed byte count. Full images for separate game versions cannot share unchanged individual files through APFS clones as legacy folder extraction did.
Found on:ANIMAL WELL, whose direct image is an empty root (56 bytes); the local Baba Is You provisional build exercises 6,374 files in a 103,473,292-byte image, 131 shaders and an unsigned IPA containing only the single image entry. Raging Loop supplies Unity resources: all 1,473 source manifest entries match the image, and every registered shader remains in the scan. Synthetic integration tests call the actual LibHac writer, read unaligned archive ranges, and exercise IPA mapping, Xcode staging and bundled-image startup.
A build folder holds hundreds of thousands of small files
Symptom: a game's <id>_build folder is slow to count, copy or delete: Metroid Dread's held 160,331 files in 48,782 folders, 84,541 of them in the compile cache.
Cause: the compile cache (shader/cache/cas) keeps a few files per shader, and Xcode's derived data (ipa/build/ios/pkg/dd) over a thousand. Both are needed to make the next build fast, and only while a build runs. The cache has since become one file read in place, and the tar remains for the derived data.
Fix:dm/tools/tree_tar.py keeps each as one uncompressed tar beside where the folder was. build_shaders unpacks the cache only when dmsh_build.py will run, and packs it after pruning; the NSP script does the same for the derived data around build_ipa.sh. Keeping real folders during the build leaves dmsh_build.py, its chain version and Xcode unchanged. A build folder from before this converts by itself. Restored files keep their times, and Xcode's incremental build holds across the round trip (1 file compiled, against 262 for a full build). A build that stops leaves the folder, and a damaged tar costs only the cache it held. regress.py's chain builds share the cache under the same lock. The cost is time on each build that runs the step, and a cache that no longer shares its libraries' blocks through APFS clones. The shader libraries (b-<chain>) stay folders: the host reads them one by one.
Found on:ANIMAL WELL (the round trip) and Metroid Dread (84,541 cache files: 51 s to pack, 22 s to restore on a T7).
The compile cache was unpacked for each build
Symptom: a build that runs the shader step writes the compile cache's files back and removes them again, on Skyward Sword HD 24,131 files and 10,150 folders each time, Tears of the Kingdom over 900,000 entries. Removing old entries took up to half an hour on an external drive.
Cause: the cache was a folder of a few files per shader, read and written by path, so a tar could only hold it between builds.
Fix:dm/tools/shader_cache.py keeps it as one SQLite file, cas.db, whose entries go in and out by name, with nothing unpacked. Bytes are stored by their hash, since a library under its MSL's digest and its shader's digest was one APFS clone. dmsh_build.py and dmsh_mtlcache.py use it directly; a shader's entry is written in one transaction, so a stop leaves all of it or none. Several writers wait for each other: dmsh_build.py's threads share a connection, and regress.py's chain builds and the NSP script share the file. Pruning is a query that gives its pages back. A cas/ or cas.tar from before this moves in by itself at the next shader step, without unpacking a tar: the Metal compile results come along. The per-shader entries do not, as the change of chain they came with means they cannot hit, and neither do the fetch variants, whose older entries hold their source's file name, so that a fresh compile's bytes differ (regress.py reports these as changed variants with no change of MSL). A damaged file costs the work it held and is made anew, and a full drive is one [RED] line. A revision from before this keeps its own folder, regress/cas.
Measured on Skyward Sword HD: a 281 MB cas.tar became a 226 MB cas.db, restoring all 2,470 shaders from the cache took 6 s (9 to 13 s with the tar), and the build folder's shader/ holds 8 files. It also ran on an exFAT volume, with three processes writing at once.
Symptom: a game's shader libraries are slow to copy or delete: Skyward Sword HD's b-<chain>/ held 12,064 files for 2,470 shaders, a third of its build folder's files, and each regress.py chain build held another set. A tar would not do: the host reads these files at every run.
Cause:dmsh_build.py writes each shader's library, reflection, source and side libraries as separate files, and the host opened them by path.
Fix:dm/tools/shader_lib.py packs the folder into one file beside its name, b-<chain>.img, in the RomFS image's format, flat, each file's data on 16 bytes. Nothing unpacks it: the host (dm/gfx/nvn/msl_image.h) keeps one descriptor and maps only a shader's pages, and the tools read it through romfs.py. dmsh_build.py still writes a folder, so the chain version does not change. The NSP script packs at the end of the shader step, also on a reused step, so a folder from before this converts without a rebuild, and removes the old image only just before a build that replaces it. regress.py packs each chain build once it has read it; a revision from before this keeps a folder, and has a chain key of its own. The Mac app and the IPA carry the one file, without the Mac's sources, as msl/b-<chain>.img. An iPhone build neither compiles nor keeps the pulling vertex libraries when nothing in it reads them (ios_wants_pulling: no pipeline archive, the profile's pso_mtl4 off, no IOS_PULL=1, no Xcode project; DMSH_IOS_PULL=0 for dmsh_build.py), so pack_ipa.py reuses the image between packs. A build that wants them again compiles them, and pack_ipa.py refuses an image without them for an app that reads them. Each file comes from a folder beside the image first, so a partial dmsh_build.py run (given keys) goes over a packed library, and the next pack takes it in; a host running from a folder goes on when it is packed. Real files remain where a tool needs them: a compile's temporary files, and the libraries metal-tt reads, for the length of its run. shader_lib.py expand gives an old kept host its folder back.
Measured on Skyward Sword HD, on an external drive, warm cache: 10,017 files to one: copy 3.72 s to 0.18 s, delete 0.40 s to 0.02 s, 150 MB to 131 MB on disk. Reading all 2,470 shaders' reflection and libraries 100 ms to 28 ms. The map must be MAP_SHARED: on a file this size a private mapping costs about 80 µs, against 0.9 µs. A cold read has not been measured, as the cache cannot be emptied without an administrator.
A shader build leaves a file per shader output while it runs or after a stop
Symptom: a game's build folder holds tens of thousands of small files while its shaders build, and keeps them when the build stops: Octopath Traveler 0's _build, stopped in its compile, held 68,230 files (38,376 in tmp/dmsh_*, 29,757 in shader/cache/b-<chain>/), against 105 once a build finished. Every later build made another tmp/dmsh_* beside the old one.
Cause:dmsh_build.py wrote each shader's outputs into b-<chain>/ and packed them into the image only at the end (the entry above); its cache check restored every cache hit into that folder before anything was translated; the translator wrote every missed shader's working files (.bin, .spv, .res.json, .metal) into one temporary folder before any was finished; and that folder went only when the build ended normally.
Fix:dm/gfx/tools/dmsh_build.py. A finished shader's files go into the compile cache (cas.db, under stage/<output folder>/) and leave the disk (commit); reads of another shader's outputs (a vertex shader's paired pixel MSL for vs_pad, the mesh step) take them from there (output). At the end the image is written from the staged outputs (publish_library, through shader_lib.write), with a partial build's image under it less the keys it built, and the folder and the staged entries go; when the image cannot be written, the staged outputs go back into the folder. Cache hits are restored when their shader is finished, not at the look. The shaders are translated and finished in batches of 4 × J (the next batch is translated while this one is finished), each in a folder of its own, and a shader's working files go as soon as it is finished. Each build's working folder holds its process ID; a later build removes a dmsh_* folder whose process is gone (one without the ID after a day). The NSP script's progress comes from a small file the build rewrites (DMSH_PROGRESS), no longer from counting its files. The outputs are the same: chain cfcc3a82c78b against the new one, Minecraft Dungeons II's 61,655 library files and Octopath Traveler 0's 41,880 are byte for byte the same (reflection without its chain field), apart from _b_cas.txt, which lists the new chain's cache digests.
Measured on an M4 Pro, every shader translated again (the chain changed), the Metal libraries from the cache: Octopath Traveler 0, 10,324 shaders, at most 405 files under shader/ and tmp/ (12 outside a build), 2:48 for the step (2:55 with batches of 4 × 128); Minecraft Dungeons II, 15,826 shaders, at most 3,279 with batches of 256 removed whole. Stopped with SIGKILL near the end of a compile: 40 files in b-<chain>/ and 7 in tmp/dmsh_*; the next run cleared them and wrote the same library from the cache in 59 s. A partial build of 42 shaders for the Mac and the iPhone leaves one image each, also when run again over them.
Symptom: a game's <id>-work folder is too slow to count: Tears of the Kingdom's held 8,893,028 files and 509 GB, and its count ran past ten minutes.
Cause:regress.py kept one whole shader build per chain version in regress/chain (45 there; Breath of the Wild held 57), and agents' investigations leave folders nothing reads again (build alone over 351,000 files).
Fix:prune_chains in scripts/regress.py removes chain builds after each chain step, beyond the 3 used last, any used within 24 hours and those of the 3 newest stored baselines; a check that needs one builds it again. tree_tar.pack_quiet packs each other top-level folder untouched for 14 days, with 100 files or more, into <name>.tar at the end of an NSP script or regress.py run. These tars keep extended attributes, so a RomFS folder's case-collision names survive, and tar --xattrs -xf restores them (plain tar -xf does not). A Python virtual environment stays a folder, as scripts run it by its path and nothing writes to it, which made it look untouched. It frees no space beyond the chain builds: a folder of APFS clones can take more as a tar, so each tar must fit with 20 GB to spare. A player's NSP folder has no work folder.
Symptom: the NSP script stops at the shader split of a registered profile with [RED] 13744 file(s) could not be read, their shaders are left out (first: AI/FileEntry/FileEntry.Product.100.byml.zs: CalledProcessError: Command ['zstd', '-d', '-c', '-q', '…/extract/v0/RomFS.img/AI/…'] returned non-zero exit status 1.).
Cause:shader_split.py's unzs read a .zs file through romfs.open, which reads inside the image, but then handed its path to the zstd command, which can only open real files (Not a directory).
Fix:unzs feeds the command the bytes it read (input=). No other tool hands an external command a path inside the image. Test: test_romfs_read.pytest_zstd_file_inside_an_image.
Found on:Super Mario Bros. Wonder: every file of its RomFS is zstd-compressed, 13,744 left out in both versions; the first image-backed build after the folder extraction.
Add-on content
DLC in Documents is ignored or removed DLC remains installed
Symptom: a game's DLC works only when packed inside its IPA, although its base RomFS can be deployed separately. After pushing a smaller DLC set, removed indices can remain reported as owned and mounted.
Cause: the iOS launcher originally checked only <App>/aoc. Merely adding Documents/aoc discovery is insufficient: devicectl device copy to merges directories by default, retaining removed indices, and skipping a build with no DLC leaves the entire previous set behind. The host counts and mounts the numeric entries under DM_AOC_DIR, so those leftovers remain active.
Fix:aoc_locate in dm/ios/Sources/main.m preserves an explicit DM_AOC_DIR from the launch environment or env.txt; otherwise it selects the first directory at Documents/aoc, then <App>/aoc, and logs boot: DLC from <path>. Documents/aoc is excluded from backups. Given resources.env, scripts/push_romfs.sh transfers the complete DM_RESOURCE_AOC directory with --remove-existing-content true, replacing old <index>.img files and legacy <index>/ trees. An empty or omitted value sends an empty directory, clearing previous content and explicitly selecting no external DLC rather than falling back to bundled content. A nonempty path that is unavailable stops before any transfer. A direct RomFS-image push has no DLC metadata and leaves DLC unchanged. Transfers are not atomic: rerun a failed push before launching the game. Usage: Quick reinstalls.
Found on: synthetic startup and transfer fixtures while reviewing PR #486, part of issue #388. The Foundation fixture runs the actual iOS locator; the push-helper fixture models devicectl's documented merge and replacement behavior, including smaller and empty DLC sets and transfer failure. Applies to every game; no device gameplay validation was performed.
Launcher metadata
No Game language row in an app packed from a NUP or a dump
Symptom: the settings of a Mac app or IPA packed from a NUP or a dump folder have no Game language row, while DM_LANG in the environment changes the language. Reported on Donkey Kong Bananza (NX2Apple#619, #659; the reporters' packed files were not available).
Cause:settings.py shows the row when the game's control.nacp lists more than one language, and two cases gave it none. A NUP update may carry no control data: the NSP script's language check fell back to the base game's file, but the Mac settings got only the update's path, and build_ipa.sh's fallback ($W/data/v0) is a folder the NSP script does not make. And there may be no NACP at all: the Yoshi and the Mysterious Book and Age of Imprisonment NUPs carry none in any package, and an FTP dump such as Bananza's has no control data.
Fix:languages_control in build_ipa_from_nsp.py picks the file once (the packed version's, else the base game's), for the check, mac_app and build_ipa.sh (CONTROL). With no list, settings.py offers every language with a note, as game_language and DM_LANG already accept any. Age of Imprisonment shows a language it lacks as English (Russian, Portuguese) and has more text languages than its voice folders suggest (Dutch, Simplified Chinese), so a game's list cannot be read from its folders.
Found on: code reading and the NUP records of the two games above, with PR #1182; boots of Age of Imprisonment to its first settings screen. Not confirmed on Bananza by the reporters.
A Mac front end's default settings change Metal execution paths
Symptom: starting a Mac app through a front end with unchanged settings enables the encode thread, and can disable Metal 4 pipelines and the pipeline archive; starting the same app from Finder keeps the host's defaults.
Cause: Mac launcher.json reused iOS defaults from settings.tsv. rec_on in nvn_rec.h defaults off on Mac and on on iOS; pso4_on in nvn_mtl.m defaults on on Mac and follows the profile on iOS. The exported defaults instead enabled the encode thread everywhere and followed the profile for Metal 4 everywhere. A front end applying those values changed the Mac host's behavior without a player override.
Fix: a setting's default may be <ios>|<mac>, selected before resolving a profile macro or build variable. Metal 4 uses DM_GAME_PSO_MTL4|1, the encode thread 1|0. The generator validates the platform and the pair. Tests cover a profile with Metal 4 on and one with it off, and both platforms; the host and shader chain are unchanged.
Found on: synthetic metadata generation for Super Mario Bros. Wonder while reviewing PR #352; checked against the native defaults, without a gameplay run.
The Mac settings window stays on screen after Play
Symptom: in a Mac app, the settings window stays on screen after Play, not responding (a spinning cursor over it), until the game ends.
Cause: the window runs in the launcher's process (dm/mac/Launcher.swift). finish ordered the window out and stopped NSApp.run; then launcher.c started the host and blocked in waitpid, so the main thread never ran another run-loop pass. A window's order-out is committed only at the end of such a pass, and its default fade-out runs over several passes, so the window never left the screen. With the run loop kept running, the same order-out took effect at once.
Fix:finish sets the window's animationBehavior to .none before the order-out, and dm_mac_launcher calls CATransaction.flush() after run() returns, before it hands back to launcher.c. Neither alone was enough. Check: a copy of the window with a timed Play and a CGWindowListCopyWindowInfo probe while the thread then blocks; the window was still listed 2 and 4 s after Play before the fix, and gone 3 times in 3 after it. test_launcher_storage.py and test_nsp_mac_app.py pass.
A game folder on an exFAT drive reads one update as two, or fails to remove a folder
Symptom: with the NSP folder on an exFAT or FAT drive, the build leaves out an update whose name has no [v…] tag (the folder seems to hold two), or a stage stops with FileNotFoundError on a ._<name> path inside <id>_build/ and the next run fails on the half-deleted folder.
Cause: these file systems have no extended attributes, so macOS keeps a file's attributes (a download's quarantine flag) in an AppleDouble file ._<name> beside it, and deletes it with the file. scan_nsps took ._Update [<title ID>800].nsp, 4 KB of metadata, for an update: its name carries the title ID. Deleting x takes ._x with it, so shutil.rmtree's later unlink of the listed ._x found it gone; before Python 3.13 that error ended the removal.
Fix:scripts/build_ipa_from_nsp.pyscan_nsps takes a file only when its header is one nx_extract opens (is_package: PFS0 at 0, or an XCI's HEAD at 0x100), and says which files it skips. The name tag alone no longer counts, so the same check also rejects any other tagged file that is not a package. Names that start with ._ are skipped without a line. The script's rmtree passes ENOENT from shutil.rmtree (the entry is gone, which is the goal) and raises every other error. dm/tools/tests/test_nsp_external_drive.py and test_nsp_scan.py cover both without game files.
Found on: the NX2Apple fork's users, who keep games on external drives (PR #476, part of issue #388); reproduced on an exFAT disk image made with hdiutil create -fs ExFAT. Applies to every game.
The c1 guard stops on an AppleDouble file after the shader build
Symptom: with the build folder on an exFAT or FAT drive, the shader step compiles every library and then stops in c1_guard.py with UnicodeDecodeError: 'utf-8' codec can't decode byte … (dmsh_build.py: [RED] c1_guard.py failed), reading a ._<key>_b.metal.
Cause: the same AppleDouble files as above, beside the shader build's outputs. c1_guard.py took every name ending in .metal for a shader's MSL and read the binary ._ file as text. cb_extent.py (every _b.json) skipped it only because it ignores what it cannot parse, and dmsh_mtlcache.py (every _b.metal) took ._<key>_b for a shader, found no readable reflection for it and so filled the Metal cache from none of that folder's libraries.
Fix: the three take only the names the shader build writes, <16 hex key>_b….metal / .json as dmsh_build.py (OUT_KEY) and regress.py already did: c1_guard.pyoutputs (which cb_extent.py uses too) and dmsh_mtlcache.pyOUTPUT. None of them is part of the chain version. dm/tools/tests/test_nsp_chain.py runs both table tools on a folder with ._ files; the old c1_guard.py stops on it. On 13 Sentinels' 751 shaders both tables are unchanged.
Found on: a user's iOS shader build on an external drive (PR #555). Applies to every NVN game.
The Mac app step stops on AppleDouble files in its inputs, its copies and its signature
Symptom: with the build folder or the app on an exFAT or FAT drive, the Mac app step stops in pack_app.py. The first form it takes is no input file at <build>/shader/cache/<chain>/._<key>_b.metallib, or no rewriter fixups for <build>/<version>/._main.patched.bin when the modules come first. When the inputs get past it, the copies stop with chflags or Permission denied, and the signing step stops with codesign failed (1): code object is not signed at all and In subcomponent: …/Contents/MacOS/._nxnative, which reads like a broken launcher build.
Cause: the same AppleDouble files as above. The module, shader, pipeline and font globs of selected_inputs took ._<name> for a module or a library, and a ._x.patched.bin has no x.fixups.tsv beside it. shutil.copy2 of the host tried to set the copy's flags on a file system without extended attributes (chflags: Operation not permitted), and cp -R carried a folder's sidecars into the bundle, where DMModuleFiles listed them. codesign then read the ._nxnative beside the launcher as a nested code object that is not signed and refused the whole bundle. Signing also sets an attribute on what it signs, so the sidecar comes back after a successful signature.
Fix:dm/mac/pack_app.pydetritus() drops those names from selected_inputs, so neither the pack nor DMModuleFiles sees them. copy() passes -X where it copies and not where it clones (cp refuses the two together), and the host goes through shutil.copy instead of copy2, so a copy creates none of them. strip_detritus() runs before codesign reads the staged bundle, runs again after xattr -cr when codesign still refuses one, and runs once more after signing. On APFS nothing changes: no sidecar exists there, and the two passes are a stat walk. dm/tools/tests/test_nsp_mac_app.py puts a ._ file in every input folder and checks that it is not selected, does not reach the bundle, and is not in DMModuleFiles.
Found on: the NX2Apple fork's users, packing Mac apps on an external drive (LNPANDY/NX2Apple#486); reproduced on an exFAT disk image made with hdiutil create -fs ExFAT, where the same inputs pack, sign and pass codesign --verify --strict with the change and stop three ways without it. Applies to every game's Mac app.
A second Mac app build stops with [Errno 45] when it replaces the old app
Symptom: with the app on a Mac OS Extended, exFAT or FAT drive, the first build into a folder works and the next one that changes anything stops in pack_app.pyreplace_app with OSError: [Errno 45] Operation not supported: '<app>'. The old app stays in place.
Cause:replace_app exchanges the new and the old app with renamex_np(RENAME_SWAP), which only APFS offers: the others return ENOTSUP. A first build has no old app to exchange, so it takes a plain rename.
Fix: on ENOTSUP or EOPNOTSUPP, swap_by_moves makes the state the swap leaves with three renames (the old app to <app>.old, the new one to <app>, the old one to <app>.partial), so the key write and its rollback do not change. A rename that fails undoes the ones before it. These volumes cannot make it atomic: a process killed between the first two renames leaves no app at <app>, with the old one at <app>.old until the next replacement clears it. APFS still swaps. Tests: dm/tools/tests/test_nsp_mac_app.pytest_volume_without_atomic_swap_replaces_by_moves, test_fallback_restores_the_old_app_when_a_move_fails and test_fallback_clears_an_old_app_a_killed_build_left_aside.
Found on: a user's rebuild onto an external Mac OS Extended (Journaled) drive (LNPANDY/NX2Apple#595, PR #1190); reproduced on HFS+ and exFAT disk images made with hdiutil create, where the PackReuse tests stop with the real errno without the change and reach the fallback with it. Applies to every game's Mac app.
Symptom: on an exFAT or FAT drive, shader splitting stops with ValueError: not a version 2 shader code archive or [RED] /glslc_capture/._…: no GLSLC output.
Cause:shader_split.py selected captured .glslc programs and decoded .ushaderbytecode archives by extension alone. macOS AppleDouble sidecars retain that extension, but their ._<name> files contain metadata rather than shader bytes.
Fix: exclude ._ names when listing both external shader directories, including DM_GLSLC_CAPTURE and DM_UE_SHADERS overrides. Other dot-prefixed names remain valid inputs, and malformed real inputs still fail their normal validation.
Found on: an external-drive build reported in issue #229's follow-up. That commenter used a different, unnamed game; this fixes the sidecar failure, not the original Metroid Prime 4 port.
Cause:nsz 4.6.1 has no --keys argument. It loads keys.txt beside sys.argv[0], or falls back to ~/.switch/prod.keys, while importing its modules, before parsing arguments (4.6.1 key loader). Version 5.0 supports --keys (argument parser); staging keys.txt also supports 4.6.1.
Fix:scripts/build_ipa_from_nsp.pydecompress_nsz prefers the current Python's module: it links the file selected by -k as keys.txt in the conversion's temporary folder and runs nsz -D there, with sys.argv[0] set to nsz before import. Without the module, nsz_command probes nsz and nsz-cli-macos-arm64 on the build's PATH and passes --keys to the first whose successful help lists that option. An unsupported or failed candidate does not hide the next one; when none works, the error lists the reasons and installation options. Executable, input and output paths are absolute because conversion changes directories. The temporary folder and link are removed on success or failure; the user's default keys are untouched.
Found on:Super Mario Odyssey, converting its v0 NSZ. Applies to NSZ and XCZ inputs for every game. Synthetic tests during PR #327 also reproduce an older console script hiding the release binary, a failed help probe containing --keys, and a relative PATH executable failing after the directory change; no game files are needed for those cases.
ZBIC-compressed NSO segments
Symptom: the NSP script or any module tool stops on main: compressed with zstd (NSO flag bit 7), which this loader does not decode yet; with the decoder but without its package, ZBIC (zstd) segments need the zbic package (python3 -m pip install zbic).
Cause: from system 22.0.0, an NSO may store its segments as ZBIC, NSO flag bit 7 for the whole module (the per-segment compression and check-hash bits keep their meaning). ZBIC is Nintendo's variant of zstd, not a container around standard frames: its frames start with the magic ZBIC and its FSE tables are written with binary interpolative coding, so neither compression.zstd nor the zstd command can read them.
Fix:dm/tools/nso.py decodes such segments with the zbic package (Kinnay's, zstd with Atmosphère's ZBIC changes), imported only for a flag-bit-7 module; every segment is still held to the header's size and SHA-256. zstd has no fixed ratio bound, so a ZBIC segment is held only to its own size, where an LZ4 one is also held to 255 bytes per stored byte. Install it with the other tools (README, step 1). The NSP script checks for it before it extracts anything, as it does for its other modules, because it first reads a module at the rewrite, after the whole RomFS is unpacked.
Check: Well Dweller's main decodes to 85,370,056 bytes with segments 0x39680d0 / 0x9f7e74 / 0x9b1930, every SHA-256 matching; dm/tools/tests/test_nso.py checks that a flag-bit-7 module whose segments are not ZBIC is an Error naming ZBIC.
A version without one of the profile's modules stops the host at boot
Symptom:FATAL open <folder>/subsdk1.fixups.tsv right after the host starts, or build_ipa.sh stops with no rewritten module …/subsdk1, for a version that has no such module.
Cause: a game's versions can differ in modules: Mario Kart World's base game has main, subsdk0 (a Mono runtime) and sdk, its update 1.8.0 main, subsdk0 (the movie player), subsdk1 (a C library) and sdk. The host, the rewrite, the IPA and the shader split all took the one modules list of the profile.
Fix:modules lists all of them in load order, and each build row names its own (game.pybuild_modules; a row naming one modules lacks is an error). The host loads the modules of the row of main's build, read from the rewriter's main.fixups.tsv (main.cprofile_modules, from DM_GAME_BUILD_MODULES), and all of modules for a build with no row. build_ipa.sh takes the same list (game.py modules <id> <folder>). The NSP script rewrites, sizes mem_used from and splits the modules the packed version's ExeFS holds (Game.modules_in), and a derived profile's row names those (derive_profile).
Found on:Mario Kart World: the base game boots with its three modules from a profile that lists four. Applies to every game. test_game_profile.py checks the rows and the list.
Package identification
An NSP without a ticket or a title ID in its name stops with "no NSP with a readable title ID"
Symptom:[RED] no NSP with a readable title ID in <folder>, although the keys open the package (the launcher's key check passes). Renaming the file to Game [0100XXXXXXXXXXXX].nsp makes the build go on.
Cause:title_of took the title ID only from the ticket's file name (<rights ID>.tik) or from a [title ID] tag in the package's file name. Many NSPs carry no ticket (standard-crypto dumps, repacks), and a download's file name often has no tag.
Fix: with neither, title_of asks the package: nx_extract <prod.keys> --title <package> opens it, decrypts the header of each meta NCA with the keys' header_key and prints the lowest title ID (the base game's before its update's or a DLC's). It works for NSP, NSZ (the meta NCA is not compressed), XCI and XCZ, and runs only when the ticket and the tag are both missing, so the common cases stay a read of the file name table. nx_extract is built into a temporary folder for it, since the build folder is named after the title ID. Key values are never printed.
A game file whose extension is in capitals is not found
Symptom:[RED] no NSP with a readable title ID in <folder> for a folder holding Game [0100XXXXXXXXXXXX].NSP, on any Mac volume; renaming it to .nsp makes the build go on. A dump folder beside Game.NSP is packed instead of the package, and Game.NUP is not found. A Game.NSZ stops with nsz made no Game.nsp.
Cause: the script listed the folder with Path.glob('*.nsp'), which compares case-sensitively on every POSIX system, a case-insensitive Mac volume included. nsz has the same limit of its own: PathTools.isCompressedGame takes only a lowercase extension, and nsz skips anything else and exits 0 with no output. It resolves a symlink before it looks.
Fix:files_with_suffix lists a folder's files by their lowercased extension, for every scan of the inputs, and a package beside a compressed one is found in that listing. A compressed file in capitals goes to nsz as a hard link with a lowercase extension in its temporary folder, or as a copy where there can be none (another volume, exFAT), so the player's file stays as it is.
A merged NSP (base game, update and DLC in one file) packs as one title
Symptom: a game folder holding one merged package ([Base + Update + DLC].nsp, or a 1G+1U cartridge dump with no update NSP beside it) stops with no base game NSP found, or packs the base game alone (v0) without its update and DLC; when the base game's own build cannot be packed (Mario Kart 8 Deluxe 1.0.0's main is 32-bit), the build stops there.
Cause:scan_nsps gave each file one title ID: its first ticket's, else its name's tag. A merged package holds a ticket per title in any order, so it counted as the base game or as the update, never as both, and its DLC was never seen. nx_extract then read the update from a separate update NSP only.
Fix: a package whose file table (an NSP's PFS0, an XCI's secure HFS0 partition) holds more than one *.cnmt.nca is asked for its content metas, nx_extract <prod.keys> --contents <package> (one content <type> <title ID> <version> line each), and counts as each Application, Patch and AddOnContent it lists, with the versions from the metas. The one file is then given to nx_extract as both the base game and the update: OpenNsp keeps the NCAs that the Application content meta lists for the base and the Patch content meta's for the update, both of the application being packed (--app), and --aoc <package> <out> <DLC title ID> takes one DLC's data NCA by its title ID. The extraction estimate, the progress total and the NSZ conversion count the file once. When the content metas cannot be read, a [RED] line says so and the package counts by its ticket as before. A DLC NSP that bundles several DLC is read the same way.
Found on: Mario Kart 8 Deluxe (0100152000022000, no profile here) from one merged NSP, in NX2Apple (PR #842, LNPANDY/NX2Apple#376). Checked with Balatro's update NSP and base NSP merged into one PFS0: it scans as base v0 and update v851968, and nx_extract given it as base and update writes the same ExeFS, control data and RomFS manifest as the two separate NSPs. Applies to every game.
A derived profile leaves out a module that moved to another slot
Symptom: a version a profile was not brought up on crashes at its first movie: UNIMPL _ZN5movie9ExtractorC1ENS_13ContainerTypeE, then CRASH sig=11 (issue #1211). The derived profile's modules has only main sdk.
Cause:derive_profile could only take modules out of the profile's list, but a module's slot belongs to a build, not to a game: Absolum's movie module is subsdk2 in 1.1.2, where subsdk0 is NVIDIA's Vulkan driver, and subsdk0 in 1.2.1.
Fix:derive_profile adds what profile_guess.main_modules finds: a module the list leaves out that exports something main imports, which no listed module and no host hook supplies, and never a GPU driver. It shares this with nro_modules (modules_supplying). The added module goes in the version's build row too, since the host loads what the row names. Checked on 9 games' real modules with their registered lists: nothing is added (Super Mario Galaxy 2's GLSLC and nn::pcm, which the host supplies, stay out), and each of the 7 subsdks taken out of a list comes back. A registered build keeps its row: the rule runs only when a profile is derived.
Found on:Absolum 1.2.1, from the reporter's logs: with the module added by hand the game reached gameplay at 60 fps. No 1.2.1 files on this Mac: not run here.
A derived profile loads only the modules its version ships
Symptom: the NSP script stops at its rewrite stage with FileNotFoundError: … extract/v0/ExeFS/subsdk1 while packing a version the profile was not brought up on; the extraction has every other module.
Cause:derive_profile in scripts/build_ipa_from_nsp.py gives the derived profile this version's build row, made only from the modules its ExeFS holds, but copied modules unchanged. A version that ships fewer modules than the registered one (an older base game before an update added a module) then names a module the rewrite and the host try to load.
Fix:derive_profile keeps in modules only those the version's ExeFS holds, with a comment naming the ones left out; a version that ships them all keeps the line as it is, and shader_only derivations are unchanged. The host is built from the derived profile, so it loads the same list.
Found on: Mario Kart World 1.0.0 (0400C3F00006E000, a base-game dump: main, subsdk0 and sdk), packed from a profile for 1.8.0, whose subsdk1 is the C library module the update added (PR #1081). With it the rewrite, the shader split (11,561 shaders) and the Mac host built, and the game drew its first course on the Mac.
Check:test_nsp_incremental (test_a_module_the_version_does_not_ship_is_left_out: the module left out with its comment and its build row, then kept once the version ships it).
Keys
Keys older than the game stop the extraction with an unhandled MissingKeyException
Symptom:nx_extract stops with exit code 134 and Unhandled exception. LibHac.Common.Keys.MissingKeyException: Unable to decrypt NCA section. with Key Name: key_area_key_application_<NN> (or titlekek_<NN>), after any NSZ conversion, the build ID lookup or the survey. An NSZ converted with such keys is written anyway: nsz -D warns master_key_<NN> missing … This will lead to corrupted output. and exits 0.
Cause: each NCA's header names the key generation its content needs; a prod.keys dumped from a console on older firmware than the game lacks that master key and every key derived from it. Nothing checked the keys against the packages before the first step that decrypts them, and LibHac's exception reached the log as a crash.
Fix:nx_extract <prod.keys> --keys <package>... opens every NCA of the packages (an NSZ's or XCZ's compressed ones by their header, which nsz keeps as it was) and applies LibHac's own test (CanOpenSection). On the newest key missing it prints [RED] <file>: prod.keys has no <key> (master key <NN>, which came with firmware <version>) … and exits with code 6; a rights-ID NCA with no common ticket is named the same way. The NSP script runs it once after scanning the folder (check_keys in package_title_reader), before nsz -D, the survey and the extraction, and stops with that line; any other failure of the check is left to those steps. Extraction, --aoc and --contents run the same check before they decrypt anything (the content metas first), so nx_extract alone also stops with exit code 6 instead of a crash. The firmware table runs to 21.0.0 (master key 0x14); a newer key is named as "a firmware after 21.0.0". The check reads only headers: the NSP script stops on Balatro's two NSPs in about 1.5 s.
Found on: NX Library (Bium23/NxLibrary), a Mac front end that checks the keys before starting a build, an idea from NX2Apple. Checked with Balatro's NSPs and an NSZ of its base game, with a copy of prod.keys without the keys of master key 0x10 and later (and one without 0x0a and later): every path names key_area_key_application_10 and firmware 17.0.0, and the complete keys pass and extract as before. Applies to every game.
Cartridge dumps
A cartridge dump with its update on it extracts the update as the base game
Symptom: the extraction of a game whose base is an XCI that also carries its update (a "1G+1U" dump) stops with [RED] nx_extract exit code -6. extract_v<version>.log lists the update's build IDs even for the base game alone, then ends in an unhandled exception from the RomFS (ResultFsNonRealDataVerificationFailed (2002-4604): Hash error! from RomFsFileSystem..ctor, or ArgumentException / InvalidOperationException from the patch layers).
Cause: the XCI's secure partition holds the update's NCAs beside the base game's, and the update's Program NCA has the program's title ID too. nx_extract took the first Program NCA of the lowest title ID as the base, which is the update's when the dump lists it first. Its data section is a patch (BKTR) over the base RomFS, so it cannot be read alone or under another patch. The ExeFS still opens, with the update's build IDs, because an update's code section is complete. The Control NCA (display version, icon) was also whichever of the two came last.
Fix:dm/tools/nx_extract/Program.csOpenNsp reads the base package's content metas (*.cnmt in its Meta NCAs) and keeps only the NCAs that its Application content meta lists, and prints how many it left out. When none of them is a Program NCA (NCA files renamed, so their names are not content IDs), it prints a [RED] line and reads every NCA as before. An NSP or a base-only XCI keeps every NCA but its meta NCA, so it extracts as before (Balatro's NSPs and Monster Hunter Rise's XCI give identical manifests). The update on the cartridge is packed too since a merged package counts as each title it holds.
Found on:The Legend of Zelda: Tears of the Kingdom, a 1G+1U XCI (base v0 with update 1.4.3, whose 524 MB Program NCA is listed before the base's 17.1 GB one) beside the 1.4.3 update NSP (PR #311). Reproduced with a package of Balatro's update NSP followed by its base NSP: the base alone and the base with the update now give the same manifests and control data as the separate NSPs. Applies to any game's XCI that carries its update.
A cartridge dump that holds two games packs only the first
Symptom: an XCI with two applications (Super Mario Galaxy + Super Mario Galaxy 2 on one cartridge dump) always packs the first game; build_ipa_from_nsp.py never reaches the second.
Cause: the scan took one title ID per file (the file name tags both games, and the first tag won), and nx_extract kept the NCAs of every Application content meta and took the Program NCA of the lowest title ID. Nothing could name the other application.
Fix: the XCI's content metas make it count as each title it holds, so the scan finds two games and stops with more than one game in the folder, naming both. --title-id <title ID> (TITLE_ID in Spyder) picks the game to pack from them; the script passes its title ID to every base extraction (nx_extract … --app <title ID>), and OpenNsp keeps only that title's Application and Patch content metas' NCAs (an --app the package has no content meta of stops with the ones it has). Each game gets its own <title ID>_build/ folder and app.
Found on:Super Mario Galaxy 2, on the same XCI as Super Mario Galaxy (PR #701, which packed both from the XCI with an earlier form of the option). This form is checked with a test XCI's file table and with Balatro's packages (--app of a title the package does not hold stops, naming the one it has); no two-game XCI is on hand.
Damaged packages
Extraction of a damaged NSP ends in "The LinkedList is empty"
Symptom:nx_extract stops with exit code 134 and Unhandled exception. System.InvalidOperationException: The LinkedList is empty. from LibHac.Tools.FsSystem.CachedStorage.GetBlock, with no [RED] hash check failed list, with or without --allow-damaged.
Cause: a damaged NSP with more than a few damaged files used up LibHac 0.19's hash cache. CachedStorage.GetBlock takes a block off its list before reading into it and does not put it back when the read fails, so the 4 blocks of a filesystem last for 4 failed reads and the next read that misses the cache throws. That exception is no hash error, so nx_extract did not catch it. A copy of SteamWorld Dig with 2 flipped bytes listed its 2 files; with 40 it crashed.
Fix: each hash error gets a new checked filesystem (Failed in dm/tools/nx_extract/Program.cs, on the image, folder and manifest paths), so a damaged NSP lists every damaged file and stops with exit code 3, or goes on with --allow-damaged. The same copy with 40 flipped bytes lists all 36 files. test_every_damaged_file_is_listed_past_the_hash_cache in dm/tools/tests/test_romfs_read.py puts a real RomFS behind a real CachedStorage on all three paths, with no NSP or keys.
Found on:Breath of the Wild (issue #810, whose reporter diagnosed the lost cache blocks and carried a local patch that reopens the filesystem). Applies to any game.
A damaged RomFS's own tables
Symptom:[RED] the NSP is damaged: a filesystem's own tables fail the NCA's hash check, exit code 3, right after the build IDs; before 2026-11-04, [RED] nx_extract exit code -6 with an unhandled ResultFsNonRealDataVerificationFailed (2002-4604) under RomfsHeader. The ExeFS reads, and the build IDs and name are printed, since only the RomFS is hit.
Cause: the damage lies in what the RomFS opens from (its header, or the hash levels above it), so no file can be listed, and --allow-damaged has no file to write as read. Hashing each NCA tells which one it is: its file name is the first 16 bytes of its SHA-256.
Fix:nx_extract says so and exits with 3, as for damaged files. A good copy of the NSP is the only cure. A base game that is intact packs alone with --package <its version>, once it has a profile.
Found on:Mario & Luigi: Brothership: update 1.0.1's Program NCA 074db5a4… (its NSP is also a repack: it holds a .cnmt.xml).
A crash from a damaged NSP's file looks like a host bug
Symptom: a build packed with --allow-damaged crashes or draws wrongly in game code, the same way on each run, at a place unrelated to the host's last work. The log gave no sign of damage; the build log's list (logs/extract_<version>.log) is the only record, and it is not part of a bug report's run log.
Cause:--allow-damaged writes files that failed the NCA's hash check as they were read, and nothing told the running host. A wrong byte in a compressed pack decodes to wrong data far from where the game then fails. A hash block also spans neighbouring small files, so a listed file's bytes may still be right.
Fix: the entry's last byte in a DMROMFS1 index (format) is 1 on a file that failed the check (img_ent.flags, romfs_image.h; nx_extract sets it, and every reader refuses another value, or a flag on a directory). The mark stays with the image in an IPA, a Mac app or a hand-copied romfs.img. The host logs [RED] fs RomFS image <path>: damaged files <N>: ... at the mount and [RED] fs rom:/<file> failed the NCA's hash check ... at each damaged file's first open. A clean image is byte-identical to one from before this mark (checked on SteamWorld Dig), so its identity does not change. Not marked: an image extracted before this change, and folder extraction, which has no index (the NSP script writes images). Search a crash's log for hash check before looking for a cause in the host.
Found on:Breath of the Wild (issue #810). dm/host/tests/t_fs.c checks the count, the single report at first open and the refused flags; a boot of SteamWorld Dig on a deliberately damaged image logged the count and the files it opened.
An update's offline HTML pages reported as a damaged NSP
Symptom: the NSP script stops at the offline HTML pages (logs/extract_html_<version>.log) with [RED] the NSP is damaged: <n> files fail the NCA's hash check and a list under /html-document/, although the program extracted cleanly and the NSZ tool verified every NCA of the update.
Cause: an update's HtmlDocument NCA can be a patch of the base game's. nx_extract --html guessed: it read the update's NCA alone first and over each base NCA second, keeping the first read that did not throw. A read that fails the hash check does not throw, though: Copy lists each file as damaged and goes on. So the read alone was kept, the patched one was never tried, and the files it listed stopped the build.
Fix:nx_extract --html (dm/tools/nx_extract/Program.cs) reads the NCA's header: a data section with a relocation table (IsPatchSection) is a patch, read over the base game's Manual NCAs, and any other is read alone, once, so damage in it is reported as damage. The header does not say which base NCA a patch is over (HtmlDocument and LegalInformation are both Manual), so Try reads over each in turn and undoes a read that adds files that fail the hash check (it deletes the folder and takes the entries off the list). A patch that reads over none is reported with [RED] an HtmlDocument NCA of the update patches none of the base game's, and the log names each patched read (html<n>: a patch of the base game's NCA).
Found on:Super Smash Bros. Ultimate update 13.0.5 (v2031616) (PR #1215): 1,641 of the NCA's files failed alone; read through the base, 2,798 files (122.2 MB) and none damaged, and the game packed and played a battle on the Mac at 60 fps. Balatro's update has a full Manual NCA: it is read alone, as before. IsPatchSection checked on Balatro's update: true on the Program NCA's RomFS, false on its ExeFS, its Manual NCA and every base section.
Sparse base games
The survey's build ID lookup stops on the base game alone with a hash error
Symptom: for a game with no profile, the NSP script stops at build ID lookup · main of the base game with [RED] nx_extract exit code -6; <title ID>_survey/logs/exefs.log ends in ResultFsNonRealDataVerificationFailed (2002-4604): Hash error! from PartitionFileSystemMetaCore.QueryMetaDataSize under Nca.OpenFileSystem, although every NCA in both NSPs matches its content ID (the SHA-256 that names it) and the base game with its update extracts cleanly.
Cause: the base game's Program NCA is sparse: its code and data sections carry a sparse layer and leave out data that the update supplies, so the base is smaller than its update (1.6 GB against 2.9 GB). Horizon mounts such an NCA only through its patch; read alone, the left-out ranges fail the hash check from the first block of the ExeFS. The lookup (same_build) tries main's build ID of the newest update first, then of the base game alone, and run stopped the build on the second attempt's failure.
Fix:dm/tools/nx_extract/Program.cs stops with exit code 4 and [RED] the base game's program is sparse: it can be read only with its update NSP when it is given a sparse base without an update (any enabled section of the Program NCA with a sparse layer). scripts/build_ipa_from_nsp.pyrun takes the exit codes a caller accepts (ok), and same_build accepts 4: the base game alone then has no main of its own and the lookup goes on. The base game with its update reads as before (LibHac layers the update over the sparse base; the whole 4.2 GB merged RomFS of the game below reads with every hash check on).
Found on:Minecraft Dungeons II, base v0 (SDK 22.2.0) with update v131072 (1.1.3.0). Applies to any game whose base Program NCA is sparse.
The offline pages of a sparse base game stop the build as a damaged NSP
Symptom: the NSP script stops right after extract v<version> with [RED] the NSP is damaged: a filesystem's own tables fail the NCA's hash check and nx_extract exit code 3 from logs/extract_html_<version>.log, although every NCA's SHA-256 matches its name.
Cause: the base game's HtmlDocument NCA can be sparse too (16 KB, beside the sparse Program NCA). --html reads the update's HtmlDocument NCAs, each over a base one when it is a patch, and then every base HtmlDocument NCA on its own; a sparse one read alone fails the hash check while its tables open.
Fix:dm/tools/nx_extract/Program.cs reads no sparse base HtmlDocument NCA on its own (a sparse HtmlDocument NCA of the base game (… bytes): read only through the update's patch, not alone), as Horizon mounts it only through its patch.
Found on:Minecraft Dungeons II, base v0 with update v131072: the update's patch (accessible-urls) and its full legal information NCA are read, the base's legal information NCA too.