Skip to content

Reject truncated PFS0 payloads before extraction - #2122

Open
thehobbit85 wants to merge 1 commit into
seedds:mainfrom
thehobbit85:codex/pfs0-payload-bounds
Open

thehobbit85 wants to merge 1 commit into
seedds:mainfrom
thehobbit85:codex/pfs0-payload-bounds

Conversation

@thehobbit85

Copy link
Copy Markdown
Contributor

What and why

Addresses the truncated licence-only DLC package reported in issue #2013. Its PFS0 table declares more payload than the file contains. The extractor previously trusted those ranges and later aborted with an unhandled LibHac ResultFsOutOfRange.

Validate every PFS0 file offset and size against the original payload before mounting it or adding the optional virtual filename terminator. Subtraction keeps corrupt 64-bit values from wrapping; an empty file exactly at EOF remains valid. Probe the XCI header only when that offset exists, so short PFS0 headers also reach validation. Invalid packages now name the file, print [RED] the NSP is damaged, and return the established damaged-input exit code 3. --allow-damaged cannot bypass missing container bytes. The source file is unchanged.

This is a partial fix, so it does not close #2013. The base game's SHAD/BTCD shader discovery remains unresolved. This does not reconstruct missing data or validate an encrypted NCA's internal size when its outer PFS0 ranges are consistent.

Checked

Apple Silicon Mac, macOS 26.6, Python 3.14.6, .NET 10 and the repository's LibHac 0.19 dependency. Synthetic containers and an empty key file only; no game files or real keys.

  • New regression fails on unchanged production code with truncated PFS0 payload accepted; passes after the fix. The existing test_pfs0_storage.py fixture compiles and exercises the real C# parser and CLI entry.
  • Both normally terminated and missing-final-NUL packages: preserve valid payloads; reject a truncated final member, offset beyond EOF, range crossing EOF and 64-bit offset/size overflow; accept a zero-length file at EOF.
  • Production CLI --aoc, with and without --allow-damaged: short payloads and a four-byte header return 3 with the package name and create no output directory, rather than throwing an unhandled read exception.
  • dotnet build dm/tools/nx_extract/nx_extract.csproj -v:q: PASS, zero warnings/errors.
  • Full tools suite: 1,423 tests, 23 skips, one inherited ABI-literal ratchet failure in unchanged nvn_mtl.m ('?' count 2→3). It was independently reproduced on pristine base 6c676444; existing test_dmsh_abi's ratchet no longer counts "?": the unknown binding kind is also what log lines write for a name they do not have, and the third such placeholder in nvn_mtl.m fails the Tool tests on main (issue #1656) #1981 addresses it. All other tests pass.
  • Documentation tests: all six pass after the final documentation placement. git diff --check: PASS. No host, graphics, shader-chain or Python production source changes.
  • Attempted regress.py check --games 0100726014352000,010015100B514000,01007EF00011E000,0100152000022000,010099C022B96000: exit 2 because there is no REGRESS_ROOT, REGRESS_GAMES or local game data. Selected Diablo II for the reported DLC (no profile), Wonder for ordinary base extraction, Breath of the Wild for update layering, Mario Kart 8 Deluxe for DLC, and Super Mario Galaxy for another packaging path. None was run; no gameplay claim.

Fresh main 6c676444 and all 1,472 open/draft/closed/merged PR descriptions plus the issue timeline were checked before publication. #886's final-name-terminator repair is on main and remains compatible; it does not validate payload ranges. No existing solution to this truncated-container defect was found.

Docs

  • Durable cause/fix and unresolved shader limitation in docs/reference/packing.md; extraction behavior in docs/development.md; symptom index regenerated.
  • No agent artifacts, game files or extracted data in the diff.

Validate file ranges without overflow before applying the virtual filename terminator. Report a named damaged-package error and exit 3 instead of an unhandled LibHac read failure.

Refs: seedds#2013
@github-actions github-actions Bot added packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers tools Tools and scripts (dm/tools, scripts) docs Documentation labels Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers tools Tools and scripts (dm/tools, scripts)

Projects

None yet

2 participants