Skip to content
Cirno-LabsPublic

Latest commit

 

History

910 Commits

Folders and files

Repository files navigation

th07

A from-scratch matching decompilation of 東方妖々夢 ~ Perfect Cherry Blossom (Touhou 7), built and verified on Linux through Wine.

The goal is C++ source that compiles, with the original toolchain, into a binary that is byte-for-byte identical to the original executable.

This repository contains no game code or data. You need your own copy of the game, see orig/README.md.

Clean room

This is an independent, clean-room implementation. It is written from the binary alone: no code, names, struct layouts or function lists are taken from other decompilations or ports of Touhou games. Names come from our own analysis or, for library code, from the libraries themselves.

Status

The standalone executable (just exe) now achieves 100% byte identity: all 650,752 bytes match the original retail 1.00b executable, including code, data, resources, headers and Rich metadata. Both files have SHA-1 80cbf99382c81ea0a9b7b85c7f9e059a75a52725. Only the link timestamp is copied from the original, and the resources, the game's data, are taken from it; the code, data, imports, headers and Rich header come out of the build.

All 694 game functions are decompiled and named, sit at their original addresses, and match raw. Ordinary-object comparison passes 702/702 functions. A cold rebuild of 37 ordinary objects plus 37 /GL inputs reproduces the same hash, and all 31 Python regression tests pass. just check-exe builds and enforces whole-file equality; just compare-exe gives the detailed report.

The match uses documented compiler/linker layout aids, including discarded helpers and static-construction adapters with 20/4 unused stack bytes. It is not a claim to have recovered the original source, and no generated code/data bytes are patched from the game executable. See executable parity for the reconstruction details, validation and earlier checkpoints.

just replay uses the shared replay harness (tools/replay/, from th06) with th07's format and tracer to compare complete playback runs. See replay parity for setup, the 36-recording world record corpus, saved-trace audits and validation limits. Replay checks require natural completion and recorded stage scores as well as exact original/rebuild trace equality. The final audit matches all 36 traces; 26 recordings pass full-route validation, while the same ten fail in the original executable.

Ghidra finds 1561 functions. As in th06, the game code is compiled without optimization, so its functions start with push ebp; mov ebp, esp; the statically linked D3DX8 and CRT code is optimized. That puts the game code at 0x401000–0x46004b: 694 functions, about 381 KB. D3DX8's tables follow it, then D3DX8 and the CRT.

just analyze   # writes build/functions_100b.csv

Progress

just progress renders it: every game function, by address and as a treemap by size, coloured by whether it is decompiled and matching. just hooks makes every commit redraw it and include it.

Target

Retail 1.00b, released 2003-10-24, the last official update:

  • 32-bit PE, image base 0x400000, entry point 0x47ea7d, .text is 0x8bd38 bytes
  • .data is 0xec9258 bytes in memory but only 0x3800 on disk, so most of it is zero-initialized globals
  • Imports d3d8, DSOUND, DINPUT8, WINMM, GDI32, USER32, KERNEL32 and ole32. Only Direct3DCreate8 is imported from D3D, so D3DX8 is statically linked.

Toolchain

Read from the Rich header (tools/peinfo.py):

Tool Build Objects Probably
C++ compiler 13.00, /GL 9466 37 Game code
C++ compiler 13.00 9466 15
C compiler 13.00 9466 129 MSVC CRT
MASM 7.00 9466 47 MSVC CRT
C++ compiler 13.00 8830 58 Static libraries, probably D3DX8
Older linker objects 8755 10 Static libraries
Import library 7.00 9210 10 VS.NET 2002 Platform SDK import libraries
C compiler 13.00 9178 3 uuid.lib's GUIDs
Alias objects 7.00 9162 3 libcmt.lib's name aliases (_mbscpy and so on)
Linker 7.00 9466

This is the th06 toolchain: Visual C++ .NET 2002 (MSVC 7.0, cl 13.00.9466) with the DirectX 8 SDK. The difference is that most C++ objects were compiled with /GL (product id 44, confirmed by linking a test object with /GL and /LTCG). So far that does not show in the code: functions match when compiled without it. It shows in the layout: link-time code generation puts each object's inline functions first, so the standalone exe is built with /GL and /LTCG. /GL needs msobj10.dll from the Visual Studio CD in the compiler's directory.

The import libraries for user32, gdi32, winmm, ole32, kernel32 and uuid are Visual Studio .NET 2002's own (Vc7\PlatformSDK\Lib, import library build 9210 in the Rich header), not a later Platform SDK's: they decide the import hints and the Rich header. They are not loose files on the CDs but sit in a cabinet inside vs_setup.msi on disc 1. just psdk-libs ZIP extracts them from the install media (a zip holding cd1/VSPROD1.iso) into C:\msvc7\PlatformSDK\Lib in the Wine prefix, which the build searches before C:\psdk\Lib.

Workflow

Everything runs headless, so it works over SSH. just lists the recipes.

  1. Analyze the binary with Ghidra in headless mode (just analyze). New functions are added to config/symbols.csv with placeholder names (Func43be50); library functions keep the names Ghidra's function ID gives them.
  2. Write C++ for a function in src/, in the style of 2003-era code. just decompile [ADDR...] writes Ghidra's pseudocode to build/decomp/<address>.c to read next to the disassembly.
  3. Record the function and every original function and global it uses in config/symbols.csv: address, size, our name, and the mangled name, which the link error for an unresolved external gives.
  4. Compare (just compare): builds with cl.exe under Wine and checks each function against the original byte for byte. Calls and global addresses are resolved through symbols.csv and compared too; only references to unnamed data such as string literals are masked. just compare SYMBOL shows a disassembly diff.
  5. Play (just run): runs the original game with the new code hooked in, headless under Xvfb, to check that the code behaves correctly as well as matching. Steps drive it and take screenshots, e.g. just run wait:12 key:Escape key:z wait:2 shot:menu writes build/shots/menu.png.
  6. Standalone (just exe): compiles the same sources again with /GL into build/obj_gl/ and links them with /LTCG, without the hook DLL, into build/th07.exe, a game that needs nothing from the original executable. /GL objects hold no machine code, so the DLL and just compare keep using the plain objects in build/obj/. The game's data and global objects are defined in the files that own them, under TH07_EXE; the DLL uses the original's instead. just exe STEPS runs it like just run. docs/exe-parity.md has how it matches.

Hooking

The decompiled code runs inside the original game. tools/patch_exe.py adds one import, th07.dll, to a copy of the executable. The DLL is built from the same objects that just compare checks. On load it overwrites each decompiled function in the original with a jump to the new code.

Calls from new code to functions that are not decompiled yet go through generated thunks (push address; ret) to the original. Globals are absolute symbols at their original addresses. Both are generated from symbols.csv. The DLL has no C runtime of its own: CRT functions belong in symbols.csv too, so the game and the new code share one heap.

Layout

orig/     your game files (not tracked) and their checksums
src/      decompiled source; src/dll/ is the hook DLL's own code
config/   symbols.csv: address, size, name and mangled symbol
tools/    the shared decomp-tools: build, compare, exe patcher, symbol
          seeding, Ghidra scripts, the replay harness and the just recipes
project.toml  th07's settings for the tools: exe, flags, link order
scripts/  th07's own: its build hook, the CRT archives, the replay
          hook, tracer, fixtures, audit and tests
replays/  world record replays for just replay, their manifest and README
docs/     progress image, executable and replay parity
build/    outputs; build/game/ is a runnable copy of the patched game

tools/ is the decomp-tools repository, shared with the other games, as a git submodule: clone with --recursive, or run git submodule update --init tools after cloning, and just tools-init to put it on its main branch. A change to the tools is a commit made in tools/ and sent with just tools-push; just tools-pull brings in the other games' changes. Either way, commit the new tools pointer here.

Requirements

  • Wine (32-bit, with its msidb), Python 3, just, binutils (objdump), 7-Zip
  • For just run and just replay: Xvfb, xdotool, ImageMagick, 32-bit Mesa OpenGL (libgl1-mesa-dri:i386, libglx-mesa0:i386) and the ja_JP.UTF-8 locale
  • Visual C++ .NET 2002, the Windows XP SP2 Platform SDK and the DirectX 8.0 SDK in the Wine prefix at .wine/ (bring your own copies)
  • Ghidra (for analysis, optional)

Legal

Touhou Project and 東方妖々夢 are by ZUN / Team Shanghai Alice. This is an unofficial fan project. It contains no assets or code from the original game, and it is not affiliated with or endorsed by ZUN or Team Shanghai Alice.

License

The source in this repository is licensed under the GNU General Public License, version 3 only (GPL-3.0-only), see LICENSE. The license covers our code, not the original game or its data.

Releases

Packages

Contributors

Languages