th08

A from-scratch matching decompilation of 東方永夜抄 ~ Imperishable Night (Touhou 8), built and verified on Linux through Wine.

The goal is C++ source that compiles, with the original toolchain, into a binary that is byte-for-byte identical to the original executable.

This repository contains no game code or data. You need your own copy of the game, see orig/README.md.

Clean room

This is an independent, clean-room implementation. It is written from the binary alone: no code, names, struct layouts or function lists are taken from other decompilations or ports of Touhou games. Names come from our own analysis or, for library code, from the libraries themselves.

Status

All 1116 game functions are decompiled and match the original byte for byte (just compare: 1130/1130, counting separately emitted inline copies and folded symbols).

The standalone executable (just exe) is built from this source alone, boots and plays, and is byte for byte the original: build/th08.exe has the original's SHA-1, 2044fea52667be4f61bcd78efc8f7d63fbbc244f. That includes every section, the import tables, the PE headers and the Rich header, which counts the original's 63 C++ objects because the source is split into 63 objects. Where th08's bytes don't show a seam, the splits follow th06's and th07's objects for the same code and data, and three have neither (each new file's first lines say which). All 136 named writable globals sit at their original addresses too, including function statics absent from the public link map (python3 scripts/check_exe_layout.py). Only the link timestamp is copied from the original, and the resources, the game's data, are taken from it; the code, data, imports, headers and Rich header come out of the build.

Getting there took the original's link inputs as well as its code. The build reconstructs game-function and readonly-data placement with the linker's /ORDER option, generated from symbols.csv, exe_constants.csv and the compiled objects; this does not establish that the original build used /ORDER. The C runtime's objects and the import objects load in the original's order, and /OPT:ICF keeps the original's copy of every identical function, because the objects mention, and give the linker, what the original's did, including code /OPT:REF dropped: exe-only Unused* functions stand in for that dropped code, and a few functions sit in a different order in their files. The original's precompiled-header layout remains unknown.

config/exe_constants.csv records 1143 compiler constant/string COMDAT addresses recovered through the original operands and our object relocations. After a vanilla build, python3 scripts/recover_exe_constants.py regenerates it; scripts/check_exe_layout.py verifies every recorded address. All recorded game readonly constants and initialized globals are identical, including the FileBase vtable's CRT __purecall pointers. just check-exe builds and enforces whole-file equality; just compare-exe gives the detailed report. See executable parity for the reconstruction details and the earlier checkpoints.

just replay uses the shared replay harness (tools/replay/, from th06) with th08's format and tracer to compare complete playback runs. See replay parity for setup, the 59-recording world record corpus, saved-trace audits and validation limits. Replay checks require natural completion and recorded stage scores as well as exact original/rebuild trace equality. All 59 recordings pass, and their traces match both original runs.

Ghidra finds 2225 functions. As in th06 and th07, the game code is mostly compiled without optimization, so its functions start with push ebp; mov ebp, esp, with a few objects optimized for size; the statically linked D3DX8 and CRT code is optimized. That puts the game code at 0x402000–0x475298: 1116 functions, about 460 KB. D3DX8's tables follow it, then the DirectX import thunks, D3DX8 and the CRT.

just analyze   # writes build/functions_100d.csv

just progress renders it: every game function, by address and as a treemap by size, coloured by whether it is decompiled and matching. just hooks makes every commit redraw it and include it.

Target

Retail 1.00d, released 2004-09-19, the last official update:

• 32-bit PE, image base 0x400000, entry point 0x4a619e, .text is 0xb1b78 bytes and starts at 0x402000, a page later than th06 and th07
• .data is 0x1413840 bytes in memory but only 0x6e00 on disk, so most of it is zero-initialized globals. A .data1 section of 0x8e0 bytes follows it.
• Imports d3d8, DSOUND, DINPUT8, WINMM, GDI32, USER32, KERNEL32, ADVAPI32 (registry reads) and ole32. Only Direct3DCreate8 is imported from D3D, so D3DX8 is statically linked.

Toolchain

Read from the Rich header (tools/peinfo.py):

Tool	Build	Objects	Probably
C++ compiler 13.00	9466	63	Game code
C compiler 13.00	9466	130	MSVC CRT
MASM 7.00	9466	48	MSVC CRT
C++ compiler 13.00	9177	69	DirectX libraries, mostly D3DX8
C compiler 13.00	9177, 9178	3, 3	Static libraries
Import library 7.00	9210	12	Platform SDK import libraries
Import library	9111	7	DirectX import libraries
Linker 7.00	9466

This is the th06 and th07 toolchain: Visual C++ .NET 2002 (MSVC 7.0, cl 13.00.9466). Unlike th07, no object was compiled with /GL. The game code still includes DirectX 8.0's headers, but the DirectX libraries are not a DirectX SDK's: they are the ones Visual Studio .NET 2002 itself installs in Vc7\PlatformSDK\lib\prerelease. Their d3dx8.lib objects say 9177 and their d3d8, dinput8 and dsound import libraries 9111, which no DirectX 8.1 SDK provides (9178 and 9210), and D3DX8's Intel-compiled objects bring the compiler banners that fill the original's 0x1e00 bytes of headers. They are not loose files on the media: they come with the merge module zDxF_pre_b.common.msm in disc 1's vs_setup.msi, which tools/msi_extract.py unpacks as tools/psdk_libs.py does for the Platform SDK. The standalone exe links them from C:\msvc7\PlatformSDK\Lib\prerelease in the Wine prefix. The compiler flags start as th07's and change as functions show otherwise.

Workflow

Everything runs headless, so it works over SSH. just lists the recipes.

1. Analyze the binary with Ghidra in headless mode (just analyze). New functions are added to config/symbols.csv with placeholder names (Func43be50); library functions keep the names Ghidra's function ID gives them.
2. Write C++ for a function in src/, in the style of 2004-era code. just decompile [ADDR...] writes Ghidra's pseudocode to build/decomp/<address>.c to read next to the disassembly.
3. Record the function and every original function and global it uses in config/symbols.csv: address, size, our name, and the mangled name, which the link error for an unresolved external gives.
4. Compare (just compare): builds with cl.exe under Wine and checks each function against the original byte for byte. Calls and global addresses are resolved through symbols.csv and compared too; only references to unnamed data such as string literals are masked. just compare SYMBOL shows a disassembly diff.
5. Play (just run): runs the original game with the new code hooked in, headless under Xvfb, to check that the code behaves correctly as well as matching. Steps drive it and take screenshots, e.g. just run wait:12 key:Escape key:z wait:2 shot:menu writes build/shots/menu.png. The patched executable is not the original, so the game's own check of it quits the game after about a minute, and rejects the score.dat records and replays a patched game wrote: FEAT=orig_checksum just run makes the DLL claim the original's checksum.
6. Standalone (just exe): links the same objects, without the hook DLL, into build/th08.exe, a game that needs nothing from the original executable. The game's data and global objects are defined in the files that own them, under TH08_EXE; the DLL uses the original's instead. just exe STEPS runs it like just run, and just compare-exe compares it with the original. docs/exe-parity.md has how it matches.

Hooking

The decompiled code runs inside the original game. tools/patch_exe.py adds one import, th08.dll, to a copy of the executable. The DLL is built from the same objects that just compare checks. On load it overwrites each decompiled function in the original with a jump to the new code.

Calls from new code to functions that are not decompiled yet go through generated thunks (push address; ret) to the original. Globals are absolute symbols at their original addresses. Both are generated from symbols.csv. The DLL has no C runtime of its own: CRT functions belong in symbols.csv too, so the game and the new code share one heap.

Layout

orig/     your game files (not tracked) and their checksums
src/      decompiled source; src/dll/ is the hook DLL's own code
config/   symbols.csv: address, size, name and mangled symbol;
          exe_constants.csv: the readonly constants' addresses
tools/    the shared decomp-tools: build, compare, exe patcher, symbol
          seeding, Ghidra scripts, the replay harness and the just recipes
project.toml  th08's settings for the tools: exe, flags, link order
scripts/  th08's own: its build hook, the exe constants' recovery and
          layout check, the replay hook, tracer, fixtures, audit and tests
replays/  world record replays for just replay, their manifest and README
docs/     progress image, executable and replay parity
build/    outputs; build/game/ is a runnable copy of the patched game

tools/ is the decomp-tools repository, shared with the other games, as a git submodule: clone with --recursive, or run git submodule update --init tools after cloning, and just tools-init to put it on its main branch. A change to the tools is a commit made in tools/ and sent with just tools-push; just tools-pull brings in the other games' changes. Either way, commit the new tools pointer here.

Requirements

• Wine (32-bit, with its msidb), Python 3, just, binutils (objdump), 7-Zip
• For just run and just replay: Xvfb, xdotool, ImageMagick, 32-bit Mesa OpenGL (libgl1-mesa-dri:i386, libglx-mesa0:i386) and the ja_JP.UTF-8 locale
• Visual C++ .NET 2002, the Windows XP SP2 Platform SDK and the DirectX 8.0 SDK in the Wine prefix at .wine/ (bring your own copies)
• Ghidra (for analysis, optional)

Legal

Touhou Project and 東方永夜抄 are by ZUN / Team Shanghai Alice. This is an unofficial fan project. It contains no assets or code from the original game, and it is not affiliated with or endorsed by ZUN or Team Shanghai Alice.

License

The source in this repository is licensed under the GNU General Public License, version 3 only (GPL-3.0-only), see LICENSE. The license covers our code, not the original game or its data.