Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 104 additions & 0 deletions tools/ingame/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# In-game run tools

One in-game run of a WrongGPU candidate build, with evidence and a byte-for-byte restore. Used for Model L in
Cyberpunk 2077 and Resident Evil Requiem (#89). Windows only, Python 3, no GPU work of their own except
`offline_create.py`.

Nothing in these tools is specific to one PC. Every path comes from a game config (`run.py --config`) or from
arguments (`offline_create.py`, `make_record.py`).

## Before a game: the offline create test

```
python offline_create.py build --work <dir>
python offline_create.py create --work <dir> --runtime-dir <built runtime> --model <L model.wgcm> --model-sha <sha256> [--extra l_post=own] [-- <GPU slot wrapper>]
```

`create` runs the interop harness once:
- `model=L` and `executor=dx12`;
- 600 frames, 1280x720 to 1920x1080, with a launch trace.

It passes only if all of these hold:
- the runtime ran on D3D12;
- the timing windows came back;
- Model L kernels appear in the trace.

Run it before every game run with a new build.

`timing` measures Model L against Model M on six shapes, interleaved, and writes `l-timing-harness.json`.

Everything after a lone `--` wraps each GPU run, for example a GPU-slot tool.

## The game run

1. Copy `examples/<game>.json` and fill in every `<placeholder>`:
- `game_root`, `exe_dir`, `exe`, `launchers`;
- `out`: an empty folder for this run;
- `launcher_package` and `payload`: from `build.py --model-l`;
- `model`: the file and its SHA-256;
- `aside`: files moved out of the game for the run, then moved back. Never opened, only hashed and renamed;
- `hash_only`: files hashed before and after but never touched, for example a script loader the game needs;
- `settings`: the file, its format (`ini` or `cp2077`) and the keys to set;
- `gpu_flag`: the GPU flag file;
- `disk_free_gb`: minimum free space per drive.

`%VARIABLES%` are expanded. `exe_dir` is relative to `game_root`. Optional `runtime_options_extra` adds runtime
options to every model choice (e.g. `l_post=own`); `--extra` on `prepare`, `use-m` or `use-l` replaces it for that
choice (`--extra ''` for none), so two variants can run in one session.

2. Run the phases:

```
python run.py --config my.json all --dry-run every phase, nothing changed
python run.py --config my.json preflight
python run.py --config my.json prepare
python run.py --config my.json launch then start the game in its launcher
python run.py --config my.json hold
python run.py --config my.json shot <label> ... use-m / use-l between runs (game closed)
python run.py --config my.json collect after the game has exited
python run.py --config my.json restore
```

What each phase does:
- **`prepare`**:
- makes a manifest of the game folder;
- backs up our install's files and the settings file;
- moves the `aside` files out and hashes the `hash_only` files;
- installs the candidate with the launcher's `update`;
- sets `runtime_options model=<model>` and OptiScaler's `[WrongGPU] UpscalerIndex`;
- adds the timing and launch-trace options;
- copies the model to `<exe dir>\WrongGPU\model\<model>\model.wgcm`.

The model goes into the game's own folder, not `%LOCALAPPDATA%`, because a tool running inside a packaged
(MSIX) app writes new `%LOCALAPPDATA%` files to a private copy that the game never sees.
- **`launch`**:
- takes the GPU flag for this one game run;
- starts `game_hold.py`, which releases the flag in `finally` when the game exits or never starts;
- starts the VRAM log.
- **`collect`**:
- copies our own logs. OptiScaler's log is never read;
- writes `proof.json` with the model option, the model file and digest, Model L kernels, the executor, dispatch
counts, refusals and the kernel counts of each launch trace (one trace per model choice,
`launch-trace-<time>.txt`).

The verdict is PASS or NOT PROVEN.
- **`restore`**:
- puts every backed-up file back;
- removes what the update added and the placed model, with the folders it created;
- moves the `aside` files back and checks the `hash_only` hashes;
- restores the settings file;
- verifies the game folder against the manifest.

3. Timing record from the runtime's own timing lines, split by game process and scene:

```
python make_record.py --log <out>\logs\wronggpu-fsr.log --scenes scenes.json --out record.json [--meta meta.json] [--checkout <src>] [--gpu-name-like "*name*"]
```

## Rules these tools keep

- Only our files and the files the config names are touched.
- Vendor files (`nvngx*`, `*dlss*`, `sl.*`, `nvapi*`) are recorded by name, size and time only. They are never opened.
- Another mod's files are hashed at most.
- The model stays in place until the game has exited. Only `restore` removes it.
- Do not use these tools on games with anti-cheat.
88 changes: 88 additions & 0 deletions tools/ingame/aside.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# SPDX-License-Identifier: MIT
"""Move files out of a game tree for a test and put them back byte for byte (e.g. another mod's files).
The files' contents are never parsed: only hashed (SHA-256), stat'ed and renamed (same volume: a rename keeps bytes,
timestamps and attributes).

python aside.py out <aside folder> <file> [<file> ...] -> moves, writes <aside folder>\\MANIFEST.json
python aside.py restore <aside folder> -> moves back to the original paths
python aside.py verify <aside folder> -> checks the original paths against the manifest
"""
import ctypes
import hashlib
import json
import os
import sys
from ctypes import wintypes

k32 = ctypes.WinDLL('kernel32', use_last_error=True)


class FILETIME(ctypes.Structure):
_fields_ = [('lo', wintypes.DWORD), ('hi', wintypes.DWORD)]


class WIN32_FILE_ATTRIBUTE_DATA(ctypes.Structure):
_fields_ = [('attrs', wintypes.DWORD), ('ctime', FILETIME), ('atime', FILETIME), ('mtime', FILETIME),
('size_hi', wintypes.DWORD), ('size_lo', wintypes.DWORD)]


def info(path):
d = WIN32_FILE_ATTRIBUTE_DATA()
if not k32.GetFileAttributesExW(path, 0, ctypes.byref(d)):
raise OSError(ctypes.get_last_error(), 'GetFileAttributesExW', path)
ft = lambda f: (f.hi << 32) | f.lo
h = hashlib.sha256()
with open(path, 'rb') as f: # hashed only, never parsed
for b in iter(lambda: f.read(1 << 20), b''):
h.update(b)
return {'size': (d.size_hi << 32) | d.size_lo, 'sha256': h.hexdigest(), 'attributes': d.attrs,
'creation_filetime': ft(d.ctime), 'last_write_filetime': ft(d.mtime)}


def out(folder, files):
os.makedirs(folder, exist_ok=False)
man = []
for i, src in enumerate(files):
src = os.path.abspath(src)
e = {'original_path': src, 'aside_name': '%02d_%s' % (i, os.path.basename(src))}
e.update(info(src))
dst = os.path.join(folder, e['aside_name'])
os.rename(src, dst) # same volume: bytes, timestamps and attributes kept
after = info(dst)
e['aside_check'] = 'ok' if all(after[x] == e[x] for x in ('size', 'sha256', 'creation_filetime', 'last_write_filetime', 'attributes')) else 'MISMATCH %r' % after
man.append(e)
print('moved %s -> %s (%s)' % (src, dst, e['aside_check']))
json.dump(man, open(os.path.join(folder, 'MANIFEST.json'), 'w', encoding='utf-8'), indent=2)
print('manifest: %s' % os.path.join(folder, 'MANIFEST.json'))


def restore(folder):
man = json.load(open(os.path.join(folder, 'MANIFEST.json'), encoding='utf-8'))
for e in man:
src = os.path.join(folder, e['aside_name'])
if os.path.exists(e['original_path']):
raise SystemExit('refusing: %s exists again' % e['original_path'])
os.rename(src, e['original_path'])
print('restored %s' % e['original_path'])


def verify(folder):
man = json.load(open(os.path.join(folder, 'MANIFEST.json'), encoding='utf-8'))
bad = 0
for e in man:
now = info(e['original_path'])
diff = [x for x in ('size', 'sha256', 'creation_filetime', 'last_write_filetime', 'attributes') if now[x] != e[x]]
bad += bool(diff)
print('%s %s%s' % ('ok ' if not diff else 'MISMATCH', e['original_path'], '' if not diff else ' ' + repr(diff)))
print('verify: %d file(s), %d mismatch(es)' % (len(man), bad))
return 1 if bad else 0


if __name__ == '__main__':
cmd = sys.argv[1]
if cmd == 'out':
out(sys.argv[2], sys.argv[3:])
elif cmd == 'restore':
restore(sys.argv[2])
else:
sys.exit(verify(sys.argv[2]))
47 changes: 47 additions & 0 deletions tools/ingame/check_record.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# SPDX-License-Identifier: MIT
"""Check a game's WrongGPU install record against the folder: every recorded 'copy' file's SHA-256 against the
record's installed hash (files owned by wg / wg-optibundle only), and list the files in the exe folder's top level and
OptiScaler\\ that the record doesn't name. Vendor-named files (nvngx*, *dlss*, sl.*, nvapi*) are never opened.
usage: check_record.py <exe folder>"""
import fnmatch
import hashlib
import os
import sys

d = sys.argv[1]
rec = open(os.path.join(d, 'wronggpu-setup', 'install.record'), encoding='utf-8').read().splitlines()
owner = {}
copies = []
for l in rec:
f = l.split('\t')
if f[0] == 'owner' and len(f) >= 4:
owner[(f[1], f[2])] = f[3]
elif f[0] == 'copy':
copies.append((f[4], f[3]))
elif f[0] == 'disable':
copies.append((f[3], None))
VENDOR = ['nvngx*', '*dlss*', 'sl.*', 'nvapi*']
bad = 0
named = set()
for rel, inst in copies:
named.add(rel.lower())
p = os.path.join(d, rel)
own = owner.get(('copy', rel), owner.get(('disable', rel), '?'))
if inst is None:
print('%-8s %-14s %s present=%s (moved aside by the install; backup in wronggpu-setup\\backup)' % ('disable', own, rel, os.path.exists(p)))
continue
if not os.path.exists(p):
print('MISSING %-14s %s' % (own, rel)); bad += 1; continue
h = hashlib.sha256(open(p, 'rb').read()).hexdigest()
ok = h == inst
bad += not ok
print('%-8s %-14s %s' % ('ok' if ok else 'CHANGED', own, rel))
print('--- not named by the record (top level and OptiScaler\\, names only):')
for sub in ('', 'OptiScaler'):
base = os.path.join(d, sub)
for n in sorted(os.listdir(base)):
rel = os.path.join(sub, n) if sub else n
if os.path.isfile(os.path.join(base, n)) and rel.lower() not in named:
tag = 'vendor' if any(fnmatch.fnmatch(n.lower(), v) for v in VENDOR) else ''
print(' %s %s' % (rel, tag))
print('record check: %d problem(s)' % bad)
22 changes: 22 additions & 0 deletions tools/ingame/examples/cyberpunk2077.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "Cyberpunk 2077",
"game_root": "<the game folder, e.g. ...\\steamapps\\common\\Cyberpunk 2077>",
"exe_dir": "bin\\x64",
"exe": "Cyberpunk2077.exe",
"launchers": ["REDprelauncher.exe", "REDlauncher.exe"],
"out": "<an empty output folder for this run>",
"launcher_package": "<the candidate's WrongGPU-<version>.exe (build.py --model-l)>",
"payload": "<the same build's payload folder>",
"model": {"name": "L", "file": "<the converted Model L model.wgcm>", "sha256": "<its SHA-256>"},
"aside": [],
"hash_only": [],
"settings": {
"file": "%LOCALAPPDATA%\\CD Projekt Red\\Cyberpunk 2077\\UserSettings.json",
"format": "cp2077",
"keys": {"ResolutionScaling": "DLSS", "DLSS": "Quality", "DLSS_D": "false", "RayTracedPathTracing": "false",
"FrameGeneration": "Off", "DLSSFrameGen": "false", "WindowMode": "BorderlessWindowed"}
},
"gpu_flag": "<the GPU-in-use flag file other GPU jobs on this PC respect>",
"flag_owner": "ingame-run",
"disk_free_gb": {"C:\\": 20}
}
22 changes: 22 additions & 0 deletions tools/ingame/examples/re9.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "Resident Evil Requiem",
"game_root": "<the game folder, e.g. ...\\steamapps\\common\\RESIDENT EVIL requiem BIOHAZARD requiem>",
"exe_dir": ".",
"exe": "re9.exe",
"launchers": [],
"out": "<an empty output folder for this run>",
"launcher_package": "<the candidate's WrongGPU-<version>.exe (build.py --model-l)>",
"payload": "<the same build's payload folder>",
"model": {"name": "L", "file": "<the converted Model L model.wgcm>", "sha256": "<its SHA-256>"},
"aside": [],
"hash_only": ["dinput8.dll", "_storage_\\dinput8.dll"],
"settings": {
"file": "<the game folder>\\config.ini",
"format": "ini",
"keys": {"UpscalingAlgorithm": "DLSS", "UpscalingQuality_DLSS": "Quality", "RayTracingSetting": "High",
"FrameGeneration": "Off", "WindowMode": "Borderless"}
},
"gpu_flag": "<the GPU-in-use flag file other GPU jobs on this PC respect>",
"flag_owner": "ingame-run",
"disk_free_gb": {"C:\\": 20}
}
57 changes: 57 additions & 0 deletions tools/ingame/game_backup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# SPDX-License-Identifier: MIT
"""Backups before an in-game run (copies, nothing moved or changed):
- every file the game's WrongGPU install record names (copy entries), the record folder wronggpu-setup\\ (with its
backup\\, which holds the owner's earlier OptiScaler dxgi.dll: copied as a file, never opened), OptiScaler.ini,
wronggpu*.ini / logs in the exe folder -> <out>\\game\\<relative path> (copy2: timestamps kept);
- %LOCALAPPDATA%\\WrongGPU except model\\ (list + SHA-256 in localappdata-before.json, copies in <out>\\lad\\).
usage: game_backup.py <exe folder> <out folder>"""
import hashlib
import json
import os
import shutil
import sys

exe_dir, out = sys.argv[1], sys.argv[2]
sha = lambda p: hashlib.sha256(open(p, 'rb').read()).hexdigest()
rels = set()
for l in open(os.path.join(exe_dir, 'wronggpu-setup', 'install.record'), encoding='utf-8').read().splitlines():
f = l.split('\t')
if f[0] == 'copy':
rels.add(f[4])
for dp, dn, fn in os.walk(os.path.join(exe_dir, 'wronggpu-setup')):
for n in fn:
rels.add(os.path.relpath(os.path.join(dp, n), exe_dir))
for n in os.listdir(exe_dir):
low = n.lower()
if os.path.isfile(os.path.join(exe_dir, n)) and (low.startswith('wronggpu') or low == 'optiscaler.ini'):
rels.add(n)
for sub in ('WrongGPU', os.path.join('OptiScaler', 'custom', 'WrongGPU')):
d = os.path.join(exe_dir, sub)
for dp, dn, fn in os.walk(d):
for n in fn:
rels.add(os.path.relpath(os.path.join(dp, n), exe_dir))
copied = []
for r in sorted(rels):
src = os.path.join(exe_dir, r)
if not os.path.isfile(src):
continue
dst = os.path.join(out, 'game', r)
os.makedirs(os.path.dirname(dst), exist_ok=True)
shutil.copy2(src, dst)
copied.append(r)
lad = os.path.join(os.environ['LOCALAPPDATA'], 'WrongGPU')
lad_list = {}
for dp, dn, fn in os.walk(lad):
rel_dp = os.path.relpath(dp, lad)
if rel_dp.split(os.sep)[0] == 'model':
continue
for n in fn:
p = os.path.join(dp, n)
r = os.path.relpath(p, lad)
lad_list[r] = {'size': os.path.getsize(p), 'sha256': sha(p)}
dst = os.path.join(out, 'lad', r)
os.makedirs(os.path.dirname(dst), exist_ok=True)
shutil.copy2(p, dst)
json.dump(lad_list, open(os.path.join(out, 'localappdata-before.json'), 'w'), indent=1, sort_keys=True)
json.dump(copied, open(os.path.join(out, 'game-backup-list.json'), 'w'), indent=1)
print('game files backed up: %d; %%LOCALAPPDATA%%\\WrongGPU files (model\\ excluded): %d' % (len(copied), len(lad_list)))
Loading
Loading