Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions clean/converter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,20 @@ with that preset's message. It also converts an **L-shaped** map
(`tests/cases.py` `l_shaped_map`): preset M's map layout with the input encoder's output linear 64 x 128, and two new
stages of L's record kinds (two linears and a C = 32 Swin block before `enc0`, a C = 32 Swin block before `dec0`),
filled with our own random data. Its stage names and placement are stand-ins; the real map defines its own. A map of
versions 1-4 must still have the sources W, D and K: should preset L have no dither table or constants, that rule
becomes per preset.
versions 1-4 must have the sources its preset requires (`required_sources` in `known_presets`): W, D and K for presets
M and L.

**Preset FG** (frame generation, issue #96): `"preset": "FG"` in a map of version 1. Its sources are the weight blobs of
the frame-generation library's two convolution networks, each a block source with its own name, length and hashes
(no keyed source, no constants, no dither table); its tensors use preset M's record kinds (`f16` reads, strides that skip
a padding area or take one copy of a stored-several-times bias). The preset requires no source by name. The source
the key is made from is searched first: when it is missing, the file has no frame-generation model (exit 12, preset FG's
message); any other missing blob is exit 13. A map without the field is still read as preset M (so it must have W, D
and K) unless the caller names the preset (`--preset FG`). The spec side's map is not delivered yet; `tests/run_tests.py`
(`test_fg`) converts a synthetic library with our own random blobs in the frame-generation layout
([`clean/fg/layout.py`](../fg/layout.py): every blob's real size, shuffled order, aligned, even and odd offsets, a decoy),
with a stand-in map (`tests/cases.py` `fg_map`) whose tensor names and axis order are placeholders. The padding areas
and the extra bias copies of the synthetic blobs hold NaN, so a map that read them would fail (exit 22).

## Preset F (map version 5, spec 57)

Expand Down Expand Up @@ -134,7 +146,8 @@ Preset K, for the spec team (48-T1, 48-T6):
| 12 | M: W not found. K: T not found | This DLSS file has no preset-M model. Version 310.5 or newer is needed. / This DLSS file has no preset-K model. Version 310.1 or newer is needed. (+ "This file is version a.b.c.d." when the file has a version resource) |
| 12 | F: W not found (spec 57 section 8) | This DLSS file has no preset-F (Ray Reconstruction) model. Version 310.7 or newer is needed. (+ the file version when there is one) |
| 12 | L: W not found (a map with `"preset": "L"`) | This DLSS file has no preset-L model. Version 310.5 or newer is needed. (+ the file version when there is one) |
| 13 | M: W found, D or K not found. K: T found, O not found (spec 48 draft 4 section 8). F: W found, another source (or every variant of IMG-IN) not found. The log names the missing source | This DLSS build is not supported yet (model data found, but other parts differ). |
| 12 | FG: the key's source not found (a map with `"preset": "FG"`) | This DLSS Frame Generation file has no frame-generation model. Version 310.1 or newer is needed. (+ the file version when there is one) |
| 13 | M: W found, D or K not found. FG: the key's source found, another blob not found. K: T found, O not found (spec 48 draft 4 section 8). F: W found, another source (or every variant of IMG-IN) not found. The log names the missing source | This DLSS build is not supported yet (model data found, but other parts differ). |
| 14 | F only (spec 57 section 8): a constant's sites disagree, a `const_f16x2` with unequal halves, a `const_f16lo` with a non-zero high half, a non-finite value where 57.6.1 or 57.6.2 forbids it, an RN16 overflow, the two reads of an `also` tensor differ, a `require` or check fails, the attention constants fail 57.6.4. The log names the tensor or check (never a value). Not an installer error: the next file is tried | This DLSS build is not supported yet (model data found, but other parts differ). |
| 20 | map invalid: unreadable, bad JSON, bad format or version, unknown op / locate / element / encoding / dtype (or one its version does not define), missing field, unknown perm, perm not dividing its axis, `axes` length ≠ `shape` length, a segment that is not a positive multiple of 4 or runs past the end of T, K sources other than T (and O), `capped_softmax` without 8 × 8 last axes, a source F that is not a keyed optional `lz4-block` block source with variants, a field outside the decoded text, variants with different numbers of fields, a `digest` list whose length is not the number of variants (or a list for a group without variants), a group that mixes tensors from F and from other sources, … Also (spec 08 section 4.3): a variant of F matched its hashes, but the block does not decompress to `decoded_length`, does not match `decoded_sha256`, or a field is not eight hexadecimal digits | Installer error: the converter data is damaged. … |
| 20 | F: every structural failure of 57 section 5.4 (an index outside its source included) and a defensive bounds failure at run time | Installer error: the converter data is damaged. … |
Expand Down
3 changes: 3 additions & 0 deletions clean/converter/src/api.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ const char* const kMsgNoPresetF =
"This DLSS file has no preset-F (Ray Reconstruction) model. Version 310.7 or newer is needed.";
// Model L (#89). The minimum version is the first DLSS version confirmed to carry a preset-L model.
const char* const kMsgNoPresetL = "This DLSS file has no preset-L model. Version 310.5 or newer is needed.";
// Frame generation (#96): the first frame-generation library version known to carry its two networks' weights.
const char* const kMsgNoPresetFG =
"This DLSS Frame Generation file has no frame-generation model. Version 310.1 or newer is needed.";
const char* const kMsgUnsupported = "This DLSS build is not supported yet (model data found, but other parts differ).";
const char* const kMsgMapDamaged = "Installer error: the converter data is damaged. Please reinstall WrongGPU.";
const char* const kMsgMapDoesNotFit =
Expand Down
1 change: 1 addition & 0 deletions clean/converter/src/api.h
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ extern const char* const kMsgNoPresetM;
extern const char* const kMsgNoPresetK;
extern const char* const kMsgNoPresetF;
extern const char* const kMsgNoPresetL;
extern const char* const kMsgNoPresetFG;
extern const char* const kMsgUnsupported;
extern const char* const kMsgMapDamaged;
extern const char* const kMsgMapDoesNotFit;
Expand Down
2 changes: 1 addition & 1 deletion clean/converter/src/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ void log_line(const std::string& line) { std::cerr << "[wronggpu-convert] " << l

void usage() {
std::cout << "usage: wronggpu-convert --map <map.json> --dll <model dll> [--dll <other> ...]\n"
" --out <dir> [--raw] [--threads N] [--preset M|K|F|L]\n"
" --out <dir> [--raw] [--threads N] [--preset M|K|F|L|FG]\n"
" --dll may be given several times; the files are tried in order: the first that converts\n"
" completely is used (for a map with an optional group, such as preset M's output filter: the\n"
" first that has the group, if any has it). The map names its preset (field \"preset\", or by its\n"
Expand Down
66 changes: 43 additions & 23 deletions clean/converter/src/map.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,10 @@ bool json_number_exact_f32(const std::string& lit, float& out) {

class Loader {
public:
explicit Loader(MapError& err) : err_(err) {}
// `hint`: the preset the caller loads the map for ("" : none). A map without the field "preset" is read as that
// preset when the preset uses the map's version, so that its own required sources are checked (load_map below makes
// every other check of the caller's name).
Loader(MapError& err, const std::string& hint) : err_(err), hint_(hint) {}

bool damaged(const std::string& what) {
err_.kind = MapErrorKind::Damaged;
Expand Down Expand Up @@ -259,7 +262,8 @@ class Loader {
m.preset_name = p->name;
m.preset_named = true;
} else {
m.preset_name = preset_for_version(version_);
const PresetInfo* h = hint_.empty() ? nullptr : find_preset(hint_);
m.preset_name = h && h->allows(version_) ? std::string(h->name) : preset_for_version(version_);
}
std::string order;
if (!need_string(root, "channel_order", "map", order)) return false;
Expand All @@ -274,9 +278,13 @@ class Loader {
if (!load_source(kv.first, kv.second, m)) return false;
}
if (!k_) {
for (const char* required : {"W", "D", "K"}) {
if (m.find_source(required) < 0) return damaged(std::string("sources: missing source ") + required);
// The sources the map's preset needs (map.h PresetInfo): W, D and K for presets M and L, none by name for FG.
const PresetInfo* p = find_preset(m.preset_name);
if (!p) return damaged("map: no preset");
for (const std::string& required : p->required_sources) {
if (m.find_source(required) < 0) return damaged("sources: missing source " + required);
}
if (m.sources.empty()) return damaged("sources: the map has no source");
} else if (!check_k_sources(m)) {
return false;
}
Expand Down Expand Up @@ -1367,6 +1375,7 @@ class Loader {

private:
MapError& err_;
std::string hint_;
int version_ = 0;
bool k_ = false; // versions 2 and 3 (preset K, spec 48)
};
Expand All @@ -1376,12 +1385,17 @@ class Loader {
const std::vector<PresetInfo>& known_presets() {
// the map versions each preset's maps use (bit v: version v), as they were before maps named their preset
static const std::vector<PresetInfo> presets = {
{"M", (1u << 1) | (1u << 4), kMsgNoPresetM}, // spec 08 (version 4 adds the source F)
{"K", (1u << 2) | (1u << 3), kMsgNoPresetK}, // spec 48
{"F", 1u << 5, kMsgNoPresetF}, // spec 57
{"M", (1u << 1) | (1u << 4), kMsgNoPresetM, {"W", "D", "K"}}, // spec 08 (version 4 adds the source F)
{"K", (1u << 2) | (1u << 3), kMsgNoPresetK, {}}, // spec 48 (its own source rules: T, O)
{"F", 1u << 5, kMsgNoPresetF, {}}, // spec 57 (its own source rules)
// Model L (#89): the same record kinds as preset M's map (no new element format, op or source kind), so the
// same map versions; its parameters are a block of their own in the same library file, found by its own hashes
{"L", (1u << 1) | (1u << 4), kMsgNoPresetL},
{"L", (1u << 1) | (1u << 4), kMsgNoPresetL, {"W", "D", "K"}},
// Frame generation (#96): the weights of its two convolution networks, a fixed set of separate float16 blobs in
// the frame-generation library, each a block source located by its own length and hashes (no keyed source, no
// constants, no dither table). Map version 1 record kinds only; the map names its own sources. The source the key
// is made from is searched first: when it is missing, the file has no frame-generation model (exit 12).
{"FG", 1u << 1, kMsgNoPresetFG, {}},
};
return presets;
}
Expand All @@ -1398,8 +1412,27 @@ std::string preset_for_version(int version) {
return "";
}

namespace {

bool load_map_hinted(const std::string& file_bytes, Map& out, MapError& err, const std::string& hint) {
out = Map();
err = MapError();
out.file_sha256 = Sha256::hash(file_bytes.data(), file_bytes.size());
Json root;
std::string jerr;
if (!parse_json(file_bytes, root, jerr)) {
err.kind = MapErrorKind::Damaged;
err.detail = "not valid JSON: " + jerr;
return false;
}
Loader loader(err, hint);
return loader.load(root, out);
}

} // namespace

bool load_map(const std::string& file_bytes, Map& out, MapError& err, const std::string& expected_preset) {
if (!load_map(file_bytes, out, err)) return false;
if (!load_map_hinted(file_bytes, out, err, expected_preset)) return false;
if (expected_preset.empty()) return true;
auto damaged = [&](const std::string& detail) {
err.kind = MapErrorKind::Damaged;
Expand All @@ -1420,19 +1453,6 @@ bool load_map(const std::string& file_bytes, Map& out, MapError& err, const std:
return true;
}

bool load_map(const std::string& file_bytes, Map& out, MapError& err) {
out = Map();
err = MapError();
out.file_sha256 = Sha256::hash(file_bytes.data(), file_bytes.size());
Json root;
std::string jerr;
if (!parse_json(file_bytes, root, jerr)) {
err.kind = MapErrorKind::Damaged;
err.detail = "not valid JSON: " + jerr;
return false;
}
Loader loader(err);
return loader.load(root, out);
}
bool load_map(const std::string& file_bytes, Map& out, MapError& err) { return load_map_hinted(file_bytes, out, err, ""); }

} // namespace wgc
8 changes: 6 additions & 2 deletions clean/converter/src/map.h
Original file line number Diff line number Diff line change
Expand Up @@ -221,9 +221,12 @@ struct Perm {
// model of that preset. A map names its preset in the optional top-level field "preset"; a map without it gets the preset
// its version had before the field existed (versions 1 and 4: M; 2 and 3: K; 5: F), so the shipped maps load as before.
struct PresetInfo {
const char* name; // "M", "K", "F", "L"
const char* name; // "M", "K", "F", "L", "FG"
unsigned versions; // bit v set: map version v is allowed for this preset
const char* no_model_message; // api.cpp kMsgNoPreset*: the DLL has no model of this preset
// Map versions 1 and 4: the sources a map of this preset must have, by name (M and L: W, D and K). Preset FG names
// none: its sources are the frame-generation library's weight blobs, each with its own name.
std::vector<std::string> required_sources;
bool allows(int version) const { return version >= 0 && version < 32 && (versions >> version & 1u) != 0; }
};
// The known presets, in a fixed order; find_preset: by name (exact, upper case), nullptr when unknown.
Expand Down Expand Up @@ -253,7 +256,8 @@ struct Map {
int find_source(const std::string& name) const;
int find_array(const std::string& name) const;
// The preset the player messages name, as one letter (the first of preset_name; without one, by the version as before:
// 'M' for versions 1 and 4, 'K' for versions 2 and 3, 'F' for 5).
// 'M' for versions 1 and 4, 'K' for versions 2 and 3, 'F' for 5). Not unique: preset FG also gives 'F'; code that
// must tell presets apart uses preset_name or preset_info().
char preset() const {
if (!preset_name.empty()) return preset_name[0];
const std::string p = preset_for_version(version);
Expand Down
112 changes: 112 additions & 0 deletions clean/converter/tests/cases.py
Original file line number Diff line number Diff line change
Expand Up @@ -589,3 +589,115 @@ def flip_bit(data, offset, bit=0):
b = bytearray(data)
b[offset] ^= 1 << bit
return bytes(b)


# ---------------------------------------------------------------- frame generation (#96): preset FG

def _fg_layout():
"""clean/fg/layout.py: the frame-generation weight layout (structure only)."""
import os
import sys
root = os.path.normpath(os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "..", ".."))
if root not in sys.path:
sys.path.insert(0, root)
from clean.fg import layout
return layout


def _small_halves(rng, n):
"""n random finite binary16 patterns with |x| < 2 (exponent field at most 15), fast."""
from array import array
a = array("H", rng.randbytes(2 * n))
a = array("H", (x & 0xBFFF for x in a))
if struct.pack("<H", 1) != b"\x01\x00":
a.byteswap()
return a.tobytes()


FG_NAN = struct.pack("<H", 0x7E00) # a quiet NaN: decoding it is an error, so an area holding it must never be read


def fg_blobs(rng):
"""Our own random blobs in the frame-generation layout (clean/fg/layout.py), in upload order: {name: bytes}.
net1: random weights; the zero area and bias copies 1..15 hold NaN (a converter that reads them fails), copy 0 of
the bias is random. net2: random weight and bias blobs. Nothing comes from a real file."""
L = _fg_layout()
out = {}
for l in L.NET1:
bias = bytearray(FG_NAN * l.bias_elements)
b0 = _small_halves(rng, l.out)
for c in range(l.out):
e = L.net1_bias_index(c)
bias[2 * e: 2 * e + 2] = b0[2 * c: 2 * c + 2]
out[l.name] = _small_halves(rng, l.weight_elements) + FG_NAN * l.weight_elements + bytes(bias)
for l in L.NET2:
out[l.name + "_w"] = _small_halves(rng, l.weight_elements)
out[l.name + "_b"] = _small_halves(rng, l.out)
for b in L.blobs():
assert len(out[b.name]) == b.nbytes
return out


def fg_map(blobs, label="synthetic FG label", key_from="net1_01"):
"""A version-1 map with "preset": "FG" for the blobs of fg_blobs: one block source per blob (map order = upload
order) and one stage per layer. The tensor layout is a stand-in (weights as [out, K] rows, net1's bias as copy 0
of each group of 8, net2's bias as [out]); the spec side's map defines the real names, axis order and stages."""
L = _fg_layout()
m = {
"format": "wronggpu-converter-map",
"version": 1,
"model": "synthetic frame-generation case",
"preset": "FG",
"channel_order": "natural",
"sources": {b.name: source_entry(b.name, "fake frame-generation weight blob", "f16le", blobs[b.name])
for b in L.blobs()},
"key": {"label": label, "from": key_from},
"perms": {},
"stages": [],
}
f16 = {"name": "f16"}
for l in L.NET1:
m["stages"].append({"stage": l.name, "digest": "", "tensors": [
tensor("W", [l.out, l.k], "float32", l.name, f16, 0, [lin(l.k), lin(1)]),
tensor("b", [l.out // L.NET1_BIAS_GROUP, L.NET1_BIAS_GROUP], "float32", l.name, f16, l.bias_offset,
[lin(L.NET1_BIAS_GROUP * L.NET1_BIAS_COPIES), lin(1)]),
]})
for l in L.NET2:
m["stages"].append({"stage": l.name, "digest": "", "tensors": [
tensor("W", [l.out, 9 * l.cin], "float32", l.name + "_w", f16, 0, [lin(9 * l.cin), lin(1)]),
tensor("b", [l.out], "float32", l.name + "_b", f16, 0, [lin(1)]),
]})
return m


def fg_case(seed=41, drop=(), decoy="net2_07_w"):
"""A synthetic frame-generation library: random bytes holding the 67 blobs of fg_blobs in a shuffled order, at
16-aligned, even and odd offsets, with a decoy of one blob (its first 64 bytes, then other bytes) before it.
drop: blob names left out of the file (the map still lists them). Returns the Case (placements: {blob: offset})."""
rng = random.Random(seed)
blobs = fg_blobs(rng)
m = fg_map(blobs)
order = list(blobs)
rng.shuffle(order)
buf = bytearray(rng.randbytes(4096))
placements = {}
for i, name in enumerate(order):
if name == decoy:
fake = bytearray(blobs[name])
fake[-1] ^= 0x01
buf += rng.randbytes(16 - len(buf) % 16) + fake
buf += rng.randbytes(rng.randrange(1, 64))
pad = (16 - len(buf) % 16) % 16 if i % 3 == 0 else (len(buf) % 2 if i % 3 == 1 else 1 - len(buf) % 2)
buf += rng.randbytes(pad)
if name in drop:
continue
placements[name] = len(buf)
buf += blobs[name]
buf += rng.randbytes(4096)
case = Case("fg")
case.file = bytes(buf)
case.map = m
case.sources = blobs
case.expected, case.key = finalize_map(m, blobs)
case.placements = placements
return case
Loading
Loading