Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions clean/converter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,17 +58,17 @@ the key is made from is searched first: when it is missing, the file has no fram
message); any other missing blob is exit 13. A map without the field is still read as preset M (so it must have W, D
and K) unless the caller names the preset (`--preset FG`).

Its map is `data/fg.v1.map.json` (the spec side's v1; 67 sources `A01`..`A17`, `B01w`/`B01b`..`B25w`/`B25b`; 42
stages `dl1.l01`..`dl2.l25`): `wronggpu-convert --map data\fg.v1.map.json --dll <frame-generation library> --out <dir>`.
It uses two map features of its own:
Its map is `data/fg.v1.map.json` (the spec side's revision v1.1; 67 sources `A01`..`A17`, `B01w`/`B01b`..`B25w`/`B25b`;
42 stages `dl1.l01`..`dl2.l25`): `wronggpu-convert --map data\fg.v1.map.json --dll <frame-generation library> --out <dir>`.
Each stage has `W` in natural order ([out, in, 3, 3] or [out, in]) and `b` [out]. It uses a map feature of its own:

- `"channel_order": "storage"`: the weight tensors (`W_storage`) are the blobs' weight spans as stored, 1-D, not yet
reshaped to natural axes (that follows in a later map revision). Accepted only in a version-1 map of preset FG; the
converter reads the tensors the same way either way.
- An axis given by **explicit offsets** (`{"offsets": [o0, o1, ...]}`, one per index, in place of `stride` and `perm`;
map versions 1 and 4): the axis contributes `offsets[i]` to the element index. The net1 biases use it to take copy 0
of each group of eight. The index range is checked when the map is loaded (smallest and largest offset); a list of
the wrong length, a non-integer, or `offsets` together with `stride` or `perm` is "converter data damaged".
map versions 1 and 4): the axis contributes `offsets[i]` to the element index. Every weight axis of the FG map uses
it, and the net1 biases use it to take copy 0 of each group of eight. The index range is checked when the map is
loaded (smallest and largest offset); a list of the wrong length, a non-integer, or `offsets` together with `stride`
or `perm` is "converter data damaged".
- `"channel_order": "storage"` (the form of the map's first revision: weight tensors as the stored spans, 1-D) is
still accepted, only in a version-1 map of preset FG; the converter reads the tensors the same way either way.

`tests/run_tests.py` pins the map's SHA-256 (`FG_MAP_SHA256`), checks its sources and stages against
[`clean/fg/layout.py`](../fg/layout.py), and converts a synthetic library with our own random blobs in exactly the map's
Expand Down
63 changes: 41 additions & 22 deletions clean/converter/tests/run_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -407,40 +407,46 @@ def test_l_shaped(r, m_map_path):


MSG_NO_PRESET_FG = "This DLSS Frame Generation file has no frame-generation model. Version 310.1 or newer is needed."
# data/fg.v1.map.json as the spec side delivered it (v1, 2026-11-14): its bytes must not change without a new digest here
FG_MAP_SHA256 = "6dbdc84945817affcbbd8e50bdc1ea6976636137e5c4edd64b5481cc0d51a940"
# data/fg.v1.map.json as the spec side delivered it (revision v1.1, natural order, 2026-11-14): its bytes must not change
# without a new digest here
FG_MAP_SHA256 = "b32d9195ab76e14f9c4e318bc55e5d8def83698e0a10d7581689fedf6e1c7ad5"


def test_fg_real_map(r, fgmap):
"""Frame generation (#96): the delivered map (data/fg.v1.map.json, the spec side's v1: map version 1, "preset": "FG",
channel_order "storage", 67 block sources A01..A17, B01w/B01b..B25w/B25b, 42 stages, bias reads by explicit offsets).
Its bytes are pinned by digest; its layout is the one of clean/fg/layout.py; it validates, and a file without the
blobs gives preset FG's message."""
"""Frame generation (#96): the delivered map (data/fg.v1.map.json, the spec side's revision v1.1: map version 1,
"preset": "FG", channel_order "natural", 67 block sources A01..A17, B01w/B01b..B25w/B25b, 42 stages; every weight
tensor W in natural order [out, in, 3, 3] or [out, in] by explicit offsets; net1 biases by offsets, net2 biases by
stride). Its bytes are pinned by digest; its layout is the one of clean/fg/layout.py; it validates, and a file without
the blobs gives preset FG's message."""
with open(fgmap, "rb") as f:
raw = f.read()
r.check("FG map: file digest", hashlib.sha256(raw).hexdigest() == FG_MAP_SHA256, hashlib.sha256(raw).hexdigest())
m = json.loads(raw)
L = cases._fg_layout()
names = list(m["sources"])
r.check("FG map: version 1, preset FG, channel_order storage, key from A01, no perms",
(m["version"], m["preset"], m["channel_order"], m["key"]["from"], m["perms"]) == (1, "FG", "storage", "A01", {}))
r.check("FG map: version 1, preset FG, channel_order natural, key from A01, no perms",
(m["version"], m["preset"], m["channel_order"], m["key"]["from"], m["perms"]) == (1, "FG", "natural", "A01", {}))
r.check("FG map: 67 sources in upload order with the layout's lengths",
names == ["A%02d" % i for i in range(1, 18)] + ["B%02d%s" % (i, x) for i in range(1, 26) for x in "wb"] and
[m["sources"][n]["length"] for n in names] == [b.nbytes for b in L.blobs()])
stages = m["stages"]
r.check("FG map: 42 stages dl1.l01..l17, dl2.l01..l25",
[s["stage"] for s in stages] == ["dl1.l%02d" % i for i in range(1, 18)] + ["dl2.l%02d" % i for i in range(1, 26)])
ok1 = all([t["name"] for t in s["tensors"]] == ["W_storage", "b"] and
s["tensors"][0]["shape"] == [l.weight_elements] and s["tensors"][0]["base"] == 0 and
def offsets_only(t):
return all(set(a) == {"offsets"} for a in t["axes"])
ok1 = all([t["name"] for t in s["tensors"]] == ["W", "b"] and
s["tensors"][0]["shape"] == ([l.out, l.cin, 3, 3] if l.taps == 9 else [l.out, l.cin]) and
s["tensors"][0]["base"] == 0 and offsets_only(s["tensors"][0]) and
s["tensors"][1]["base"] == l.bias_offset and
s["tensors"][1]["axes"] == [{"offsets": [L.net1_bias_index(c) for c in range(l.out)]}]
for s, l in zip(stages[:17], L.NET1))
r.check("FG map: net1 stages read the weight span and copy 0 of each bias group, skipping the zero area", ok1)
ok2 = all([t["name"] for t in s["tensors"]] == ["W_storage", "b_storage"] and
r.check("FG map: net1 stages read natural weights and copy 0 of each bias group, skipping the zero area", ok1)
ok2 = all([t["name"] for t in s["tensors"]] == ["W", "b"] and
[t["src"] for t in s["tensors"]] == ["B%02dw" % l.index, "B%02db" % l.index] and
s["tensors"][0]["shape"] == [l.weight_elements] and s["tensors"][1]["shape"] == [l.out]
s["tensors"][0]["shape"] == [l.out, l.cin_used, 3, 3] and offsets_only(s["tensors"][0]) and
s["tensors"][1]["shape"] == [l.out] and s["tensors"][1]["axes"] == [{"stride": 1}]
for s, l in zip(stages[17:], L.NET2))
r.check("FG map: net2 stages read the whole weight and bias blobs", ok2)
r.check("FG map: net2 stages read natural weights (layer 1: its 10 used inputs) and the whole bias blob", ok2)
fake = r.write("fg-real/random.dll", random_bytes_for_test(1 << 20))
code, so, se = r.run(fgmap, [fake], r.path("fg-real", "out", "x")[:-2])
r.check("FG map: validates; a file without the blobs -> exit 12, preset FG's message",
Expand Down Expand Up @@ -480,13 +486,21 @@ def test_fg(r, fgmap):
del plain["preset"]
pp = r.write("fg/map-no-field.json", plain)
code, so, se = r.run(pp, [dll], r.path("fg", "out-fail", "x")[:-2])
r.check("FG map without its field and without --preset: read as preset M, whose maps are natural -> damaged",
code == EXIT["map_damaged"] and MSG["map_damaged"] in so and "channel_order" in se, so + se[-300:])
natural = copy.deepcopy(plain)
natural["channel_order"] = "natural"
code, so, se = r.run(r.write("fg/map-no-field-natural.json", natural), [dll], r.path("fg", "out-fail", "x")[:-2])
r.check("...and as natural, preset M's sources W, D and K are missing -> damaged",
code == EXIT["map_damaged"] and "missing source W" in se, so + se[-300:])
r.check("FG map without its field and without --preset: read as preset M, W, D and K missing -> damaged",
code == EXIT["map_damaged"] and MSG["map_damaged"] in so and "missing source W" in se, so + se[-300:])
storage = copy.deepcopy(plain)
storage["channel_order"] = "storage"
code, so, se = r.run(r.write("fg/map-no-field-storage.json", storage), [dll], r.path("fg", "out-fail", "x")[:-2])
r.check("...and with channel_order storage, which only preset FG may use -> damaged",
code == EXIT["map_damaged"] and "channel_order" in se, so + se[-300:])
named_storage = copy.deepcopy(case.map)
named_storage["channel_order"] = "storage" # the v1 form: names what the tensors hold, read the same way
nsp = r.write("fg/map-storage.json", named_storage)
out3 = r.path("fg", "out-storage", "x")[:-2]
code, so, se = r.run(nsp, [dll], out3)
r.check("FG map with channel_order storage (map v1's form): converts the same tensors", code == 0, so + se[-300:])
with open(nsp, "rb") as f:
check_output(r, "FG storage order", case, out3, f.read())
out2 = r.path("fg", "out-hint", "x")[:-2]
code, so, se = r.run(pp, [dll], out2, extra=["--preset", "FG"])
r.check("FG map without its field, --preset FG: converts", code == 0 and "preset FG" in se, so + se[-300:])
Expand Down Expand Up @@ -514,7 +528,12 @@ def test_fg(r, fgmap):

# a map that reads net1's zero area (or a bias copy other than 0) meets the test's NaN: exit 22. This shows the
# success case above really skipped them.
for label, fn in (("zero area", lambda t: t.update(base=t["shape"][0])),
def zero_area_base(t): # net1 blob: the zero area starts right after the out * K weights
n = 1
for d in t["shape"]:
n *= d
t.update(base=n)
for label, fn in (("zero area", zero_area_base),
("bias copy 1", lambda t: t.update(base=t["base"] + 8))):
bad = copy.deepcopy(case.map)
st = bad["stages"][0]
Expand Down
42 changes: 22 additions & 20 deletions clean/fg/FORMATS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,21 +17,23 @@ delivered by the spec side; the code never guesses it (it raises, or takes the c
- Splitting a blob into its parts: [`reference/blobs.py`](reference/blobs.py). net1: the weights in stored order, the
zero area checked to hold only +0 / -0, the bias as copy 0 of each group of eight with the other fifteen copies
checked to be bit-identical (`check=False` skips both checks). net2: weight and bias blob sizes checked. The weights
stay flat until the in-blob axis order (OPEN item 2) is delivered.
come out flat in stored order; [`reference/natural.py`](reference/natural.py) puts them in natural order with the
map's own offsets.
- net1's blob 2 is a 1 x 1 convolution (32 -> 64). net2's first layer stores 16 input channels and reads 10: its
natural weights are [32, 10, 3, 3].

## Converter map

- Map version 1 with `"preset": "FG"` (converter README, "Preset FG"): one block source per blob, located by length
and hashes, `f16` reads. net1's zero area is skipped by the tensor's base; its bias is read as copy 0 of each group of
eight (`layout.net1_bias_index`).
- The map is `data/fg.v1.map.json` (the spec side's v1; digest pinned in `tests/test_fg_map.py` and the converter's
tests): sources `A01`..`A17` (net1) and `B01w`/`B01b`..`B25w`/`B25b` (net2), stages `dl1.l01`..`dl1.l17` and
`dl2.l01`..`dl2.l25`, in upload order, exactly the blobs and spans of `layout.py`. Per stage: `W_storage` (the
weight span as stored, 1-D) and `b` (net1: copy 0 of each bias group, natural order) or `b_storage` (net2: the bias
blob as stored).
- `"channel_order": "storage"`: the weights are not yet in natural axes. The natural view comes with a later map
revision (v1.1), with the same shapes and spans; whether net2's stored bias order is the natural channel order is
confirmed then too (OPEN until it is).
- The map is `data/fg.v1.map.json`, the spec side's revision v1.1 (digest pinned in `tests/test_fg_map.py` and the
converter's tests): sources `A01`..`A17` (net1) and `B01w`/`B01b`..`B25w`/`B25b` (net2), stages `dl1.l01`..`dl1.l17`
and `dl2.l01`..`dl2.l25`, in upload order, exactly the blobs and spans of `layout.py`. `"channel_order": "natural"`.
Per stage: `W` in natural order, [out, in, 3, 3] for 3 x 3 layers and [out, in] for 1 x 1 layers, every axis by
explicit offsets into the blob; `b` [out] (net1: copy 0 of each bias group; net2: the bias blob, natural order).
- The first revision (v1) gave the weights as stored (`"channel_order": "storage"`, 1-D `W_storage`); the converter
still accepts that form for preset FG.
- A conversion of a real library is checked by digests only (the converter compares every stage digest the map
gives); no value or byte of it is ever written into the repository or any public text. `tools/real_check.py` runs
that check and keeps only the verdict and digests (`python -m clean.fg.tools.real_check --exe <converter> --dll
Expand All @@ -40,7 +42,8 @@ delivered by the spec side; the code never guesses it (it raises, or takes the c
## CPU reference (`reference/ops.py`)

- Tensors are channel-first `x[c, y, x]`; weights use the module's own canonical order `w[out, ky, kx, in]`
(`weight_shape(layer)` for every layout layer). Converting a blob into that order waits for item 2 below.
(`weight_shape(layer)` for every layout layer). `natural.py` turns the map's natural `W` ([out, in, ky, kx] or
[out, in]) into that order (`to_canonical`), and stored blob weights into natural ones with the map's offsets.
- Settled and implemented: float16 operands and outputs; convolution with bias, one float16 rounding of the output,
then ReLU; 2x pooling over 2 x 2 blocks; bilinear 2x upscale in the align-corners form (output sample i reads input
position i (n - 1) / (2n - 1)); element-wise add and channel concatenation; net1's features: luma with fixed RGB
Expand All @@ -56,17 +59,16 @@ delivered by the spec side; the code never guesses it (it raises, or takes the c

## Forward pass (`reference/forward.py`)

- Parameters per layer (layout names `net1_01`..`net2_25`): storage-order weights and the bias, from blobs
(`params_from_blobs`) or from the converter's `--raw` output of the preset-FG map (`params_from_raw`, stages
`dl1.lNN` / `dl2.lNN`; every value checked to be exact in float16).
- Parameters per layer (layout names `net1_01`..`net2_25`): canonical natural weights `[out, k, k, in]` and the bias,
from blobs (`params_from_blobs`: stored weights gathered with the map's offsets) or from the converter's `--raw`
output of the preset-FG map (`params_from_raw`, stages `dl1.lNN` / `dl2.lNN`, tensors `W` and `b`; every value
checked to be exact in float16).
- `run_layer`: one convolution with `ops.conv2d`. `run_graph`: a network as a list of steps (`conv` with its layer and
ReLU flag, `pool` with its kind, `upscale`, `add`, `concat`, `slice`), checked before it runs (known steps, exact
fields, buffers written once before they are read, each layer used at most once). `net1_forward` adds the 2 x 2
fold before and the unfold (16 values per cell to 4 per pixel) after; `net2_forward` returns the named buffers.
- Required arguments (OPEN): `wiring` (the steps); `weight_view` (storage to natural `[out, k, k, in]`: map v1 holds
the weights as stored, the natural view comes with v1.1; `view_by_axes` covers a plain axis reorder); `pad`; `blob2`
("1x1" runs net1's blob 2 as a 1 x 1 convolution, "2x2" raises `OpenItem`); `fold_order`. The real wiring plugs in
as data; the layers do not change.
- Required arguments (OPEN): `wiring` (the steps), `pad`, `fold_order`. The weight order and net1's blob 2 are settled
by map v1.1 and are no longer arguments. The real wiring plugs in as data; the layers do not change.

## Configuration (`reference/config.py`)

Expand All @@ -81,10 +83,10 @@ delivered by the spec side; the code never guesses it (it raises, or takes the c

## OPEN (waiting for the spec side)

1. The converter map's natural-order revision (v1.1). v1 is delivered (`data/fg.v1.map.json`, weights as stored).
2. The in-blob axis order of the weights (tap, input channel, output channel, any tiling), for both networks.
1. (closed: map v1.1, natural order, is `data/fg.v1.map.json`.)
2. (closed: the in-blob axis order is the map's offsets.)
3. Which launch reads which blob (upload order is known; launch order is not).
4. net1's blob 2: a 1 x 1 convolution or a 2 x 2 one.
4. (closed: net1's blob 2 is a 1 x 1 convolution.)
5. Both networks' grid sizes relative to the output size.
6. What the UI-recomposition preset changes in the stage order.
7. Launch order, the motion-field scatter, the pull pyramid and the UI handling (part 3).
Expand Down
6 changes: 4 additions & 2 deletions clean/fg/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,12 @@ front end or the launcher yet. Status, open items and formats: [FORMATS.md](FORM
| `tests/test_fg_layout.py` | the table against the totals the spec side gives |
| `reference/ops.py` | NumPy CPU reference of the networks' building blocks: float16 convolution (3 x 3, 1 x 1) with bias and ReLU, 2x pooling, bilinear 2x upscale (align corners), element-wise combines, net1's input features and 2 x 2 fold; each open behaviour is a required argument or raises `OpenItem` |
| `tests/test_fg_reference.py` | the building blocks against plain loop versions, on synthetic data |
| `reference/forward.py` | both networks' forward pass: each layer from its stage's parameters (from blobs or the converter's raw output), and a network run from a wiring the caller supplies (conv / pool / upscale / add / concat / slice steps); the wiring, the natural weight order, the border rule and net1's blob 2 are required arguments |
| `reference/forward.py` | both networks' forward pass: each layer from its stage's natural parameters (from blobs or the converter's raw output), and a network run from a wiring the caller supplies (conv / pool / upscale / add / concat / slice steps); the wiring, the border rule and the fold order are required arguments |
| `tests/test_fg_forward.py` | every layer of both networks against a direct NumPy convolution on synthetic weights; each step kind; wiring checks; stand-in wirings against layers composed by hand; parameters from blobs and from raw files |
| `../converter` (preset FG) | the converter reads a version-1 map with `"preset": "FG"`: one block source per weight blob; see the converter's README, "Preset FG" |
| `../../data/fg.v1.map.json` | the spec side's converter map v1 (weights as stored); `tests/test_fg_map.py` pins its digest and checks it against `layout.py` |
| `../../data/fg.v1.map.json` | the spec side's converter map, revision v1.1 (weights in natural order); `tests/test_fg_map.py` pins its digest and checks it against `layout.py` |
| `reference/natural.py` | the natural weight order from the map itself: its offsets as gather indices (stored blob weights to natural), natural to canonical `[out, k, k, in]` |
| `tests/test_fg_natural.py` | natural weights scattered into synthetic blobs with the map's gathers come back unchanged, for every layer; the gathers cover each weight span |
| `tools/real_check.py` | runs the converter once on one given frame-generation library (execute-only) and records only the verdict, the map's digest, the file version and size, the matching stage digests and the converted file's digest; deletes the converted file. Offsets and bytes of the library are never recorded |
| `tests/test_fg_real_check.py` | the tool on made-up converter output and a stand-in converter |
| `reference/config.py` | the settled per-call configuration: the multi-frame intermediate time (one single-precision division), the index checks and flags, UI recomposition (FG preset A / B) with the profile values that force it |
Expand Down
Loading
Loading