Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 19 additions & 10 deletions clean/k/FORMATS.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,24 +113,33 @@ The per-frame runtime runs `KFrame`'s frame on its HIP stream (the interop's Fra
| Step | Kernels | Notes |
|---|---|---|
| input copy (option `input_copy=1`) or the compact input pass (interop default for K) | none: the K kernels read the copies | the game's RGBA16F colour (`KIO_C16`, not divided by π: model K applies κ), R32F depth and FP16 motion of 2 or 4 channels (`KIO_M16`, m = ms · r + mo on load: FP32 multiply then add, as `input_convert` computes it) in their 256-byte pitched rows; `KFrameIn.mch`, `rmsx`, `rmsy`, `rmox`, `rmoy`; the same values as the FP32 slot of the input pass, so the same output bits (GPU test) |
| exposure | `exposure_auto_diet` (`exposure.co`, spec 07, when auto-exposure is asked; `exposure_auto_diet_c16` on the RGBA16F copy), `k_expo_scalars` (`k_io.co`) | the exposure pair `[e, e_prev, last e]` of spec 10 (value, the game's texel on the GPU, or spec 07), then `xs` |
| exposure | `exposure_auto_diet` (`exposure.co`, spec 07, HDR only, when the game asked for auto-exposure at creation or the front end asks this frame; `exposure_auto_diet_c16` on the RGBA16F copy), `k_expo_scalars` (`k_io.co`) | E_cur, E_prev, E_aux of spec 42 draft 3 section 4 (the game's value or texel as E_cur, else 1; the measured value as E_aux), then `xs` |
| preparation | `k_out_color`, then `k_in_tiled` when supported; otherwise `k_in_color` plus `k_in_base`/`k_in_reset` | `KFrameIn.flags` = [HDR] from `WgRtConfig.hdr` (front end `WG_CREATE_HDR`), always [MVlo], [inv], reset |
| network | the eleven `k_stage_*` / `k_stagew<NG>_*` launches of `k_net.py` variant "fused" | window offsets `k_stage_offsets(f, net/window_offsets)`; the G2 d0 also emits hidden history and, by default, the compact pixel head instead of the head map (`k_stage2_d0_hp`, `k_stage2h_d0_hp`) |
| output stage | `k_out_pixel_c` (compact pixel head, default) or `k_out_pixel` (runtime option `k_head=full`) | game output omode 1 (FrameIo.out_half_pitch > 0) or 2 (interop dense RGB) |

`k_expo_scalars(ex, src, xs, KExpoArgs)` (one lane): with `pair` = 1 it first updates the pair as `exposure_pair`
(e = src[0] or `value`, e_prev = e on the first frame, else the previous e); then E_cur = e, E_prev = e_prev (E_cur on
a reset frame), E_aux = E_cur, each rounded to FP16 with 0 → 1, and the FP32 operations of `k_io.scalars()` with
e₁ = e₂ = 1 and `inv_pi` = RN32(1/π) (π: the pre-exposure, 1 if missing, ≤ 0 or not finite). The kernels then read
the same values the host would give for that exposure (bit-identical to `KFrame` given that E_cur).
`KExpoArgs` = {float value; int pair, use_src, first, hdr, reset; float e1, e2, inv_pi} (36 bytes).

State: Hc, L, Hh double-buffered, zero at create; reset on the game's flag, the first frame and a render-size change;
`k_expo_scalars(st, src, measured, xs, KExpoArgs)` (one lane), spec 42 draft 3 section 4 (the host rule of
`clean/k/reference/exposure.py`): [HDR] E_cur = `value` (`src_mode` 0: the front end's exposure value, 1 when the
game gives none), src[0] (`src_mode` 1: the game's exposure texel) or 1 (`src_mode` 2: no exposure from the game,
auto-exposure asked); the auto-exposure value never takes its place. E_aux = measured[0] when auto-exposure ran this
frame (`aux` = 1), else E_cur. E_prev = st[3] (the previous frame's E_cur), E_cur on the first frame and on a reset.
[LDR] all 1. The raw values go to st = [E_cur, E_prev, E_aux, E_cur for the next frame]; then each is rounded to FP16
with 0 → 1, and the FP32 operations of `k_io.scalars()` with e₁ = e₂ = 1 (the front-end interface carries no
exposure scale) and `inv_pi` = RN32(1/π) (π: the pre-exposure, 1 if missing, ≤ 0 or not finite): κ = e₁E_cur/π,
λ = e₂E_aux/(e₁E_cur), ρ = e₁E_cur/(e₁E_prev). With auto-exposure and no texture, E_cur = E_prev = 1 and the measured
value acts only through λ. The kernels then read the same values the host would give for those exposures
(bit-identical to `KFrame` given the same E_cur, E_aux). Runtime option `k_expo_rule=v051` (A/B only): the 0.5.1 rule
(the measured value as E_cur, E_aux = E_cur). `KExpoArgs` = {float value; int src_mode, aux, first, hdr, reset; float
e1, e2, inv_pi} (36 bytes).

State: Hc, L, Hh double-buffered, zero at create; reset on the game's flag and the first frame only (a render-size
change keeps the history: the histories are at output and base resolution; model M's spec 10 size rule is not K's);
frame index f = 0 on a reset frame, else the previous f + 1; `WgRtFrame.reset = 2` (a parked upscaler taken over by
a new context) is a reset that also starts the host frame state over as at create (frame counter, so the first-frame
exposure; f; history parity; last size; jitter), so the frames from it on equal a new runtime's; filter-shape sets of all eight levels uploaded once at
create (the frame takes level `k_filter_level(rw, rh, W, H)`); `exp2tab` allocated and zero (the GPU uses v_exp_f16).
s = ½, base `k_base_size(W, H, ½)`. Buffers for `wg_rt_buffer`: `exposure.pair`, `k.scalars` (xs), `k.x0`, `k.N`
s = ½, base `k_base_size(W, H, ½)`. Buffers for `wg_rt_buffer`: `exposure.pair` ([E_cur, E_prev, E_aux] of the last
frame, raw), `exposure.measured` (the last auto-exposure value), `k.scalars` (xs), `k.x0`, `k.N`
(network output, the 40-channel head map: only with the runtime option `k_head=full`; the default frame writes the
compact pixel head instead, buffer `k.Np`, and asking for `k.N` fails with a message naming the option; `k.Np` in
turn fails with `k_head=full`),
Expand Down
15 changes: 13 additions & 2 deletions clean/k/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,10 @@ one rounding instead of two, and 46 of 3.7 M colour-history pixels differed by 1
### Whole frame: `clean/k/runner/k_frame.py`

`KFrame(dev, func, output_size, max_render_size, params, flags)` keeps the histories (double-buffered) and the frame
index of spec 44 (0 on a reset: the game's flag, the first frame, or a render-size change). `frame(x)` returns the
index of spec 44 (0 on a reset: the game's flag or the first frame; a render-size change keeps the history). The frame
dict gives E_cur, E_aux, e1, e2, or the inputs of the exposure rule of spec 42 draft 3 section 4
(`clean/k/reference/exposure.py`: `game_exposure` (None: the game passes none), `measured` (the auto-exposure value
when it runs), `exposure_scale`), which the per-frame runtime applies on the GPU. `frame(x)` returns the
image, and `time_frame(x)` the GPU ms of the input stage, network, output stage and total. `params` are the
converter's tensors (`k_model.load_raw`) including `net/window_offsets`.

Expand Down Expand Up @@ -184,6 +187,14 @@ Model K ships in the same `wronggpu_runtime.dll` as model M; nothing changes for
compact one), `input_copy=1` (copies instead of the input pass where the formats allow), `timing_log=1`,
`k_diag=input2|copyin` (timing diagnostics only), `k_head=full` (the 40-channel head map between d0 and the
output stage instead of the compact pixel head; needed to read the buffer `k.N`).
The input stage is exact by default: the input-stage kernels from `k_io_exact.co` (every operation correctly
rounded: bit for bit spec 42 and `KFrame(exact_input=True)`); the output colour taps then come from `k_out_color`.
`k_in_exact=1` names the default; `k_in_exact=0` is the same as `k_fast_in=1`, which selects `k_io.co`'s GPU log2 /
2^x / reciprocal estimates instead (not bit-exact, test use). `k_in_exact=1` is refused with `k_fast=1` or
`k_fast_in=1`.
Exposure (spec 42 draft 3 section 4): E_cur = the game's exposure texture or value, 1 if it passes none; the
auto-exposure value (spec 07, run when the game asks for it at creation, HDR only) is E_aux and acts only through
λ. `k_expo_rule=v051` (A/B only) restores the 0.5.1 rule (the measured value as E_cur and E_aux).

In game (Cyberpunk 2077 benchmark, 1440p Quality, RX 9070 XT; the project's measurements): model K
GPU time about 1.8 ms per frame (`fused`; 1.72 ms with `fused16`), the game at ~120 fps against ~117 fps with model M
Expand Down Expand Up @@ -472,7 +483,7 @@ The 1.3 ms objective remains unmet. See

## Reset-frame input path

History resets (first frame, explicit reset, or render-size change) select `k_in_reset`
History resets (first frame or the game's reset flag; not a render-size change) select `k_in_reset`
after the unchanged colour prepass. Spec 42 sections 6.1, 6.5 and 7 require each
slot's channels to be `(current, current, 0, 0)` on a reset. The kernel therefore
reads only the current own-sample channel; it skips history, motion/depth, colour
Expand Down
50 changes: 27 additions & 23 deletions clean/k/kernels/k_io.hip
Original file line number Diff line number Diff line change
Expand Up @@ -1874,45 +1874,49 @@ WG_KERNEL_N(k_in_tiled_color_uniform_f, 256)(KInColorUniformParams P0) {

// Host-side arguments of k_expo_scalars (must match KExpoArgs in clean/k/runtime/runtime_k.cpp and k_io.py).
struct KExpoArgs {
float value; // pair mode: this frame's exposure value (when use_src = 0)
int pair; // 1: update the exposure pair first (as kernels/exposure.hip exposure_pair: e = src[0] or value,
// e_prev = e on the first frame, else the previous e); 0: the pair is already written
// (auto-exposure: exposure_auto_diet)
int use_src, first;
float value; // E_cur when src_mode = 0: the exposure value the game passes
int src_mode; // where E_cur comes from: 0 value, 1 src[0] (the game's exposure texel), 2 none (E_cur = 1)
int aux; // 1: E_aux = measured[0] (spec 07 auto-exposure ran this frame); 0: E_aux = E_cur
int first; // the first frame of the runtime (E_prev = E_cur)
int hdr, reset;
float e1, e2; // exposure factors of spec 42 section 4 (1 unless the game supplies an exposure scale)
float e1, e2; // exposure factors of spec 42 section 4 (the game's exposure scale, else 1)
float inv_pi; // RN32(1 / pi), pi the pre-exposure (host)
};
static_assert(sizeof(KExpoArgs) == 36, "KExpoArgs");

// Spec 42 section 4 / spec 43 section 3.7 from the exposure pair ex = [e, e_prev, last e] (the model M pair buffer of
// the runtime, spec 07 / spec 10 section 3): E_cur = e, E_prev = e_prev (E_cur on a reset), E_aux = E_cur (the
// runtime has one exposure value per frame: spec 07's result, the game's texel or value), each read as FP16 (nearest
// even) with 0 -> 1; then exactly the FP32 operations of clean/k/runner/k_io.py scalars() and out_uniforms()
// (kinv = RN(1/kappa); FP32 division is correctly rounded on the GPU). Writes xs = (kappa, rho, lambda, kinv), which
// the stage kernels read through KFrameIn.xs. One lane.
// Spec 42 draft 3 section 4 / spec 43 section 3.7. Which value is which (HDR): E_cur = the game's exposure texel or
// value, 1 if the game passes none (the auto-exposure value never takes its place); E_aux = the auto-exposure value
// when it runs, else E_cur; E_prev = the previous frame's E_cur, E_cur itself on the first frame and on a reset.
// LDR: every value and factor 1. The state st = [E_cur, E_prev, E_aux, E_cur kept for the next frame] holds the raw
// values; each is then read as FP16 (nearest even) with 0 -> 1, and the scalars are exactly the FP32 operations of
// clean/k/runner/k_io.py scalars() and out_uniforms() (kinv = RN(1/kappa); FP32 division is correctly rounded on the
// GPU). Writes xs = (kappa, rho, lambda, kinv), which the stage kernels read through KFrameIn.xs. One lane.
WG_FN float kx_ev(float e) {
const float h = (float)kh_round(e);
return h == 0.0f ? 1.0f : h;
}
WG_KERNEL_N(k_expo_scalars, 32)(float* ex, const float* src, float* xs, KExpoArgs A) {
WG_KERNEL_N(k_expo_scalars, 32)(float* st, const float* src, const float* measured, float* xs, KExpoArgs A) {
if (wg_lane_id() != 0) return;
if (A.pair) {
const float e = A.use_src ? src[0] : A.value;
const float e_prev = A.first ? e : ex[2];
ex[0] = e;
ex[1] = e_prev;
ex[2] = e;
float Ecr = 1.0f, Epr = 1.0f, Ear = 1.0f;
if (A.hdr) {
Ecr = A.src_mode == 1 ? src[0] : A.src_mode == 0 ? A.value : 1.0f;
Ear = A.aux ? measured[0] : Ecr;
Epr = (A.first || A.reset) ? Ecr : st[3];
}
const float Ec = kx_ev(ex[0]), Ep = kx_ev(A.reset ? ex[0] : ex[1]), Ea = Ec;
const float ce = A.e1 * Ec;
st[0] = Ecr;
st[1] = Epr;
st[2] = Ear;
st[3] = Ecr;
const float e1 = A.hdr ? A.e1 : 1.0f, e2 = A.hdr ? A.e2 : 1.0f;
const float Ec = kx_ev(Ecr), Ep = kx_ev(Epr), Ea = kx_ev(Ear);
const float ce = e1 * Ec;
float kappa = 1.0f, rho = 1.0f;
if (A.hdr) {
kappa = A.inv_pi * ce;
if (kappa == 0.0f) kappa = 1.0f;
rho = ce / (A.e1 * Ep);
rho = ce / (e1 * Ep);
}
const float lam = (A.e2 * Ea) / ce;
const float lam = (e2 * Ea) / ce;
xs[0] = kappa;
xs[1] = rho;
xs[2] = lam;
Expand Down
23 changes: 14 additions & 9 deletions clean/k/runner/k_frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,15 @@
"""Model K, one whole frame on one device: input stage (spec 42) -> the eleven network stages (spec 41 / 44) ->
output stage (spec 43), with the history images kept between frames (spec 42 / 43 inputs Hc, Hh, L, double-buffered)
and the frame index of spec 44 (0 on every reset frame, then +1 per frame; it selects the window offsets of m, e4, d4
from the converted model's table).
from the converted model's table). A reset frame: the game's reset flag or the first frame; a render-size change keeps
the history (spec 44; model M's spec 10 rule of a reset on a size change does not apply to K). E_prev: the previous
frame's E_cur, E_cur on a reset frame (spec 42 draft 3 section 4).

kf = KFrame(dev, func, output_size, max_render_size, params, flags) # flags: hdr, mv_lowres, depth_inverted
out = kf.frame(x) # x: color, motion, depth, render_size, jitter, mv_scale, mv_offset, pre_exposure,
# reset, E_cur, E_aux (FP16 values), e1, e2 -> {"out": [H, W, 4] RGBA float32}
# reset, E_cur, E_aux (FP16 values), e1, e2 -> {"out": [H, W, 4] RGBA float32};
# or instead of E_cur, E_aux, e1, e2 the inputs of the exposure rule
# (clean/k/reference/exposure.py): game_exposure, measured, exposure_scale
ms = kf.time_frame(x) # {"input", "network", "output", "total"}: median GPU ms

reference_frame(state, x, params, flags) runs the same frame with the NumPy references (state: the histories and the
Expand All @@ -17,6 +21,7 @@
import numpy as np

from clean.k.reference import instage, network, outstage
from clean.k.reference.exposure import apply_rule
from clean.k.reference.layout import base_size
from clean.k.reference.outstage import ceil16
from clean.k.runner import k_io
Expand Down Expand Up @@ -93,12 +98,12 @@ def _frame_dict(self, x):
rw, rh = (int(v) for v in x["render_size"])
if rw > self.max_rw or rh > self.max_rh:
raise ValueError(f"render size {(rw, rh)} above the maximum {(self.max_rw, self.max_rh)}")
reset = bool(x.get("reset")) or self.first or self.last_render != (rw, rh)
d = dict(x)
reset = bool(x.get("reset")) or self.first
d = apply_rule(x, self.flags["hdr"])
# E_prev: the previous frame's E_cur, unless the frame gives it (spec 49's test sequences); E_cur on a reset
e_prev = x.get("E_prev") if x.get("E_prev") is not None else self.E_prev
d.update(self.flags, output_size=(self.W, self.H), base=(self.Bw, self.Bh), reset=reset, s=self.s,
E_prev=x["E_cur"] if reset or e_prev is None else e_prev)
E_prev=d["E_cur"] if reset or e_prev is None else e_prev)
return d

def _launches(self, d, full_head=False):
Expand Down Expand Up @@ -272,16 +277,16 @@ def new_reference_state(output_size, base=None):
def reference_frame(state, x, params, flags):
"""The NumPy references for one frame; state (new_reference_state) is updated like KFrame's."""
rw, rh = (int(v) for v in x["render_size"])
reset = bool(x.get("reset")) or state["first"] or state["last_render"] != (rw, rh)
d = dict(x)
reset = bool(x.get("reset")) or state["first"] # the game's flag or the first frame (not a render-size change)
d = apply_rule(x, bool(flags.get("hdr", False)))
d.update({k: bool(flags.get(k, False)) for k in ("hdr", "mv_lowres", "depth_inverted")},
output_size=state["output_size"], base=state["base"], reset=reset,
E_prev=x["E_cur"] if reset or (x.get("E_prev") is None and state["E_prev"] is None)
E_prev=d["E_cur"] if reset or (x.get("E_prev") is None and state["E_prev"] is None)
else (x["E_prev"] if x.get("E_prev") is not None else state["E_prev"]))
f = 0 if reset else state["f"]
x0 = instage.input_stage(dict(d, Hc=state["Hc"], Hh=state["Hh"]))
outs = network.run_network(x0, params, f)
N = outs["d0"]["head"]
o = outstage.output_stage(dict(d, N=N, L=state["L"], Hc=state["Hc"]))
state.update(Hc=o["Hc"], L=o["L"], Hh=o["Hh"], f=f + 1, first=False, E_prev=x["E_cur"], last_render=(rw, rh))
state.update(Hc=o["Hc"], L=o["L"], Hh=o["Hh"], f=f + 1, first=False, E_prev=d["E_cur"], last_render=(rw, rh))
return {"out": o["out"], "x0": x0, "N": N, "f": f, "reset": reset}
2 changes: 1 addition & 1 deletion clean/k/runner/k_io.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ class KInColorParams(ctypes.Structure):

class KExpoArgs(ctypes.Structure):
"""Host arguments of k_expo_scalars (the per-frame runtime's GPU exposure scalars, clean/k/runtime/runtime_k.cpp)."""
_fields_ = [("value", ctypes.c_float)] + [(n, ctypes.c_int) for n in ("pair", "use_src", "first", "hdr", "reset")] + \
_fields_ = [("value", ctypes.c_float)] + [(n, ctypes.c_int) for n in ("src_mode", "aux", "first", "hdr", "reset")] + \
[(n, ctypes.c_float) for n in ("e1", "e2", "inv_pi")]


Expand Down
Loading
Loading