Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions clean/dx12/kernel_sets.json
Original file line number Diff line number Diff line change
Expand Up @@ -428,10 +428,18 @@
"k_out_color",
"k_io.co"
],
[
"k_out_color__a2",
"k_io.co"
],
[
"k_out_color_direct",
"k_io.co"
],
[
"k_out_color_direct__a2",
"k_io.co"
],
[
"k_out_pixel",
"k_io.co"
Expand All @@ -440,10 +448,18 @@
"k_out_pixel_c",
"k_io.co"
],
[
"k_out_pixel_c__a2",
"k_io.co"
],
[
"k_out_pixel_c_direct",
"k_io.co"
],
[
"k_out_pixel_c_direct__a2",
"k_io.co"
],
[
"k_out_pixel_direct",
"k_io.co"
Expand Down Expand Up @@ -584,13 +600,25 @@
"kx/k_in_tiled",
"k_io_exact.co"
],
[
"kx/k_in_tiled__a1",
"k_io_exact.co"
],
[
"kx/k_in_tiled_direct",
"k_io_exact.co"
],
[
"kx/k_in_tiled_direct__a1",
"k_io_exact.co"
],
[
"kx/k_in_tiled_uniform",
"k_io_exact.co"
],
[
"kx/k_in_tiled_uniform__a1",
"k_io_exact.co"
]
]
}
5 changes: 5 additions & 0 deletions clean/k/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,11 @@ Model K ships in the same `wronggpu_runtime.dll` as model M; nothing changes for
`k_in_exact=1` names the default; `k_in_exact=0` is the same as `k_fast_in=1`, which selects `k_io.co`'s GPU log2 /
2^x / reciprocal estimates instead (not bit-exact, test use). `k_in_exact=1` is refused with `k_fast=1` or
`k_fast_in=1`.
`k_approx=a1|a2|a1,a2` (native D3D12 device only; opt-in, not bit-exact, judged by image quality against the exact
path; `clean/k/dx12/README.md`): `a1` the tiled input stage's transcendental unit forms, fast-close where the stage
supports it: a frame on the legacy input path (a render size above 0.68 x the output, a jitter outside [-0.5, 0.5],
`input_tiled=0`) runs the exact input stage, and the first such frame of a context logs one line; `a2` the output
stage's division forms (needs the compact head).
Exposure (spec 42 draft 3 section 4): E_cur = the game's exposure texture or value, 1 if it passes none; the
auto-exposure value (spec 07, run when the game asks for it at creation, HDR only) is E_aux and acts only through
λ. `k_expo_rule=v051` (A/B only) restores the 0.5.1 rule (the measured value as E_cur and E_aux).
Expand Down
25 changes: 25 additions & 0 deletions clean/k/dx12/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,3 +112,28 @@ identical.

Known limit, shared with Model M's ports: D3D12 flushes FP32 denormals (inputs and results) to zero and HIP keeps them.
`k_expo_scalars` only meets one with a pre-exposure above about 8.5e37, so the check's inputs keep clear of them.

## Close approximations (runtime option `k_approx`, opt-in)

Every port above is exact, and nothing set is the exact path. `k_approx` selects close approximations of two stages on
the native D3D12 device (HIP and the hybrid device refuse the option); they are judged by image quality against the
exact path, not byte for byte, and taking one adds a note of create.

- `a1`, the tiled input stage (`kx/k_in_tiled*__a1`, `hlsl/k_in_common.hlsli` `KI_HWMATH`): its correctly rounded log2,
2^x and divisions (FP32 double-word arithmetic with a Ziv test and a binary64 fallback) become `v_log_f32`,
`v_exp_f32` (about 1 FP32 ulp) and `v_rcp_f32` times the dividend (scaled for |b| >= 2^126). Fast-close where the
stage supports it: frames that take the legacy input kernels (a render size above 0.68 x the output, a jitter outside
[-0.5, 0.5], `input_tiled=0`) run the exact input stage (never refused: exact is always within quality); the first
such frame of a context logs one line (`wg_rt_take_frame_note`, logged by the D3D12 executor).
- `a2`, the output stage (`k_out_color*__a2`, `k_out_pixel_c*__a2`, `hlsl/k_out.hlsli` `KO_APPROX`): its exact divisions
become `v_rcp_f32` times the dividend plus one correction of the quotient from its exact residual, and two refined
reciprocals drop their Newton step; every opaque barrier stays. It needs the compact head (`k_head=compact`, the
default): the full head's `k_out_pixel` has no `a2` form.

`k_approx=a1,a2` takes both. Create refuses, in one line, an empty value or list item, an unknown tag, `a1` without the
exact input kernels, and `a2` with `k_fast=1` or `k_head=full`. It prepares the `__a1` / `__a2` forms (and checks their
manifest digests) only when they are selected, and the D3D12 executor runs no warm-up of its own for model K, so an
unusable approximate shader cannot affect a context that does not select it. `test_k_runtime_d3d12.py`
(`test_k_approx_*`, `test_native_k_approx_frames`) checks the refusals, the notes, the per-frame kernel selection and
loose closeness bounds on the synthetic model; `test_k_approx_executor.py` checks the executor with a changed
approximate shader, selected and not.
22 changes: 22 additions & 0 deletions clean/k/dx12/hlsl/k_in_common.hlsli
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,32 @@
#pragma once

// ---- the exact input stage's arithmetic (k_io.hip with KI_FAST = 0) --------------------------------------------------
#ifndef KI_HWMATH
#define KI_HWMATH 0
#endif
#if KI_HWMATH
// CLOSE-APPROXIMATION variant (opt-in, never the default): the transcendental unit instead of the correctly rounded
// sequences: v_log_f32 / v_exp_f32 (about 1 FP32 ulp), v_rcp_f32 and a product for the divisions. The results meet the
// input stage's FP16 roundings, so they differ from the exact form only where an FP32 result lies within about one
// FP32 ulp of an FP16 rounding midpoint.
float ki_log2(float x) { return log2(x); }
float ki_exp2(float x) { return exp2(x); }
float ki_rcp(float x) { return rcp(x); }
// a rcp(b), branch free. For |b| >= 2^126, 1 / b would flush to zero (e.g. the score's g / nu for a colour range near
// FLT_MAX, where the quotient is about 1/8): there b is scaled by s = 2^-64 first and the product scaled back,
// (a rcp(b s)) s, which cannot overflow (|a / b| <= 2^2 there) and flushes only where a / b itself is below 2^-126.
// s = 1 elsewhere (the same value as a rcp(b)); b = 0, +-inf and NaN give the IEEE quotient's special value.
float ki_div(float a, float b)
{
const float s = abs(b) < 8.5070592e+37f ? 1.0f : 5.4210109e-20f; // 2^126, 2^-64
return (a * rcp(b * s)) * s;
}
#else
float ki_log2(float x) { return kf_log2(x); }
float ki_exp2(float x) { return kf_exp2(x); }
float ki_rcp(float x) { return kf_div(1.0f, x); }
float ki_div(float a, float b) { return kf_div(a, b); }
#endif
// spec 42 decode factor D(t)
float ki_decode(float t)
{
Expand Down
28 changes: 28 additions & 0 deletions clean/k/dx12/hlsl/k_out.hlsli
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,16 @@ WG_ARGS_END
// opaque zero (the launch's workgroup count is below 2^31): x ^ ko_z is x, but the D3D12 shader compiler cannot see
// through it. Used where the driver was seen to regroup chains of products by a shared / uniform factor or to fuse
// FP16 multiply + add (wg_math.hlsli "DRIVER EXCEPTIONS"); HIP keeps every such product as written (contract off).
#ifndef KO_APPROX
#define KO_APPROX 0
#endif
// KO_APPROX 1: CLOSE-APPROXIMATION variant a2 (runtime option k_approx=a2; opt-in, never the default): ko_div is
// v_rcp_f32 times the dividend plus one correction of the quotient from its exact residual (q + r (a - b q), two fma)
// instead of the exact division, and ko_div_axis and ko_rcp_refined drop their Newton step. ko_rcp_output_bounded and
// every opaque barrier (kof, koh, kp_opq, the conversion input) stay: without them, measured, one-FP16-ulp differences
// of mid-range intermediates reach outputs near zero (a few at 4 ulp). Not byte-identical; measured on the port-check
// output family (6 seeds, plain and special-value inputs): every FP16 output within 1 ulp of exact, about 1e-5 of the
// values differ.
#define ko_z (wg_grid_ >> 31)
float kof(float x) { return asfloat(asuint(x) ^ ko_z); }
double kod(double x) { uint lo, hi; asuint(x, lo, hi); return asdouble(lo ^ ko_z, hi); }
Expand Down Expand Up @@ -139,6 +149,17 @@ half kh_mul(half a, half b) { return koh(a * b); }
// scaled back by the exact power of two 2^(127 - eb); every later step is then wg_div's step times a power of two with
// all values normal (quotient within 2^+-71, residuals >= 2^-82), so each rounds to the scaled bits of wg_div's result,
// and wg_div's exponent reassembly gives that same value. Other operands (zero, inf, NaN, far exponents): wg_div.
#if KO_APPROX
// a2: a rcp(b), branch free (|b| >= 2^126 scaled by 2^-64 first, see k_in_common.hlsli ki_div), plus one correction of
// the quotient from its exact residual: within about half an FP32 ulp
float ko_div(float a, float b)
{
const float s = abs(b) < 8.5070592e+37f ? 1.0f : 5.4210109e-20f;
const float bs = b * s, r = rcp(bs);
const float q = a * r;
return mad(mad(-bs, q, a), r, q) * s;
}
#else
float ko_div(float a, float b)
{
const uint ua = asuint(a), ub = asuint(b);
Expand All @@ -153,6 +174,7 @@ float ko_div(float a, float b)
[branch] if (ea - 92u > 70u || eb - 92u > 70u) q = wg_div(a, b);
return q;
}
#endif
half kh_div(half a, half b) { return kh_round(ko_div((float)a, (float)b)); }

// ---- binary64 log / 2^x (k_fp.h kd_*) ---------------------------------------------------------------------------------
Expand Down Expand Up @@ -433,6 +455,9 @@ void ko_color_tap(float4 c, out uint2 tap, out uint tT)
// beta quotient of the axis coefficients (ko_div_axis: v_rcp_f32, one Newton step, residual correction)
float ko_div_axis(float a, float b)
{
#if KO_APPROX // a2: without the Newton step
return a * rcp(b);
#endif
float r = rcp(b);
r = mad(mad(-b, r, 1.0f), r, r);
const float q = a * r;
Expand Down Expand Up @@ -496,6 +521,9 @@ float ko_cubic_unit(float Am1, float A0, float A1, float A2, KoAxis a)
// the IEEE division elsewhere
float ko_rcp_refined(float x)
{
#if KO_APPROX // a2: without the Newton step
return rcp(x);
#endif
[branch] if (x >= asfloat(0x2f800000u) && x <= asfloat(0x4f800000u)) {
const float r = rcp(x);
return mad(mad(-x, r, 1.0f), r, r);
Expand Down
5 changes: 5 additions & 0 deletions clean/k/dx12/kernels/k_in.hlsl
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,11 @@
// @wg-kernel kx/k_in_tiled_uniform co=k_io_exact.co block=256 wave=64 args=s248:p4.b128.p5.b48 defines=K_UNIFORM=1
// @wg-kernel kx/k_in_tiled co=k_io_exact.co block=256 wave=64 args=s200:p4.b128.p5
// @wg-kernel kx/k_in_tiled_direct co=k_io_exact.co entry=k_in_tiled block=256 wave=64 args=s200:p4.b128.p5 rs=direct defines=WG_DIRECT_TEXTURES=1
// The __a1 forms: the opt-in close approximation a1 (runtime option k_approx=a1, never the default): log2 / 2^x /
// divisions on the transcendental unit (hlsl/k_in_common.hlsli KI_HWMATH); not byte-identical to HIP.
// @wg-kernel kx/k_in_tiled_uniform__a1 co=k_io_exact.co entry=k_in_tiled_uniform block=256 wave=64 args=s248:p4.b128.p5.b48 defines=K_UNIFORM=1,KI_HWMATH=1
// @wg-kernel kx/k_in_tiled__a1 co=k_io_exact.co entry=k_in_tiled block=256 wave=64 args=s200:p4.b128.p5 defines=KI_HWMATH=1
// @wg-kernel kx/k_in_tiled_direct__a1 co=k_io_exact.co entry=k_in_tiled block=256 wave=64 args=s200:p4.b128.p5 rs=direct defines=WG_DIRECT_TEXTURES=1,KI_HWMATH=1
#include "wg_platform.hlsli"
#ifndef K_UNIFORM
#define K_UNIFORM 0
Expand Down
7 changes: 7 additions & 0 deletions clean/k/dx12/kernels/k_out.hlsl
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,13 @@
// @wg-kernel k_out_color_direct co=k_io.co entry=k_out_color block=128 wave=64 args=s312:p4.b12.x24.b52.x12.b24.x4.p4.x8.p4.b8.p2.b56 rs=direct defines=WG_DIRECT_TEXTURES=1
// @wg-kernel k_out_pixel_c_direct co=k_io.co entry=k_out_pixel_c block=128 wave=64 args=s312:p4.b12.x24.b52.x12.b24.x4.p4.x8.p4.b8.p2.b56 rs=direct defines=KO_COMPACT=1,WG_DIRECT_TEXTURES=1
// @wg-kernel k_out_pixel_direct co=k_io.co entry=k_out_pixel_c block=128 wave=64 args=s312:p4.b12.x24.b52.x12.b24.x4.p4.x8.p4.b8.p2.b56 rs=direct defines=KO_COMPACT=0,WG_DIRECT_TEXTURES=1
// The __a2 forms: the opt-in close approximation a2 (runtime option k_approx=a2, never the default): divisions on
// v_rcp_f32 with one quotient correction, two reciprocals without their Newton step, every opaque barrier kept
// (hlsl/k_out.hlsli KO_APPROX); not byte-identical to HIP.
// @wg-kernel k_out_color__a2 co=k_io.co entry=k_out_color block=128 wave=64 args=s312:p4.b12.x24.b52.x12.b24.x4.p4.x8.p4.b8.p2.b56 defines=KO_APPROX=1
// @wg-kernel k_out_pixel_c__a2 co=k_io.co entry=k_out_pixel_c block=128 wave=64 args=s312:p4.b12.x24.b52.x12.b24.x4.p4.x8.p4.b8.p2.b56 defines=KO_COMPACT=1,KO_APPROX=1
// @wg-kernel k_out_color_direct__a2 co=k_io.co entry=k_out_color block=128 wave=64 args=s312:p4.b12.x24.b52.x12.b24.x4.p4.x8.p4.b8.p2.b56 rs=direct defines=WG_DIRECT_TEXTURES=1,KO_APPROX=1
// @wg-kernel k_out_pixel_c_direct__a2 co=k_io.co entry=k_out_pixel_c block=128 wave=64 args=s312:p4.b12.x24.b52.x12.b24.x4.p4.x8.p4.b8.p2.b56 rs=direct defines=KO_COMPACT=1,WG_DIRECT_TEXTURES=1,KO_APPROX=1
#include "wg_platform.hlsli"
#if defined(WG_DIRECT_TEXTURES)
#include "native_direct.hlsli"
Expand Down
Loading
Loading