Repository navigation
Lumen follow-up: benchmark harness for SSGI quality / convergence #24
Description
Activity
Migration note: #128 now implements the canonical harness requested here.
tools/quality/scenes.tomlowns fixed Sponza/PBR/etc. cameras and settings;tools/quality/run.pyruns fixed-step, uncapped sustained warm-up/measurement windows, records native per-pass GPU timestamps and CPU frame work, captures named graph intermediates, writesresult.jsonplus Markdown/HTML evidence, verifies same-machine reproducibility, and applies explicit visual/performance gates. Active HW/SW GI fallback and native adapter/backend metadata are recorded rather than inferred.The old
tools/validate.shworkflow is documented as legacy exploration because it embeds cameras, caches references, resizes throughsips, and lacks the fixed-step/warm-up/adapter/timing/baseline-governance contract.bloom-reference,bloom-diff, and Cycles reference tooling are reused by the new workflow rather than duplicated.Metal evidence and remaining baseline/hardware qualification status are recorded in #128. Leave this issue open until #128 receives human baseline approval and the labelled hardware workflow has produced its retained artifacts; at that point this can close as superseded by #128.
A focused SSGI qualification finding from
ad8e7c4should become part of this benchmark/convergence ticket.Observed on the deterministic GPU corpus:
- An immediate-draw, Hi-Z-only scene remained at zero current probe radiance and a black resolved SSGI target.
- A retained static receiver/emitter scene warmed through the production
hiz-screentosdf-clipmaptransition and then produced 2047 current-radiance probe texels, 4096 retained-history texels, zero non-finite resolved pixels, and max resolved luminance 0.9426.
The landed checkpoint deliberately does not change the Hi-Z marcher: exploratory marcher changes lacked a stable before/after quality and timing oracle and were reverted. It adds the raw
ssgicapture, probe-domain history diagnostics, and a deterministic SDF/card-backed convergence gate needed to investigate this safely.Recommended next scope for this issue:
- Add a deterministic scene that forces and verifies the
hiz-screenbackend without warming into SDF/card tracing. - Gate nonzero raw trace and resolved indirect diffuse separately, with fixed-camera convergence across a short sequence.
- Record SSGI pass timing and image metrics before changing march/thickness behavior.
- Land a Hi-Z fix only with improved quality, finite output, and no pass/resource or measured performance regression.
Hi-Z SSGI qualification/fix checkpoint landed in draft PR #147 at
7076bea.Root causes
- The software marcher stopped on any coarse Hi-Z overlap even when the mip-zero refinement rejected it. Nearby foreground in a coarse min-depth footprint therefore terminated almost every ray close to its origin.
- Trace output already stores cosine-weighted incident radiance over 64 octahedral directions, but resolve sampled one normal-indexed direction as if it were diffuse irradiance. Sparse valid Hi-Z hits could therefore remain visually black.
Production change
- Continue marching after a coarse-footprint false positive; accept only a mip-zero-confirmed front-depth crossing.
- Cosine-convolve all 64 directional samples once in the existing temporal workgroup.
- Cache the diffuse result in the existing probe header and reuse the header load resolve already needs for bilateral weights, removing the per-sample probe-history texture lookup.
- Keep the existing four production probe passes and both history textures. No new texture, bind group, dispatch, draw, or graph pass.
- Probe headers grow from 32 to 48 bytes: +16 bytes/probe, or 32,640 bytes at a 1920x1080 render size.
Deterministic quality evidence
Immediate-only room, forced/verified
hiz-screenbackend:Resolution Build Current trace texels Retained history Non-finite pixels Resolved max luma 256x256 before 0 4,096 0 0.000000 256x256 7076bea2,043 4,096 0 0.239963 1280x720 before 0 58,880 0 0.000000 1280x720 7076bea29,381 58,880 0 0.240076 The raw resolved target has zero isolated HDR outliers. The existing retained-scene SDF gate also remains healthy: 4,096 accepted, 0 refreshed, 2,047 current, 4,096 retained, 0 non-finite, max luma 0.9728.
GPU timing
Apple M1 Max / Metal, 1280x720 render size, 24 warm-up frames, 120 timestamped frames per run, frozen pre/post binaries interleaved for 10 pairs:
- broken baseline four-probe-pass mean: 466.012 us
- corrected build four-probe-pass mean: 465.647 us
- delta: -0.366 us / -0.078% (noise-level, no measured regression)
Validation
- release native library: 297 passed, 0 failed, 1 ignored
- release golden/temporal corpus: 43 passed, 0 failed, 2 hardware-only ignored
- new probe-header ABI unit contract: passed
- FFI parity: 0 failures, 0 warnings
- wasm32 shared check and full
wasm-pack build --release --target web: passed - formatting, WGSL parsing through unit/golden coverage, diff whitespace, and file-line ratchet: passed
This completes the focused Hi-Z qualification/fix scope proposed in the previous comment. #24 remains open under its existing dependency on #128 human baseline approval and labelled hardware evidence; this checkpoint does not claim those external acceptance items are complete.
Detailed Bistro camera-motion stabilization checkpoints have been pushed on
codex/rendering-wip-integrated-20260727:6cc995f(fix(renderer): prevent delayed SSGI light flares)70fc0ea(fix(renderer): bound SSGI probe solid-angle outliers)
Verified causes and production changes
The delayed broad light/shadow fragments appeared after roughly 20 seconds because the 1,176-placement detailed Bistro scene admits one BLAS per frame. Mesh-Card and cardless hit lighting therefore changed progressively while camera-fitted direct-shadow data was still being consumed.
6cc995fmakes the completed Mesh-Card direct-light bake coherent and world-space, keeps pre-bake hits on stable sky/emissive terms, and makes the bounded cardless fallback sky/emissive-only instead of applying one facade-sized direct visibility sample.After that fix, the remaining localized colored streaks/sparkles came from a fixed 1/64-sphere probe ray intersecting a tiny bright texture/light.
70fc0eaapplies a 5x-mean solid-angle outlier bound inside the existing temporal workgroup. The measured maximum ratio for a smooth cosine field on the fixed 8x8 octahedral grid is 4.2102x, so the bound retains smooth sky/sun fields while suppressing energy too concentrated for the represented solid angle.Neither checkpoint adds a render pass, ray, texture, allocation, or bind group.
Visual verification
Interactive native-resolution detailed-Bistro checks with TAA/SSGI/SSR enabled found:
- the former large camera-moving light/shadow regions were removed;
- the delayed flare after scene admission was removed;
- the severe red/green streaks and most sparkling were drastically reduced;
- toggling SSGI no longer exposes the stale resolve image.
The reviewer described the successive result as “100% better”, then “1000% better”. A smaller residual colored/view-dependent variation remains under active investigation, so this is a stable checkpoint rather than a claim that all Bistro SSGI work is complete.
Deterministic evidence
- SSGI GPU corpus: 5/5 passed, finite output retained.
- Full detailed-Bistro same-endpoint oracle after 1,200 settle frames (all 1,176 placements resident):
- raw SSGI SSIM:
0.9999911339 - final output SSIM:
0.9999980638 - raw/final outlier pixel fraction:
0.0 - raw/final maximum path-dependent delta:
1/7(8-bit capture units)
- raw SSGI SSIM:
- Settled raw SSGI: zero non-finite pixels and zero isolated local HDR outliers.
- Native glossy slow-pan regression coverage is separately landed in
8990169(mean SSIM0.991087, minimum0.990060, derivative error0.084115).
The full-settle endpoint result rules out unbounded long-lived temporal accumulation as the remaining small effect; current work is isolating saturated-ray energy and camera-position reconstruction without increasing ray count or adding renderer resources.
New test gap verified in detailed Bistro on 2026-08-13: the current settle-at-identical-endpoint oracle can report near-perfect SSIM while nearby camera viewpoints show a large coherent red indirect-light patch at different world locations.
The corpus currently proves finiteness, history acceptance/rejection, and repeatability at one endpoint. It does not prove that the reconstructed indirect irradiance field is world-stable across a camera sweep.
Required addition: capture raw SSGI and final HDR at a short sequence of nearby poses looking at the facade/awning case, reproject samples to a common world-space receiver plane, and gate low-frequency chroma/luminance drift. Also retain a 15-20 second sweep to catch accumulated/reconstruction failures that do not appear in a short settle test.
SSGI qualification update for pushed checkpoint
33fa720:- 26/26 renderer shader tests passed
- 5/5 SSGI GPU corpus passed with finite output and retained history
- detailed Bistro same-endpoint camera-path test passed: raw SSGI SSIM 0.999936807, final output SSIM 0.999993387, no outlier pixels
- user visual test confirms a material reduction in the camera-moving red facade fragment, with a residual soft red tint still present
The attempted generic synthetic colored-bounce oracle was deliberately not committed: card-light admission caused it to exercise the neutral fallback instead of Bistro's colored Mesh-Card feed, so it could have passed vacuously. The real 1,176-placement Bistro oracle remains the trusted path/history gate. A future nearby-viewpoint oracle needs either world-point reprojection or a deterministic Mesh-Card-backed fixture.
SSGI reconstruction checkpoint
7b6a526is pushed oncodex/rendering-wip-integrated-20260727.Root cause and production change:
- Raw Bistro
ssgi.pngexposed exact 16-half-resolution-pixel dark rows: the strict 2x2 bilateral resolve rejected every surrounding probe on shallow, finely tessellated ground and returned zero energy. - When the strict footprint is empty, resolve now reconstructs only if all four surrounding probes pass a broader symmetric world-plane/normal compatibility test. Unsupported silhouettes and mixed surfaces remain black.
- The fallback reuses each already-loaded probe header and its authoritative clamped diffuse history. It adds no pass, allocation, binding, texture sample, or radiance fetch.
Qualification:
- SSGI GPU corpus: 8/8 passed.
- Native HD settled-probe stability: passed; finite output, max luma 0.347170.
- Matched performance against pushed parent
8a78331, five alternating HD runs: normalized median resolve ratio 2.851 baseline versus 2.811 candidate (no measured regression). - Matched Bistro ground-row diagnostic at 400x225 density: near-zero rows 16 -> 3; rows below 0.05 luma 17 -> 6.
- Exact shader source/parse contract passed and pins one radiance fetch, no filtered lookup, and four-probe geometry support.
A newly exercised full 1,176-placement same-endpoint camera-path oracle still detects small path-dependent raw probe history with no endpoint settle: raw SSIM 0.997584, final output SSIM 0.999910. The pushed parent produces essentially the same result (0.997610 / 0.999910), so this is not introduced by the reconstruction change. Preserved diagnostics show identical current ray/current integrated values but differing retained probe history and confidence. That upstream temporal-retention work remains open under #135; this checkpoint does not claim complete SSGI temporal convergence.
- Raw Bistro
Accepted detailed-Bistro SSGI checkpoint pushed as
76a8ac3(fix(renderer): settle SSGI transport and resolve gaps).What changed:
- Replaced the endlessly advancing angular phase with a finite 16-phase ring of complete diffuse estimates (32 cosine-weighted rays per phase, 512 effective directions). Static transport becomes stationary after a complete cycle instead of retaining low-amplitude Monte Carlo shimmer.
- Kept the geometry-clamped temporal RGB history unfiltered and stored a scalar spatial/history luminance ratio in the existing probe-header W lane. Empty-footprint reconstruction can now preserve the spatially filtered energy without another texture read, binding, allocation, or header byte.
- Kept the strict 2x2 bilateral reconstruction unchanged. Its empty-footprint fallback now admits at least two world-plane/normal-compatible probes with a bounded corner-weight floor, filling the remaining shallow-ground support gaps while unsupported silhouettes stay black.
- Added capture-only resolve support, geometry, and plane-ratio diagnostics. They create no persistent normal-frame resources.
Detailed Bistro visual result:
- The previously severe delayed red/white facade projections and moving indirect patches are gone.
- The residual shallow sidewalk strip is now subtle enough that the reviewer accepted SSGI with: “pretty good - let's call it done for now!”
- A separate small flicker around paving, foliage silhouettes, hard shadow/contact edges, and the scooter remains under TAA investigation; controlled SSGI/SSR isolation does not attribute that residual to SSGI.
Current M1 Max / Metal evidence:
- Hi-Z SSGI: 3,648 current probes, 3,648 retained probes, 0 non-finite pixels, max luma 0.158882.
- Stationary TAA+SSGI: SSIM 0.9853471438, luminance RMSE 0.0178549145.
- Current four-probe-pass GPU total: 531.701 us in the full corpus (resolve 358.053 us).
- The scalar fallback-energy read remained within <1% interleaved timing noise; the final admission change adds arithmetic only and no pass, ray, sample, resource, or allocation.
- Native glossy slow pan: mean SSIM 0.990825, minimum SSIM 0.989956, derivative error 0.083868.
- Full validation at the pushed commit plus isolated TAA worktree: 447 library tests passed (1 intentional ignore); 77 golden/GPU tests passed (2 intentional hardware-oracle ignores).
This closes the reported detailed-Bistro SSGI artifact for now. #24 remains open for its broader #128 baseline/hardware qualification dependency.
SSGI motion-stability checkpoint is pushed on codex/rendering-wip-integrated-20260727:
- c0f195b — fix(renderer): stabilize SSGI reconstruction during motion
- b92668f — test(renderer): average complete GPU profile windows
Production result:
- The probe lattice is now 8x8 half-resolution pixels with eight rays per probe, preserving the established total ray-query density while quadrupling receiver sites.
- A finite 16-phase ring retains 128 angular samples per receiver. Static probe placement remains non-jittered.
- Hardware ray-query history uses a complete geometry-clamped 5x5 probe reconstruction; Hi-Z/SDF remain variance-adaptive to avoid amplifying their approximate phase changes.
- Resolve now keeps geometry/depth-validated per-pixel ping-pong history so shallow ground support moves with world surfaces instead of sticking to the display.
- Capture-only support/acceptance diagnostics were added without persistent normal-frame resources.
Visual verification on the 1,176-placement Bistro scene: the camera-fixed floor grid is significantly reduced. The reviewer accepted moving on to other quality work, while explicitly retaining a faint residual grid/line artifact as open follow-up; this checkpoint does not claim that last residual is eliminated.
Qualification on M1 Max / Metal:
- Full library suite: 448 passed, 1 intentional ignore.
- Full golden/GPU suite: 77 passed, 2 intentional hardware-oracle ignores.
- Full SSGI corpus: 9/9 passed.
- Detailed Bistro same-endpoint path: raw SSIM 0.999998728; displayed SSIM 0.999929724; zero outlier pixels.
- SSGI capture: 14,592 retained probes, 0 non-finite pixels; 14,439 moving resolve-history acceptances.
- Native glossy slow pan: mean SSIM 0.990844, minimum SSIM 0.989877, derivative error 0.077052.
- Five interleaved same-binary timing pairs: candidate median 675.732 us versus pushed-parent median 680.397 us, so no measured GPU regression.
Issue remains open for the broader #128 qualification dependency and the faint accepted residual.
Current-HEAD SSGI foreground revalidation is complete and pushed in evidence checkpoint
b54eba9(qualified renderer revision6677c28).Fresh evidence on Apple M1 Max / Metal:
- Exact 1,176-placement
BistroReference.gltf, scene SHA-256217b6cf37bfb435eb6d164a6107e19f69f4bbf4d1bfb022713aa7076a8752927. - Canonical 512x288, 32-frame five-owner matrix passes: largest >8-level adjacent component 25 px against the permanent <=32 px gate.
- TAA-off removes 94.20% of accumulated range, 98.11% of adjacent-frame change, and 92% of the coherent component. Matched SSGI-off changes those measures by only 4.44%, 2.41%, and 0%, so the old SSGI projection has not returned.
- Fresh clean executable SHA-256
e7e8ba551364efe63b9188c78860af76ad2348e1c1abeddba80c3f64a82d6d8b: 1600x900, scale 1.0, Ultra, HW ray query, TAA/SSGI/SSR on, sharpen and motion blur off. The bright-area/paving foreground sweep was accepted by the reviewer as “looking good all around!”; no failure capture was generated. - Five alternating historical 256x256 fixture pairs versus accepted checkpoint
b92668f: baseline median 682.258 us, current 662.301 us (-2.93%), with identical image metrics and zero non-finite pixels. - Five alternating fully admitted Bistro endpoint pairs: baseline median 6,371.515 us, current 6,266.866 us (-1.64%).
- Release library: 482 passed, 0 failed, 1 ignored. GPU golden corpus: 79 passed, 0 failed, 2 hardware-only ignored. All 20 profiled runs passed.
Durable evidence:
docs/evidence/issue-24-ssgi-foreground-revalidation-v1.{md,json}.Conclusion: the world-surface cache and moving reconstruction fixes remain intact through the later visibility/virtual-geometry work, with no measured GPU regression. No renderer change is required by this revalidation. #24 remains open only for its broader #128 baseline-governance and labelled cross-hardware dependency.
- Exact 1,176-placement
Across 014 and 016, the Metal drawable-stall flake on macOS made steady-state FPS comparisons unreliable — every version change required waiting 5-10 minutes between runs for the OS to release its hold on the swapchain. This made it impossible to attach solid numbers to V1-V4 perf claims like "32 rays/probe + importance sampling matches 32 rays/probe uniform". Visual captures (
docs/perf/ticket-*-after.png) are the de-facto acceptance signal.A proper harness would let us actually measure:
Scope
intel-sponzaexample) that:Stretch
Context
Flake documentation lives informally in the V5-V14 commit messages. Originally in the 014 closure: "FPS ≥ 40 fps ... was flake-bound throughout V8-V14 testing". A repeatable harness would close that loop.