NRD in DSLE (Dark Souls 2 & 3 PT)

[Ganaboy2k]

Hello Dzhdan,

Thanks for all your support so far. As discussed we can use this consolidated thread from here on. I will start with this template so is easier.

Problem Statement :

1. Passing Confidence inputs to NRD - Reblur so that it can disocclude regions of shading change temporally. Most noticeable with Shadows of Moving/Animated objects when Reblur is used to denoise Direct and Indirect lighting together from LocalLightSources (Point/Spot/Area Lights)

Solution Plan:

1. During SHARC Rendering(5x5 Downscale). Output the following properties Visibility Buffer - HitInstance, PrimitiveId, Radiance Buffer Raw. This pass technically need not be with SHARC, but is useful to combine
2. Fetch Previous Buffers output from 1(Visibility buffer, Radiance). GetPosition in PreviousFrame, Run the path tracer loop again, note we already know the previous frame seed used. Compared the results and output to a buffer.
3. Blur the Output from 2 multiple times, do a 1 frame temporal accumulation with Confidence Output Buffer from Previous. Finally Upscale the output to full resolution(Bilinear), which will be confidence input to NRD.

PS : Maybe I can reuse the 2nd path trace loop to update the SharcCache as well. Improving cache quality. As the SHARC cache operates in world space

Let me know what you think of my plan.

Regards,
Ganesh

[dzhdanNV]

Yes. This should work. I think that utilizing SHARC pass is a good idea (I thought about it previously too).

Problematic areas:

• since resolution is 1/5 there will be issues on geometry edges, so the blur pass should respect edges
• "confidence" should fade out to 1 in areas where indirect lighting prevails

Hope it works.

[Ganaboy2k]

Problematic areas:

• since resolution is 1/5 there will be issues on geometry edges, so the blur pass should respect edges
• "confidence" should fade out to 1 in areas where indirect lighting prevails

Thanks. Will try and post back here

[Ganaboy2k]

Hello Dzhdan,

I implemented it this way

1. Compute Gradients during SHARC pass. Output Gradients - (LDiffuseDiff, LSpecDiff, LDiffuseAbsolute, LSpecAbsolute). Also output BlurGuide - (OctNormal, Roughnes, ViewZ) all at 5x5 downscaled
2. Run Gradients Blur using the BlurGuide x4 (5x5 resolution) (Atrous)
3. Convert to Confidence inputs, blend with previous frame . (Upscale done using Bilaterial, BlurGuide/FullResDepth). BlendFactor - 1.0/(1.0 + 2.0). I dont disocclude history in this temporal blend but maybe I should.

Without

With

Confidence view

Shadow Ghosting is reduced significantly when in motion. Can't seem to upload videos unfortunately

Regards,
Ganesh

[Ganaboy2k]

One more easier example comp Without/With

Without

With

[dzhdanNV]

I like it! I'm happy to review the code and suggest something hopefully useful (and even steal some ideas, crediting your honest name for sure). Please, do share :)

Well, I see the following actions for me:

• implement something similar in the NRD sample, since mitigating lags is the MOST critical issue for NRD. Since SHARC is a good friend of NRD integrating an anti-lag solution inside the "SHARC update" pass seems to be a simple and robust solution
• briefly describe the idea in the NRD docs and reference implementation in the NRD sample
  • bad practice: RESTIR-based algos compute confidence using TLAS and prev-TLAS, obviously keeping alive prev-TLAS is a huge no-go.

Can't seem to upload videos unfortunately

Try YouTube :) Let me know if nothing helps, probably I could arrange a shared folder (for us, not public).

[Ganaboy2k]

• implement something similar in the NRD sample, since mitigating lags is the MOST critical issue for NRD. Since SHARC is a good friend of NRD integrating an anti-lag solution inside the "SHARC update" pass seems to be a simple and robust solution

• briefly describe the idea in the NRD docs and reference implementation in the NRD sample

  • bad practice: RESTIR-based algos compute confidence using TLAS and prev-TLAS, obviously keeping alive prev-TLAS is a huge no-go.

Try YouTube :) Let me know if nothing helps, probably I could arrange a shared folder (for us, not public).

Yeah that could work nicely. We can keep the upload file short but high quality

Yeah keeping previous -TLAS is huge no-go for me as well. It interferes with lot of things like async building synchronization and what not. Let Share the code files. A lot of Common function is based on NRD/NRD Sample. I still live in stone age and don't have my code on Github yet

I have Twitter link of the demo though
https://x.com/DS2LightingMod/status/2001387811069661205

I can upload source code files in the private folder. Can't seem to attach here

[Ganaboy2k]

Gradients.zip

Oh I was able to compress to zip and upload the relevant shaders. :)

[dzhdanNV]

https://x.com/DS2LightingMod/status/2001387811069661205

Watched the video: INSANE \O/

Trying to arrange a folder...

[dzhdanNV]

I need to know your email. Please, write me a short message using this https://github.com/NVIDIA-RTX/NRD/blob/master/Include/NRD.h#L14 (you will figure it out ^_^)

[Ganaboy2k]

I need to know your email. Please, write me a short message using this https://github.com/NVIDIA-RTX/NRD/blob/master/Include/NRD.h#L14 (you will figure it out ^_^)

Sent a short message to you. You will find the code I shared super familiar as it is pretty much based on NRD sample.

[dzhdanNV]

Thanks for the code. Comments & suggestions (omitting minor stuff, like, comparing uint >= 0 ^_^):

• set "out of screen" confidence values to 1 (NRD handles this case well)
• probably a gradient itself should be computed "after tonemap" and not in "radiometric units". We don't care if there is a huge radiance change if the tone-mapping maps these values to ~same output color (i.e. no visual difference)

A generic suggestion - for NV HW better use [numthreads(16, 16, 1)] CTA size by default and, in some cases, 16x8/8x16 for better cache utilization (an NSIGHT GRAPHICS - GPU Trace feature can help here). 8x8 is good only in specific cases (for example, 16x16 consumes too much shared memory, limiting overall occupancy).

[dzhdanNV]

Ganesh, you have inspired me: I will implement something like this in the NRD sample very soon. Well, I wanna do it RIGHT now, but have other work to do first.

[dzhdanNV]

Confidence factors in SHARC Update Pass conclusions:

• it works! thanks for the implementation
• I think, you can switch to other tasks...
• ...because soon I implement a cleaner approach in the NRD sample (I hope to do this before X-mass, hope there is no a strict deadline)

More feedback:

• watched the high-res video: some artefacts can be suppressed by switching to 5x5 blur (using a tone-mapped color should help
  • SHARC update is already (1/5; 1/5), so 3x3 blur feels a bit "not enough"
• if "SHARC accum buffer" contains data for the current frame, then we can dump it into a texture and on the next frame compare (with reprojection) with the current one. It's like "SHARC curr" vs "SHARC prev" (before resolve, i.e. temporal part). If it works, the benefits are:
  • no 2nd ray
  • even in the current frame SHARC is accumulated: that's what we need (we sacrifice a bit of spatial resolution to stability, what is important for gradients)

[dzhdanNV]

An action item for me:
I have just realized that "confidence factors" must not be necessarily at the screen resolution. It's better, yes. But NRD should support a low res too and apply a linear texture sample to the input resource under the hood. Will do soon too and update docs.