Importance-based/guided adaptive sampling
Current adaptive sampling works by sampling a given pixel until a value (a function of variance and brightness) drops below a specified threshold. I propose an adaptive sampling mode which uses a an importance map, derived from the variance and brightness data of the rendered image, to allocate pixel samples probabilistically. Instead of having a maximum samples/pixel value, we can use an average samples/pixel value. Blender already importance samples sky textures to cast shadow rays, so it is entirely possible to importance sample an image.
This method has the following advantages:
- Minimum samples can be very low or even zero
- Probabilistic nature makes it easy for new high-variance pixels to be discovered (reduced under-sampling)
- The last samples of the render won't be focused exclusively on clearing up fireflies/high-variance spots (reduced over-sampling)
- Time limit rendering will work better with adaptive sampling
- Settings could be more intuitive, or even the same as without adaptive sampling
- Potentially greater image quality, as importance sampling is very effective at allocating samples in other domains of rendering (tests will be needed)
I've had this idea for over a year now, and I haven't been able to think of any major drawbacks or limitations that threshold-based adaptive sampling doesn't already have.
For example: how would threads be allocated when most samples are concentrated in a single region of the image? The same way as they are when most pixels have converged in the existing method.
I'm sure there are some specific scenes and edge-cases where the old method will work better though.
2 Comments
This seems like a worse version of metropolis light transport. The "edge case" of MLT was that it becomes unusable for animations as it will flicker a lot, even if it can produce a much less noisy-looking image by itself.
Much better algorithms with path guiding have already been implemented in Blender. 3.6 has path guiding, and resampled importance sampling is in 4.0 already. Look up videos on ReSTIR (even if no temporal for Blender) or people's results with RIS already in the forums to see the massive benefits this can provide. The drawback is that they're CPU-only... for now.
I'd much rather work gets done on making this proven extremely good algorithm work on GPU.
My proposal is intended to fix a different issue than RIS and other path guiding improvements. Those algorithms increase the quality of samples, which is very useful, but I'm looking to allocate samples in a way that avoids wasting system resources on samples which are closer to converging.
I decided to try RIS on some scenes that I believe my method would be useful in. It helped clean up a few fireflies, but other than that, I couldn't see any difference. This isn't because RIS is a bad algorithm (It's a great one, actually), It's because noisy lighting wasn't the issue. The scene contained scattering volumes and subsurface scattering materials against a relatively noiseless diffuse backdrop. The backdrop received just as many high-quality RIS samples as the noisy materials did.
Also, about MLT and low-frequency noise:
The low-frequency noise is MLT comes from the fact that it is a "random walk" algorithm. It starts with an initial path or pixel, and then perturbs it according to a distribution centered on the current sample. My idea completely re-shuffles all samples, meaning that they wont self-correlate like in MLT.