Real-Time Noise Filtering For Light Simulations | Two Minute Papers #181

Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. In this series, we talk a lot about photorealistic rendering, which is one of the most exciting areas in computer graphics research. Photorealistic rendering means that we put virtual objects in a scene, assign material models to them, and then run a light simulation program to create a beautiful image. This image depicts how these objects would look like in reality. This is particularly useful for the film industry because we can create highly realistic scenes and set them up in a way that we couldn't do in real life. We can have any possible object we can imagine, light sources that we wouldn't ever be able to buy, change the time of the day, or even the planet we're on. Practically, we have an infinite budget. That's amazing! However, creating such an image takes a long time, often in the order of hours to days. Here you can see me render an image and even if the footage is sped up significantly, you can see that this is going to take a long, long time. Spp means samples per pixel, so this the number of light rays we compute for every pixel. The more spp, the cleaner, more converged image we get. This technique performs spatiotemporal filtering. This means that we take a noisy input video and try to eliminate the noise and try to guess how the final image would look like. And it can create almost fully converged images from extremely noisy inputs. Well, as you can see, these videos are created with one sample per pixel, which is as noisy as it gets. These images with the one sample per pixel can be created extremely quickly, in less than ten milliseconds per image, and this new denoiser also takes around 10 milliseconds to reconstruct the final image from the noisy input. And yes, you heard it right, this is finally a real-time result. This all happens through decoupling the direct and indirect effect of light sources and denoising them separately. In the meantime, the algorithm also tries to estimate the amount of noise in different parts of the image to provide more useful information for the denoising routines. The fact that the entire pipeline runs on the graphics card is a great testament to the simplicity of this algorithm. Whenever you see the term SVGF, you'll see the results of the new technique. So we have these noisy input videos with 1 spp. And... look at that! Wow! This is one of those papers that looks like magic. And, no neural networks or learning algorithms have been used in this work. Not so long ago, I speculated, or more accurately, hoped that real-time photorealistic rendering would be a possibility during my lifetime. And just a few years later, this paper appears. We know that the rate of progress in computer graphics research is just staggering, but this is, this is too much to handle. Super excited to see where the artists will take this, and of course, I'll be here to show you the coolest followup works. Thanks for watching and for your generous support, and I'll see you next time!