Google's Enhance AI - Super Resolution Is Here! 🔍

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today we are going to grow people out of… noise of all things. So, I hear you asking, what is going on here? Well, what this work performs is something that we call super resolution. What is that? Simple. The enhance thing. Have a look at this technique from last year. In goes a course image or video, and this AI-based method is tasked with…this! Yes. This is not science fiction. This is super resolution, which means that the AI synthesized crisp details onto the image. Now, fast forward a year later, and let’s see what this new paper from scientists are Google Brain is capable of. First, a hallmark of a good technique is when we can give it a really coarse input and it can still do something with it. In this case, this image will be 64 by 64 pixels, which…is almost nothing I’m afraid, and, let’s see how it fares. This will not be easy. And, well, the initial results are…not good. But don’t put too much of a stake in the initial results, because this work iteratively refines this noise, which means that you should hold on to your papers, and…oh yes, it means that it improves over time…it’s getting there…. whoa, still going. And, wow. I can hardly believe what has happened here. In each case, in goes a really coarse input image, where we get so little information. Look, the eye color is often given by only a couple pixels, and we get a really crisp, and believable output. What’s more, it can even deal with glasses too. Now, of course, this is not the first paper on super resolution. What’s more, it is not even the hundredth paper performing super resolution. So, comparing to previous works is vital here. We will compare this to previous methods in two different ways. One, of course, we are going to look. Previous, regression-based methods perform reasonably well, however, if we take a closer look, we see that the images are a little blurry. High-frequency details are missing. And now, let’s see if the new method can do any better. Well, this looks great, but we are Fellow Scholars here, we know that we can only evaluate this result in the presence of the true image. Now let’s see. Nice. We would have to zoom in real close to find out that the two images are not the same. Fantastic. Now, while we are looking at these very convincing high-resolution outputs. Please note that we are only really scratching the surface here. The heart and soul of a good super resolution paper is proper evaluation and user studies, and the paper contains a ton more details on that. For instance, this part of the study shows how likely people were to confuse the synthesized images with the real ones. Previous methods, especially PULSE, which is an amazing technique reached about 33%. Which means that most of the time, people found out the trick, but, whoa, look here. The new method is almost at the 50% mark. This is the very first time that I see a super resolution technique where people can barely tell that these images are synthetic. We are getting one step closer to this technique getting deployed in real-world products. It could improve the quality of your Zoom meetings, video games, online images, and much, much more. Now, note that not even this one is perfect…look, as we increase the resolution of the output of the image, the users are more likely to find out that these are synthetic images. But still, for now, this is an amazing leap forward in just one paper. I can hardly believe that we can take this image, and make it into this image using a learning-based method. What a time to be alive!

Thanks for watching and for your generous support, and I'll see you next time!