Google’s New AI: Blurry Photos No More!

Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. Today we are going to end the rule of   ugly, blurry photos with a new  AI technique developed by Google. You see, most smartphones today have multiple  cameras, and can take these beautiful portrait   mode photos where the subject is in focus,  and the rest of the image is blurred out.    This is all well and good if we have shot the  image perfectly, and we have in focus exactly   the things that we want before pressing the  button. If we missed our chance, unfortunately,   that’s it. Refocusing these images would  be amazing, if we could do it. So, can we? Well, unfortunately we can. Now I hear you asking  Doc, why do you say, unfortunately we can? Well,   we have to use light fields to properly refocus  an image. This unfortunately requires an expensive   and inconvenient array of cameras to capture all  the required information in and out of focus. Or, we can try focus stacking. That is,  capturing a series of images with different   focus points. Focusing on different things  in different photos. This is not expensive,   but unfortunately, it is time  consuming, and not trivial to perform. But, wait a minute. I just said at the start  of the video that most smartphone brands today   have multiple cameras. That typically  means a wide and an ultrawide lens.   If we take a photo with both of these  lenses at the same time, we get two   images with different parts of the real scene  in focus. If you think that sounds reassuring,   it gets worse, look. The two lenses give quite  different results. The colors are different,   the focus is different. Hmm. Perhaps a  learning-based algorithm could stitch   these two together into an image that we  can refocus, but it looks quite difficult. And today, Fellow Scholars, fortunately, that  is a possibility. This is the result from a   work named DRBNet from just one year ago. This  is supposed to deblur the whole image and give   us a sharp result. Well, it could have been a  lot better, look here. Not too great. But now,   hold on to your papers Fellow Scholars, and have  a look at the new technique. So much better. And when we try the old technique for  refocusing an image, we get this. But,   with the new technique, we get this instead. That  is a huge difference just one more paper down the   line, especially that the previous  work was published just a year ago. And with this new work, we can now refocus all  of these beautiful images. And even though none   of them are as perfect as having a real camera  focusing at all the objects, I find these results   to be absolutely incredible. Not perfect by any  means, but remember, the two lenses are really   different. This technique is like a classic music  conductor having two different orchestras playing   together in harmony. Neither of the orchestras  are perfect by themselves because they are   both missing some instruments, but together, if  directed well, they can play the perfect symphony.    You can now take in this symphony, not  with your ears, but with your eyes. Now two more things that I absolutely loved  about this paper. One is that it describes a   method that is not specific to  a particular device. And two,   these ultrawide lenses are often of lower  quality than the other lenses in smartphone   cameras and are typically underused  and overlooked. But not here. Love it. So, from now on, unusable  blurry images of important   family moments shall be banished  forever. What a time to be alive!

Thanks for watching and for your generous  support, and I'll see you next time!