AI Creates 3D Models From Faces | Two Minute Papers #149

Dear fellow scholars, this is two minute papers with Kohaa Eher. How cool would it be to be able to place a character representing us in a digital film or a computer game? Of course, it would clearly be an extremely laborious task to digitize the 3D geometry and the albido map of our face. This albido map means a texture, a colored pattern that describes how our skin reflects and absorbs light. Capturing such a representation is clearly a very lengthy and expensive process. So get this completely crazy idea. This technique creates this full digital representation of any face from no more than one simple photograph. We can even get historical figures in our digital universe. All we need is one photograph of them. And now feast your eyes on these incredible results. After taking a photograph, this technique creates two of these albido maps. One is a complete low-frequency map which records the entirety of the face but only contains the rough details. The other albido map contains finer details but in return is incomplete. Do you remember the texture synthesis methods that we discussed earlier in the series? The input was a tiny patch of image with a repetitive structure and after learning the statistical properties of these structures, it was possible to continue them indefinitely. The key insight is that we can also do something akin to that here as well. We take this incomplete albido map and try to synthesize the missing details. Pretty amazing idea indeed. The authors of the paper invoke a classical learning algorithm, a convolutional neural network to accomplish that. The deeper the neural network we use, the more high frequency details appear on the outputs or in other words, the crisper the image we get. In the paper, you will find a detailed description of their crowdsource user study that was used to validate this technique, including the user interface and the questions being asked. There are also some comparisons against patch match. one of the landmark techniques for texture synthesis that we have also talked about in an earlier episode. It's pretty amazing to see this twominut papers knowledge base grow and get more and more intertwined. I hope you're enjoying the process as much as I do. Also, due to popular request, the two-minut papers t-shirts are now available. This time, we are using a different service for printing these shirts. Please give us some feedback on how you liked it. I've put my email address in the video description. If you attach a photo of yourself wearing some cool twominut papers merch, we'll be even more delighted. Just open 2minutpapers. com and you'll immediately have access to it. This link will bring you to the service that ships to the US. The link for shipping outside the US is available in the video description. Thanks for watching and for your generous support, and I see you next time. Heat.