# What Is 3D Photography? 🎑 ## Метаданные - **Канал:** Two Minute Papers - **YouTube:** https://www.youtube.com/watch?v=BjkgyKEQbSM - **Дата:** 15.12.2020 - **Длительность:** 5:27 - **Просмотры:** 82,375 ## Описание ❤️ Check out Weights & Biases and sign up for a free demo here: https://www.wandb.com/papers ❤️ Their mentioned post is available here: https://wandb.ai/authors/One-Shot-3D-Photography/reports/Paper-Summary-One-Shot-3D-Photography--VmlldzozNjE2MjQ 📝 The paper "One Shot 3D Photography" is available here: https://facebookresearch.github.io/one_shot_3d_photography/ 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Haro, Alex Serban, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bruno Mikuš, Bryan Learn, Christian Ahlin, Eric Haddad, Eric Lau, Eric Martel, Gordon Child, Haris Husic, Jace O'Brien, Javier Bustamante, Joshua Goller, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Michael Albrecht, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Ramsey Elbasheer, Robin Graham, Steef, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. If you wish to support the series, click here: https://www.patreon.com/TwoMinutePapers Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: https://discordapp.com/invite/hbcTJu2 Károly Zsolnai-Fehér's links: Instagram: https://www.instagram.com/twominutepapers/ Twitter: https://twitter.com/twominutepapers Web: https://cg.tuwien.ac.at/~zsolnai/ #3dphotos ## Содержание ### [0:00](https://www.youtube.com/watch?v=BjkgyKEQbSM) Segment 1 (00:00 - 05:00) Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. This is a standard color photo made with a smartphone. Hence, it contains only a 2D representation of the world, and when we look at it, our brain is able to reconstruct the 3D information from it. And I wonder, would it be possible for an AI to do the same, and go all the way and create a 3D version of this photo that we can rotate around? Well, this new learning-based method promises exactly that, and if that is at all possible, even more. These are big words, so let’s have a look if it can indeed live up to its promise. So, first, we take a photograph, and we’ll find out together in a moment what kind of phone is needed for this. Probably an amazing one, right? For now, this will be the input, and now, let’s see the 3D photo as an output. Let’s rotate this around. And…wow. This is amazing. And you know what is even more amazing? Since pretty much every smartphone is equipped with a gyroscope, these photos can be rotated around in harmony with the rotation of our phones, and wait a second…is this some sort of misunderstanding, or do I see correctly that we can even look behind a human if we wanted to. That content was not even part of the original photo! How does this work? More on that in a moment. Also, just imagine putting on a pair of VR glasses, and looking at a plain 2D photo and get an experience as if we were really there. It truly feels like we are living in a science fiction world. If we grab our trusty smartphone and use these images, we can create a timeline full of these 3D photos and marvel at how beautifully we can scroll such a timeline here. And now, we have piled up quite a few questions here. How is this wizardry possible? What kind of phone do we need for this? Do we need a depth sensor? Maybe even LiDAR? Let’s look under the hood and find out together. This is the input. One colored photograph, that is expected, and let’s continue…goodness! Now this is unexpected… the algorithm creates a depth map by itself. This depth map tells the algorithm how far different parts of the image are from the camera. Just look at how crisp the outlines are. My goodness, so good. Then, with this depth information, it now has an understanding of what is where in this image, and creates these layers. Which is, unfortunately, not much help because, as you remember, we don’t have any information on what is behind the person. No matter, because we can use a technique that implements image inpainting to fill in these regions with sensible data. And now, with this, we can start exploring these 3D photos. So…if it created this depth map from the color information, this means that we don’t even need a depth sensor for this. Just a simple, color photograph. But wait a minute…this means that we can plug in any photo from any phone or camera that we or someone else took, at any time…and I mean at any time, right? Just imagine taking a black and white photo of a historic event, colorizing it with a previous learning-based method, and passing this color image to this new method, and then, this happens. My goodness. So, all this looks and sounds great, but how long do we have to wait for such a 3D photo be generated? Does my phone battery get completely drained by the time all this computation is done? What is your guess? Please stop the video and leave a comment with your guess. I’ll wait. Alright so is this a battery killer? Let’s see. The depth estimation step takes…whoa, a quarter of a second, inpainting, half a second, and after a little housekeeping, we find out that this is not a battery killer at all, because the whole process is done in approximately one second. Holy mother of papers. I am very excited to see this technique out there in the wild as soon as possible. ### [5:00](https://www.youtube.com/watch?v=BjkgyKEQbSM&t=300s) Segment 2 (05:00 - 05:00) Thanks for watching and for your generous support, and I'll see you next time! --- *Источник: https://ekstraktznaniy.ru/video/14017*