New AI: Photos Go In, Reality Comes Out! 🌁

5:26

New AI: Photos Go In, Reality Comes Out! 🌁

Two Minute Papers 16.11.2021 159 568 просмотров 9 290 лайков

Machine-readable: Markdown · JSON API · Site index

Смотреть на YouTube

Поделиться Telegram VK Бот

Транскрипт Скачать .md

Анализ с AI

Описание видео

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The paper "ADOP: Approximate Differentiable One-Pixel Point Rendering" is available here: https://arxiv.org/abs/2110.06635 ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers - https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, Christian Ahlin, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Bustamante, John Le, Jonas, Kenneth Davis, Klaus Busse, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Mark Oates, Michael Albrecht, Michael Tedder, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Steef, Taras Bobrovytsky, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi. If you wish to appear here or pick up other perks, click here: https://www.patreon.com/TwoMinutePapers Thumbnail background design: Felícia Zsolnai-Fehér - http://felicia.hu Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: https://discordapp.com/invite/hbcTJu2 Károly Zsolnai-Fehér's links: Instagram: https://www.instagram.com/twominutepapers/ Twitter: https://twitter.com/twominutepapers Web: https://cg.tuwien.ac.at/~zsolnai/

Оглавление (8 сегментов)

Introduction

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today we are going take a collection of photos like these, and magically, create a video where we can fly through these photos. How is this even possible? Especially that the input is only a handful of photos. Well, we give it to a learning algorithm and ask it to synthesize a photorealistic video where we fly through the scene as we please. That sounds impossible. Especially that some information is given about the scene, but this is really not much.

Synthesis

And everything in between these photos has to be synthesized here. Let’s see how well this new method can perform that, but don’t expect too much…and…. wow. It took a handful of photos and filled in the rest so well that we got a smooth and creamy video out of it. So, the images certainly look good in isolation. Now let’s compare it to the real world images that we already have, but have hidden from the algorithm, and hold on to your papers…wow, this is breathtaking. It is not a great deal different, is it? Does this mean…yes! It means that it guesses what reality should look like almost perfectly. Now note that this mainly applies for looking at these images in isolation. As soon as we

artifacts

weave them together into a video and start flying through the scene, we will see some flickering, artifacts, but that is to be expected. The AI has to create so much information from so little, and the tiny inaccuracies that appear in each image are different, and when played abruptly after each other, this introduces these artifacts. So, which regions should we look at to find these flaws? Well, usually regions where we have very little information in our set of photos,

challenges

and a lot of variation when we move our head. For instance, visibility around thin structures is still a challenge. But of course, you know the joke - how do you spot a Two Minute Papers viewer? They are always looking behind thin fences. Shiny surfaces are a challenge too as they reflect their environment and change a lot as we move our head around. So how does it compare to previous methods? Well, it creates images that are sharper and

comparisons

more true to the real images. Look! What you see here is a very rare sight. Usually, when we see a new technique like this emerge, it almost always does better on some datasets, and worse on others. The comparisons are almost always a wash. But here? Not at all. Not in the slightest. Look, here you see four previous techniques, four scenes, and three different ways of measuring the quality of the output images. And almost none of it matters, because the new technique reliably outperforms all of them, everywhere. Except here, in this one case, depending on how we measure how good a solution is. And even then, it’s quite close. Absolutely amazing.

paper

Make sure to also have a look at the paper in the video description to see that it can also perform filmic tone mapping, change the exposure of the output images, and more. So, how did they pull this off? What hardware do we need to train such a neural network?

hardware

Do we need the server warehouses of Google or OpenAI to make this happen? No, not at all! And here comes the best part…if you have been holding on to your papers so far, now, squeeze that paper, because all it takes is a consumer graphics card and 12 to 24 hours of training. And after that, we can use the neural network for as long as we wish. So, recreating

conclusion

reality from a handful of photos with a neural network that some people today can train at home themselves? The pace of progress in AI research is absolutely amazing. What a time to be alive! Thanks for watching and for your generous support, and I'll see you next time!

Другие видео автора — Two Minute Papers

Ctrl+V

Экстракт Знаний в Telegram

Экстракты и дистилляты из лучших YouTube-каналов — сразу после публикации.

Подписаться

Лучшие методички за неделю — каждый понедельник