# Google’s New AI: Fly INTO Photos…But Deeper! 🐦

## Метаданные

- **Канал:** Two Minute Papers
- **YouTube:** https://www.youtube.com/watch?v=H-pTZf1zsa8
- **Дата:** 03.10.2022
- **Длительность:** 5:49
- **Просмотры:** 125,282
- **Источник:** https://ekstraktznaniy.ru/video/13429

## Описание

❤️ Train a neural network and track your experiments with Weights & Biases here: http://wandb.me/paperintro

📝 The paper "InfiniteNature-Zero Learning Perpetual View Generation of Natural Scenes from Single Images" is available here:
https://infinite-nature-zero.github.io/

❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: 
- https://www.patreon.com/TwoMinutePapers
- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Eric Martel, Geronimo Moralez, Gordon Child, Ivo Galic, Jace O'Brien, Jack Lukic, John Le, Jonas, Jonathan, Kenneth Davis, Klaus Busse, Kyle Davis, Lorin Atzberger, Lukas Biewald, Luke Dominique Warner, Matthew Allen Fisher, Michael Albrecht, Michael Tedder, Nevin Spoljaric, Nikhil Velpanur, Owen Campbell-Moore, Owen S

## Транскрипт

### Intro []

Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. Today we are going to use an AI to fly  into this photo. And we will see how   much better this technique has become  in just one year. It will be insanity. Yes, in a previous video, we explored an insane  idea: what if we could take just one photograph   of a landscape, and then, we would fly into this  photo like a bird. Of course, that is a big ask,   because to be able to do this, we would need to  invent at least 3 things. One is image inpainting.    Look, when we start flying, the regions between  the trees are missing. We need to generate those.

### The Problem [0:45]

Two, information is also missing not just within,  but around the photo. This is a huge problem,   because completely new regions should  also appear that are beyond the image.    This means that we also need  to perform image outpainting,   creating these new regions from scratch.   Continuing the image, if you will. And three, as we fly closer to these new regions,   we will be looking at fewer and fewer pixels  and from closer and closer, which means…this.    For this, we would need super resolution - in  goes a coarse image or video, and this AI-based   method is tasked with…this! Yes. This is not  science fiction. This is super resolution,   where the AI starts out from noise and  synthesizes crisp details onto the image.

### Super Resolution [1:40]

So, last year, scientists at Google  created an amazing AI that was able   to learn and fuse all these three  techniques together, to create this.    Wow, so this is possible after all. Well, hold on  to your papers, because it is not only possible,   but the followup paper is already here, we are  now one more paper down the line, in less than   a year later! I know you’re itching to see  it, me too, so let’s compare them together!

### Comparison [2:08]

These methods will all start from the same point,  and oh my, look how quickly they deviate. The two   earlier methods quickly break down, and this  is the work that we talked about a few weeks   ago. This is clearly much better, however, as  every single one of you Fellow Scholars can see,   it lacks temporal coherence. What is that?   Well, this means that the AI does not have

### What is that [2:38]

a long-term vision of what it wishes to do, and  barely remembers what it did just a few frames   ago. As a result, these landscapes start morphing  into something completely different very quickly. And now, you know what’s coming, so hold onto  your papers and let's look at the new technique!

### New technique [3:03]

My goodness, I love it! So much  improvement in just a year. Now,   everyone can see that these are also not perfect,   but this kind of improvement just one more  paper down the line is nothing short of amazing. Especially that it doesn’t only have better  quality. No-no, it can do even more. It offers

### More control [3:27]

us more control too! Now we can turn the camera  around and whenever we see something interesting,   we can decide which direction we wish to go.   And the result is that now, we can create and   also control these beautiful, long aerial  videos where after the first few frames,   every single thing is synthesized by an AI.   How cool is that? What a time to be alive!

### Video synthesis [3:52]

And, it doesn’t stop there. It gets even better.   If you have been holding on to your papers so far,   now squeeze that paper, because this new AI  synthesizes these videos…without ever having   seen one. That’s right, it had never seen a video.   The previous work was trained on drone videos,   but training this one only requires  a collection of single photographs.    Multiple views and the camera position  are not required. That is insanity. This

### Conclusion [4:26]

AI is so much smarter than the previous  one that was published just a year ago,   and it requires training data that is much  easier to produce at the same time. And,   I wonder what we will be capable of just  two more papers down the line. So cool! Thanks for watching and for your generous  support, and I'll see you next time!