# This New AI Creates Lava From Water! 🌊

## Метаданные

- **Канал:** Two Minute Papers
- **YouTube:** https://www.youtube.com/watch?v=S-Jj3ybaUNg
- **Дата:** 04.01.2022
- **Длительность:** 6:59
- **Просмотры:** 57,006
- **Источник:** https://ekstraktznaniy.ru/video/13710

## Описание

❤️ Check out the Gradient Dissent podcast by Weights & Biases: http://wandb.me/gd 

📝 The paper "VGPNN: Diverse Generation from a Single Video Made Possible" is available here:
https://nivha.github.io/vgpnn/

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, Christian Ahlin, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Bustamante, John Le, Jonas, Kenneth Davis, Klaus Busse, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Mark Oates, Michael Albrecht, Michael Tedder, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Peter Edwards, Rajarshi Nigam, Ramsey Elbasheer, Steef, Taras Bobrovytsky, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: https://www.patreon.com/TwoMinutePapers

Thumbnail background design: Felícia Zso

## Транскрипт

### <Untitled Chapter 1> []

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. This new learning-based method is capable of generating new videos, creating lava from water, enhancing your dance videos, adding new players to a football game, and more. This one will feel like it can do absolutely anything! For instance, have a look at these videos. Believe it or not, some of these were generated by an AI from this new paper. Let’s try to find out together which ones are real, and which are generated. Now before we start, many of the results that you will see here will be flawed, but I promise that there will be some gems in there too. This one doesn’t make too much sense. One down. Two, the bus appears to be changing length from time to time. Three, the bus pops out of existence. Four…wait…this actually looks pretty good…until we find out that someone is about to get rear-ended. This leaves us with two examples. Are they real, or are they fakes? Now, please stop the video and let me know in the comments. So, this one is real footage. Okay, what about this guy? Well, get this…this one is actually one of the generated ones. Just this one was real. And this is what this new technique is capable of. We give it a sample video, and it reimagines variations of it. Of course, the results are hit or miss, for instance, here, people and cars just pop in and out of existence, but if we try it on the billiard balls. Wow, now this is something. Look at how well it preserved the specular highlights, and also, the shadows move with the balls in the synthetic variations too. Once again, the results are generally quite hit or miss, but if we look through a few results, we often find some gems out there. But this is nothing compared to what is to come. Let’s call this first feature video synthesis, and we have 4 more really cool applications with somewhat flawed, but sometimes amazing results. Now, check this out, here comes number two, video analogies.

### Video Analogies [2:20]

This makes it possible for us to mix two videos that depict things that follow a similar logic. For instance, here, only four of these sixteen videos are real, the rest are all generated. Now, here comes feature number three - time retargeting.

### Time Retargeting [2:39]

For instance, we can lengthen or shorten videos. Well, that sounds simple enough, no need for an AI for that…but here is the key! This does not mean just adding new frames to the end of the video. Look! They are different. Yes, the entirety of the video is getting redesigned here. We can use this to lengthen these videos without really adding new content to it. Absolutely amazing. We can also use it to shorten a video. Now, of course, once again, this doesn’t mean that it just chops off the end of the video. No-no! It means something much more challenging. Look! It makes sure that all of your killer moves make it into the video, but, in a shorter amount of time. It makes the video tighter if you will. Amazing feature number 4.

### Four Sketch to Video [3:34]

Sketch to video. Here, we can take an input video, add a crude drawing, and the video will follow this drawing. The result is, of course, not perfect, but, in return, this can handle many number to number transitions. And now, feature number five. Video inpainting on steroids. Previous techniques can help us delete part of an image, or even a video, and generate data in these holes that make sense given their surroundings. But this, this one does something way better! Look, we can cut out different parts of a video, and mark the missing region with a color, and then, what happens? Oh yes! The blue regions will contain a player from the blue team, the white region white team, or, if we wish to get someone out of the way, we can do that too. Just mark them green. Here, the white regions will be inpainted with clouds, and the blues with birds. Loving it. But wait a second, we noted that there are already good techniques out there for image inpainting. There are also good techniques for video time retargeting. So, what is so interesting here? Well, two things. One, here all of these things are being done with just one technique. Normally, we would need a collection of different methods to perform all of these. But here, just one AI. Two, now hold on to your papers, and look at this! Whoa. Previous techniques take forever to generate these results. 144p means an extremely crude, pixelated image, and even then, they take from hours…to, my goodness. Even days. So, what about this new one? It can generate much higher resolution images, and, in a matter of minutes. That is an incredible leap in just one paper. Now, clearly, most of these results aren’t perfect. I would argue that they are not even close to perfect. But some of them already show a lot of promise. And, as always, dear Fellow Scholars, do not forget to apply the First Law of Papers. Which says, do not look at where we are. will be two more papers down the line. And line, I am sure that not two, but five, or maybe even six out of these six videos will look significantly more real. Thanks for watching and for your generous support, and I'll see you next time!