# This New AI Makes DeepFakes... For Animation Movies! 🧑‍🎨 ## Метаданные - **Канал:** Two Minute Papers - **YouTube:** https://www.youtube.com/watch?v=4lQkQSmA8nA - **Дата:** 12.04.2022 - **Длительность:** 6:26 - **Просмотры:** 359,461 - **Источник:** https://ekstraktznaniy.ru/video/13599 ## Описание ❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.com/papers 📝 The paper "Stitch it in Time: GAN-Based Facial Editing of Real Videos" is available here: https://stitch-time.github.io/ ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers - https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Bustamante, John Le, Jonas, Jonathan, Kenneth Davis, Klaus Busse, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Michael Albrecht, Michael Tedder, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Paul F, Rajarshi Nigam, Ramsey Elbasheer, Steef, Taras Bobrovytsky, Ted Johnson, Thomas Krcma ## Транскрипт ### Intro [] Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today we are going to use this new AI film director technique to reimagine videos and even movies just by saying what we want to add to it. Yes, really! And you will see that the results are absolute insanity. Now, previous techniques are already able to take a photo of someone, add a beard, make ### This new work is different [0:23] them young or old, but this work, this is something else. Why? Well, this can do the same for not only one still image, but for an entire video. Just imagine how cool it would be to create a more smiling version of an actor without actually having to re-record the scene. ### Obama speech reimagined [0:50] Hold on to your papers, and look at how we can take the original video of an Obama speech, and make it a little happier by adding a smile to his face. Or even make him younger, or older. These are absolutely amazing. I love the grey hair, I love the young man hair, I love how the gestures and head turns are going through. This looks like something straight out of a science fiction movie. ### Temporal coherence [1:26] Loving it. Now, what is the key here? The key here is that this AI knows about temporal coherence. What does that mean? Well, it means that it does this in a way when it synthesizes the second image, it remembers what it did to the first image, and takes it into consideration. That is not trivial at all. And if you feel that this does not sound like a huge deal, let’s have a look at an earlier technique that doesn’t remember what it did a moment ago. Prepare your eyes for a fair bit of flickering. This is an AI-based technique from about 6 years ago that performs style transfer. And when creating image 2, it completely disregards what it did in image 1. As you see, this really isn't the way to go. And, with all this newfound knowledge, let’s look at the new footage of the new AI and look at the temporal coherence together. Just look at that. That is absolutely incredible. I am seeing close to perfect temporal coherence. My goodness. Now, not even this technique is perfect. ### Not perfect [2:41] Look, when trying it here, Mark Zuckerberg’s hair seems to come to a life of its own. I also find the mouth movements a bit weaker than with the other results. ### A DeepFake of ... me! [2:56] Now, if you allow me, let me also enter the fray. Well, this is not me, but how the AI reimagines me as a young man. Good job little man, keep those papers coming! This is incredible, I love it. And this is what I actually look like. Okay, just kidding. This is old man Károly as imagined by the AI, who is vigorously telling you that papers were way better back in his day. This is the original. And this is my sister, Karolina filling in when I am busy. I would say that the temporal coherence is weaker here, but it is understandable because the AI has to do a ton of heavy lifting by synthesizing so much more than just a smile. And even with all these amazing results, so far you have seen nothing yet. No sir. It can do even more. ### It works on animated characters too [3:58] If you have been holding on to your papers so far, now, squeeze that paper, because it works not only on real humans, but, animated characters too. And just look at that. We can order these characters to smile more, be angrier, or happier. And we can even add lipstick to a virtual character, without having to ask any of the animators and modelers of this movie. ### So much improvement in so little time [4:30] How cool is that? And remember, we were lucky to be able to do this to one still image just a couple papers ago. And now, all this, with temporal coherence. And, I hope you are enjoying these results, many of which you can only see here on Two Minute Papers. Nowhere else. And, of course, I would like to send a huge thank you to the authors who took time off their busy day to generate these just for us. ### It has its limits [5:00] Now, even the animated movie synthesis has some weaker results, in my opinion, this is one of them, but still, very impressive. And don’t even get me started about the rest, these are so good! And just imagine what we will be able to do just a couple more papers down the line. My goodness. What a time to be alive! So, what would you use this for? What do you expect to happen a couple more papers down the line? Please let me know in the comments below. I’d love to hear your thoughts. Thanks for watching and for your generous support, and I'll see you next time!