# New Face Swapping AI Creates Amazing DeepFakes!

## Метаданные

- **Канал:** Two Minute Papers
- **YouTube:** https://www.youtube.com/watch?v=duo-tHbSdMk
- **Дата:** 21.09.2019
- **Длительность:** 4:13
- **Просмотры:** 129,985
- **Источник:** https://ekstraktznaniy.ru/video/14250

## Описание

📝 The paper "FSGAN: Subject Agnostic Face Swapping and Reenactment" is available here:
https://nirkin.com/fsgan/

❤️ Pick up cool perks on our Patreon page: https://www.patreon.com/TwoMinutePapers

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Brian Gilman, Bruno Brito, Bryan Learn, Christian Ahlin, Christoph Jadanowski, Claudio Fernandes, Daniel Hasegan, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, John De Witt, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Matthias Jost,, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, Steve Messina, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten

## Транскрипт

### <Untitled Chapter 1> []

Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. Recently, we have experienced an abundance of papers on facial reenactment in machine

### Real-time Facial Reenactment [0:06]

learning research. We talked about a technique by the name Face2Face back in 2016, approximately 300 videos ago. It was able to take a video of us and transfer our gestures to a target subject. This was kind of possible at the time with specialized depth cameras, until Face2Face appeared and took the world by storm as it was able to perform what you see here with a regular consumer camera. However, it only transferred gestures, so of course, scientists were quite excited about the possibility of transferring more than just that. But, that would require solving so many more problems - for instance, if we wish to turn the head of the target subject, we may need to visualize regions that we haven’t seen in these videos, which also requires an intuitive understanding of hair, the human face and more. This is quite challenging. So, can this be really done? Well, have a look at this amazing new paper!

### Qualitative Face Swapping Results [1:12]

You see here the left image, this is the source person, the video on the right is the target video, and our task is to transfer not just the gestures, but the pose, gestures and appearance of the face on the left to the video on the right. And, this new method works like magic. Look! It not only works like magic, but pulls it off on a surprisingly large variety of cases, many of which I haven’t expected at all. Now, hold on to your papers, because this technique was not trained on these subjects, which means that this is the first time it is seeing these people. It has been trained on plenty of people, but not these people. Now, before we look at this example, you are probably saying, well, the occlusions from the microphone will surely throw the algorithm off, right? Well, let’s have a look. Nope, no issues at all. Absolutely amazing, love it! So how does this wizardry work exactly? Well, it requires careful coordination between no less than four neural networks, where each of which specializes for a different task. The first two is a reenactment generator that produces a first estimation of the reenacted face, and a segmentation generator network that creates this colorful image that shows which region in the image corresponds to which facial landmark. These two are then handed over to the third network, the inpainting generator, which fills the rest of the image, and since we have overlapping information, in comes the fourth, blending generator to the rescue to combine all this information into our final image. The paper contains a detailed description of each of these networks, so make sure to

### Comparison with Face2Face (Thies et al. 2016) [3:05]

have a look! And if you do, you will also find that there are plenty of comparisons against previous works, of course, Face2Face is one of them, which was already amazing, and you can see how far we’ve come in only three years. Now, when we try to evaluate such a research work, we are curious as to how much these individual puzzle pieces, in this case, the generator networks contribute to the final results. Are really all of them needed? What if we remove some of them? Well, this is a good paper, so we can find the answer in Table 2, where all of these components are tested in isolation. The downward and upward arrows show which measure is subject to minimization and maximization, and if we look at this column, it is quite clear that all of them indeed improve the situation, and contribute to the final results. And remember, all this from just one image of the source person. Insanity. Thanks for watching and for your generous support, and I'll see you next time!