# AI Learns Facial Animation in VR

## Метаданные

- **Канал:** Two Minute Papers
- **YouTube:** https://www.youtube.com/watch?v=hkSfHCtpnHU
- **Дата:** 24.08.2019
- **Длительность:** 4:02
- **Просмотры:** 50,495
- **Источник:** https://ekstraktznaniy.ru/video/14264

## Описание

❤️ Check out Linode here and get $20 free credit on your account:
https://www.linode.com/papers

📝 The paper "VR Facial Animation via Multiview Image Translation" is available here:
https://research.fb.com/publications/vr-facial-animation-via-multiview-image-translation/

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
313V, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Brian Gilman, Bruno Brito, Bryan Learn, Christian Ahlin, Christoph Jadanowski, Claudio Fernandes, Daniel Hasegan, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, John De Witt, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, St

## Транскрипт

### Segment 1 (00:00 - 04:00) []

Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. One of the main promises of Virtual Reality, VR in short is enhancing the quality of our remote interactions. With VR, we could talk with our colleagues and beloved ones through telepresence applications that create a virtual avatar of us - much like the ones you see here. Normally, this requires putting sensors all over our faces to be able to reconstruct the gestures we make. A previous work used a depth camera that was hanging off of the VR headset, thus having a better look of the entirety of our face, while a later work used a mouth camera to solve this problem. This new paper attempts to capture all of our gestures by using a headset without these additional complexities by using no more than three infrared cameras. No extra devices hanging off of the headset, nothing. All of them are built into the headpiece. This means two key challenges: one is the fact that the sensor below sees the face in an uncomfortable, oblique angle, below you see exactly the data that is being captured by the three sensors. And two, the output of this process should be a virtual avatar, but it is unclear what the correspondence between all this data and the animated character should be. So the idea sounds great, the only problem is that this is near impossible. So how did the researchers end up doing this? Well, what they did is they built a prototype headset with 6 additional sensors. Now, wearing this headset would perhaps not be too much more convenient than the previous works we’ve looked at a moment ago. But don’t judge this work just yet, because this additional information is required to create the output avatar, and then, the smaller three-sensor headset can be trained by dropping these additional views. In short, the augmented, more complex camera is used as a crutch to train the smaller headset. Amazing idea, I love it. Our more experienced Fellow Scholars also know that there is a little style transfer magic being done here, and finally, all of these partial views are then stitched together into the final avatar. You can also see here that it smokes the competition, uses only three sensors, and does all this in real time. Wow. What an amazing paper. If you want to show your friends how you are about to sneeze in the highest possible quality video footage, look no further. Now, I am a research scientist by day, and I also run my own projects where I cannot choose my own hosting provider, and every time I have problems with it, I tell my wife that I wish we could use Linode. Linode is the world’s largest independent cloud hosting and computing provider, and they just introduced a GPU server pilot program. These GPU instances are tailor-made for AI, scientific computing and computer graphics projects. Yes, exactly the kind of works you see here in this series. If you feel inspired by these works and you wish to run your own experiments or deploy your already existing works through a simple and reliable hosting service, make sure to join over 800,000 other happy customers and choose Linode. Note that this is a pilot program with limited availability. To reserve your GPU instance at a discounted rate, make sure to visit https://www. linode. com/papers or click the link in the description and use the promo code “PAPERS20” to get 20 dollars free on your account. You also get super fast storage and proper support if you have any questions. Give it a try today! Our thanks to Linode for supporting the series and helping us make better videos for you. Thanks for watching and for your generous support, and I'll see you next time!