NVIDIA’s AI: Amazing DeepFakes And Virtual Avatars!

Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. Today we are going to see that the  research papers that you see here   in this series are real. Here you see NVIDIA’s  game changing videoconferencing AI. So what does   this do? Why is this so interesting? How does  this transfer a video of us over the internet?    Well, here is a crazy idea. It doesn’t! What?   Transmitting video without transmitting video?    How is that even possible?   Well, now, it is possible! What they do in this work, is take only the  first image from the video, and they throw away   the entire video afterwards! But before that,  it stores a tiny bit of information from it,   which is, how our head is moving over time,  and how our expressions change. That is an   absolutely outrageous idea, except the fact that  it works. It not only works, but it works really   well. And because this is an amazing paper,  it does not stop there - it can do even more. Look at these two previous methods  trying to frontalize the input video.    This means that we look to the side a little,  and the algorithm synthesizes a new image of   us as if the camera was right in front of us.   That sounds like a science fiction movie, except   that it seems absolutely impossible given how  much these techniques are struggling with the   task…until we look at the new method. My goodness.   There is some jumpiness in the neck movement   in the output video here, and some warping issues  here, but otherwise, very impressive results.    Now if you have been holding on to your papers  so far, now, squeeze that paper, because these   previous methods are not some ancient papers  that were published a long time ago. Not at   all! Both of them were published within the same  year as the new paper. How amazing is that. Wow. And it could also perform these deepfakes too.   Look! We only need one image of the target person,   and we can transfer all of our gestures to  them, in a way that is significantly better   than most previous methods. Now, of course,  not even this technique was perfect,   it still struggled a great deal in the presence  of occluder objects, but still, just the fact   that this is now possible feels like we are  living in the future. What a time to be alive! Now, I said that today we are going to  see that this paper is as real as it gets,   so, what does that mean? Well, today,   anyone can try this technique. Yes, just  one year after publishing this paper,   and it is now available as a demo. The link to  it is available in the video description below. And, get this, some people at NVIDIA are already  using it for their virtual meetings. And by “it”,   I mean both the compression engine, where  the previous industry standard compression   algorithm could do this when given very little  data. That is not much. But the new technique,   with the same amount of data, can now  do this. That is insanity. Loving it. And, they also use their own gestures to create  these deepfakes and make virtual characters   come alive, or to frontalize their videos when  talking to each other. It almost feels like we   are living in a science fiction movie.   And all this is out there for us to use. Also, these technologies will soon  be part of the NVIDIA Video Codec SDK   as the AI Face Codec, which means that it will be  soon deployed to an even wider audience to use.    These companies are already using it. So,  this is one more amazing example that shows   that the papers that you see here in Two Minute  Papers are real. Sometimes so real that we can go   from a research paper to a product in just  a year. That is absolutely miraculous. So, what do you think? What would you use  this for? Let me know in the comments below!

Thanks for watching and for your generous  support, and I'll see you next time!