# This AI Creates A Moving Digital Avatar Of You

## Метаданные

- **Канал:** Two Minute Papers
- **YouTube:** https://www.youtube.com/watch?v=NlZJlFCh8MU
- **Дата:** 11.12.2019
- **Длительность:** 5:28
- **Просмотры:** 125,516
- **Источник:** https://ekstraktznaniy.ru/video/14210

## Описание

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 

📝 The paper "Neural Volumes: Learning Dynamic Renderable Volumes from Images" is available here: 
https://research.fb.com/publications/neural-volumes-learning-dynamic-renderable-volumes-from-images/

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Haro, Anastasia Marchenkova, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Benji Rabhan, Brian Gilman, Bryan Learn, Christian Ahlin, Claudio Fernandes, Daniel Hasegan, Dan Kennedy, Dennis Abts, Eric Haddad, Eric Martel, Erik de Bruijn, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, John De Witt, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Grah

## Транскрипт

### <Untitled Chapter 1> []

Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. In this series, we talk about research on all kinds of physics simulations, including fluids, collision physics, and we have even ventured into hair simulations. We mostly talk about how the individual hair strands should move, and how they should look, in terms of color and reflectance. Creating these beautiful videos takes getting many-many moving parts right, for instance, before all of that, the very first step is not any of those steps. First, we have to get the 3D geometry of these hairstyles into our simulation system. In a previous episode, we have seen an excellent work that does this well for human hair. But what if we would like to model not human hair, but something completely different? Well, hold on to your papers, because this new work is so general, that it can look at an input image or video, and give us not only a model of the human hair, but human skin, garments, and of course, my favorite, smoke plumes, and more. But if you look here, this part begs the following question - the input is an image, and the output also looks like an image, and we need to make them similar - so what’s the big deal here? A copying machine can do that, no? Well, not really. Here’s why. On the output, we are working with something that indeed looks like an image, but it is not an image. It is a 3 dimensional cube, in which we have to specify color and opacity values everywhere. After that, we simulate rays of light passing through this volume, which is a technique that we call ray marching, and this process has to produce the same 2D image through ray marching as what was given as an input. That’s much, much harder than building a copying machine. As you see here, normally, this does not work well at all, because, for instance, a standard algorithm sees lights in the background, and it assumes that these are really bright and dense points. That is kind of true, but they are usually not even part of the data we would like to reconstruct. To solve this issue, the authors propose learning to tell the foreground and background images apart, so they can be separated before we start the reconstruction of the human. And this is a good research paper, which means that if it contains multiple new techniques, each of them are tested separately to know how much they contribute to the final results. We get the previously seen, dreadful results without the background separation step, here are the results with the learned backgrounds, we can still see the lights due to the way the final image is constructed, and the fact that we have so little of this halo effect is really cool. Here you see the results with the true background data where the background learning step is not present. Note that this is cheating, because this data is not available for all cameras and backgrounds, however, it is a great way to test the quality of this learning step. The comparison of the learned method against this reveals that the two are very close, which is exactly what we are looking for. And finally, the input footage is also shown for reference. This is ultimately what we are trying to achieve, and as you see, the output is quite close to it.

### Example Reconstructions [3:31]

As you see here, the final algorithm excels at reconstructing volume data for toys, smoke plumes, and humans alike. And the coolest part is that it works for not only stationary inputs, but for animations

### Warping Method Comparison - "Hair Swing" dataset [3:41]

as well. Wait, actually, there is something that is perhaps even cooler, with the magic of neural networks and latent spaces, we can even animate this data. Here you see an example of that where an avatar is animated in real-time by moving around this magenta dot. A limiting factor here is the resolution of this reconstruction - if you look closely

### Hybrid Rendering with 3D Meshes [4:02]

you see that some fine details are missing, but you know the saying…given the rate of progress in machine learning research, two more papers down the line, and this will likely be orders of magnitude better. If you feel that you always need to take your daily dose of papers, my statistics show that many of you are subscribed, but didn’t use the bell icon. If you click this bell icon, you will never miss a future episode and can properly engage in your paper addiction. This episode has been supported by Lambda. If you're a researcher or a startup looking for cheap GPU compute to run these algorithms, check out Lambda GPU Cloud. I've talked about Lambda's GPU workstations in other videos and am happy to tell you that they're offering GPU cloud services as well. The Lambda GPU Cloud can train Imagenet to 93% accuracy for less than $19! Lambda's web-based IDE lets you easily access your instance right in your browser. And finally, hold on to your papers, because the Lambda GPU Cloud costs less than half of AWS and Azure. Make sure to go to lambdalabs. com/papers and sign up for one of their amazing GPU instances today. Our thanks to lambda for helping us make better videos for you. Thanks for watching and for your generous support, and I'll see you next time!