This AI Makes Digital Copies of Humans! 👤

Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. Today we are going to look at a  paper with two twists. You know what,   I’ll give you twist number one right away. This  human isn’t here. This human isn’t here either.    And, neither is this human here.    Now, you are probably asking Károly,  what are you even talking about? Now hold on to your papers,  because I am talking about this.    Look. This is the geometry of this virtual  human inside a virtual world. Whoa. Yes,   all of these people are in a synthetic video, and  in a virtual environment that can be changed with   a simple click. And more importantly, as we change  the lighting or the environment, it also simulates   the effect of that environment on the character,  making it look like they are really there. So that sounds good, but how do we take  a human and make a digital copy of them? Well, first, we place them in a capture  system that contains hundreds of led lights   and an elaborate sensor for capturing  depth information. Why do we need these?    Well, all this gives the system plenty  of data on how the skin, hair and the clothes   reflect light. And, at this point, we know  everything we need to know and can now proceed   and place our virtual copy in a computer  game or even a telepresence meeting. Now, this is already amazing, but two things  really stick out here. One you will see   when you look at this previous competing work.   This had a really smooth output geometry,   which means that only few high-frequency  details were retained. This other work was   better at retaining the details, but, look, tons  of artifacts appear when the model is moving. And, what does the new one look like?   Is it any better? Let’s have a look.    Oh my! We get tons of fine details, and  the movements have improved significantly.    Not perfect by any means, look here,  but still, an amazing leap forward. Two, the other remarkable thing here  is that the results are so realistic   that objects the in virtual scene can cast a  shadow on our model. What a time to be alive! And now, yes, you remember that I promised  two twists. Where is twist number two? Well,   it has been here all along for the entirety of  this video. Have you noticed? Look, all this   is from 2019. From two years ago. Two years is  a long time in machine learning and computer   graphics research, and I cannot wait to see how it  will be improved two more papers down the line. If   you are excited too, make sure to subscribe and  hit the bell icon to not miss it when it appears. Thanks for watching and for your generous  support, and I'll see you next time!