New AI: Impossible Creatures Come Alive!

Alright, two top-tier, absolutely amazing AI  papers today. This AI animates creatures it   has never seen before. And hereAnd here,  even a dinosaur can learn something from   a mystery creature. And this one makes a  poorly made drawing into a fully-fledged   3D animation. Both solve problems  that I thought were impossible. Oh my. So, what is this? People talk a great deal about  breathing life into virtual characters. That is   computer animation. But whenever they talk about  computer animation, they are talking about moving   virtual humanoids around. Humans, mostly. But  what about these guys? Can we still animate them? Okay, the problem is the following: you  are not an animator, so you give an AI   the shape of the character that they call  skeleton, and out comes how it would move. And then of course, these skeletons  are just an underlying structure,   and the final appearance of the  character is of course, on top of that. Well, people tried to do that  with automated techniques, but   the results were…unsatisfactory to say the least.    But with this new method… surprise! These are  really believable motions on a wide variety of   icky creatures. That would already be good, but  this is nothing compared to what is coming now. These animals don’t just move around, they  can learn from each other. For instance,   this dinosaur is doing something…interesting.   Standing on one leg? Hmm…where did it learn   this from? Oh yes, it learned it from a  flamingo. Imagine that! And note that the   insane part is that this is not just  copying, it has learned the motion,   and has adapted it to its own body. That is  absolutely incredible. And with this system,   a chicken can also learn how to jump  from an ostrich as well. Loving it. So, adaptation is possible. But how far can  we push it? Well, let’s give it a try. Here,   scientists have excluded the cat and the komodo  dragon from the training set, so the AI has never   seen them. And if we do that with a previous  technique, unfortunately it seems completely   clueless. Like a confused pile of digital  limbs having an existential crisis. No wonder,   because it hasn’t seen this kind of animal  before. Now let’s see what the new one can do…I   can’t believe it. Motions generated  for unseen animals. This is insanity! So how on earth does it do that? Well, it has  looked at a bunch of animals moving around,   and there are parts in different animals that are  semantically similar. Parts that have a similar   function. For instance, arms and legs are a  general concept, and if you look at a new animal,   having seen a lot of different kinds of arms and  leg morphologies in action, you may be able to   infer how it is used. You Fellow Scholars can do  that, of course, because you are all brilliant.    Man, I love my audience. But the incredible  part is that here, a machine is doing that.    Generalizing knowledge to unseen examples.   And to me that sounds like intelligence. Wow. You’ll love this one. Look. So we can even peer  into what it is thinking about these motions.    It understands when this animal is relaxed, and  when it is attacking. And to demonstrate that it   really understands this, it shows similar motions  in completely differently shaped animals. Yes,   it has learned the concept of attacking.   Same vibe, different body. So cool! And it also understands when other, when  different kinds of motions like falling   and growling start and when they end.   And how they look in different animals. But it gets even better. Now hold  on to your papers Fellow Scholars,   because it can even take an input motion that  is incomplete, and when it switches to orange,   you see how the AI synthesized the rest of the  motion. Goodness, this is like image inpainting,   filling out missing parts of an  image, only for animation. Brilliant. And now…look. Are you kidding me? Here, they took  an animation of some body parts and left out the   rest, and the orange part, for instance, the  upper body motion, is then simulated by the AI. The source code is freely available for all  of us to experiment with. Thank you so much! Now, imagine combining it with a technique  that could take a character, in this case,   humanoids. Then, you just draw a line to indicate  what it should be doing, and it would perform the

motion accordingly. That is of course possible,  and that technique is called, becoming an artist   who is excellent at their job. But doing this  automatically with AI? Not a chance. Why? Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. I’ll show you why. For instance, let’s  punch something this way and see if that   is possible. Here is the line, now,  little AI, you do the rest. Well,   it is loading up, but then…hey! Nothing  happens. Where is the punch? This other one,   look carefully. Why did it do that? Here is the  problem: we, humans, understand the world in 3D,   even if shown just a piece of drawing on  a 2D paper, but this technique doesn’t.    It does not understand that we meant the line is  forward, not sideways. But of course, once again,   how on earth would it understand it? If we wanted  to move backwards, we would use the same line. So   it would kind of have to understand the  intention behind the drawing as well,   which is of course, impossible. No AI  can read minds, so even when we would   try a new method to do tha…. wait a minute. It  understood! This is exactly what we wanted! Wow. So, yes, with this other technique, we can create  a simple storyboard without any artistic training,   and, it works incredibly well already. One  more problem that used to be impossible,   which is now, possible. It is truly stunning  what it is capable of, and once again. Yup.    Look at that. These papers might as well be  some secret superprojects, because almost   nobody knows about them. This is why I am making  Two Minute Papers, and I hope you appreciate it   too. This is arcane knowledge of the impossible.   I don’t think you hear about these anywhere else.