This AI Stuntman Just Keeps Getting Better! 🏃

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today we are going to see how an AI can learn crazy stunts…from just one video clip. And if even that’s not enough, it can even do more. This agent is embedded in a physics simulation, and first, it looks at a piece of reference motion, like this one. And then, after looking, it can reproduce it. That is already pretty cool, but it doesn’t stop there. I think you know what’s coming…yes! Not only learning, but improving the original motion. Look, it can refine this motion a bit…and then, a bit more…and then, a bit more. And this just keeps on going, until…wait a second. Hold on to your papers…because this looks impossible! Are you trying to tell me that it’s improved the move so much, that it can jump through this? Yes, yes it does. Here is the first reproduction of the jump motion, and the improved version side by side. Whoa. The difference speaks for itself. Absolutely amazing. We can also give it this reference clip to teach it to jump from one box to another. This isn’t quite difficult. And now comes one of my favorites from the paper! And that is testing how much it can improve upon this technique. Let’s give it a try! It also learned how to perform a shorter jump, a longer jump…and now, oh yes, the final boss. Wow, it could even pull off this super long jump. It seems that this super bot can do absolutely anything! Well…almost. And, it can not only learn these amazing moves, but it can also weave them together so well, that we can build a cool little playground, and it gets through it with ease… well, most of it anyway. So at this point, I was wondering how general the knowledge is that it learns from these example clips? A good sign of an intelligent actor is that things can change a little and it can adapt to that. Now, it clearly can deal with a changing environment, that is fantastic, but do you know what else it can deal with? And now, if you have been holding on to your papers, squeeze that paper, because it can also deal with changing body proportions. Yes, really. We can put it in a different body, and it will still work. This chap is cursed with this crazy configuration, and can still pull off a cartwheel. If you haven’t been exercising lately, what’s your excuse now? We can also ask it to perform the same task with more or less energy, or to even apply just a tiny bit of force for a punch, or to go full Mike Tyson on the opponent. So how is all this wizardry possible? Well, one of the key contributions of this work is that the authors devised a method to search this space of motions efficiently. Since it does it in a continuous reinforcement learning environment, this is super challenging. At the risk of simplifying the solution, their method solves this by running both an exploration phase to find new ways of pulling off a move, and, with blue you see that when it found something that seems to work, it also keeps refining it. Similar endeavors are also referred to as the exploration-exploitation problem, and the authors proposed a really cool new way of handling it. Now, there are plenty more contributions in the paper, so make sure to have a look at it in the video description. Especially given that this is a fantastic paper, and a presentation is second to none. I am sure that the authors could have worked half as much on this project and this paper would still have been accepted, but they still decided to put in that extra mile. And I am honored to be able to celebrate their amazing work together with you Fellow Scholars. And, for now, an AI agent can look at a single clip of a motion, and can not only perform

it, but it can make it better, pull it off in different environments, and it can even be put in a different body and still do it well. What a time to be alive! Thanks for watching and for your generous support, and I'll see you next time!