DeepMind’s AI Athletes Play In The Real World!

Dear Fellow Scholars, this is Two Minute Papers  with Dr. Károly Zsolnai-Fehér. Dr. Papers. Today you are going to see an insane paper. One of  the most fun papers I’ve read in a while. Earlier,   scientists at DeepMind were looking to teach  a bunch of AI agents to play football/soccer.    Unfortunately, they started out like this.   Not very impressive. But then, they trained   them for 5 years. What? 5 years? Well, their  training took what felt like 5-years for them,   but only 3 days for us, because these are not  real robots, and thus, on a fast computer,   they can be simulated faster than real  time. So what happened in that time? This happened! Whoa! Now they can  not only play amazingly well, but,   they can even think a few steps ahead,  and they are even anticipating the   behavior of their teammates and  position themselves accordingly. The saves they are capable of are  also outstanding. And as we don’t   really have a referee around, they also  beat up each other. And when I say that,   I really mean it. It’s bad. Real bad. Also,  the only thing they like more than throwing   each other is throwing tantrums. And  it even gets worse. This lovely chap   almost seems like someone who is faking an  injury, but then, look! A miracle happened! And today, scientists at DeepMind  brought this work to the next level. How? In a different paper, scientists at OpenAI  simulated a robot hand in a video game world.    Then, the things it learned in this simulation  could be brought into life by deploying it into   the real world. So train in a simulation, and then  play in the real world. Yes, that is fantastic,   but, why am I saying this? Oh wait, is is  possible that the same happened here too? Real   robots playing football? Yes! That is exactly what  happened. So with that, let the insanity begin. The new environment contains 2 real robots with  20 actuated joints each. So what did they learn   first? Well, first, they learned to get up, and  also, score goals. All this in a simulated robot   body. That is very impressive already, but wait,  because that’s just the start. Now it is time for   random perturbations. What does that mean? It  means that forces are randomly exerted on the   AI to make sure it can recover from an impact.   And, are you thinking what I am thinking? Oh yes,   now let’s go to the real robots, and let’s test  that impact. Aww! That is not very nice. But   then…wow! Look at that! What it had learned  in a simulation now really applies to a real   life robot. It can get hit, and it is able to  get up over and over again. My goodness. But,   there is a problem. What is the problem? Well,  their knees get injured all the time. So what   did scientists at DeepMind do with that?   Well, hold on to your papers Fellow Scholars,   because they taught the robot to get up and move  in a way that is not too hard on the knees. Yes,   that’s right! And thus, they didn’t suffer that  many knee injuries. What a time to be alive! So with that, finally, they can now run, and  score goals. But wait, playing a match and   winning takes more than just running and kicking.   They need to have a variety of other skills and   combine them in order to win. So, did they? Oh  yes they did! They can now control the ball,   turn with it, block the ball with their  bodies, shoot accurately, even in the   case of a moving ball, which is even harder.   And that is the miracle of research papers:   such incredible progress can happen  just one more paper down the line. And, yes, I know, I know. You are  asking the question, okay, but Károly,   did they beat up each other here too like in the  previous paper? And the answer is no. You see,   they are penalized for getting too close to  each other, so luckily that did not happen.    But this kind of improvement in just one paper, I  am wondering what these little AIs will be capable   of two more papers down the line? Well, what do  you think? Let me know in the comments below!

Thanks for watching and for your generous  support, and I'll see you next time!