DeepMind’s AI Plays Catch…And So Much More! 🤖

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today we are going to see how an AI can win a complex game that it has never seen before. Zero prior training on that game. Yes, really! Now, before that, for context, have a look at this related work from 2019, where scientists at OpenAI built a super fun hide and seek game for their AI agents to play. And, boy, did they do some crazy stuff. Now, these agents learn from previous experiences, and to the surprise of no one, for the first few million rounds, we start out with…pandemonium. Everyone just running around aimlessly. Then, over time, the hiders learned to lock out the seekers by blocking the doors off with these boxes and started winning consistently. I think the coolest part about this is that the map was deliberately designed by the OpenAI scientists in a way that the hiders can only succeed through collaboration. But then, something happened. Did you notice this pointy, doorstop-shaped object? Are you thinking what I am thinking? Well, probably, and not only that, but later, the AI also discovered that it can be pushed near a wall and be used as a ramp, and, tadaa! Got’em! Then, it was up to the hiders again to invent something new. So, did they do that? Can this crazy strategy be defeated? Well, check this out. These resourceful little critters learned that since there is a little time at the start of the game when the seekers are frozen, apparently, during this time, they cannot see them, so why not just sneak out and steal the ramp, and lock it away from them. Absolutely incredible. Look at those happy eyes as they are carrying that ramp. But today is not 2019, it is 2021, so I wonder what scientists at the other amazing AI lab, DeepMind have been up to. Can this paper be topped? Well, believe it or not, they have managed to create something that is perhaps even crazier than this. This new paper proposes that these AI agents look at the screen, just like a human would, and engage in open-ended learning where the tasks are always changing. What does this mean? Well, it means that these agents are not preparing for an exam. They are preparing for life! And hence, hopefully they learn more general concepts, and, as a result, maybe excel at a variety of different tasks. Even better, these scientists at DeepMind claim that their AI agents not only excel at a variety of tasks, but they excel at new ones they have never seen before! Those are big words, so, let’s see the results! The red agent here is the hider, and the blue is the seeker. They both understand their roles, the red agent is running, and the blue is seeking. Look, its viewing direction is shown with this lightsaber-looking line pointing at the red agent. No wonder it is running away! And, look, it manages to get some distance from the seeker, and finds a new, previously unexplored part of the map and hides there. Excellent. And you would think that the Star Wars references end here? No! Not even close. Look, in a more advanced variant of the game, this green seeker lost the two other hiders, and what does he do. Ah yes, of course, grabs his lightsaber, and takes the high ground. Then, it spots the red agent and starts chasing it. All this without ever having played this game before. That is excellent. In this cooperative game, the agents are asked to get as close to the purple pyramid as they can. Of course, to achieve that, they need to build a ramp. Which they successfully realize. Excellent. But it gets better! Now note that we did not say that the task is to build a ramp. get as close to the purple pyramid as we can. Does that mean that? …Yes, yes it does. Great job bending the rules, little AI! In this game, the agent is asked to stop the purple ball from touching the red floor. At first, it tries its best to block the rolling of the ball with its body, then, look!

It realizes that it is much better to just push it against the wall. And it gets even better, look, it learned that best is to just chuck the ball behind this slab. It is completely right, this needs no further energy expenditure, and the ball never touches the red floor again. Great! And finally, in this King of the Hill game, the goal is to take the white floor and get the other agent out of there. As they are playing this game for the first time, they have no idea where the white floor is. As soon as the blue agent finds it, it stays there…so far so good. But, this is not a cooperative game, we have an opponent here. Look! Boom! A quite potent opponent indeed who can take the blue agent out, and, it understands that it has to camp in there and defend the region. Again. Awesome! So, the goal here is not to be an expert in one game, but to be a journeyman in many games. And these agents are working really well at a variety of games without ever having played them. So, in summary, OpenAI’s agent - expert in a narrower domain. DeepMind’s agent - journeyman in a broader domain. Two different kinds of intelligence. Both doing amazing things. Loving it. What a time to be alive! Scientists at DeepMind have knocked it out of the park with this one. They have also published AlphaFold this year, a huge breakthrough that makes an AI predict protein structures. Now, I saw some of you asking why we didn’t cover it. Is it not an important work? Well, quite the opposite! I am spellbound by it and I think that paper is a great gift to humanity, however. I try my best to educate myself on this topic, however, I don’t feel that I am qualified to speak about it. Not yet anyway. So, I think it is best to let the experts who know more about this take the stage! This is, of course, bad for views, but no matter, we are not maximizing views here. We are maximizing meaning. Thanks for watching and for your generous support, and I'll see you next time!