DeepMind’s New Gaming AI Does The Impossible!

I am stunned, this is one of the most exciting  research works I have seen in quite a while. We   thought that this was impossible, but scientists  at Google DeepMind wrote an AI called Genie that   makes a game from scratch. Almost. You  give it an input image, and look at that,   you can now play with it. The input can  be a drawing, or almost anything you want. But all that is nothing compared to  what just happened. That was version one   that only supported these silly little platformer  games. But that was just nine months ago,   and now here is Genie version 2. What can  this do? Look at that…in goes an image,   even a real or made up landscape, and out  comes a much more sophisticated game. Wow. Now we always say The First Law Of Papers  says that research is a process. Do not   look at where we are, look at where we  will be two more papers down the line.    And now we are one more paper down the  line, so here is the moment of truth. And by the way, by the end of the  video, you’ll find that this AI is   not really made because of video games.   Not at all. I’ll tell you about that too. So let’s take Genie 2 out for a spin and see what  it can do through 10 amazing examples. And then,   we will also break it unexpected,  and funny ways. You’ll love it. Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. One, do you remember the Oasis AI? Creating  a playable Minecraft world from scratch? That   was amazing, but look. I mean look away. And  now look back. You’re suddenly somewhere else.    Why? Because it did not have long-term memory.   This Genie 2 came out just a few weeks later,   and look at that. Look away, look back.   yup. Same scene. This one has longer-term   memory. All this progress appears just a few  weeks later. I can’t believe this. Bravo. Two, this can create games with a variety  of art styles and environments. So good. Three, you can control these characters with a  keyboard, directional buttons and jump too, and   you get these lovely custom character animations.   But you will see that there is more here. Four, finally, these are actual 3D  scenes you can navigate and play in. That’s great news because five,   you can even interact with these  objects. You can open that door,   shoot a barrel, or become a samurai. A really  pixelated samurai, but a samurai nonetheless. And six, NPCs, non-player characters  and interactions with them too,   so you can get a sidekick or make a  huge boss level for your players. I   do not know what happened here. If you  do, let me know in the comments below. And no, we are not going to forget about the Two  Minute Papers special. Seven, physics and smoke   simulations. Wow. Yes, low resolution, but once  again, just think about what we will be capable   of two more papers down the line. Gravity works  too. So be careful out there, Fellow Scholars. Eight, look at that, are you seeing  what I am seeing? Lighting effects   work really well, and so do  reflections. Chef’s kiss! So with all this, nine, you can do really  rapid prototyping. Go from idea to game   real quick. Should the game be about a paper  plane? Nope, it should be a hawk. Or wait,   a dragon it is. Or just us. Anything goes,  you don’t have to invest a year and find out   that the idea wasn’t so good because you can  create all of these games super quickly. Now hold on to your papers Fellow Scholars,  because all this was for show, because ultimately,   this whole thing is not about gaming. Yes,  you heard it right. They built this AI not   just to play video games, although that is quite  nice with a Google DeepMind salary I gotta say,   but the real goal is to learn a world  model. To be able to write a game,   you have to understand how the world works,  or even better, how a variety of different   worlds with different rules work. And then,  what do you do? Oh my. Here is SIMA. Yes,   finally! They did exactly what I was hoping  that they would do. They let their gaming AI   play in this environment. We can instruct it to  go up the stairs or go through the middle door. This will be fantastic, because there are  areas within AI that suffer from a lack of   training data. Text is not one of them, just  read the internet. We also have lots of video   data, there is Youtube and many  other websites for that, however,

what they don’t give you is counterfactual  information. That means what if situations   that start from the same starting point, but  then, different scenarios play out. And, yes,   that is exactly what this system can give us. A training environment that can give you for all   intents and purposes unlimited data. This could be  used to train robots or self-driving cars safely   in life-like environments so that they would be  able to adapt to new situations in real life. Now, wait. We are not done yet, the  best is yet to come because sometimes   unexpected and funny things happen in  these environments. Fellow Scholars,   you are going to love this. You think you made  a game about snowboarding? Nope, parkour it is!    Or if you let loose a magic spell, you have  to be ready for the consequences. Goodness. Or get this, imagine going out  for a coffee and reading a paper,   and when you come back…wait what? Is that  a ghost? That is insanity! But, in return,   we will have self-driving cars that can deal  with ghosts and destructive magic spells.    I’ll take that any day of the week thank  you very much. What a time to be alive! So, what do you think? What  would you Fellow Scholars   use this for? Let me know in the comments below.