DeepMind’s New AI Makes Games From Scratch!

goodness deep mind's new work might be one of the best papers of the year so what is going on here well today we can use AI techniques to generate images from text videos from text but wait are you thinking what I am thinking it is great to look at all this but this is a game I don't just want to look I want to play and deep mind's amazing new paper is about exactly that dear fellow Scholars this is two minute papers with Dr Caro now believe it or not we already looked at an earlier Nvidia paper that did something like this so is this a solved problem what did it do exactly and what does deep mind's new work do that's perhaps even better well this is nvidia's game G normally if we wish to write a computer game we first Envision the game in our mind then we sit down and do the programming but this paper did this completely differently it first looked at someone playing the game and then it was able to code up the game so that it not only looks like it but it also behaves the same way to our key presses you see it at work here yes this means that we can even play with it and it learns the internal rules of the game and the graphics just by looking at some gameplay we don't need access to the source code or the internal workings of the game as long as we can just look at it can learn the rules and scientists a deep mind just put out a paper that made my jaw drop I mean look at the list of authors this is essentially a super group wow I am very excited so what does this do well it doesn't even need to look at an or existing game because it makes a game from scratch oh my this sounds not like text to image not even text to video but text to video game so here's the promise in goes a piece of text the text goes into a text to image AI that produces an image and now hold on to your papers as we can now start playing with that image wow just look at that a fully AI assisted workflow it recognizes who should be the playable character and which of these is the environment creates the controls for this character like moving around and jumping it also learned the parallx effect so it knows which the foreground and background is how far away they are and how quickly they should move compared to each other Bravo this is already incredible but it gets better so far the input was eventually an image and that image can also be a photo from The Real World you add the photo and out comes a playable game yes we will talk about the fact that this is quite pixelated in a moment but you see we don't even necessarily need a photo from The Real World we can also use a sketch just draw something and you get a game out of it my goodness isn't that the dream so good and it does all this watching videos on the internet and let's have a look at how it relates to previous techniques those required additional information for instance they needed to know the buttons that were pressed but this one look this one is completely unsupervised that means that we don't even need to label the videos and show which is the playable character and what buttons were pressed nothing like that just Chuck in the videos and now little AI it is your time to figure it all out and so it does lovely now this does not go as quickly as you see here this runs at one frame per second and as all of you fellow scholar can see it is also heavily pixelated but we are wise Scholars here so we are not fooled by the low pixel count it is not like the super high resolution detailed output like open AI Sora can produce however I feel that we are currently at a dolly1 moment of text to game what does that mean well this is what Dolly one was capable of creating and approximately a year later version two could do this it was a huge leap in capabilities and just imagine what this

technique will look like by the time it will go to version three I cannot wait to have a look at that just imagine using this together with the Miracles that computer Graphics researchers are capable of today for instance yes what you are seeing here is not reality this is a completely madeup scene in a computer simulation written by hand and it computes how the water and everything should move and a retracing based technique simulates what this should look like and now fellow scholar imagine two more papers down the line This AI looking at footage like this and then we would play with that oh my goodness what a time to be alive I have written a comment paper on this in nature physics it is very short and an easy and fun read and it is freely available for all of you fellow Scholars it is about simulations that look like reality the link is available in the video description and every good paper goes beyond its own scope a little and it is no different for this new work yes this paper may also help robots that move around in this world that is fantastic because robotics has a huge data problem and this work might give this research area just what it needs to improve quicker and it can also learn a bit about defamations we can play with this bag of chips and it knows what is supposed to happen so this creates the games of today and it trains the robots of Tomorrow what an excellent paper Bravo if you're looking for inexpensive Cloud gpus for AI Lambda now offers the best prices in the world for GPU Cloud compute no commitments or negot negotiation required just sign up and launch an instance and hold on to your papers because with the Lambda GPU Cloud you can now get on demand h100 instances and they are one of the first Cloud providers to offer publicly available on demand h100 XS did I mention they also offer persistent storage so join researchers at organizations like apple MIT and Caltech in using lamb Cloud instances workstations or servers make sure to go to lamb. com slapers to sign up for one of their amazing GPU instances today