Worlds FIRST Fully AI Generated Game Is Here (Oasis AI Minecraft Explained)

the game play you're currently seeing is 100% AI generated that means that this game play isn't pre-recorded it's being generated as someone inputs the keyboard now I want to talk to you guys about something groundbreaking that this company has recently achieved can you imagine a world where every frame every interaction and every piece of the game environment is generated by AI in real time responding instantly to what you do well that world is here and it's called Oasis so Oasis is the first playable realtime open world AI model game that generates an entire game albeit in Minecraft frame by frame using Transformer technology it's not just a game it's an interactive experience where everything from the physics to the full game rule and Graphics come alive as you play now what's notable about Oasis in Minecraft is that your keyboard and mouse inputs Drive everything in real time you can move you can jump you can pick up items and even break blocks which is all interestingly powered by an AI model that learned just by watching the gameplay now this isn't just a giant leap for gaming but a step towards AI models that can simulate complex interactive worlds and potentially replacing our old and traditional game engines with something more adaptive more fluid and more Dynamic now what's crazy about all of this is that the company is actually releasing the code model weights for local use and of course they released a live playable demo which you can see the gaml right now this is exactly my gameplay I know it's not that interesting now you may be wondering who on Earth is behind this kind of crazy technology well it's actually two companies one of them is called decart now decart ai is an efficient AI platform that provides a order of magnitude Improvement in both training and inferencing of the largest generative models now leveraging these Advanced capabilities decart is able to train foundational generative interactive models and make it accessible for everyone in real time this company has raised $21 million in seed funding from prominent investors such as sequa capital and Z Ventures and they aim to continue pushing the boundaries of generative AI now this company collaborated with another company called etched doai now et. is where things get absolutely insane because htoi was founded in 2022 by Harvard dropouts Gavin Uberti and Chris Zoo now this company has actually positioned itself insanely as a challenger to Nvidia in the AI Hardware Space by focusing on Specialized chips rather than general purpose gpus and they've raised $120 million in funding to develop their incredible sooo chip which they claim can outperform in video's gpus by a significant margin when it comes to Transformer models now you might be wondering okay they've got a sooo chip sounds pretty crazy but what is that so the soo chip is an Asic designed specifically for Transformer models which are basically the backbone of many generative AI applications for example text Generation image generation and video generation unlike general purpose gpus that need to support a wide variety of architectures so who is just completely optimized only for Transformers and this specialization allows it to achieve a much higher performance and efficiency by eliminating the unnecessary components and control flow logic that general purpose chips require so they've basically created a chip that is just completely focused on what AI is doing especially with our current technology and that's where they view the future to be now we can see this in a short clip from the CEO actually talking about why he's betting big on his chips well so who's a kind of AI chip that is only able to do one very specific thing an accelerator from Nvidia or Google is flexible it can be programmed to run many different kinds of AI models like convolutional networks lstms or Transformers right our chips are different they are only able to run this one very narrow class of model which we call Transformers but when running these kinds of models they are vastly more performant more than 20 times higher throughput well we're able to burn the Transformer algorithm into the chip we are building our own silicon and our own servers and this enables us to get more than 20 times higher throughput in terms of output words per second from Models like uh chat GPT or llama uh but you're also able to get much better latency numbers if you measure the time to the first token we can cut that by a of 20

as well we are taking a bet unlike every other company in the market our chips can only run Transformers so if Transformers change dramatically or go away then we'll be in a bad place uh but if we're right and Transformers keep being the dominant way that AI models work will be the most performant chip on the market by an order of magnet so now you might be wondering okay how the hell does this even work well the idea behind Oasis is how the AI generates every part of the game in real time imagine you have a blank canvas and each time you press a button the AI draws the next part of the game frame by frame as you play and the technology behind Oasis has two main approaches so essentially we have number one which is the diffusion models and diffusion models are essentially a kind of reverse noise removal so if you start with a blurry picture full of random static and the AI gradually cleans up until you get a clear image for Oasis this process isn't just used to create pictures but also to generate video frames making the game play smooth and interactive then of course we have Transformers and attention Transformers are like a very attentive friend who remembers everything you just did and in Oasis the Transformer AI pays attention to both what happened and before and what you're doing now and this way it can generate realistic video frames that make sense in the context of the actual game like if you jump it remembers where you landed and uses that to generate what comes next now of course since this would take a ton of computing power to generate every Fame full quality Oasis uses something called a vit vae which is essentially just a fancy name for a compression method and they use this to shrink down the amount of data that the AI has to handle at one time and this way it focuses on the important stuff rather than worrying about every single Pixel so the big breakthrough with Oasis is that it's actually combining all of these Technologies to make an entire interactive game world come to life in real time based entirely on what you're doing without relying on pre-programmed Graphics or rules it's essentially an entirely new way to create and experience games now thanks to their efforts they can now create each frame in about 47 millisecs and this means that realtime gaming becomes possible although it's pretty demanding on current Hardware the realtime Game Changer SL the real game Cher will be a new type of Hardware called Soo which is being developed especially to run these kinds of AI models much faster and at a lower cost while Oasis can run on current gpus at 360p here this the soo Hardware could make it possible to run in full 4K allowing for a much better experience plus it could make running erases cheaper allowing more people to use it without high costs so in total it looks like this is just the beginning of what we're starting to see with real time generated AI games now the crazy thing about all of this is that this company isn't the first company to do this there was actually another company that most of you guys know called Google Now Google did this with everyone's favorite game which is called Doom now they called it game Eng and it's an Innovative game engine created by the researchers of Google that operat it's using a pretty similar technology so essentially game and gen which is quite similar to what we have today Works to essentially generate images or videos but adapted for interactive gameplay and with game n gen what they wanted to do to G demonstrate the capabilities they used it to simulate the video game Doom now Doom was chosen because it's iconic and many people have done crazy different things with doom and of course it's an open- Source version that the researchers could use now what's crazy is that you know the gameplay you're seeing on screen right now is completely AI generated once again and it was you know performing impressively generating game scenes at 20 frames per second which is fast enough for realtime gameplay and the AI not only recreated the graphics of Doom but managed the game mechanics like keeping the players's health ammo and interactions even with the world and such as opening doors or fighting enemies which is completely crazy now they kind of trained this in a different way they actually used an AI agent which was trained to play the game and then record its actions and resulting game States as it plays and then the recording was basically the training data for the next stage the second stage basically involves what we spoke about before using the diffusion model and then of course using that to make interactive gameplay now this was pretty crazy of course what you're about to see this is incredible because it allows for Real Time game but there is one problem that these models do suffer with which I'm about to show you guys in my gam playay so of course this is our first area where we can look and decide

at where we like to explore now you can see we've got Village output dense forest rugus Coastline desert expanse and I got to be honest it doesn't really matter which one you pick because this game is AI generated you can easily transport around which I will show you so immediately you are beamed off into this place that seems like a Minecraft world and of course like I said before this is completely AI generated now one of the first things that you'll notice is that when you move your cursor left and right it's actually quite remarkably slow that's of course being the fact that this game is generating these frames whilst I'm inputting them so I don't expect it to be too fast now of course you can see that the resolution is quite low as of course this is something that makes sense considering the fact that it's AI generated and getting extremely high quality outputs is going to be very hard now of course in the P but they speak about how you do get some rough outputs right now that's exactly what you can see here and one of the favorite things that I really do like about this model is the fact that anytime you would like to teleport around you can simply look at the ground and then look up again and you'll see that you're in a different location so for example let's say we are now in this snow biome you can see that if I want to teleport to a different biome if I just look down at the ground I then look back up you can see that the environment has changed so yeah I don't really like that let's look down again and then if I look up you can see that there's some wood next to me so it might build a structure and now we're right beneath a sand biome so this is a game that is basically generating the next frame based on the previous frame so that's why when you look at the ground and then you look back up you can see that the entire environment has changed now if this game it manages to get more memory where it's able to take in not just the last one frame or two frames but the last 20 or 30 or maybe the last two to three minutes of the game then you would get a lot more character consistency now surprisingly you can actually manage to you know interact with blocks you can pick a stone pickaxe you can enter different blocks and interestingly enough you can see that the Minecraft sand physics just aren't there which I find to be remarkable and kind of interesting because when we look at how Minecraft sand is essentially what usually happens is that the sand will drop once one of those blocks have been uh you know broken so I'm surprised that this uh engine didn't manage to get that down I guess potentially what could have happened is that in the training data you may have not got that recorded so all you know AI you know models and stuff like that they're all based on the training data and well if you're trying to be a microf pro this is essentially Sandstone so that might be why it's not there but essentially what I'm trying to say is that like this gameplay depends on the AI generated data that you have so if you have data that's super high quality and you have data that is very specific to certain biomes then that data is going to be reflected in the game play so for example right here what I'm going to try to do is destroy this bottom block and then this top block here is supposed to drop down but like I said before this is showing you that in their data I'm guessing that they didn't account for this because uh the block just yeah like right there it just spawned back in but it's really uncanny because um I can like change my hot Bar menu um I am trying to eat but I don't have any hunger bars um so I'm guessing that's why I can't move much but you can also place blocks which is really cool so for example uh what I can do is I can jump down and I can place blocks here uh some that some disappear some turn into sand as you guys can see um and then yeah um I'm I'm out of blocks now because the value changed as it was being AI generated and I'm guessing that um if I continue to do this is going to spawn me in various different locations I wonder how high I can get if I continue to go even higher now this is pretty confusing because we can see from the bar that it looks like I'm in survival mode but I'm not really in survival mode so um it's kind of interesting it's kind of weird it's it's really trippy it's kind of like a dream uh world if you're dreaming about Minecraft um but yeah it's definitely a little bit weird but you can interact with these you can look up if you also want to change as well and you can see you can get into random areas and wow there's like a crafting table here I didn't see that so uh I want to see if I can interact with the crafting table um let me do that can I interact oh there's a bunch of different items there wow that's super weird oh wait I can continue I can end the game I mean I've got 36 seconds left so uh I don't remember how to do that how to get out of this okay uh we got fences is this food okay NOP it's fences so yeah when you're building stuff and you're destroying stuff things do kind of mesh together it is kind of weird but like I said before this is just a quick part of the game play um that I wanted to show you guys how strange this entire thing is but I think we'll get more of this in the future in a way that actually um makes sense