How to Build the 3D World of Your Dreams with a Text Prompt | Kiran Bhat | TED

I'm excited to present a new way to design 3D games with AI. Now, many of you may have used AI to produce text or images or even code, but 3D, with that extra dimension, is incredibly hard. So today, in the next few minutes, you and I are going to create a playable 3D game in Roblox, that the world has never seen before, because we are going to imagine it together and use AI to build it. Now these are super early days, but let's see what AI can do. OK. I'm going to prompt to create an Asian village in winter. So how about a stylized Japanese village in winter. Maybe a central pagoda. A few houses with pine trees around it. Maybe a traditional red gate and lanterns. Some prompts. So when I say generate, what happens is an AI model takes this prompt, analyzes it, and produces the code that you see on the right. This is all happening live. And what it's doing is it's transforming that prompt into spatial boxes where the objects are going to land. So that's the location of the different objects. And soon you'll see, behind me, in my avatar, the objects transforming into their shapes, and then the textures. These are all happening live. So while this is all happening, I need your help to help me build this out. So what would you guys want to add to this scene? What did you say? Wolves, OK. Add a wolf. Let me do an anime... a cute wolf -- (Laughter) Near me. How about some vehicles? Snowmobile, OK. A snowmobile. And a few... (Audience: Palm trees!) Oh, palm trees, OK, wow. Near the gate. OK. A palm tree, somebody said? Palm tree next to the house, maybe. And a Santa? (Laughter) How about something else? (Laughter) OK, a few animals. A few animals next to the trees, OK. Alright, that's good enough. We get a sense of it. So while this is happening, let me show you what we've built so far. So right off the bat, you can see how this -- it got the overall aesthetic of the scene, right. You know, everything fits well globally. It got the environment, got the background. It got the pagoda to be the tallest structure. You have these houses next to it and you have the trees behind. So all of that sounds good. But I see my friend Maurice joining in from California. He's also watching this live and say, "Hi, Maurice." Isn't it fun to basically connect, build and play with friends? This is the whole point of this thing. So obviously he's excited. Oh wow, look at some of these things it's created. So the one interesting thing is if you don't like something, you can just move it around. So I'm going to move this object. It's a huge truck. Well let's move it away. Obviously this is the fun of doing the live demos. And then let's take a look at all the things it created. Oh, I see this fox here or a wolf, right, the cute anime wolf. Let's see. Come back here. Let's take a look at you. OK, well, let's move you here. What do you guys think? (Applause) Well, unlike images, because this is a 3D world, we can build on a game engine. We can add effects to it. So you can actually make this much more immersive. Maybe I can make it snow because it's the winter village, so you'll start seeing snow in a second. And in addition to creating objects, the AI model can also produce code to make these objects functional. So I can actually go up to this car or maybe let me try this snowmobile

and it lets me ride it. So if I press this, I can kind of go around and ride it around and you can see how this can get really cool. And I'm sure Maurice is going to do some fun driving around. Look at that, he's already there. (Laughter) OK, Maurice, take it easy. One last thing I'd like to show you is with these worlds, I love seeing them at different times of the day, so I can switch it around, I can make it into sunset. Or actually I can see it at night. And night, it's kind of really cool because you start seeing the moon and the sky. But another thing that happens is see how these lanterns that we added, they start lighting up. And this is another example of the functionality which we call 4D, that the model adds. So it knows that the lanterns emit light. And so it makes them, you know, lit. So these are some of the things that you can do, which bring a scene to life. And let me switch back to day. And the lights turn off. Anyway, I could be doing this all day long with you guys, but sadly, I think I'm out of time. Hopefully you guys had a preview of the transformative power of AI for 3D. We think this is the future of 3D gaming. Thank you. (Applause)