NVIDIA's New AI Makes Cars Fly...Sort Of!

Now let’s teach cars to fly with an AI. Sort  of. How? Well, imagine taking just one image   of the real world, and then, this new AI  would imagine it not just as one image,   but as a scene where you can walk around. And  it gets better. It even understands that water   should reflect its surroundings, wonderful,  and…oh my. Look at that! I can’t believe this.    It not only imagines the reflection off of  the water, but even the water moving itself. I spent years and years trying to learn fluid   dynamics to write liquid simulations like  this, but now, through the power of AI,   you just get that for free. That is kind of  mind blowing. Wow. So, what else can it do? Well, you will see soon that it can even  teach a car to fly. What? How? Well, this   builds on NVIDIA’s Cosmos, which was  first unveiled here on Two Minute Papers,   and we showed how it can generate  lots and lots of useful videos.    Why? To train self-driving cars and all  kinds of useful robots in a video game,   and when they demonstrate that they are safe,  they can be tried out in the real world. So, now let’s use not an image, but a  video as an input, nice - once again it   demonstrates that it understands the scene,  and now, Fellow Scholars, let’s change the   camera trajectory. But not in the way you  think…oh my goodness, is this an error? Nope,   they are indeed going to make this car fly,  aren’t they? Let’s have a look together. Wow,   that is absolutely incredible, loving it. And  this is really tough, the AI needs to imagine   and continue this world so now we see things  that we didn’t see before. All dreamed up by   the AI. Crazy. And when the original footage  ends, suddenly, the AI-imagined footage appears   and the jump should not be jarring. You can’t  just suddenly jump to a poor quality video. I just want to fly everywhere with this. So  cool. So this way, you can take just one piece   of footage, and create many what if situations.   What if we changed lanes? Well, you can generate   all this information without leaving the house,  and train an AI in a simulation safely until   it demonstrates that it is good enough to  handle a variety of difficult situations. Now that would already be good, but it gets even  better. How? Well, hold on to your papers Fellow   Scholars because this one can do not only this  self-driving thing. Not even close! Look, this   one seems like it can do everything. When  given an image of this selfie doggie,   we can now look behind it and it looks  absolutely seamless. And I am a light   transport researcher by trade, that is  ray tracing if you will, so I couldn’t   help but marvel at the reflections  here, which are just sublime. Yummy. And, look at that. Caustics! Caustics  are beautiful, bright patterns of light   you see when light bends through or bounces off  curved surfaces like water or glass. Once again,   writing a computer program that simulates  this took me years and years of studying,   and once again, here, you get it for free.   It even understands transparency and thus,   did really well with the dust particles here. Just imagine all the creation  we will be able to do with this. Although, wait, in this candle scene, you see  that it is unlikely that the intention of the   artist was to fry the horns of this animal. Oh  yes, you see that its understanding of physics   is a bit lacking, we talked about this in  our previous episode. So why is that? Why   not just add more training data so it finally  understands? Dear Fellow Scholars, this is Two   Minute Papers with Dr. Károly Zsolnai-Fehér. Well,  interestingly, that wouldn’t work. And the reason   it wouldn’t work is because these systems are  trained to generate footage, but not necessarily   to understand it and answer questions. Thus,  in a way, there are systems out there that can   create absolutely beautiful videos, yet they  don’t quite understand what they are doing. That is so interesting! So this  can generate entirely new parts   of cities, but it does not necessarily  understand how a city should function,   just roughly what they look like. And  the next generation of AI techniques   will have a proper understanding of  that too. What a time to be alive! And the pace of progress in AI research is  just absolutely incredible. Just imagine what   we will be capable of just two more  papers down the line. For instance,   remember, this representation is a point  cloud. Point clouds can now be converted   to 3D geometry in a way where you can  even see them being born from nothing. Now, not even this technique is perfect,  you’ve heard about some limitations already,

the resolution could be a bit higher,  I am sure it will be soon, also,   this muscular person seems  somehow a bit off to me. And the full paper on how  to do all this is available,   and the source code will be available  too, for free, for everyone. You know,   everyone today is talking about this new Manus  AI, yes, you can get a lot of views with it,   but really, all we know about it is…almost  nothing. Speculation. And then, more speculation   about the speculation. Here, we work differently.   We are Scholars, we like research papers,   we send joy and positivity into the world,  and show people how amazing research is. So please subscribe and hit the bell icon if  you appreciate that and would like to see real   science explained beautifully and simply.   Well, hopefully, I am trying my best here.    And for the first time ever, I’ll be at the GTC  conference soon, look for the Fellow Scholar   with this badge, come and say hi and I’ll give  you a gift…until I run out of gifts of course.