NVIDIA’s New AI: Wow, 8x Better Text To 3D!

I am very excited about this paper. So what does  it do? Magic, according to the authors! That is,   not text to image, where we write a text  prompt and get a beautiful photo or painting,   but something else: text to 3D. Hmm…now that  would be fantastic, I would say it qualifies

as magic. We write something and out comes 3D  geometry that we can put into a video game or   any kind of virtual world. Teleconferencing.   Anything you wish. Dear Fellow Scholars,   this is Two Minute Papers with Dr. Károly  Zsolnai-Fehér. Now, wait a second. The previous,   DreamFusion technique was capable of  doing that. And this was a paper that   was published just a year ago. Now, yes, but  this one has two problems. Problem number one,   it takes a long time, and problem  number two, we still get coarse results. Now, does this paper really have the magic they  promised? Let’s have a look together! Whoa!    Look at that difference. I immediately  see that this has so much more detail,   it has 8 times higher resolution than the previous  method. And it gets better, we noted earlier that   they are not only coarse, but slow too. The  coarseness is gone, fantastic, but I imagine   that this is now probably even slower in return.   We don’t get all this quality for free, do we?    That cannot be true. Whoa! It is not only free, it  is even twice as fast as the previous technique. I also love the variety of objects it can conjure  up. We can even ask for more imaginative things,   like a car made out of sushi. That’s good. Now,  clearly, as all of you Fellow Scholar see right   here, this is not the quality of geometry that you  would put into those fancy triple A quality games,   but at this point I feel that we are just two  more papers away from that kind of detail. Now, there is plenty to love here, and somehow,  it gets even better. We can even write a prompt,   and if we would like to refine the results, we can  rewrite parts of the prompt and get a very similar

result. It works great with this baby bunny, but  I’ll note that if we modify the prompt too much,   we might get an entirely different scene,  so this one requires a light touch. And it gets even better. We can have  a collection of photos of our cat,   and then, ask the AI to create  a virtual version of this cat   but riding on a bike. Imagine meeting  your own cat in a video game! Love it. So, when can we try it? Well, NVIDIA  is planning to include it in Picasso,

their generative AI framework, which is  hopefully going to be widely available   soon. It will be part of the Adobe suite too,  quite honestly, this will be everywhere real   soon. What a time to be alive! If you cannot  wait to give it a go, and you wish to get   notified when it becomes available, that is a  possibility through the link in the description. And one more thing. I have surprising news.   This technique has already been surpassed. Yes,   you heard it right. The pace of progress in AI  and computer graphics research is so incredible,   that there is already something even better  than this out there. I’ll give a hint:   it is a Gaussian splatting-based technique.   I can’t wait to tell you all about it in   the next episode. Subscribe and hit  the bell icon to not miss out on it.