NVIDIA’s New AI Grows Stuff Out Of Nothing!

I present to you a stunning piece of research work  today. In most computer games and animated movies,   whenever we see a scene, what lies under  is hours and hours of work from a skilled   artist creating these meshes, often done by  hand using a modeler program like Blender. But in the age of AI, we can write a text  prompt and get an image, even a piece of video,   so what about 3D geometry? Can we  create virtual worlds from thin air? You would think that it is possible,  and there is research on that,   so let’s have a look. Oh my. Yes, you can  get a mesh, but it’s not really a good one,   is it? You still have to work on this one to  be able to use it, and that almost defeats   the purpose, because you need a skilled  artist for that. What about the rest of us? But it gets worse, even if you get  an object that is poorly constructed,   we like to say that it is poorly tessellated,  so even if you wanted to do something with it,   you can’t move and edit the  parts intuitively. And that   is why I was super excited for this  paper that promises something better. Now let’s see together… that is so much  better tessellation, this camera can be   taken apart and edited easily, and it requires  fewer elements, a sparser mesh to do that. But   you might not even need that because look,  you needed to perform surgery on this poor   little penguin with the previous method, and  with the new one…not so much. Loving this. But I am still a bit skeptical. You see,  previous methods can sometimes generate   you something that you don’t need a surgery  for, however, look. The surfaces are uneven,   they have these weird artifacts that make  them look like as if they took a beating.    So what about this new one? Oh my, that is not  perfect, but this geometry is so much cleaner. And every paper in this area says that  they create high-quality geometry,   and I got to say this is the first time ever  in a paper where I think I have to agree. And here comes my favorite part - you can  even have a look at the AI building the   3D geometry. It really feels like the whole  thing is being born from thin air. Absolutely   amazing. Dear Fellow Scholars, this is Two  Minute Papers with Dr. Károly Zsolnai-Fehér. Now, wait a second. You see this bluish thing on  the left…so is this geometry coming from thin air,   I mean from just a text prompt? No, but almost  yes. You see, the input is a point cloud. And   to that I say thank goodness. You see, during my  years as a PhD student, I sat through hundreds and   hundreds of research presentations on how to take  a point cloud and create a nice mesh from it. I’ll   give you the summary in short: it seems impossible  and every single technique brings its own issues.    But this new one can take a point cloud and create  relatively high-quality geometry that we might   not even need to touch. That is fantastic, bravo.   But it is still not a text prompt. Now no matter,   because generating a proper point cloud with an AI  from a text prompt is very easy to do, it is just   generating coordinates in space. But generating  the mesh geometry itself would be harder because   it has to have proper surface connectivity  and has topological constraints too. It also   has to be easily editable. So, here is the genius  part: instead of doing the hard thing, geometry,   we do the easy task, point clouds, and then use  this AI to do the hard part for us. So good! And on occasion, it doesn’t just  reconstruct the point cloud into   a mesh, no-no, it can even fix the  problems with the point cloud itself. So I am already delighted by this paper, but  it turns out, it gets even better. Look. Oh   my! You can even choose to have a lower  or a higher polygon count. More coarse,   or more detailed models. That is  perfect because for a real-time game,   you might want to get geometry  that is a bit quicker to render,   or if you are rendering an animated movie and  you have all the time in the world for rendering,   just get a denser, more detailed one. In  fact, hold on to your papers Fellow Scholars,   because this can generate a mesh that is up to 40  times more detailed than previous techniques. Wow. And the results are absolutely sublime,   so much better than anything many  of us could put together by hand. You can also choose whether the mesh should  be built from triangles or quads. And it does   all this while requiring 50% less memory,  and runs 2. 5 times faster than previous   methods. What an absolute blockbuster  paper. Wow. And you can also play and

rotate around these nice, shaded results  on their website in the description. Now, a little criticism. Not even this technique  is perfect, if you want a super detailed model,   you still have to wait for a while. Missing parts  and holes can still occur, and you still can’t   just use a text prompt directly, you would need a  separate tool to create a point cloud. I say it is   a fantastic deal and I can’t even imagine what we  will be capable of two more papers down the line. And we are Fellow Scholars here, we love  the papers, and I recommend having a look   at this one. It shows you how they reimagine  these token-based AI assistant models to be   able to create geometry, which is amazing.   And they use this thing that they call an   hourglass neural architecture. I first thought,  okay, but this looks like an autoencoder,   an idea that dates back almost 40 years  now. However, this is not the case, nope,   this is new exciting stuff, not just  an AI thrown at the problem. Loving it. Now, one more thing. I don’t do  real-life appearances very often,   last time I saw you Fellow Scholars in London  and San Francisco. And I am coming over again.    I will be at the GTC conference, look for  a Fellow Scholar marked with a Two Minute   Papers badge. I can assure you that it is  not an AI, it is me. If you come say hi,   I will give you a small gift, limited  edition, while supplies last. See you there! So, what do you think? What would  you Fellow Scholars use this   for? Let me know in the comments below.