NVIDIA’s New AI: 20% Faster Game Graphics!

This paper has a ton of promise. Amazing  promises. Get this: it promises to reconstruct

3D objects from a bunch of images. It  promises better generative 3D modeling,   even creating an animated 3D model of us moving,  tetrahedral mesh geometry for physics simulation

for squishy things, and can also recommend  geometry models that are easier to 3D print. Now, wait wait. This is not  new. In fact, this is old stuff,   because this was possible for 36 years now. What?   Yes, really. The legendary computer graphics paper   by the name Marching Cubes also set out to do  that. It is a simple handcrafted technique,

and it can be used for tomography,  magnetic resonance imaging and more.    It gives you everything you want. Or, Neural  Dual Contouring. A paper from just a year ago.

So why not just use them? Well, here is why.   Look. We are starting out from this sphere and   would like to morph it into this object.   Can we? Oof! Not quite! The previous Dual   Contouring technique blows up immediately.   The neural version is more well behaved,   but it can’t give us our desired shape. Marching  cubes, comes to the rescue…until it falters in   being flexible enough to reconstruct the  final shape. A good try though. So, what   about the new technique? Wow, that is a seemingly  perfect reconstruction of the desired geometry. So, what else can it do? Well, hold  on to your papers Fellow Scholars,   because here in goes a video of a  physics simulation, this is really tough,   because but the desired geometry is  not stationary anymore. And once again,   it starts out from a ball, and over time,  wow, look! Out comes 3D geometry that we can   use in our video games and all kinds of virtual  worlds. All this from an input video. So cool! And, it gets even better! When combined with a  text to 3D technique from just 2 years ago, here,   we can do something similar to text to image, we  write something, and out comes a painting of that,   but here, instead we write something, and out  comes…can that really be? Out comes this 3D   geometry. And the difference is that finally,  we are now getting high-quality geometry. And get this, even creating animations is also  possible with this. That is a huge step forward. And because this is a differentiable  mesh generation technique,   it means that it starts out from  something completely different,   and it slowly morphs into the final result over  time. I absolutely love these animations. And   this one shows that the new technique once  again, converges to the desired geometry,   we have seen that, okay, but. Wait a minute.   That is not only a better piece of geometry,   but it also uses about 20% fewer little  elements, little triangles to accomplish that.    This means that these models will require less  memory, and can be rendered faster. Fantastic! And there is an problem here. Do you see  the problem? Well, Marching Cubes has some   more issues, for instance in the nose region here.   Thus, this geometry is typically not 3D printable,   or if so, not out of the box. And finally,  the new one can generate pieces of geometry   that don’t have these problems, and  are now more suitable for 3D printing. Now, clearly, these are not yet the kinds  of assets that you would directly put into   those triple A quality games, not even close.   But all this is a heck of a lot of value in   just one paper. And just imagine what we will be  capable of just two more papers down the line.