AI Creates 3D Models From Images | Two Minute Papers #186

Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. Today we're going to talk about a task that humans are remarkably good at, but learning algorithms mostly flounder. And that is creating 3D geometry by looking at a 2D color image. In video games and animation films, this is a scenario that comes up very often - if we need a new weapon model in the game, we typically give the artist a photo, who will sit down with a 3D modeler program, and spends a few hours sculpting a similar 3D geometry. And I will quickly note that our binocular vision is not entirely necessary to make this happen. We can look at 2D images all day long and still have a good idea about the shape of an airplane, even with one eye closed. We had previous episodes on this problem, and the verdict was that the results with previous techniques are great, but not very detailed. Mathematicians like to say that this algorithm has a cubic complexity or cubic scaling, which means that if we wish to increase the resolution of the 3D model just a tiny bit, we have wait not a tiny bit longer, but significantly longer. And the cubic part means that this tradeoff becomes unbearable even for moderately high resolutions. This paper offers a technique to break through this limitation. This new technique still uses a learning algorithm to predict the geometry, but it creates these 3D models hierarchically. This means that it starts out approximating the coarse geometry of the output, and restarts the process by adding more and more fine details to it. The geometry becomes more and more refined over several steps. Now, this refinement doesn't just work unless we have a carefully designed algorithm around it. The refinement happens by using additional information in each step from the created model. Namely, we imagine our predicted 3D geometry as a collection of small blocks, and each block is classified as either free space, occupied space, or as a surface. After this classification happened, we have the possibility to focus our efforts on refining the surface of the model, leading to a significant improvement in the execution time of the algorithm. As a result, we get 3D models that are of higher quality than the ones offered by previous techniques. The outputs are still not super high resolution, but they capture a fair number of surface detail. And you know the drill, research is a process, and every paper is a stepping stone. And this is one of those stepping stones that can potentially save many hours of work for 3D artists in the industry. Thanks for watching and for your generous support, and I'll see you next time!