Geometric Detail Transfer | Two Minute Papers #141

dear fellow scholars this is two minute papers with károly on AFA here in the world of digital 3d modeling it often occurs that we are looking for surfaces that are not perfectly smooth but have some sort of surface detail wrinkles engravings grain on a wooden table are excellent examples of details that we can add to our models and computer graphics people like to collectively call these things displacement maps artists often encounter cases where they like the displacements on one object but the object itself is not really interesting however it could be that there is a different piece of geometry these details would look great on consider this problem solved because in this piece of work the input is two 3d models one with interesting geometric details and the other is the model onto which we transfer these surface details the output will be our 3d geometric shape with two of these models fused together the results look absolutely amazing I would love to use this right away in several projects the first key part is the usage of metric learning weight technical term what does this mean exactly metric learning is a classical technique in the field of machine learning where we are trying to learn distances between things where distance is mathematically ill-defined let's make it even simpler and go with an example for instance we have a database of human faces and we would like to search for faces that are similar to a given input to do this we specify a few distances by hand for instance we could say that a person with a beard is a short distance from one with a mustache and the larger no facial hair if we hand many examples of these distances to a learning algorithm it will be able to find people with similar beards and in this work this metric learning is used to learn the relationship between objects with and without these rich surface details this helps in the transferring process as to creating the new displacements on the new model there are several hurdles to overcome one we cannot just grab the displacements and shove them onto a different model because it can potentially look different have different curvatures and sizes the solution to this would be capturing the statistical properties of the surface details and use this information to synthesize new ones on the target model note that we cannot just perform this texture synthesis in 2d like we do for images because as we project the result to a 3d model it introduces severe distortions to the displacement patterns it is a bit like putting a rubber blanket onto a complicated object different regions of the blanket will be distorted differently make sure to have a look at the paper where the authors present quite a few more results and of course the intricacies of this technique are also described in detail I hope some public implementations of this method will appear soon I would be quite excited to use this right away and I am sure there are many artists who would love to create these wonderfully detailed models for the animated films and computer games of the future in the meantime we have a completely overhauled software and hardware pipeline to create these videos we have written down our joyful and perilous story of it on patreon if you are interested in looking a bit behind the curtain as to how these episodes are made make sure to have a look it is available in the video description thanks for watching and for your generous support now see you next time