AI Learns Semantic Style Transfer | Two Minute Papers #177

Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. Style transfer is an amazing area in machine learning and AI research, where we take two images. Image number one is an input photograph, and image number two, is the desired style. And the output of this process is the content of image number one with the style of image number two. This first paper opened up an incredible new area of research. As a result, a ton of different variants have emerged in the last two years. Feedforward style transfer for close to real-time results, temporally coherent style transfer for videos, and much, much more. And this one not only outperforms previously existing techniques, but also broadens the horizon of possible style transfer applications. And obviously, a human would be best at doing this because a human has an understanding of the objects seen in these images. And now, hold on to your papers, because the main objective of this is method to create semantically meaningful results for style transfer. It is meant to do well with input image pairs that may look completely different visually, but have some semantic components that are similar. For instance, a photograph of a human face and a drawing of a virtual character is excellent example of that. In this case, this learning algorithm recognizes that they both have noses and uses this valuable information in the style transfer process. As a result, it has three super cool applications. First, the regular photo to style transfer that we all know and love. Second, it is also capable of swapping the style of two input images. Third, and hold on to your papers because this is going to be even more insane. Style or sketch to photo. And we have a plus one here as well, so, fourth, it also supports color transfer between photographs, which will allow creating amazing time-lapse videos. I always try to lure you Fellow Scholars into looking at these papers, so make sure to have a look at the paper for some more results on this. And you can see here that this method was compared to several other techniques, for instance, you can see the cycle consistency paper and PatchMatch. And this is one of those moments when I get super happy, because more than a 170 episodes into the series, we can not only appreciate the quality of these new results, but we also had previous episodes about both of these algorithms. As always, the links are available in the video description, make sure to have a look, it's going to be a lot of fun. The source code of this project is also available. We also have a ton of episodes on computer graphics in the series, make sure to have a look at those as well. Every now and then I get e-mails from viewers who say that they came for the AI videos, and just in case, watched a recent episode on computer graphics and were completely hooked. Thanks for watching and for your generous support, and I'll see you next time!