Amazing AI Caricatures Are Here!

Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. Today we are going to create these cartoon-style  DeepFakes, and find out how much better this   new AI is than previous techniques. I think you  will be very surprised when you see the results. So, what is exactly happening here? Well, in goes  a photo of us, and we also provide a target style,   and bam. We become a cartoon, an anime  character, or a caricature of ourselves. That sounds absolutely amazing, so let’s  have a look at all three of these one by one. Now hold on to your papers, Fellow Scholars, and  let’s start with my favorite first. Caricatures.    These are meant to exaggerate the facial  features of the test subject and we can   also provide a style. Here, Mr Bean  has a funny face, and to my delight,   all of the caricatures also have a funny feel  to them. That is absolutely amazing. A+ work   right there, little AI. Most of these also pass  the identity test - that is, if we were to cover   the photo of the real person, and just look at the  AIs interpretation, in almost all cases, we could   tell exactly who it is. That is absolutely  incredible. Just think about how difficult   it is to retain the identity of the input photo  while transforming it according to this style. Also, glasses work. A variety  of hairstyles work. Finally,   an AI that creates amazing caricatures.   This is incredible. Loving it. Two, next up, cartoons. The results  are still high-quality images,   I like that. Glasses and  hairstyles are still supported.    The identity test is still kind of okay here,  Harry Potter is definitely recognizable, in come   cases, Mr Bean too. Traces of the stubble of Mr.   DiCaprio are visible on some of the results, but   they are gone from most of the rest, and hence,  it would be difficult to tell exactly who this is. Three, if we are looking to become  some anime characters, we see an even   more powerful demonstration of this effect. Sometimes it feels like our input photos are   only taken into consideration to get the exact  position of the face, and hence I feel like we   are seeing the same one character in different  poses. Now this will make a very interesting   case study. Why? Well, perhaps the reason for  this is that these example images contain too   few detailed facial features to be able to redo  a real person into an anime version. Remember,   caricatures were extremely recognizable and  passed the identity test easily. So here,   I feel that we ran not into the limitations  of this AI, but into the limitations of the   concept itself. Actually, we can find  out about that ourselves. How? Well,   let’s compare this to what previous techniques  were capable of and see if this one is better. First, we can inspect these images ourselves.   These are three previous techniques, and this   is the new method. Hmm. This one is called  GNR…Are you thinking what I am thinking? Yes,   this paper is called GANs N’ Roses. Yes, that  just happened. I am not kidding. This did not   perform that well at all, but StarGAN2…this  is actually not so bad. In some cases,   the quality of these images seems higher  than the new technique…until we realize   that is it not doing exactly what we asked  for. Remember, this is the input photo,   and this is the target. The input photo is meant  to be reimagined in the style of the target, and,   oh yes, there we are. Sometimes, this just keeps  giving us images that are good, but good is not   good enough here. These would have to be close  to the target style, and often, they aren’t. Hence, I am really happy to see the  comparisons against the unsupervised   image-to-image translation technique because it  is doing something very similar to this technique,   but the quality differences are quite  striking. The new one is so much better. Let’s see the user study from the paper and see  if others agree too. What? Is that really true?

Goodness! That is one of the cleanest wins for  a followup paper I have seen in a while. Wow.    Just look at that! And the incredible thing  here is that all of these techniques were   published in research papers from the last 2 to  3 years. These are from not so long ago at all! You see, the progress in AI research is  nothing short of incredible. These amazing   new papers are popping up every year, and in  some fields, almost every week. So, with that,   we can soon become any character  when playing in a virtual a world,   and we can also choose how much of our identity  we wish to reveal. What a time to be alive! So what do you think? What kind of character would   you like to become? Let me  know in the comments below! Thanks for watching and for your generous  support, and I'll see you next time!