Can An AI Create Original Art? 👨‍🎨

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Approximately 7 months ago, we discussed an AI-based technique called StyleGAN2, which could synthesize images of human faces for us. As a result, none of the faces that you see here are real, all of them were generated by this technique. The quality of the images and the amount of detail therein is truly stunning. We could also exert artistic control over these outputs by combining aspects of multiple faces together. And as the quality of these images improve over time, we think more and more about new questions to ask about them. And one of those questions is, for instance, how original are the outputs of these networks? Can these really make something truly unique? And believe it or not, this paper gives us a fairly good answer to that. One of the key ideas in this work is that in order to change the outputs, we have to change the model itself. Now that sounds a little nebulous, so let’s have a look at an example. First, we choose a rule that we wish to change, in our case, this will be the towers. We can ask the algorithm to show us matches to this concept, and indeed, it highlights the towers on the images we haven’t marked up yet, so it indeed understands what we meant. Then, we highlight the tree as a goal, place it accordingly onto the tower, and a few seconds later, there we go! The model has been reprogrammed such that instead of towers, it would make trees. Something original has emerged here, and, look, not only on one image, but on multiple images at the same time. Now, have a look at these human faces. By the way, none of them are real and were all synthesized by StyleGAN2, the method that you saw at the start of this video. Some of them do not appear to be too happy about the research progress in machine learning, but I am sure that this paper can put a smile on their faces. Let’s select the ones that aren’t too happy, then copy a big smile and paste it onto their faces. See if it works! It does, wow! Let’s flick between the before and after images and see how well the changes are adapted to each of the target faces. Truly excellent work. And now, on to eyebrows. Hold on to your papers while we choose a few of them, and now, I hope you agree that this mustache would make glorious replacement for them. And there we go. Perfect! And note that with this, we are violating Betteridge's law of headlines again in this series, because the answer to our central question is a resounding yes, these neural networks can indeed create truly original works, and what’s more, even entire datasets that haven’t existed before. Now, at the start of the video, we noted that instead of editing images, it edits the neural network’s model instead.

If you look here, we have a set of input images created by a generator network. Then, as we highlight concepts, for instance, the watermark text here, we can look for the weights that contain this information, and rewrite the network to accommodate these user requests, in this case, to remove these patterns. Now that they are gone, by selecting humans, we can again, rewrite the network weights to add more of them, and finally, the signature tree trick from earlier can take place. The key here is that if we change one image, then we have a new and original image, but if we change the generator model itself, we can make thousands of new images in one go. Or even a full dataset. Loving the idea. And perhaps the trickiest part of this work is minimizing the effect on other weights while we reprogram the ones we wish to change. Of course, there will always be some collateral damage, but the results, in most cases still seem to remain intact. Make sure to have a look at the paper to see how it’s done exactly.

Also, good news, the authors also provided an online notebook where you can try this technique yourself. If you do, make sure to read the instructions carefully and regardless of whether you get successes or failure cases, make sure to post them in the comments section here! In research, both are useful information. So, after the training step has taken place, neural networks can be rewired to make sure they create truly original works, and all this on not one image, but on a mass scale. What a time to be alive! Thanks for watching and for your generous support, and I'll see you next time!