# An AI That Makes Dog Photos - But How? 🐶

## Метаданные

- **Канал:** Two Minute Papers
- **YouTube:** https://www.youtube.com/watch?v=bnm7skt2aYE
- **Дата:** 23.03.2021
- **Длительность:** 8:51
- **Просмотры:** 92,851

## Описание

❤️ Check out Weights & Biases and sign up for a free demo here: https://www.wandb.com/papers 
❤️ Their mentioned post is available here: https://wandb.ai/ayush-thakur/ada/reports/Train-Generative-Adversarial-Network-With-Limited-Data--Vmlldzo1NDYyMjA

📝 The paper "Training Generative Adversarial Networks with Limited Data" is available here:
Paper: https://arxiv.org/abs/2006.06676
Pytorch implementation: https://github.com/NVlabs/stylegan2-ada-pytorch

📝 My thesis with the quote is available here:
https://users.cg.tuwien.ac.at/zsolnai/gfx/photorealistic-material-learning-and-synthesis/

Unofficial StyleGAN2-ADA trained on corgis (+ colab notebook):
https://github.com/seawee1/Did-Somebody-Say-Corgi

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Aleksandr Mashrabov, Alex Haro, Alex Serban, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bruno Mikuš, Bryan Learn, Christian Ahlin, Eric Haddad, Eric Martel, Gordon Child, Haris Husic,  Ivo Galic, Jace O'Brien, Javier Bustamante, John Le, Jonas, Joshua Goller, Kenneth Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Mark Oates, Michael Albrecht, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Ramsey Elbasheer, Robin Graham, Steef, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: https://www.patreon.com/TwoMinutePapers

Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: https://discordapp.com/invite/hbcTJu2

Károly Zsolnai-Fehér's links:
Instagram: https://www.instagram.com/twominutepapers/
Twitter: https://twitter.com/twominutepapers
Web: https://cg.tuwien.ac.at/~zsolnai/

#nvidia #stylegan2

## Содержание

### [0:00](https://www.youtube.com/watch?v=bnm7skt2aYE) Segment 1 (00:00 - 05:00)

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today, we are going to explore a paper that improves on the incredible StyleGAN2. What is that? StyleGAN2 is a neural network-based learning algorithm that is not only capable of creating these eye-poppingly detailed images of human beings that don’t even exist, but, it also improved on its previous version in a number of different ways. For instance, with the original StyleGAN method, we could exert some artistic control over these images, however, look, you see how this part of the teeth and eyes are pinned to a particular location and the algorithm just refuses to let it go, sometimes to the detriment of its surroundings. The improved StyleGAN2 method addressed this problem, you can see the results here. Teeth and eyes are now allowed to float around freely, and perhaps this is the only place on the internet where we can say that and be happy about it. It could also mix two images together, and it could do it not only for human faces, but for cars, buildings, horses, and more. And get this, this paper was published in December 2019, and since then, it has been used in a number of absolutely incredible applications and followup works. Let’s look at three of them. One, for instance, the first question I usually hear when I talk about an amazing paper like this is “okay, great, but when do I get to use this”? And the answer is, right now, because it is implemented in Photoshop in a feature that is called Neural Filters. Two, artistic control over these images has improved so much that now, we can pin down a few intuitive parameters and change them with minimal changes to other parts of the image. For instance, it could grow Elon Musk a majestic beard…and, Elon Musk was not the only person who got an algorithmic beard, I hope you know what’s coming…yes, I got one too! Let me know in the comments whose beard you liked better! Three, a nice followup paper that could take a photo of Abraham Lincoln and other historic figures, and could restore their images as if we were time travelers and took these photos with a more modern camera. The best part here was that it leveraged the superb morphing capabilities of StyleGAN2 and took an image of their siblings, a person who has somewhat similar proportions to the target subject, and morph them into a modern image of this historic figure. This was brilliant, because restoring images is hard, but with StyleGAN2, morphing is now easy, so the authors decided to trade a difficult problem for an easier one. And the results speak for themselves. We cannot know for sure if this is what these historic figures really looked like, but for now, it makes one heck of a thought experiment. And now, let’s marvel together at these beautiful results with the new method, that goes by the name, StyleGAN2-ADA. While we look through these results, all of which were generated with the new method, here are three things that it does better. One, if often works just as well as StyleGAN2 but requires ten times fewer images for training. This means that now, it can create these beautiful images, and this can be done by training a set of neural networks from less than 10 thousand images at a time. Whoa. That is not much at all. Two, it creates better quality results. The baseline here is original StyleGAN2, the numbers are subject to minimization and are a measure of the quality of these images. As you see from the bolded numbers, the new method not only beats the baseline method substantially, but it does it across the board. That is a rare sight indeed. And three, we can train this method faster, it generates these images faster, and in the meantime, also consumes less memory, which is usually in short supply on our graphics cards. Now, we noted that the new version of the method is called StyleGAN2-ADA. What is ADA? This part means Adaptive Discriminator Augmentation. What does that mean exactly? This means that the new method endeavors to squeeze as much information out of these training datasets as it can. Data augmentation is not new, it has been done for many years now, and essentially this means that we rotate, colorize, or even corrupt these images during the training process.

### [5:00](https://www.youtube.com/watch?v=bnm7skt2aYE&t=300s) Segment 2 (05:00 - 08:00)

The key here is that with this, we are artificially increasing the number of training samples the neural network sees. The difference here is that they used a greater set of augmentations, and the adaptive part means that these augmentations are tailored more to the dataset at hand. And now comes the best part, hold on to your papers, and let’s look at the timeline here. StyleGAN2 appeared in December 2019, and StyleGAN2-ADA, this method came out just half a year later. Such immense progress, in just 6 months of time. The pace of progress in machine learning research is absolutely stunning these days. Imagine what we will be able to do with these techniques just a couple more years down the line. What a time to be alive! But this paper also teaches a very important lesson to us that I would like to show you. Have a look at this table that shows the energy expenditure for this project for transparency, but it also tells us the number of experiments that were required to finish such an amazing paper. And that is more than 3300 experiments. 255 of which were wasted due to technical problems. In the foreword of my PhD thesis, I wrote the following: “Research is the study of More precisely, research obtaining new knowledge through failure. A bad researcher fails 100% of the time, while a good one fails only 99% of the time. Hence, what you see written here (and in most papers) is only 1% of the work that has been done. I would like to thank Felícia, my wife, for providing motivation, shielding me from distractions, and bringing sunshine to my life to endure through many of these failures. ” This paper is a great testament to show how difficult the life of a researcher is. How many people give up their dreams, after being rejected once, or maybe two times? Ten times? Most people give up after 10 tries. And just imagine having a 1000 failed experiments and still not even being close to publishing a paper yet. And, with a little more effort, this amazing work came out of it. Failing doesn’t mean losing. Not in the slightest. Huge congratulations to the authors for their endurance, and for this amazing work, and I think this also shows that these Weights and Biases experiment tracking tools are invaluable, because it is next to impossible to remember what went wrong with each of them, and what should be fixed. Thanks for watching and for your generous support, and I'll see you next time!

---
*Источник: https://ekstraktznaniy.ru/video/13953*