# NVIDIA’s New AI: Superb Details, Super Fast! 🤖 ## Метаданные - **Канал:** Two Minute Papers - **YouTube:** https://www.youtube.com/watch?v=eaSTGOgO-ss - **Дата:** 15.02.2022 - **Длительность:** 6:53 - **Просмотры:** 147,534 ## Описание ❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.com/papers ❤️ Their mentioned post is available here (thank you Soumik Rakshit!): https://wandb.ai/geekyrakshit/poegan/reports/PoE-GAN-Generating-Images-from-Multi-Modal-Inputs--VmlldzoxNTA5MzUx 📝 The paper "Multimodal Conditional Image Synthesis with Product-of-Experts GANs" (#PoEGAN) is available here: https://deepimagination.cc/PoE-GAN/ ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers - https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, Christian Ahlin, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Bustamante, John Le, Jonas, Jonathan, Kenneth Davis, Klaus Busse, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Michael Albrecht, Michael Tedder, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Steef, Taras Bobrovytsky, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi. If you wish to appear here or pick up other perks, click here: https://www.patreon.com/TwoMinutePapers Thumbnail background design: Felícia Zsolnai-Fehér - http://felicia.hu Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: https://discordapp.com/invite/hbcTJu2 Károly Zsolnai-Fehér's links: Instagram: https://www.instagram.com/twominutepapers/ Twitter: https://twitter.com/twominutepapers Web: https://cg.tuwien.ac.at/~zsolnai/ #nvidia ## Содержание ### [0:00](https://www.youtube.com/watch?v=eaSTGOgO-ss) Introduction Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. Today we are going to look at NVIDIA’s spectacular  new AI that can generate beautiful images for us. ### [0:12](https://www.youtube.com/watch?v=eaSTGOgO-ss&t=12s) Image Generation But, this is not image generation of any kind,   no-no! This is different.   Let’s see how it is different. For instance, we can write a description,  and the appropriate image comes out. Snowy   mountains, pink cloudy sky. Checkmark. Okay, we can give it art direction.    Make no mistake, this is fantastic, but this  has been done before, nothing new here. Yet.   Or, we can create a segmentation map,  this tells the AI what things are.    The sea is down there, mountains,  and sky up there. Looking great, but   this has been done before too. For  instance, NVIDIA’s previous GauGAN paper   could do this too. Nothing new here. Yet. Or, we can tell the AI where things are   by sketching. This one also  works, but this has been done too. So, is there nothing new in this paper? Well, of  course there is! And now, hold on to your papers,   and watch as we fuse all of  these descriptions together. We tell it where things are and what  things are. But I wonder if we can make   this mountain snowy and a pink cloudy sky  on top of all things…. yes we can! Oh wow,   I love it. The sea could have a little more  detail, but the rest of the image is spectacular. So, with this new technique, we  can tell an AI where things are,   what things are, and on top of it,  we can also give it art direction.    And here is the key, all of these  can be done, in any combination! Now, did you see it? Curiously, an image  popped up at the start of the video when we   unchecked all the boxes. Why is that? Is that  a bug? I’ll tell you in a bit what that is. So far, these were great examples, but let’s try  to push this to its limits and see what it can do. For instance, how quickly can we iterate with  it? How quick is it to correct mistakes or   improve the work? Oh boy. Super quick!   When giving art direction to the AI,   we can update the text, and the output  refreshes almost as quickly as we can type. The sketch to image feature is also  a great tool by itself. Of course,   there is not only one way to satisfy these labels,  there are many pictures that this could describe,   so, how do we control what we get? Well,  it can even generate variants for us. ### [2:58](https://www.youtube.com/watch?v=eaSTGOgO-ss&t=178s) Drawing Rocks With this new work, we can even draw a  piece of rock within a within the sea,   and the rock will indeed appear. Not only  that, but it understands that the waves   have to go around it too. An understanding  of physics. That is insanity. My goodness. ### [3:16](https://www.youtube.com/watch?v=eaSTGOgO-ss&t=196s) Drawing Trees and Leaves Or, better, if we know in advance that we are  looking for tall trees and autumn leaves, we can   start with the art direction. And then, when  we add our labels, they will be satisfied,   we can have our river, but the trees and  the leaves will always be there. Finally,   we can sketch on top of this to have  additional control over the hills and clouds. ### [3:46](https://www.youtube.com/watch?v=eaSTGOgO-ss&t=226s) How does it work And, get this - we can even edit real images. So, how does this black magic work? Well, we  have four neural networks, four experts if you   will. And the new technique describes how to fuse  their expertise together into one amazing package. And the result is a technique that outperforms  previous techniques in most of the tested cases.    So, are these some ancient methods from many  years ago that are outperformed, or are they   cutting edge? And here comes the best part. If  you have been holding on to your papers so far,   now, squeeze that paper, because these  techniques are not some ancient methods.    Not at all. Both of these methods are from the  same year as this technique. The same year.    Such improvement in less than a year. That  is outstanding. What a time to be alive! ### [4:42](https://www.youtube.com/watch?v=eaSTGOgO-ss&t=282s) Results Now, I made a promise to you early in the  video. And the promise is explaining this. Yes,   the technique generates, even when  not given a lot of instruction. Yes,   this was the early example when we unchecked all  the boxes. What quality images can we expect then?    Well, here are some uncurated examples, this  means that the authors did not cherrypick here,   just dumped a bunch of results here. And…oh my  goodness, these are really good! The details are   really there, the resolution of the image could  be improved, but we saw with the previous GauGAN   paper that this can improve a great deal in just a  couple years. Or, with this paper, in less than a   year. I would like to send a huge congratulations  to the scientists at NVIDIA - bravo! And, you Fellow Scholars just saw how much of an  improvement we can get just one more paper down   the line, so imagine what the next paper could  bring. If you are curious, make sure to subscribe   and hit the bell icon to not miss it, when the  followup paper appears on Two Minute Papers. Thanks for watching and for your generous  support, and I'll see you next time! --- *Источник: https://ekstraktznaniy.ru/video/13655*