# Is Style Transfer For Fluid Simulations Possible? 🌊 ## Метаданные - **Канал:** Two Minute Papers - **YouTube:** https://www.youtube.com/watch?v=b8sCSumMUvM - **Дата:** 03.06.2020 - **Длительность:** 6:25 - **Просмотры:** 112,534 - **Источник:** https://ekstraktznaniy.ru/video/14121 ## Описание ❤️ Check out Weights & Biases and sign up for a free demo here: https://www.wandb.com/papers Their showcased post is available here: https://app.wandb.ai/wandb/huggingtweets/reports/HuggingTweets-Train-a-model-to-generate-tweets--VmlldzoxMTY5MjI 📝 The paper "Lagrangian Neural Style Transfer for Fluids" is available here: http://www.byungsoo.me/project/lnst/index.html 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Haro, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bruno Mikuš, Bryan Learn, Christian Ahlin, Daniel Hasegan, Eric Haddad, Eric Martel, Javier Bustamante, Lorin Atzberger, Lukas Biewald, Michael Albrecht, Nader S., Owen Campbell-Moore, Owen Skarpness, Robin Graham, Steef, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh More info if you would like to appear here: https://www.patreon.com/TwoMinutePapers Meet and discuss your ideas with other Fellow Scholars ## Транскрипт ### Intro [] Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Style transfer is a technique in machine learning research where we have two input images, one for content, and one for style, and the output is our content image reimagined with this new style. The cool part is that the content can be a photo straight from our camera, and the style can be a painting, which leads to the super fun results that you see here. An earlier paper had shown that the more sophisticated ones can sometimes make even art curators think that they are real. This previous work blew me away as it could perform style transfer for smoke simulations. I almost fell out of the chair when I have first seen these results. It could do fire textures, starry night, you name it. It seems that it is able to do anything we can think of. Now let me try to explain two things. One, why is this so difficult, and two, the results are looking really good, so, are there any shortcomings? Doing this for smoke simulations is a big departure from 2D style transfer, because that one takes an image, where this works in 3D, and does not deal with images, but with density fields. A density field means a collection of numbers that describe how dense a smoke plume is at a given spatial position. It is a physical description of a smoke plume, if you will. So how could we possibly apply artistic style from an image to a collection of densities? The solution in this earlier paper was to first, downsample the density field to a coarser version, perform the style transfer there, and upsample this density field again with already existing techniques. This technique was called Transport-based Neural Style Transfer, TNST in short, please ### Source [1:52] remember this. Now, let’s look at some results from this technique. This is what our simulation would look like normally, and then, all we have to do is show this image to the simulator, and, what does it do with it? Wow. My goodness. Just look at those heavenly patterns! So what does today’s new, followup work offer to us that the previous one doesn’t? How can this seemingly nearly perfect technique be improved? Well, this new work takes an even more brazen vantage point to this question. If style transfer on density fields is hard, then try a different representation. The title of the paper says Lagrangian style neural style transfer. So what does that mean? It means particles! This was made for particle-based simulations, which comes with several advantages. One, because the styles are now attached to particles, we can choose different styles ### Two Smoke Jets [2:52] for different smoke plumes, and they will remember what style they are supposed to follow. Because of this advantageous property, we can even ask the particles to change their ### Time-varying Stylization [3:04] styles over time, creating these heavenly animations. In these 2D examples, you also see how the texture of the simulation evolves over time ### Liquid Colorization [3:15] and that the elements of the style are really propagated to the surface and the style indeed follows how the fluid changes. This is true, even if we mix these styles together. Two, it not only provides us these high-quality results, but, it is fast. And by this, I mean blazing fast. You see, we talked about TNST, the transport-based technique approximately 7 months ago, and ### Lagrangian Neural Style Transfer for Fluids [3:46] in this series, I always note that two more papers down the line, and it will be much, much faster. So here is the Two Minute Papers Moment of Truth, what do the timings say? For the previous technique, it said more than 1d. ### Comparison to TNST [4:01] What could that 1D mean? Oh, goodness! This thing took an entire day to compute. So, what about the new one? What, really? Just one hour? That is insanity. So, how detailed of a simulation are we talking about? Let’s have a look together. M slash f means minutes per frame, and as you see, if we have tens of thousands of particles, we have 0. 05, or in other words, three seconds per frame, and we can go up to hundreds of thousands, or even millions of particles, and end up around thirty seconds per frame. Loving it. Artists are going to do miracles with this technique, I am sure. The next step is likely going to be a real-time algorithm, which may appear as soon as one or two more works down the line, and you can bet your papers that I’ll be here to cover it, so make sure to subscribe, and hit the bell icon to not miss it when it appears. The speed of progress in computer graphics research is nothing short of amazing. ### Regularization on Particle Position [5:05] Also, make sure to have a look at the full paper in the video description, not only because it is a beautiful paper and a lot of fun to read, but because you will also know what this regularization step here does exactly ### Regularization on Particle Density [5:21] to the simulation. Thanks for watching and for your generous support, and I'll see you next time!