Virtual Reality Fluid Drawing Is Here! 🥛

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today we are going to control the fate of liquids in virtual worlds.

This footage is from one of my earlier papers where I attempted fluid control. This means that we are not only simulating the movement of a piece of fluid, but we wish to coerce it to flow into a prescribed shape. This was super challenging and I haven’t really seen a satisfactory solution that I think, artist could use in the industry yet. And now that we have these modern, neural-network based algorithms, we are now able to solve problems that we never even dreamed of solving just a few years ago. For instance, they can already perform this kind of style transfer for smoke simulations. Which is, incredible. So, are you thinking what I am thinking? Can one of those maybe tackle fluid control too? Well, that’s a tough call… just to showcase how difficult this problem is. If we wish to have any control over our fluid simulations, if we are a trained artist, we can sculpt the fluid directly ourselves. Of course, this requires a great deal of expertise, and often, hours of work. Can we do better? Well, yes, kind of. We can use a particle system built into most modern 3D modeling programs, with which we can try to guide these particles to a given direction. This took about 20 minutes, and it still requires some artistic expertise. So, that’s it then? No, hold on to your papers, and check this out! The preparation for this work takes place in virtual reality, where we can make these sketches in 3D, and look at that! The liquid magically takes the shape of our sketch. So, how long did this take? Well, not one hour, and not even 20 minutes. It took one minute. Now we’re talking! And even better, we can embed this into a simulation, and it will behave like a real piece of fluid should. So, what is all this good for? Well, my experience has been in computer graphics is that if we put a powerful tool like this into the hands of capable artist, they are going to create things that we never even thought of creating. With this, they can make a heart from wine. Or create a milky skirt. Several variants even, if we wish. Or a liquid butterfly. I am loving these solutions, and don’t forget, all of these can be then embedded into a virtual

world and simulated as a real liquid. Now, we talked about three solutions, and how much they take, but we didn’t see what they looked like. Clearly, it is hard to compare these mathematically, so this is going to be, of course, subjective. So, this took an hour. It looks very smooth, and is perhaps the most beautiful of the three solutions. That is great. However, as a drawback, it does not look like a real-world water splash. The particle system took 20 minutes, it creates a more lifelike version of our letter, but the physics is still missing. This still looks like a trail of particles, not a physics system. And, let’s see the new method. Yes, this took only a minute, and it finally looks like a real splash. Now, make no mistake, all three of these solutions can be excellent, depending on our artistic vision. So, how does all this magic happen? What is the architecture of this neural network? Well, this behavior emerges not from one, but from the battle of two neural networks. A generator neural network creates new splashes, and the discriminator finds out whether these splashes are real or fake. Over time, they challenge each other, and they teach each other to do better. The technique also goes the extra mile beyond just sketching. Look, for instance, your brush strokes can also describe velocities. With this, we can not only control the shape, but even the behavior of the fluid too. So, there we go.

Finally, a learning-based technique gives us a proper solution for fluid control. And here comes the best part - is it not only quicker than previous solutions, it can also

be used by anyone. No artistic expertise is required! What a time to be alive! Thanks for watching and for your generous support, and I'll see you next time!