# Can An AI Heal This Image?👩‍⚕️

## Метаданные

- **Канал:** Two Minute Papers
- **YouTube:** https://www.youtube.com/watch?v=vx7H7GrE5KA
- **Дата:** 08.06.2021
- **Длительность:** 8:00
- **Просмотры:** 293,634
- **Источник:** https://ekstraktznaniy.ru/video/13894

## Описание

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.com/papers 
❤️ Their mentioned post is available here: https://wandb.ai/wandb/getting-started/reports/Debug-Compare-Reproduce-Machine-Learning-Models--VmlldzoyNzY5MDk?utm_source=karoly

📝 The paper "Self-Organising Textures" is available here:
https://distill.pub/selforg/2021/textures/

Game of Life animation source: https://copy.sh/life/
Game of Life image source: https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Aleksandr Mashrabov, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, Christian Ahlin, Eric Haddad, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Bustamante, John Le, Jonas, Kenneth Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Mark Oates, Michael Albrecht, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Ramsey Elbasheer, Steef, Taras Bobrovy

## Транскрипт

### <Untitled Chapter 1> []

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today we are going to play with a cellular automaton and fuse it together with a neural

### Cellular Automaton [0:06]

network. It will be quite an experience! But of course, first, what are these things anyway? You can imagine a cellular automaton as a small game where we have a bunch of cells, and a set of simple rules that describe when a cell should be full, and when it should be empty. What you see here is a popular example called Game of Life, which simulates a tiny world

### Game of Life [0:32]

where each cell represents a little life form. So why is this so interesting? Well, this cellular automaton shows us that even a small set of simple rules can give rise to remarkably complex life forms, such as gliders, spaceships, and even John von Neumann’s universal constructor, or in other words, self-replicating machines. Now, this gets more interesting, a later paper fused this cellular automaton with a neural network. It was tasked to grow, and even better, maintain a prescribed shape. Remember these two words, grow and maintain shape. And the question was, if it can recover from undesirable states, can it perhaps... regenerate when damaged? Well, here, you will see all kinds of damage…and then, this happens. Nice! The best part is that this thing wasn’t even trained to be able to perform this kind of regeneration! The objective for training was that it should be able to perform its task of growing and maintaining shape, and it turns out, some sort of regeneration is included in that. This sounds very promising, and I wonder if we can we apply this concept to something where healing is instrumental? Are there such applications in computer science? If so, what could those be? Oh yes, yes there are! For instance, think about texture synthesis. This is a topic that is subject to a great deal of research in computer graphics, and those folks have this down to a science. So, what are we doing here?

### Texture Synthesis [2:21]

Texture synthesis typically means that we need lots of concrete or gravel road, skin, marble, create unique stripes for zebras, for instance, for a computer game or the post-production of a movie, and we really don’t want to draw miles and miles of these textures by hand. Instead, we give it to an algorithm to continue this small sample, where the output should be a bigger version of this pattern, with the same characteristics. So how do we know if we have a good technique at hand? Well, first, it must not be repetitive, checkmark, and it has to be free of seams. This part means that we should not be able to see any lines or artifacts that would quickly give the trick away. Now, get this, this new paper attempts to do the same with neural cellular automatons. What an insane idea! We like those around here, so let’s give it a try! How? Well, first, by trying to expand this simple checkerboard pattern. The algorithm is starting out from random noise, and as it evolves…well, this is a disaster. We are looking for squares, but we have quadrilaterals. They are also misaligned. And they are also inconsistent. But, luckily, we are not done yet. And now, hold on to your papers, and observe how the grid cells communicate with each other to improve the result. First, the misalignment is taken care of, then, the quadrilaterals become squares, and then, the consistency of their placement is improved. And the end result is, look! In this other example, we can not only see these beautiful bubbles grow out of nowhere. But the density of the bubbles remains roughly the same over the process. Look! As two of them get too close to each other, they coalesce, or pop. Damaged bubbles can also regrow. Very cool! Okay, it can do proper texture synthesis, but so can a ton of other handcrafted computer graphics algorithms, so why is this interesting? Why bother with this? Well, first, you may think that the result of this technique is the same as other techniques, but it isn’t. The output is not necessarily just an image, but can be an animation too! Excellent. Here, it was also able to animate the checkerboard pattern, and, even better, it can not only reproduce the weave pattern, but the animation part extends to this too. And now comes the even more interesting part. Let’s ask why does it output an animation and not an image? The answer lies within these weaving patterns. We just need to carefully observe them. Let’s see. Yes, again, we start out from noise, where some woven patterns emerge, but then, it almost looks like a person who started weaving them until it resembles the initial sample. Yes, that is the key! The neural network learned to create not an image, not an animation, but no less than a computer program to accomplish this kind of texture synthesis! How cool is that? So, armed with all that knowledge, do you remember the regenerating iguana project? Let’s try to destroy these textures too and see if it can use these computer programs to recover and get us a seamless texture. First, we delete parts of the texture, then, it fills in the gap with noise, and now, let’s run that program! Wow! Resilient, self-healing texture synthesis. And in every case, it starts out from a solution that is completely wrong, improves it to be just kind of wrong, and after further improvement, there you go. Fantastic! What a time to be alive! And, note that this is a paper in the wonderful Distill journal, which not only means that it is excellent, but also interactive, so you can run many of these experiments yourself right in your web browser. The link is available in the video description. Thanks for watching and for your generous support, and I'll see you next time!