This Neural Network Regenerates…Kind Of 🦎

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today we are going to play with a cellular automaton. You can imagine these automata as small games where we have a bunch of cells, and a set of simple rules that describe when a cell should be full, and when it should be empty. These rules typically depend on the state of the neighboring cells. For instance, perhaps the most well-known form of this cellular automaton is John Horton Conway’s Game of Life, which a simulates a tiny world where each cell represents a little life form. The rules, again, depend on the neighbors of this cell - if there are too many neighbors, they will die due to overpopulation, if too few, underpopulation, and if they have just the right amount of neighbors, they will thrive, and reproduce. So why is this so interesting? Well, this cellular automaton shows us that a small set of simple rules can give rise to remarkably complex life forms, such as gliders, spaceships, and even John von Neumann’s universal constructor, or in other words, self-replicating machines. I hope you think that’s quite something, and in this paper today, we are going to take this concept further. Way further! This cellular automaton is programmed to evolve a single cell to grow into a prescribed kind of life form. Apart from that, there are many other key differences from other works, and we will highlight two of them today. One, the cell state is a little different because it can either be empty, growing, or mature, and even more importantly, two, the mathematical formulation of the problem is written in a way that is quite similar to how we train a deep neural network to accomplish something. This is absolutely amazing. Why is that? Well, because it gives rise to a highly-useful feature, namely that we can teach it to grow these prescribed organisms. But wait, over time, some of them seem to decay, some of them can’t stop growing…and, some of them will be responsible for your nightmares, so, from this point on, proceed with care. In the next experiment, the authors describe an additional step in which it can recover from these undesirable states. And now, hold on to your papers, because this leads to the one of the major points of this paper. If it can recover from undesirable states, can it perhaps... regenerate when damaged? Well, here, you will see all kinds of damage…and then, this happens. Wow! The best part is that this thing wasn’t even trained to be able to perform this kind of regeneration! The objective for training was that it should be able to perform its task of growing and maintaining shape, and it turns out, some sort of regeneration is included in that. It can also handle rotations as well, which will give rise to a lot of fun, as noted a moment ago, some nightmarish experiments. And, note that this is a paper in the Distill journal, which not only means that it is excellent, but also interactive, so you can run many of these experiments yourself right in your web browser. If Alexander Mordvintsev, the name of the first author rings a bell, he worked on Google’s Deep Dreams approximately 5 years ago. How far we can some since, my goodness. Loving these crazy, non-traditional research papers and am looking forward to seeing more of these. This episode has been supported by Weights & Biases. Here, they show you how you can visualize the training process for your boosted trees with XGBoost using their tool. If you have a closer look, you’ll see that all you need is one line of code. Weights & Biases provides tools to track your experiments in your deep learning projects. Their system is designed to save you a ton of time and money, and it is actively used in projects at prestigious labs, such as OpenAI, Toyota Research, GitHub, and more. And, the best part is that if you are an academic or have an open source project, you can use their tools for free. It really is as good as it gets. Make sure to visit them through wandb. com/papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights & Biases for their long-standing support and for helping us make better videos for you. Thanks for watching and for your generous support, and I'll see you next time!