AI Learning Morphology and Movement...at the Same Time!

Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. Reinforcement learning is a class of learning algorithms that chooses a set of actions in an environment to maximize a score. Typical use-cases of this include writing an AI to master video games or avoiding obstacles with a drone and many more cool applications. What ties most of these ideas together is that whenever we talk about reinforcement learning, we typically mean teaching an agent how to navigate in an environment. A few years ago, a really fun online app surfaced that used a genetic algorithm to evolve the morphology of a simple 2D car with the goal of having it roll as far away from a starting point as possible. It used a genetic algorithm that is quite primitive compared to modern machine learning techniques, and yet it still does well on this, so how about tasking a proper reinforcement learner to optimize the body of the agent? What’s more, what if we would jointly learn both the body and the navigation at the same time? Ok, so what does this mean in practice? Let’s have a look at an example. Here we have an ant that is supported by four legs, each consisting of three parts that are controlled by two motor joints. With the classical problem formulation, we can teach this ant to use these joints to learn to walk, but in the new formulation, not only the movement, but the body morphology is also subject to change. As a result, this ant learned that the body can also be carried by longer, thinner legs and adjusted itself accordingly. As a plus, it also learned how to walk with these new legs and this way, it was able to outclass the original agent. In this other example, the agent learns to more efficiently navigate a flat terrain by redesigning its legs that are now reminiscent of small springs and uses them to skip its way forward. Of course, if we change the terrain, the design of an effective agent also changes accordingly, and the super interesting part here is that it came up with an asymmetric design that is able to climb stairs and travel uphill efficiently. Loving it! We can also task this technique to minimize the amount of building materials used to solve a task, and subsequently, it builds an adorable little agent with tiny legs that is still able to efficiently traverse this flat terrain. This principle can also be applied to the more difficult version of this terrain, which results in a lean, insect-like solution that can still finish this level that uses about 75% less materials than the original solution. And again, remember that not only the design, but the movement is learned here at the same time. While we look at these really fun bloopers, I’d like to let you know that we have an opening at our Institute at the Vienna University of Technology for one PostDoctoral researcher. The link is available in the video description, read it carefully to make sure you qualify, and if you apply through the specified e-mail address, make sure to mention Two Minute Papers in your message. This is an excellent opportunity to read and write amazing papers, and work with some of the sweetest people. This is not standard practice in all countries so I’ll note that you can check the salary right in the call, it is a well-paid position in my opinion, and you get to live in Vienna. Also, your salary is paid not 12, but 14 times a year. That’s Austria for you — it doesn't get any better than that. Deadline is end of January. Happy holidays to all of you! Thanks for watching and for your generous support, and I'll see you early January.