# Should AI Research Try to Model the Human Brain? ## Метаданные - **Канал:** Two Minute Papers - **YouTube:** https://www.youtube.com/watch?v=QPwhEnAILa0 - **Дата:** 29.05.2019 - **Длительность:** 7:00 - **Просмотры:** 94,065 ## Описание ❤️ Pick up cool perks on our Patreon page: https://www.patreon.com/TwoMinutePapers ₿ Crypto and PayPal links are available below. Thank you very much for your generous support! › PayPal: https://www.paypal.me/TwoMinutePapers › Bitcoin: 1a5ttKiVQiDcr9j8JT2DoHGzLG7XTJccX › Bitcoin Cash: qzy42yt06xqr5f83khnhcxm2mf53cllvtytyp6ndw5 › Ethereum: 0xbBD767C0e14be1886c6610bf3F592A91D866d380 › LTC: LM8AUh5bGcNgzq6HaV1jeaJrFvmKxxgiXg 📝 The paper "Reinforcement Learning, Fast and Slow" is available here: https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(19)30061-0 The Bitter Lesson: https://www.youtube.com/watch?v=wEgq6sT1uq8 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: 313V, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Brian Gilman, Bruno Brito, Bryan Learn, Christian Ahlin, Christoph Jadanowski, Claudio Fernandes, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, Ivelin Ivanov, James Watt, Javier Bustamante, John De Witt, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Marcin Dukaczewski, Marten Rauschenberg, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Richard Reis, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, Steve Messina, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Zach Boldyga. https://www.patreon.com/TwoMinutePapers Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu Károly Zsolnai-Fehér's links: Facebook: https://www.facebook.com/TwoMinutePapers/ Twitter: https://twitter.com/karoly_zsolnai Web: https://cg.tuwien.ac.at/~zsolnai/ ## Содержание ### [0:00](https://www.youtube.com/watch?v=QPwhEnAILa0) Segment 1 (00:00 - 05:00) Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. AI research has come a long-long way in the last few years. I remember that not so long ago, we were lucky if we could train a neural network to understand traffic signs, and since then, so many things happened: by harnessing the power of learning algorithms, we are now able to impersonate other people by using a consumer camera, generate high-quality virtual human faces for people that don’t exist, or pretend to be able to dance as a pro dancer by using an external video footage and transferring it onto ourselves. Even though we are progressing at a staggering pace, there is a lot of debate as to which research direction is the most promising going forward. Roughly speaking, there are two schools of thought. One, we recently talked about Richard Sutton’s amazing article by the name, “The Bitter Lesson”, in which he makes a great argument that AI research should not try to mimic the way the human brain works - he argues that instead, all we need to do is formulate our problems in a general manner, so that our learning algorithms may find something that is potentially much better suited for a problem than our brain is. I put a link to this video in the description if you’re interested. And two, a different school of thought says that we should a good look at all these learning algorithms that use a lot of powerful hardware and can do wondrous things, like playing a bunch of Atari games at a superhuman level. Note that they learn orders of magnitude slower than the human brain does, so it should definitely be worth it to try to study and model the human brain, at least until we can match it in terms of efficiency. This school of thought is what we are going to talk about in this video. As an example, let’s take a look at deep reinforcement learning in the context of playing computer games. This technique is a combination of a neural network that processes the visual data that we see on the screen, and a reinforcement learner that comes up with the gameplay-related decisions. Absolutely amazing algorithm, a true breakthrough in AI research. Very powerful, however, also quite slow. And by slow, I mean that we can sit for an hour in front of our computer and wonder why our learner does not work at all, because it loses all of its lives almost immediately. If we remain patient, we find out that it works, it just learns at a glacial pace. So, why is this so slow? Well, two reasons. Reason number one is that the learning happens through incremental parameter adjustment. What does that mean? If a human fails really badly at a task, the human would know that a drastic adjustment to the strategy is necessary, while the deep reinforcement learner would start applying tiny, tiny changes to its behavior and test again if things got better. This takes a while, and as a result, this seems unlikely to have a close relation to how we, humans think. The second reason for it being slow is the presence of weak inductive bias. This means that the learner does not contain any information about the problem we have at hand, or in other words, has never seen the game we’re playing before and has no other previous knowledge about games at all. This is desirable in some cases, because we can reuse one learning algorithm for a variety of problems. However, because this way, the AI has to test a stupendously large number of potential hypotheses about the game, we will have to pay for this convenience by a mighty inefficient algorithm. But is this really all true? Does deep reinforcement learning really have to be so slow? And what on earth does this have to do with our brain? Well, this paper proposes an interesting counterargument that this is not necessarily true and argues that with two well thought out changes, the efficiency of deep reinforcement learning may be drastically improved, and get this, it also tells us that these changes are also possibly based in neuroscience. So what are the two changes? One is using episodic memory, which stores previous experiences to help estimating the potential value of different actions, and this way, drastic parameter adjustments become a possibility. And it not only improves the efficiency, but there is more to it, because there are recent studies that show that using episodic memory indeed contributes to the learning of real humans and animals alike. And two, it is beneficial to let the AI implement its own reinforcement learning algorithm, a concept often referred to as “learning to learn” or meta reinforcement learning. This also helps obtaining more general knowledge that can be reused across tasks, further improving the efficiency of the agent. Here you see a picture of an fMRI, and some regions are marked with yellow and orange ### [5:00](https://www.youtube.com/watch?v=QPwhEnAILa0&t=300s) Segment 2 (05:00 - 07:00) here. What could these possibly mean? Well, hold on to your papers, because these highlight neural structures that implement a very similar meta reinforcement learning scheme within the human brain. It turns out that meta reinforcement learning, or this “learning to learn” scheme may not just be something that speeds up our AI algorithms, but may be a fundamental principle of the human brain as well. So these two changes to deep reinforcement learning not only drastically improve its efficiency, but it also suddenly maps quite a bit better to our brain. How cool is that? So, which school of thought are you most fond of? Should we model the brain, or should we listen to Richard Sutton’s Bitter Lesson? Let me know in the comments. Also, make sure to have a look at the paper, I found it to be quite readable, and you really don’t need to be a neuroscientist to enjoy it and learn quite a few new things. Make sure to have a look at it in the video description! Now, I think you noticed that this paper doesn’t contain the usual visual fireworks, and is more complex than your average Two Minute Papers video, and hence, I expect it to get significantly fewer views. That’s not a great business model, but no matter, I made this channel so I can share with you all these important lessons that I learned during my journey. This has been a true privilege and I am thrilled that I am still able to talk about all these amazing papers without worrying too much whether any of these videos will go viral or not. This has only been possible because of your unwavering support on Patreon. com/TwoMinutePapers. If you feel like chipping in, just click the Patreon link in the video description. If you are more like a crypto person, we also support cryptocurrencies like Bitcoin, Ethereum and Litecoin, the addresses are also available in the description. Thanks for watching and for your generous support, and I'll see you next time! --- *Источник: https://ekstraktznaniy.ru/video/14307*