DeepMind’s New AI Finally Enters The Real World! 🤖

Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. Finally, today, DeepMind’s amazing AI,  MuZero that plays Chess and other games   has now finally entered the real world and has  learned to solve important real-world problems. This is a reinforcement learning technique that  works really well on games. Why? Well, in Chess,   Go and Starcraft, the controls are clear, we use  the mouse to move our units around or choose where   to move our pieces. And the score is also quite  clear: we get rewarded if we win the game. That   is going to be our score. To say that these worked  really well would be an understatement: DeepMind’s   MuZero is one of the best in the world in Chess,  Go, and Stacraft 2 as well. But one important   question still remains. Of course, they did not  create this AI to play video games. They created   it to be able to become a general-purpose AI  that can solve not just games, but many problems.    The games are just used as an excellent  testbed for this AI. So, what else can it do? Well, finally here it is! Hold on to your papers,  because scientists at DeepMind decided to start   using their MuZero AI to create a real solution  to a very important problem. Video compression.    And here comes the twist - they said, let’s  imagine that video compression is a video game.    Okay, that’s crazy, but let’s accept it for now.   But then, two questions: what are the controls,   and what is the score? How do we  know if we won video compression? Well, the video game controller in our hand will  be is choosing the parameters of the video encoder   for each frame. Okay, but there needs to  be a score. So what is the score here?    How do we win? Well, we win if we are able to  choose the parameters such that the quality of   the output video is as good as with the previous  compression algorithms, but, the size of the video   is smaller. The smaller the output video,  the better. That is going to be our score. And, it also uses self-competition, which  is now a popular concept in video game AIs.    This means that the AI plays against previous  versions of itself, and we measure its improvement   by it being able to defeat these previous  versions. If it can reliably do that, we   can conclude that the AI is indeed improving. This  concept works on boxing, playing catch, Starcraft   and I wonder how this would  work for video compression? Well, let’s see. Let’s immediately  drop it into deep waters.    Yes, we are going to test this against a  mature, state of art video compression algorithm   that you are likely already using at this very  moment as you are watching this on Youtube. Well,   good luck little AI, but I’ll be honest, there  is not much hope here. These traditional video   compression algorithms are a culmination  of decades of ingenious human research.    Can a newcomer AI beat it? I am not  sure. And now, hold on to your papers,   and let’s see together. How did it go? So, a 4%  difference. So, a learning-based algorithm that   is just 4% worse than decades of human innovation?   That is great! But…wait a second, it's actually   not worse. Can it be that…yes! It is not 4% worse.   It is even 4% better. Holy mother of papers, that   is absolutely incredible. Yes, this is the corner  of the internet where we get super excited by a 4%   better solution, and understand why that matters  a great deal. Welcome to Two Minute Papers! But wait, we are experienced Fellow Scholars  over here, we know that it is very easy to   be better by 4% in size at the cost of decreased  quality. But having the same quality and save 4%   is insanely difficult. So, which one is it? Let’s  look together. I am flicking between the state

of the art and the new technique, and, yes, my  goodness, the results really speak for themselves. So, let’s look a bit under the hood and see  some more about the decisions the AI is making.    Whoa. That is really cool. So what is this? Here,  we see the scores for the previous technique   and the new AI, and here, they appear to be  making similar decisions on this cover song video,   but the AI makes somewhat better  decisions overall. That is very cool.    But, look at that! In the second half  of this gaming video, MuZero makes   vastly different, and, vastly  better decisions. I love it. And to have a first crack at such a mature  problem, and manage to improve it immediately,   that is almost completely unheard of. Yet, they  have done it with protein folding, and now,   they seem to have done it for video  compression too. Bravo, DeepMind! And note the meaning in the magnitude of the  difference here. For instance, OpenAI’s Dall-E 2   was this much better than Dall-E 1. That’s  not 4% better, if that was a percentage,   this would be several hundred percent  better. So, why get so excited about 4%?    Well, the key is that 3-4% more compression is  incredible given how well polished these state   of the art techniques are. VP9 compressors  are not some first crack at the problem,   no-no. This is a mature field  with decades of experience,   where every percent of improvement requires blood,  papers and tears, and of course, lots of compute   and memory. And this is just the first crack  at the problem for DeepMind, and we get not 1%,   but 4% essentially for free. That is absolutely  amazing. My mind is blown by this result. Wow. And, I also wanted to thank you for watching  this video. I truly love talking about these   amazing research papers, and I am really  honored to have so many of you Fellow Scholars   who are here every episode, enjoying these  incredible works with me. It really means a lot.    Every now and then I have  to pinch myself to make sure   that I really get to do this every day.   Absolutely amazing. Thank you so much! So, what do you think? What else could this be   useful for? What do you expect to happen  a couple more papers down the line?    Please let me know in the comments  below. I’d love to hear your thoughts. Thanks for watching and for your generous  support, and I'll see you next time!