# OpenAI's Bot Beats DOTA World Champion Dendi | Two Minute Papers #180 ## Метаданные - **Канал:** Two Minute Papers - **YouTube:** https://www.youtube.com/watch?v=cLC_GHZCOVQ - **Дата:** 16.08.2017 - **Длительность:** 7:35 - **Просмотры:** 32,768 ## Описание Some updates and clarifications follow: Update 1: we seem to have conflicting information on the training times - both 24 hours and 2 weeks was mentioned. We'll make sure to address this when the official paper appears. Update 2: more from OpenAI - https://blog.openai.com/more-on-dota-2/ Update 3: more reddit discussion on how to trick the bot into defeat: https://www.reddit.com/r/DotA2/comments/6t8qvs/openai_bots_were_defeated_atleast_50_times/ (thanks to nikre for the link) Update 4: an OpenAI employee provides more clarification on the training process - https://news.ycombinator.com/item?id=15001521 Apologies for the inaccuracies - I've watched every video and interview I could get my hands on and found quite a bit of conflicting information. I'll take this into consideration next time when something comes up without an official research paper. OpenAI's materials on their DOTA bot: https://blog.openai.com/dota-2/ Day9's DOTA learning videos are available here: https://www.youtube.com/playlist?list=PLgmCLtUkEutILNA9EM0BON6ShoQGZhd3P WE WOULD LIKE TO THANK OUR GENEROUS PATREON SUPPORTERS WHO MAKE TWO MINUTE PAPERS POSSIBLE: Andrew Melnychuk, Christian Lawson, Dave Rushton-Smith, Dennis Abts, e, Eric Swenson, Esa Turkulainen, Kaben Gabriel Nanlohy, Michael Albrecht, Michael Orenstein, Steef, Sunil Kim, Torsten Reil. https://www.patreon.com/TwoMinutePapers Two Minute Papers Merch: US: http://twominutepapers.com/ EU/Worldwide: https://shop.spreadshirt.net/TwoMinutePapers/ Music: Antarctica by Audionautix is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/) Artist: http://audionautix.com/ Thumbnail background image source: https://blog.openai.com/dota-2/ Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu Károly Zsolnai-Fehér's links: Facebook → https://www.facebook.com/TwoMinutePapers/ Twitter → https://twitter.com/karoly_zsolnai Web → https://cg.tuwien.ac.at/~zsolnai/ ## Содержание ### [0:00](https://www.youtube.com/watch?v=cLC_GHZCOVQ) dear fellow scholars this is two minute papers with károly life ahead it is time for some minds to be blown dota 2 is a multiplayer online battle arena game with a huge cult following and World Championship events with a prize pool of over 20 million dollars in this game players form two teams and control a hero each and use their strategy and special abilities to defeat the other team open AI who recently created an AI for this game that is so good that they challenged the best players in the world now note that this program is not playing the full feature set of the game but a version that is limited to one versus one encounters with several other elements of the game disabled since lots of strategy is involved we always discuss in these episodes that long-term planning is the Achilles heel of these learning algorithms a small blunder in the early game can often snowball out of control by the end of the match and it is hard for the AI and sometimes to even humans to identify these cases and this game is a huge challenge because unlike chess and go it has lots of incomplete information and even the simplified 1vs1 mode involves a reasonable amount of long-term planning it also involves attacks trickery and deceiving an opponent and can be imagined as a strategy game that also requires significant technical prowess to pull off the most spectacular moves this game is also designed in a way that new and unfamiliar situations come up all the time which require lots of experience and split-second decision-making to master this is a true test for any kind of AI and note that this AI wasn't told anything about the game not even the rules and was just instructed to try to find a way to win the algorithm was trained in 24 hours and during this time it not only learned the rules and objectives of the game but it also pulls off remarkable tactics for instance other players were very surprised that the bots didn't take the bait which typically means a smart tactic involving giving up a smaller battle in favor of winning a bigger objective the AI has a ton of experience playing the game and typically sees through these shenanigans in this game there are also neutral units that we call creep when killed they grant precious gold and experience to our opponent so we typically try to deny that if these units encounter an obstacle they go around it so players develop the technique by the name creep blocking which is the art of holding them up by the hero character to minimize the distance traveled by them in a unit of time and the AI has not only learned about the existence of this technique by itself but it also executed with stunning precision which is quite remarkable and again during the training phase it had never seen any human play the game and do something like this the other remarkable thing is that when a player disappears in the darkness the AI predicts what he could be doing plans around it and strikes where the player is expected to show up if you remember deep Minds initial go algorithm contained the bootstrapping step where it was fed a large amount of games by players to grasp the basics the truly remarkable thing is that none of that happened here the algorithm was trained for only 24 hours and it only played against itself when it finally played against dandy the reigning world champion the first match was such a treat and I was shocked to see that the AI has outplayed him in the second game the player tried to create a situation that he thought the AI hasn't encountered before by giving up some crit to it the program ruthlessly took advantage of this mistake and defeated him almost immediately open es bought not only one but apparently also broke the will of dandy who tapped out after two matches I can hear you now den do we control it I give up giving up yeah I don't think I'm skiing is like break is that sweet alright it's gonna drop me I've seen this quite a lot this week okay so what you want to do like that's to our five or your outfit I feel like someone being hit by a sledgehammer I didn't even know this was ### [4:30](https://www.youtube.com/watch?v=cLC_GHZCOVQ&t=270s) Last Hitting Dealing the final blow to an enemy unit in order to be awarded extra gold. being worked on this is such a remarkable achievement usually the first argument I hear is that of course the a I can play non-stop without bathroom breaks or sleep while admittedly this is ### [4:43](https://www.youtube.com/watch?v=cLC_GHZCOVQ&t=283s) Manipulating Creep Aggro Attacking the enemy hero to draw their creeps to you, letting you manipulate their positions. also true for some players the algorithm was only trained for 24 hours note that the still means a stupendous amount of games played but in terms of training time given that these algorithms typically take from weeks to months to train properly 24 hours is ### [4:58](https://www.youtube.com/watch?v=cLC_GHZCOVQ&t=298s) Creep Blocking Slowing your creeps down to control their positioning. nothing the second argument that I often hear is that the AI should of course win every time because it has close to zero reaction time and can perform thousands of actions every second for instance if ### [5:12](https://www.youtube.com/watch?v=cLC_GHZCOVQ&t=312s) Zoning Preventing your opponent from getting close to your creeps, making it hard for them to gain experience and gold. we would play a game where the goal is to perform the most amount of actions per minute clearly humans with biological limitations would stand no ### [5:23](https://www.youtube.com/watch?v=cLC_GHZCOVQ&t=323s) Raze Dodging Dodging damaging spells by understanding the lag between their casting animation and impact. chance against the computer program however in this case the number of actions that this algorithm performs in a minute is comparable to that of a human player this means that these results stem from superior technical abilities and planning and not from the fact that we are talking about the computer we can look at this result from two different directions one could be saying well no big deal because this is only a highly limited and hamstrung version of the game which is way less ### [5:52](https://www.youtube.com/watch?v=cLC_GHZCOVQ&t=352s) Raze Faking Executing part of a raze animation to trick the enemy into dodging it. complex than the fully fleshed 5vs5 team match or two we could say that the algorithm had shown a remarkable aptitude for learning highly ### [6:02](https://www.youtube.com/watch?v=cLC_GHZCOVQ&t=362s) New Situations Adapting to situations never encountered during training, like the opponent buying 3 couriers, rather than 1. sophisticated technical maneuvers and longer-term strategy in a difficult game and the rest is only a matter of time in fact in five versus five there is even more room for a highly intelligent program to shine and create new tactics that we've never thought of I would bet that if anything we are going to be even more surprised by the five versus five results later we are still lacking in details of it but I have contacted the open AI guys who noted that there will ### [6:29](https://www.youtube.com/watch?v=cLC_GHZCOVQ&t=389s) Chasing Pursuing an enemy across very long distances in order to secure a win. be more information available in the next few days whenever something new appears I'll be here to cover it for you fellow scholars if you're new to the series and enjoyed this episode make sure to subscribe and click the bell icon for two super fun science videos a week and if you find yourself interested in dota 2 and admittedly it's hard not to and would like to catch up a bit on the basics make sure to visit the 9th channel who has a really nice playlist about the fundamentals of the game there's a link in the description for it check it out if you go to his channel make sure to live in a kind scholarly comment let the world see how courteous the two minute papers listeners are thanks for watching and for your generous support and I'll see you next time you --- *Источник: https://ekstraktznaniy.ru/video/14607*