# Breaking DeepMind's Game AI System | Two Minute Papers #135

## Метаданные

- **Канал:** Two Minute Papers
- **YouTube:** https://www.youtube.com/watch?v=QFu0vZgMcqk
- **Дата:** 12.03.2017
- **Длительность:** 3:43
- **Просмотры:** 23,220
- **Источник:** https://ekstraktznaniy.ru/video/14696

## Описание

Our Patreon page is available here: https://www.patreon.com/TwoMinutePapers

The paper "Adversarial Attacks on Neural Network Policies" is available here:
http://rll.berkeley.edu/adversarial/

WE WOULD LIKE TO THANK OUR GENEROUS PATREON SUPPORTERS WHO MAKE TWO MINUTE PAPERS POSSIBLE:
Claudio Fernandes, Daniel John Benton, Dave Rushton-Smith, Sunil Kim, VR Wizard.
https://www.patreon.com/TwoMinutePapers

Recommended for you:
Breaking Deep Learning Systems With Adversarial Examples - https://www.youtube.com/watch?v=j9FLOinaG94

Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz

Music: Antarctica by Audionautix is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/)
Artist: http://audionautix.com/ 

Thumbnail background image credit: https://pixabay.com/photo-1837125/
Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu

Károly Zsolnai-Fehér's links:
Facebook → https:

## Транскрипт

### Segment 1 (00:00 - 03:00) []

Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. Not so long ago, Google DeepMind introduced a novel learning algorithm that was able to reach superhuman levels in playing many Atari games. It was a spectacular milestone in AI research. Interestingly, while these learning algorithms are being improved at a staggering pace, there is a parallel subfield where researchers endeavor to break these learning systems by slightly changing the information they are presented with. Fraudulent tampering with images or video feeds, if you will. Imagine a system that is designed to identify what is seen in an image. In an earlier episode, we discussed an adversarial algorithm, where in an amusing example, they added a tiny bit of barely perceptible noise to this image, to make the deep neural network misidentify a bus for an ostrich. Machine learning researchers like to call these evil forged images adversarial samples. And now, this time around, OpenAI published a super fun piece of work to fool these game learning algorithms by changing some of their input visual information. As you will see in a moment, it is so effective that by only using a tiny bit of information, it can turn a powerful learning algorithm into a blabbering idiot. The first method adds a tiny bit of noise to a large portion of the video input, where the difference is barely perceptible, but it forces the learning algorithm to choose a different action that it would have chosen otherwise. In the other one, a different modification was used, that has a smaller footprint, but is more visible. For instance, in pong, adding a tiny fake ball to the game to coerce the learner into going down when it was originally planning to go up. The algorithm is able to learn game-specific knowledge for almost any other game to fool the player. Despite the huge difference in the results, I loved the elegant mathematical formulation of the two noise types, because despite the fact that they do something radically different, their mathematical formulation is quite similar, mathematicians like to say that we're solving the same problem, while optimizing for different target norms. Beyond DeepMind's Deep Q-Learning, two other high-quality learning algorithms are also fooled by this technique. In the white box formulation, we have access to the inner workings of the algorithm. But interestingly, a black box formulation is also proposed, where we know much less about the target system, but we know the game itself, and we train our own system and look for weaknesses in that. When we've found these weak points, we use this knowledge to break other systems. I can only imagine how much fun there was to be had for the authors when they were developing these techniques. Super excited to see how this arms race of creating more powerful learning algorithms, and in response, more powerful adversarial techniques to break them develops. In the future, I feel that the robustness of a learning algorithm, or in other words, its resilience against adversarial attacks will be just as important of a design factor as how powerful it is. There are a ton of videos published on the authors' website, make sure to have a look! And also, if you wish to support the series, make sure to have a look at our Patreon page. We kindly thank you for your contribution, it definitely helps keeping the series running. Thanks for watching and for your generous support, and I'll see you next time!