# Breaking Deep Learning Systems With Adversarial Examples | Two Minute Papers #43 ## Метаданные - **Канал:** Two Minute Papers - **YouTube:** https://www.youtube.com/watch?v=j9FLOinaG94 - **Дата:** 03.02.2016 - **Длительность:** 3:25 - **Просмотры:** 15,447 ## Описание Artificial neural networks are computer programs that try to approximate what the human brain does to solve problems like recognizing objects in images. In this piece of work, the authors analyze the properties of these neural networks and try to unveil what exactly makes them think that a paper towel is a paper towel, and, building on this knowledge, try to fool these programs. Carefully crafted adversarial examples can be used to fool deep neural network reliably. _______________ The paper "Intriguing properties of neural networks" is available here: http://arxiv.org/abs/1312.6199 The paper "Explaining and Harnessing Adversarial Examples" is available here: http://arxiv.org/abs/1412.6572 Image credits: Thumbnail image - https://www.flickr.com/photos/healthblog/8384110298 (CC BY-SA 2.0) Shower cap - Code Words / Julia Evans - https://codewords.recurse.com/issues/five/why-do-neural-networks-think-a-panda-is-a-vulture MNIST - hxhl95 Andrej Karpathy's online convolutional neural network: http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu Károly Zsolnai-Fehér's links: Patreon → https://www.patreon.com/TwoMinutePapers Facebook → https://www.facebook.com/TwoMinutePapers/ Twitter → https://twitter.com/karoly_zsolnai Web → https://cg.tuwien.ac.at/~zsolnai/ ## Содержание ### [0:00](https://www.youtube.com/watch?v=j9FLOinaG94) Segment 1 (00:00 - 03:00) Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. Artificial neural networks are computer programs that try to approximate what the human brain does to solve problems like recognizing objects in images. In this piece of work, the authors analyze the properties of these neural networks and try to unveil what exactly makes them think that a paper towel is a paper towel, and, building on this knowledge, try to fool these programs. Let's have a look at this example. One can grab this input image, and this noise pattern, and add these two images together similarly as one would add two numbers together. The operation yields the image you see here. I think it's fair to say that the difference is barely perceptible for the human eye. Not so much for neural networks, because the input image we started with is classified correctly as a bus, and the image that you see on the right is classified as an ostrich. In simple terms, bus + noise equals an ostrich. The two images look almost exactly the same, but the neural networks see them quite differently. We call these examples adversarial examples because they are designed to fool these image recognition programs. In machine learning research, there are common datasets to test different classification techniques on, one of best known example is the MNIST handwriting dataset. It is a basically a bunch of images depicting handwritten numbers that machine learning algorithms have to recognize. Long ago, this used to be a difficult problem, but nowadays, any half-decent algorithm can guess the numbers correctly more than 99% of the time after learning for just a few seconds. Now we'll see that these adversarial examples are not created by chance: if we add a lot of random noise to these images, they get quite difficult to recognize. Let's engage in modesty and say that I, myself, as a human can recognize approximately half of them, but only if I look closely and maybe even squint. A neural network can guess this correctly approximately 50% of the time as well, which is a quite respectable result. Therefore, adding random noise is not really fooling the neural networks. However, if you look at these adversarial examples in the even columns, you see how carefully they are crafted as they look very similar to the original images, but the classification accuracy of the neural network on these examples is 0%. You heard it correctly. It gets it wrong basically all the time. The take home message is that carefully crafted adversarial examples can be used to fool deep neural network reliably. You can watch them flounder on many hilarious examples to your enjoyment. "My dear sir, the Queen wears a shower cap you say? I beg your pardon? " If you would like to support Two Minute Papers, we are available on Patreon and offer cool perks for our Fellow Scholars - for instance, you can watch each episode around 24 hours in advance, or even decide the topic of the next episodes. How cool is that?! If you're interested, just click on the box below on the screen. Thanks for watching and for your generous support, and I'll see you next time! --- *Источник: https://ekstraktznaniy.ru/video/14880*