# What Can We Learn From Deep Learning Programs? | Two Minute Papers #75 ## Метаданные - **Канал:** Two Minute Papers - **YouTube:** https://www.youtube.com/watch?v=ZBWTD2aNb_o - **Дата:** 22.06.2016 - **Длительность:** 5:34 - **Просмотры:** 8,442 ## Описание The paper "Model Compression" is available here: https://www.cs.cornell.edu/~caruana/compression.kdd06.pdf There is also a talk on it here: http://research.microsoft.com/apps/video/default.aspx?id=103668&r=1 Discussions on this issue: 1. https://www.linkedin.com/pulse/computer-vision-research-my-deep-depression-nikos-paragios 2. https://www.reddit.com/r/MachineLearning/comments/4lq701/yann_lecuns_letter_to_cvpr_chair_after_bad/ Recommended for you: Neural Programmer Interpreters - https://www.youtube.com/watch?v=B70tT4WMyJk WE WOULD LIKE TO THANK OUR GENEROUS PATREON SUPPORTERS WHO MAKE TWO MINUTE PAPERS POSSIBLE: David Jaenisch, Sunil Kim, Julian Josephs. https://www.patreon.com/TwoMinutePapers We also thank Experiment for sponsoring our series. - https://experiment.com/ Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz The thumbnail background image was created by John Lord - https://flic.kr/p/nVUaB Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu Károly Zsolnai-Fehér's links: Facebook → https://www.facebook.com/TwoMinutePapers/ Twitter → https://twitter.com/karoly_zsolnai Web → https://cg.tuwien.ac.at/~zsolnai/ ## Содержание ### [0:00](https://www.youtube.com/watch?v=ZBWTD2aNb_o) Segment 1 (00:00 - 05:00) ‏ dear fellow scholars this is two minut papers with kol feh i have recently been witnessing a few heated conversations regarding the submission of deep learning papers to computer vision conferences the forums are up in arms about the fact that despite some of these papers showcase remarkably good results they were rejected on the basis of from what i have heard not adding too much to thee of that we understand is neural networks ‏ cannot really learn anything new from them i'll try to understand and rephrase their argument differently we know exactly how to train a neural network it's just that as an output of this process we get a model of something that resembles a brain as a collection of neurons and circumstances under which these neurons are activated we store these in a file that can take up to several gigabytes and the best solutions are often not intuitively understandable for us ‏ instance in this video we are training a neural network to classify these points correctly but what exactly can we learn if we look into these neurons now imagine that in practice we don't have a handful of these boxes but millions of them and more complex than the ones you see here let's start with a simple example that hopefully helps getting a better grip of this argument now i'll be damned if this video won be more than a couple minutes so this is going to be one of those slightly extended two minute papers episode ‏ i hope you don't mind the grammatical rules of my native language a lot of them are contained in enormous tomes that everyone has to go through during their school years rules are important they give the scaffolding for constructing sentences that are grammatically correct can we explain or even enumerate these rules well unless you are a linguist the answer is no almost no one really remembers more than a few rules but every native speaker knows how their ‏ sent that andar mak proper sentence and what is gibberish this is exactly what neural networks do they are trained in a very similar way in fact they are so effective at it that if we try to forcefully insert some of our knowledge in there the solutions are going to get worse appropriate time to askit a paper and ‏ define as scientific progress what if we have extremely accurate algorithms where we don't know what is going on under the hood or simpler more intuitive algorithms that may be subpar in accuracy if we have a top tier scientific conference where only a very limited number of papers get accepted which ones shall we accept i hope that this question will spark a productive discussion and hopefully scientific research venues will be more vigilant about this question in the future ‏ ok so the question is crystal clear knowledge or efficiency how about possible solutions can we extract scientific insights out of these neural networks model compression is a way to essentially compress the information in this brain thing this collection of neurons we described earlier to demonstrate why this is such a cool idea let's quickly jump to this program by deep mind that plays atar games at an amazingly high level in breakout the ‏ solution program that you see here is essentially an enormous table that describes what the program should do when it sees different inputs it is so enormous that it has many millions of records in there a manual of many pages if you will it is easy to execute for a computer but completely impossible for us to understand why and how it works however if we intuitively think about the game itself we could actually write a super simple program ‏ in one line of code that would almost be as good as this all we need to do is try to follow the ball with the paddle one line of code and pretty decent results not optimal but quite decent from such a program we can actually learn something about the game essentially what we could do with these enormous tables is compressing them into much smaller ones that are so tiny that we can actually build an intuition from them this way ‏ machar tech fin out procedure be knowledge insight if think about such algorithm essentially do research by at first it would randomly try experimenting and after a large amount of observations are collected these observations would explained small number of rules definition ‏ research and perhaps ### [5:00](https://www.youtube.com/watch?v=ZBWTD2aNb_o&t=300s) Segment 2 (05:00 - 05:00) this is one of the more interesting future frontiers of machine learning research and by the way earlier we have talked about a fantastic paper on neural programmer interpreters that also aim to output complete algorithms that can be directly used and understood the link is available in the description box thanks for watching and for your generous support and i'll see you next time ه --- *Источник: https://ekstraktznaniy.ru/video/14810*