# Photorealistic Images from Drawings | Two Minute Papers #80 ## Метаданные - **Канал:** Two Minute Papers - **YouTube:** https://www.youtube.com/watch?v=a3sgFQjEfp4 - **Дата:** 20.07.2016 - **Длительность:** 3:05 - **Просмотры:** 17,921 ## Описание The Two Minute Papers subreddit is available here: https://www.reddit.com/r/twominutepapers/ By using a convolutional neural networks (a powerful deep learning technique), it is now possible to build an application that takes a rough sketch as an input, and fetches photorealistic images from a database. ___________________________________ The paper "The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies" and the online demo is available here: http://sketchy.eye.gatech.edu/ The paper "Signature verification using a Siamese time delay neural network" is available here: https://scholar.google.hu/scholar?cluster=4400768003729787411&hl=en&as_sdt=0,5 The paper "Learning Fine-grained Image Similarity with Deep Ranking" is available here: https://arxiv.org/abs/1404.4661 Our deep learning-related videos are available here: https://www.youtube.com/playlist?list=PLujxSBD-JXglGL3ERdDOhthD3jTlfudC2 WE WOULD LIKE TO THANK OUR GENEROUS PATREON SUPPORTERS WHO MAKE TWO MINUTE PAPERS POSSIBLE: David Jaenisch, Sunil Kim, Julian Josephs, Daniel John Benton. https://www.patreon.com/TwoMinutePapers We also thank Experiment for sponsoring our series. - https://experiment.com/ Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz The thumbnail background image was drawn by Felícia Fehér. Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu Károly Zsolnai-Fehér's links: Facebook → https://www.facebook.com/TwoMinutePapers/ Twitter → https://twitter.com/karoly_zsolnai Web → https://cg.tuwien.ac.at/~zsolnai/ ## Содержание ### [0:00](https://www.youtube.com/watch?v=a3sgFQjEfp4) Segment 1 (00:00 - 03:00) Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. When we were children, every single one of us dreamed about having a magic pencil that would make our adorable little drawings come true. With the power of machine learning, the authors of this paper just made our dreams come true. Here's the workflow: we provide a crude drawing of something, and the algorithm fetches a photograph from a database that depicts something similar to it. It's not synthesizing new images from scratch from a written description like one of the previous works, it fetches an already existing image from a database. The learning happens by showing a deep convolutional neural network pairs of photographs and sketches. If you are not familiar with these networks, we have some links for you in the video description box! It is also important to note that this piece of work does not showcase a new learning technique, it is using existing techniques on a newly created database that the authors kindly provided free of charge to encourage future research in this area. What we need to teach these networks is the relation of a photograph and a sketch. For instance, in an earlier work by the name Siamese networks, the photo and the sketch would be fed to two convolutional neural networks with the additional information whether this pair is considered similar or dissimilar. This idea of Siamese networks was initially applied to signature verification more than 20 years ago. Later, Triplet networks were used provide the relation of multiple pairs, like "this sketch is closer to this photo than this other one". There is one more technique referred to in the paper that they used, which is quite a delightful read, make sure to have a look! We need lots and lots of these pairs so the learning algorithm can learn what it means that a sketch is similar to a photo, and as a result, fetch meaningful images for us. So, if we train these networks on this new database, this magic pencil dream of ours can come true. What's even better, anyone can try it online! This is going to be a very rigorous and scholarly scientific experiment - I don't know what this should be, but I hope the algorithm does. Well, that kinda makes sense. Thanks, algorithm! For those Fellow Scholars out there who are endowed with better drawing skills than I am, well, basically all of you - if you have tried it and got some amazing, or maybe not so amazing results, please post them in the comments section! Or, as we now have our very own subreddit, make sure to drop by and post some of your results there so we can marvel at them, or have a good laugh at possible failure cases. I am looking forward to meeting you Fellow Scholars at the subreddit. Flairs are also available. Thanks for watching, and for your generous support, and I'll see you next time! --- *Источник: https://ekstraktznaniy.ru/video/14801*