# Refocusing Videos With Neural Networks | Two Minute Papers #173 ## Метаданные - **Канал:** Two Minute Papers - **YouTube:** https://www.youtube.com/watch?v=EGnbAgbRIh4 - **Дата:** 23.07.2017 - **Длительность:** 3:45 - **Просмотры:** 24,634 ## Описание The paper "Light Field Video Capture Using a Learning-Based Hybrid Imaging System" and its implementation is available here: https://arxiv.org/abs/1705.02997 https://github.com/junyanz/light-field-video Recommended for you: Amazing Slow Motion Videos With Optical Flow - https://www.youtube.com/watch?v=7aLda2E0Yyg Two Minute Papers Merch: US: http://twominutepapers.com/ EU/Worldwide: https://shop.spreadshirt.net/TwoMinutePapers/ WE WOULD LIKE TO THANK OUR GENEROUS PATREON SUPPORTERS WHO MAKE TWO MINUTE PAPERS POSSIBLE: Andrew Melnychuk, Christian Lawson, Dave Rushton-Smith, Dennis Abts, e, Esa Turkulainen, Kaben Gabriel Nanlohy, Michael Albrecht, Michael Orenstein, Steef, Sunil Kim, Torsten Reil, VR Wizard. https://www.patreon.com/TwoMinutePapers Music: Antarctica by Audionautix is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/) Artist: http://audionautix.com/ Thumbnail background image credit: https://pixabay.com/photo-272263/ Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu Károly Zsolnai-Fehér's links: Facebook → https://www.facebook.com/TwoMinutePapers/ Twitter → https://twitter.com/karoly_zsolnai Web → https://cg.tuwien.ac.at/~zsolnai/ ## Содержание ### [0:00](https://www.youtube.com/watch?v=EGnbAgbRIh4) Segment 1 (00:00 - 03:00) Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. Whenever we take an image with our camera, and look at it after an event, we often feel that many of them are close to perfect, if only it was less blurry, or the focus distance was a bit further away. But the magic moment is now gone, and there is nothing to do other than cursing at the blurry footage that we're left with when showing it to our friends. However, if we have access to light fields, we can change some camera parameters after the photo was taken. This includes changing the focal distance, or even slightly adjusting the viewpoint of the camera. How cool is that! This can be accomplished by a light field camera which is also referred to as a plenoptic camera. This tries to record not only light intensities, but the direction of incoming light as well. Earlier, this was typically achieved by using an array of cameras. That's both expensive and cumbersome. And here comes the problem with using only one light field camera: because of the increased amount of data that they have to record, current light field cameras are only able to take 3 frames per second. That's hardly satisfying if we wish to do this sort of post-editing for videos. This work offers a novel technique to remedy this situation by attaching a standard camera to this light field camera. The goal is that the standard camera has 30, so tons of frames per second, but with little additional information, and the light field camera, which has only a few frames per second, but, with a ton of additional information. If we stitch all this information together in a smart way, maybe it is a possibility to get full light field editing for videos. Earlier, we have talked about interpolation techniques that can fill some of the missing frames in videos. This way, we can fill in maybe every other frame in a footage, or we can be a bit more generous than that. However, if we're shown 3 frames a second, and we have to create a smooth video by filling the blanks would almost be like asking an algorithm to create a movie from a comic book. This would be awesome, but we're not there yet. Too much information is missing. This stitching process works with a bit more information than this, and the key idea is to use two convolutional neural networks to fill in the blanks: one is used to predict flows, which describe the movements and rotations of the objects in the scene, and one to predict the final appearance of the objects. Basically, one for how they move, and look. And the results are just absolutely incredible. It is also blazing fast and takes less than a tenth of a second to create one of these new views. Here, you can see how the final program is able to change the focal distance of any of the frames in our video, or we can even click on something in the image to get it in focus. And all this is done after the video has been taken. The source code of this project is also available. With some more improvements, this could be tremendously useful in the film industry, because the directors could adjust their scenes after the shooting, and not just sigh over the inaccuracies and missed opportunities. And this is just one of many possible other applications. Absolutely amazing. If you enjoyed this episode, don't forget to subscribe to Two Minute Papers and also make sure to click the bell icon to never miss an episode. Thanks for watching and for your generous support, and I'll see you next time! --- *Источник: https://ekstraktznaniy.ru/video/14621*