# Learning to Fill Holes in Images | Two Minute Papers #130

## Метаданные

- **Канал:** Two Minute Papers
- **YouTube:** https://www.youtube.com/watch?v=psOPu3TldgY
- **Дата:** 22.02.2017
- **Длительность:** 2:56
- **Просмотры:** 15,094

## Описание

The paper "Scene Completion Using Millions of Photographs" is available here:
http://graphics.cs.cmu.edu/projects/scene-completion/scene-completion.pdf

WE WOULD LIKE TO THANK OUR GENEROUS PATREON SUPPORTERS WHO MAKE TWO MINUTE PAPERS POSSIBLE:
Claudio Fernandes, Daniel John Benton, Dave Rushton-Smith, Sunil Kim.
https://www.patreon.com/TwoMinutePapers

Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz

Music: Dat Groove by Audionautix is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/)
Artist: http://audionautix.com/

Thumbnail background image credit: https://pixabay.com/photo-1984308/
Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu

Károly Zsolnai-Fehér's links:
Facebook → https://www.facebook.com/TwoMinutePapers/
Twitter → https://twitter.com/karoly_zsolnai
Web → https://cg.tuwien.ac.at/~zsolnai/

## Содержание

### [0:00](https://www.youtube.com/watch?v=psOPu3TldgY) Segment 1 (00:00 - 02:00)

Dear fellow scholars, this is two minute papers with Kohaa. This paper is from 2007 from 10 years ago and I'm sure you'll be surprised by how well it is holding up to today's standards. For me, it was one of the works that foreshadowed the incredible power of datadriven learning algorithms. So, let's grab an image and cut a sizable part out of it and try to algorithmically fill it with data that makes sense. Removing a drunk photobombing friend from your wedding picture or a building blocking a beautiful view to the sea are excellent and honestly painfully real examples of this. This problem we like to call image completion or image in painting. But mathematically this may sound like crazy talk. Who really knows what information should be there in these holes? Let alone a computer. The first question is why would we have to synthesize all these missing details from scratch? Why not start looking around in an enormous database of photographs and look for something similar? For instance, let's unleash a learning algorithm on 1 million images. And if we do so, we could find that there may be photographs in the database that are from the same place. But then what about the illumination? The lighting may be different. Well, this is an enormous database. So then we pick a photo that was taken at a similar time of the day and use that information. And as we can see in the results, the technique works like magic. Awesome. It doesn't require user-made annotations or any sort of manual labor. These results were way, way ahead of the competition. And sometimes the algorithm proposes a set of solutions that we can choose from. The main challenge of this solution is finding similar images within the database. And fortunately, even a trivial technique that we call nearest neighbor search can rapidly eliminate 99. 99% of the dissimilar images. The paper also discusses some of the failure cases which arise mostly from the lack of highlevel semantic information. For instance, when we have to finish people, which is clearly not what this technique is meant to do, unless it's a statue of a famous person with many photographs taken in the database. Good that we are in 2017, and we know that plenty of research groups are already working on this, and I wouldn't be surprised to see a generative adversarial network-based technique to pop up for this in the very near future. Thanks for watching and for your generous support, and I see you next time. Heat.

---
*Источник: https://ekstraktznaniy.ru/video/14706*