Artistic Style Transfer For Videos | Two Minute Papers #68

3:55

Artistic Style Transfer For Videos | Two Minute Papers #68

Two Minute Papers 22.05.2016 56 658 просмотров 1 201 лайков

Machine-readable: Markdown · JSON API · Site index

Смотреть на YouTube

Поделиться Telegram VK Бот

Транскрипт Скачать .md

Анализ с AI

Описание видео

Artificial neural networks were inspired by the human brain and simulate how neurons behave when they are shown a sensory input (e.g., images, sounds, etc). They are known to be excellent tools for image recognition, any many other problems beyond that - they also excel at weather predictions, breast cancer cell mitosis detection, brain image segmentation and toxicity prediction among many others. Deep learning means that we use an artificial neural network with multiple layers, making it even more powerful for more difficult tasks. This time they have been shown to be apt at reproducing the artistic style of many famous painters, such as Vincent Van Gogh and Pablo Picasso among many others. All the user needs to do is provide an input photograph and a target image from which the artistic style will be learned. And now, onto the next frontier: transferring artistic style to videos! _________ The paper "Artistic style transfer for videos" is available here: http://arxiv.org/abs/1604.08610 The implementation of this technique is also available: https://github.com/manuelruder/artistic-videos Recommended for you: Deep Neural Network Learns Van Gogh's Art - https://www.youtube.com/watch?v=-R9bJGNHltQ Deep Learning Program Learns to Paint - https://www.youtube.com/watch?v=UGAzi1QBVEg From Doodles To Paintings With Deep Learning - https://www.youtube.com/watch?v=jMZqxfTls-0 Sintel Movie copyright: Blender Foundation https://durian.blender.org/sharing/ WE WOULD LIKE TO THANK OUR GENEROUS SUPPORTERS WHO MAKE TWO MINUTE PAPERS POSSIBLE: Sunil Kim. https://www.patreon.com/TwoMinutePapers Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz The thumbnail background image was taken from the corresponding paper. Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu Károly Zsolnai-Fehér's links: Facebook → https://www.facebook.com/TwoMinutePapers/ Twitter → https://twitter.com/karoly_zsolnai Web → https://cg.tuwien.ac.at/~zsolnai/

Оглавление (13 сегментов)

<Untitled Chapter 1>

dear fellow Scholars this is 2minute papers with Caro we have previously talked about a technique that used a deep neural network to transfer the artistic style of a painting to an arbitrary image for instance to a photograph as always if you're not familiar with some of these terms we have discussed them in previous episodes and links are available in the description box make sure to check them out style transfer is possible on Ste images as there is currently no technique to apply this to videos it is hopeful abundantly clear that a lot of potential still lies dormant inside but can we apply this artistic style transfer to videos would it work if we would simply try for an experienced researcher it is flagrantly obvious that it's an understatement to say that it wouldn't work it would fail in a spectacular manner as you can see here but with this technique it apparently works quite well to be frank the results look gorgeous so how does it work now

Freiburger Münsterplatz

don't be afraid you'll be presented with a concise but deliberately obscure statement this technique preserves temporal coherence when applying the artistic style by incorporating the optical flow of the input video now the only question is what temporal coherence and Optical flow means temporal coherence is a term that was used by physicists to describe for instance how the behavior of a wave of light changes or stays the same if we observe it at different times in computer Graphics it is also an important term because often

Star Wars

times we have techniques that we can apply to one image but not necessarily to a video because the behavior of the technique changes drastically from frame to frame introducing a disturbing flickering effect that you can see in this video here we have the same if we do the artistic style transfer because

The Jungle Book (1967)

there is no communication between the individual images of the video the technique has no idea that most of the time we're looking at the same things and if so the artistic style would have

Ice Age (2002)

to be applied the same way over and over to these regions we are clearly lacking temporal coherence now onto Optical

Cloud Atlas (2012)

flows imagine a flying drone that takes a series of photographs while hovering and looking around above us to write sophisticated navigation algorithms the Drone would have to know which object is which across many of these photographs

a Independent per-frame processing

if we have slightly turned most of what we see is the same and only a small part of this new image is new information but the computer doesn't know that as all it sees is a bunch of pixels Optical flow algorithms help us achieving this by describing the possible motions that give us photograph B from photograph a in this application what this means is that there is some interframe communication the algorithm will know

a With temporal constraint

that if I colored this person this way a moment ago I cannot drastically change the style of that region on a whim it is now easy to see why naively applying such techniques to many individual

No constraint, init prev warped

frames would be a flippant attempt to create beautiful smooth looking videos so now it hopefully makes a bit more sense this technique preserves temporal

Stylized video

coherence when applying the artistic style by incorporating the optical flow

Original video and style image

of the input video such great progress in so little time loving it last time I've mentioned crem from the comment section and this time I i' like to commend related jff for his insightful comments thanks for being around and I've definitely learned from you fellow Scholars I am really loving the

a Short-term flow only

respectful and quality discussions that take place in the comment section and it

b With long-term flow

is really cool that we can both learn from each other thanks for watching and for your generous support and I'll see you next time

Другие видео автора — Two Minute Papers

Ctrl+V

Экстракт Знаний в Telegram

Экстракты и дистилляты из лучших YouTube-каналов — сразу после публикации.

Подписаться

Лучшие методички за неделю — каждый понедельник