NVIDIA's AI Makes Amazing Slow-Mo Videos! 🚘

dear fellow scholars this is two minute papers with károly on IFA here how about some slow-motion videos if we would like to create a slow-motion video and we don't own an expensive slow-mo camera we can try to shoot a normal video and simply slow it down this sounds good on paper however the more we slow it down the more space we have between our individual frames and at some point our video will feel more like a slideshow to get around this problem in a previous video we discussed two basic techniques to fill in these missing frames one was a knife technique called frame blending that basically computes the average of two images in most cases this doesn't help all that much because it doesn't have an understanding of the motion that takes place in the video the other one was optical flow now this one is much smarter as it tries to estimate the kind of translation and rotational motions that take place in the video and they typically do much better however the disadvantage of this is that it usually takes forever to compute and it often introduces visual artifacts so now we are going to have a look at Nvidia's results and the main points of interest are always around the silhouettes of moving objects especially around regions where the foreground and the background meet keep an eye out for these regions throughout this video for instance here is one example I found let me know in the comment section if you have found more this technique builds on you net a super fast convolutional neural network architecture that was originally used to segment biomechanical images from limited training data this neural network was trained on a bit over a thousand videos and computes multiple approximate optical flows and combines them in a way that tries to minimize artifacts as you see in the side-by-side comparisons it works amazingly well some artifacts still remain but are often hard to catch and this architecture is blazing fast not real-time yet but creating a few tens of these additional frames takes only a few seconds the quality of the results is also evaluated and compared to other works in the paper make sure to have a look as the current commercially available tools are super slow and take forever I cannot wait to be able to use this technique to make some more amazing slow-motion footage for you fellow scholars thanks for watching and for your generous support now see you next time