Can Self-Driving Cars Learn Depth Perception? 🚘

dear fellow scholars this is two minute papers with dr. Khurana if I hear when we humans look at an image or a piece of video footage such as this one we all understand that this is just a 2d projection of the world around us so much so that if we have the time and patience we could draw the depth map that describes the distance of each object from the camera this information is highly useful because we can use it to create real-time defocus effects for virtual reality and computer games

or even perform this Ken Burns effect in 3d or in other words zoom and pan around in a photograph but with a beautiful twist because in the meantime we can reveal the depth of the image however when we show the same images to a machine all it sees is a bunch of numbers fortunately with the ascendancy of neural network based learning algorithms we now have a chance to do this reasonably well for instance who discussed this depth perception neural network in an earlier episode which was trained using a large number of input-output pairs where the inputs are a bunch of images and the outputs are their corresponding depth maps for the neural network to learn from the authors implemented this with a random scene generator which creates a bunch of these crazy configurations with a lot of occlusions and computes via simulation the appropriate depth map for them this is what we call supervised learning because we have all these input-output pairs the solutions are given in the training set to guide the training of the neural network this is supervised learning machine learning with crutches we can also use this depth information

to enhance the perception of self-driving cars but this application is not like the previous two I just mentioned it is much harder because in the earlier supervised learning example we have trained a neural network in a simulation and then we also use it later in a computer game which is of course another simulation we control all the variables and the environment here however self-driving cars need to be deployed in the real world these cars also generate a lot of video footage with their sensors which could be fed back to the neural networks as additional training data if we had the deaf maps for them which of course unfortunately we don't and now with this we have arrived to the concept of unsupervised learning is proper machine learning where no crutches are allowed we just unleash the algorithm on a bunch of data with no labels and if we do it well the neural network will learn something useful from it is very convenient because any video we have may be used as training data that would be great but we have a tiny problem and that tiny problem is that this sounds impossible or it may have sounded impossible until this paper appeared this work promises us no less than unsupervised depth learning from videos since this is unsupervised it means that during training all it sees is unlabeled videos from different viewpoints and somehow figures out a way to create these depth maps from it so how is this even possible well it is possible by adding just one ingenious idea the idea is that since we don't have the labels we can teach the algorithm how to be right but instead we can teach it to be consistent that doesn't sound like much does it well it makes all the difference because if we ask the algorithm to be consistent it will find out that the good way to be consistent is to be right while we are looking at some results to make this clearer let me add one more real-world example that demonstrates how cool this idea is imagine that you are a university professor overseeing an exam in mathematics and someone tells you that for one of the problems most of the students give the same answer if this is the case there is a good chance that this was the right answer it is not a hundred percent chance that this is the case but if most of the students have the same answer it is much more unlikely that they have all failed the same way there are many different ways to fail but there is only one way to succeed therefore if there is consistency often there is success and this simple but powerful thought leads to far reaching conclusions

conclusions let's have a look at some more results whoo-hoo now this is something let me explain why I am so excited for this is the input image and this is the perfect depth map that is concealed from our beloved algorithm and is there for us to be able to evaluate its performance these are two previous works both used crutches the first was trained via supervised learning by showing it input-output image pairs with depth maps and it does reasonably well while the other one gets even less supervision the worse crutch if you will and it came up with this now the unsupervised new technique was not given any crutches and came up with this holy mother of papers it looks like a somewhat coarser but still very accurate version of the true depth maps so what do you know this neural network based method looks at unlabeled videos and finds a way to create depth maps by not trying to be right but trying to be consistent this is one of those amazing papers where one simple brilliant idea can change everything and make the impossible possible what a time to be alive

alive what you see here is an instrumentation of this depth learning paper we have talked about this was made by weights and biases I think organizing these experiments really showcases the usability of their system also weights and biases provides tools to track your experiments in your deep learning projects their system is designed to save you a ton of time and money and it is actively used in projects at prestigious labs such as open AI Toyota research github and more and the best part is that if you are an academic or have an open-source project you can use their tools for free it is really as good as it gets make sure to visit them through w and be calm slash papers or just click the link in the video description and you can get a free demo today our thanks to weights and biases for their long-standing support and for helping us make better videos for you thanks for watching and for your generous support and I'll see you next time