OpenAI Sora: The Age Of AI Is Here!

buckle up fellow Scholars because what you are going to see today is something that might be the craziest thing I've been able to show you in more than 800 videos this is the kind of video an AI could create yesterday and today it can do this holy mother of papers yes open AI just released their own textto video AI Sora and it is so far beyond any anything else I have ever seen it is hard to put into words dear fellow scholar this is 2minute papers with Dr Caro when I first saw these results I thought this was some April Fool's joke no this is not a video coming from a real camera this is a video that was synthesized pixel by a new AI so let's give this one a spin these AI videos we will evaluate by three criteria one quality this is shocking the quality of these works is out of this world if we are not actively seeking for errors in the footage in many cases we may not even know that they are made by an AI and it gets better their dolly3 system that is an expert at creating images and I just stop these videos here and there and the still images are often as good or even better than what Dolly 3 can make this is beating the king in its own game unbelievable two temporal coherence this means that the AI understands exactly how each image in the video should follow each other this is what it looks like if you don't have temporal coherence a paper from just a few years ago and now we have this once again temporal coherence Second To None wow and three wait this may still not be a great technique now I hear you asking caroy why is that well it has to follow our prompt correctly it has to be true to what we asked for you see there are techniques out there that give you really good quality coherent videos however they don't care too much about the prompts that we write and what about this technique goodness that is exactly what the prompt is asking for I am out of words but it gets better it even has a hint of imagination for instance we can ask for a corgi that's also a vlogger an otter on a surfboard an Italian pop you name it just ask and it will do it imagination in a machine what a time to be alive hm wait I just noticed that we need to take a look at a fourth thing from now on and that is object permanence and consistency with previous techniques when something got occluded and is now visible again the AI might not remember and it might look completely different but here let's see wow this has a consistent World model so much so that even when we move around in 3D space everything remains where it should be and this can do so much more we can even transform an existing video into a completely new one by just writing one text prompt and now hold on to your papers fellow Scholars because it can also synthesize Virtual Worlds whether that will be something that already exists like Minecraft or a completely new game made from scratch up to you just one more paper down the line and it might be that you don't even need to develop your own games you can maybe just hook up a controller write a text prompt and open AI Sora will give that game to you immediately so how does all this magic work well one of the key ideas is that the synthesis takes place in a latent space what is that it looks something like this is one of my papers where you can walk around in this 2D latent space and each point in this space represents a material for a virtual world and here comes the key the latent space works well if you can guarantee that when exploring a nearby point you get similar material models the link to the paper is available in the video description and this concept also works for creating new fonts and now to create new videos too and

they come in full HD resolution wow so is this concept any good so far well let's have a look wait a second that is not even close to what we've seen what happened well one word compute happened you see if you don't have enough computational power this is what you get if you have four times more you get this m and if you have 16 times more you get this oh yes so the concept Comes Alive only with a sufficient amount of compute the virtual brain if you will has to be developed enough to imagine all of these videos in high quality and my goodness this is perhaps the biggest jump in quality between two research works that I have ever seen and this video Series has been around for more than 800 episodes now and now it's time for what you ask of course it is time to invoke the first law of papers says that research is a process do not look at where we are will be two more papers down the line and here is the one more paper down the line now exercise leave a comment about what you think we will be able to do just two more papers down the line I'd love to know what you fellow Scholars think especially now because once again we share one of those Sweet Moments where we witness history in the making in his excellent video which I highly recommend MKBHD says that since this is trained on videos made by humans it likely cannot go beyond what it had seen from humans I would like to note that in some cases we see AI papers that have proper zero shot performance what is that this means that leaning on all this knowledge like a human it can try to create new things it hasn't seen before for instance you could ask for a new kind of vehicle for T-Rexes and it could infer that T-Rexes have these tiny little hands so it would have to have a weel that is suitable for their little hands we will be able to test that and so much more as soon as it is out there and we will soon be back with a video on a different AI video system that is not as good as this but it is more controllable and it is something that you will be able to try out for free right away we will also have a more in-depth video about the capabilities of this new technique soon too so make sure to subscribe and hit the Bell icon to not miss out experiment tracking model evaluation and production monitoring for your deep learning projects and llm apps this is what weights and biases does and it is the best everyone is using it try it out now at wb. me/ papers or click the link in the description below