Runways Text To Video "GEN 3 ALPHA"" Actually STUNNED The Entire Industry!

so Runway actually just introduced their gen 3 Alpha it's a New Frontier High Fidelity controllable video generation model it is truly impressive and trust me when I say there are a few details that we're going to go over that will show you why this is so impressive and of course what sets them apart from the other video models so you can see here that they state that gen 3 Alpha is the first of an upcoming series of models trained by runway on a new infrastructure built for large scale multimodal training it is a major Improvement in Fidelity consistency and motion over generation 2 and a step to building General World models we'll talk about General World models later but I truly do find this intriguing because some of the key examples that Runway are going to show you they show us a lot of different things that are really impressive in terms of what the model is able to do and a few things that I literally haven't seen any other model do so let's get into things so one example that we can see here is of an astronaut running through an alley in Rio de Janeiro I think this one's pretty good and as someone who like pays attention to how videos are produced what I always want to do is focus on the subtleties that are apparent like for example you know how of course we can see that you know the hand is morphing a bit right there I'm not sure if you guys can see that but if you do focus on the hand you can actually see a few mistakes in terms of the morphing that's not the kind of things I want to focus on what I do like to look at is things like the reflection and of course the background in terms of how it is moving with the times cuz not with the times but like with the actual motion so those are the things that I do want to focus on okay um and what we can see here flying through an underground neighborhood this is something that looks really nice like this one right here is of course generative in the sense that like you know you don't really get these kinds of footages within the training data and combining them is going to be really difficult now this is arguably one of the most impressive ones and I think this one is really impressive for a few reasons so like I said I always like to focus on the subtleties of the model but one of the things that we can see here about this clip that makes it just so impressive is the fact that it uses Dynamic lighting so you can see here that at the start there's not that much Lighting in the background and so the character's face like I truly do like if I saw this I wouldn't believe this was AI if no one can tell me but you can see that it is dark and we can see that the lighting from here it it's not you know on the character's face you know like these areas right here we don't see that kind of lighting that you would expect there to be if there was a light but as soon as we get like you know the darkness right here you can see that things go completely dark we can see the shadows and everything looks pretty accurate which is really nice and then of course as things start to brighten up we can see that the lighting changes to accurately represent what we're seeing so you can see that the yellow you know Hues which kind of looks like a sunset kind of thing we can see that it's you know going onto all these kinds of features of the person's face and it's just really accurately mapping how the light Reflections are of course we can see as well at the back something that I find really impressive is of course how the light you know breaks through the hair right here we can see that the light looks really effective and then as well on the skin we can see that the lighting is also pretty impressive and around the edges of the skin we can see the impressive lighting features I'm not sure on the exact technical terms of them you know I could get some of them wrong but what I'm saying here is that this Dynamic lighting where we're seeing the face in real time be able to I guess you could say adapt to the lighting shown I just think that this is truly impressive as all of the different backgrounds come into play so this is definitely one of the most impressive scenes this is something that not only it has the background moving you know really nicely the character in the middle they stay extremely consistent and the lighting behind the scene it really does update it now this does kind of remind me of that one Sora clip but I would argue that this is just a little bit more impressive because we're seeing at how much the lighting is accurately done here so this is something that is really impressive and honestly a very good example of what makes this video model incredible now they initially speak about how trained jointly on videos and images gen 3 Alpha will power runways text to video image to video and text to image tools and existing control modes such as motion brush and Advanced Camera controls and of course director mode as well as upcoming tools for more fine grain tool for more fine grained control over structure style and motion and this is one of the things I like about Runway is because they actually allow you to I guess you could say focus on what is relatively important Runway if you don't know they have most brush and they have

Advanced Camera controls as well as director modes and these modes actually allow you to do a lot more things than a traditional text to image generator that's probably why they haven't released this just yet is because the motion brush feature they probably haven't you know updated it and integrated it into the Gen 3 Alpha of course with the Advanced Camera controls that is going to be something that they'll need to work on as well but I do personally think that Runway is probably going to become the One-Stop shop for everything text to video because it's very impressive on certain things that it's been able to do the existing control tools that people will actually want I do believe that you know this is going to be something that allows more controllability if you're someone that uses generative AI the problem with a lot of these clips is that there's not a lot of control with as to what happens in the clip you enter a text prompt you can describe it as best as you want but the problem is that you know having real control over where the actors may go over where the light you know it goes like for example if you wanted to just change out the yellow light for a blue light those kinds of things are really hard to do in video models now I know that Sora did have a demo where they showcase that you know you could switch a car and do stuff like that but I do think that Runway is positioned better because their thing has been focusing on that from the beginning so they're likely building these models with that in mind so you can also see here some of the more impressive ones there is also one that I really want to show you that is more impressive you can see here that this is a train going around a track and the temporal consistency is relatively impressive this one is one of my favorite examples this is a closeup of a living flame and it looks really effective like we can see all of the different lights going on here and we can see the background the characters do look pretty consistent in terms of their motion and I mean something like this is relatively hard for things to do and what's very impressive about this is that this was only released just after you know and this is just the demo this was only released just after Luma laps video and it really does show us that we are you know advancing across the board here because this is far more impressive than lum's AI video of course right now Luma does give us access but I think runway's you know presentation here is truly impressive this is also something that I thought was pretty incredible you can see that we start like in this abandoned area and then this thing pops up for it to be covered in Vines and then literally it's like a whole new portal into another dimension which is pretty cool so overall I think that in terms of the actual video model that Runway has here it's truly remarkable and truly impressive and one of the things that I think we can see from Runway is that they definitely wanted to work on this model a lot because we know that they've been working on video models for quite some time and although you know other companies may have beaten them to the punch initi I do think that Runway will offer a more comprehensive system in terms of what you'll be able to do for this AI area of text a video now this is where we get into the fine grained temporal control and this is why I say I really like how Runway has managed to do their system so right here they say that gen 3 Alpha has been trained with highly descriptive temporarily dense captions enabling imaginative Transitions and precise key framing of elements in the screen which is really nice which is what I talked about so they says an extreme closeup of an ant emerging from a nest and the camera pulls back revealing a neighborhood beyond the anthill now this is one of my favorite Clips because it shows the ability to go from something extraordinarily close up and then reveal the background in the entire scene which shows us just how incredible it is I mean it was truly incredible to see this scene back here just going entirely from this initial scene right here so so this kind of consistency that we do see here is something that is really hard to do because you're trying to maintain like the entire scene without actually having you know access to it at first you can see that right here of course you just have you know this and then of course you have to smoothly transition you know and what's even crazier is that you can see this even comes out of focus and then slowly goes into focus and we can see this entire Horizon up here and it still maintains its entire consistency which is truly impressive on a scale that I didn't think was even possible now of course here I think one of the very impressive things that we can have and that you can see is of course the water simulations now simulating water if you've ever done CGI you'll know that it takes so long to get water simulations to do like if you're trying to do them on your personal computer if you don't do it you know if you don't hand it off to like a render Farm it's going to take a really long time and I think for the most part these water simulations do

look pretty impressive considering how hard it is to get something like this to be right and the prompt for this was a tsunami coming through an alley in Barcelona and I think that it does look pretty cool considering the fact that this is really hard to do so things like this are definitely going to present a lot of you know use in the future because like I said one of the Practical applications of this is that like I said water simulations take a lot of time like a very long period of time and in the future when we can get to a period where you can have an image or you can have like for example an environment and then you can just simulate the water using generative AI I think it might be industry practice to do that because these water simulations and a lot of these physics simulations do take an extended period of time so that's going to be something fascinating that I do want to see in the future with as to how it works and there were some other video demos that showed the water simulations and how well they looked too and this one was the one that they actually showcased on their Twitter account it showcased you know opening a door into a waterfall in this kind of room and it is really comprehensive like when I saw this I was like there is no way that this is absolutely text a video but you can see here that one of the key things that Runway have focused on is they focused on making sure that when they were doing this model they ensured that the transitions were actually really accurate and effective and we can see that because we start with a scene where the doors are basically entirely shut and then as they open you can see that the water it ripples out the lighting is reflecting off this wooden panel here which looks really nice and then you can see that the water is you know it's flowing it looks really nice and we can see that all of these water physics at the bottom here like around this area it looks really reflective um you know we can see the reflections on the plants that look really nice as well we can even see in the background I'm not sure you know if it's like some sort of weird house or whatever but we can see that the lights from there um and it's just really effective like in terms of how everything is looking so I think this example shows you where runway's Focus was in terms of the overall model because they're trying to focus on making sure that their transitions were also very effective and whatever you know training mechanism they used for the water as well definitely looks really effective because as the door opens as well you can see that the water kind of you know underneath the door it's kind of hard to see you can see that the water kind of gets pushed to the side so it's definitely something that I think you know they deserve props for there was also this one of an fpv drone flying through a cliff and we can see that the ability to enter these environments with markable accuracy is really cool and one of the things that most people actually might not notice about this is that you know from here everything looks pretty normal but as you can see as we start to enter this area you can see like the little fish ey lens coming into effect it is pretty hard to see but that is something that is notable on these drones and I find that to be really cool you know that kinds of gives us that realism of it actually being a drone and then of course we do have the Flames flickering inside and it still like maintains its consistency which looks so effective because what it has to be able to do as well is the lighting in a very effective manner so that is something that you know being done here it shows us just how effective this model is now one of the areas where they actually speak about and this is why I was stating before that I wouldn't have known if it was AI if they didn't say is that you know they Excel generating expressive human characters with a wide range of actions gestures and emotions unlocking new storytelling opportunities so basically one of the things that they wanted to focus on was photorealistic humans and this is something that is really hard to do like if you've seen the game Industries and the trailers of when they're actually focusing on these kinds of things and trying to make you know humans as photorealistic as possible you'll know that this is not easy to do at all even with game engines and we can see right here that as this human manages to move his eyes around and even the blinking it looks extraordinarily realistic like this does look a lot higher quality than a lot of these other clips that we've seen like if you compare this clip right now which looks extraordinarily realistic like if someone showed me this I would say maybe it is a weird clip but if we go back to the top and if we compare it to you know this clip for example we can see that there's you know on the background it sometimes morphs a bit you could see right there like I don't know if you saw that like just

watch out for this um you can see that there's a bit of morphing here but this is something you know those are minor things those will get like etched out in the next update but the point is that they managed to actually focus on photorealistic humans which makes this model really really impressive in terms of it being able to stand out in certain areas and I think this is going to be a key area for Runway like for example this one right here is truly impressive like this genuinely looks like a stock you know footage where you do see someone like this one is I don't know if it's better than Sora quality like I would argue it's a little bit better because it looks photorealistic and photo realism is hard to achieve because there are certain details that like you know video generators just don't capture but whatever runways is like if you genuinely compare this clip to any one of these Sor ones they don't look photorealistic like they don't look like as photo realistic as this is and I know you guys might think that this is just you know like me exaggerating but trust me I've analyzed the Sor Eclipse but this one is definitely better in terms of overall humans like I would say this is by far what whatever this is right now you know in terms of generating photorealistic humans this is truly impressive because they've managed to capture the Nuance in the details of the skin of the eyes I think there they did get the Sun a bit wrong like the sun it comes out before a little bit there on the background but you can see that the video model is still pretty effective like for example if we take a look at this piece of skin right here we can see that as the thing you know rotates we can see that the Shadows from the hair are effectively you know planted on the skin it's just really really nice and something that we also do see as well is that we don't see the hair meshing about and looking weird and as for the skin the tattoos look remarkably intact like they actually look very very effective like there are no weirdness about you know how these humans look so the photo realism I would say is a 10 out of 10 for this model like I'm not sure what they could truly improve at this point because it just looks absolutely impressive you can also see this example right here which is another one that looks remarkably impressive the emotions of the human look really impressive although there's not much emotions here it just doesn't look AI generated there is nothing about this clip that tells me that this is AI generated and I've been looking at this stuff all of the weirdness that you get I mean even if you look at the fine hairs on her forehead like right here like you would expect because it's an AI generated video model for there to be you know small changes like into weaving and stuff like that with the other textures but we don't get that at all it's remarkably consistent which is pretty impressive and then of course we get this clip of this old man and you can see as well that everything here looks remarkably impressive Ive and extremely photorealistic so I would say the biggest thing that I've seen from Runway when I was you know trying to make this video and put certain things together and look at things beforehand is that the human aspect is definitely there like you can see as well the wrinkles of the you know face as this character manages to not smile but you know move up his cheeks a little bit and it's stuff like that is really hard to get because you know simulating human skin and all of the and all of that stuff is very hard so this right here is super impressive I'm guessing that their training data maybe they focused on just you know having a lot of examples of photorealistic humans we know that they've been working on this for quite some time like probably longer than opening ey which is why this looks so like actually photorealistic and guys remember there is a difference between photo realism and something that looks high quality like there is a complete difference like you get Photoshop pictures like if you've seen a Photoshop picture and it looks high quality but it doesn't look photorealistic there is a complete difference so photorealism is something that looks actually real like you can't tell whether or not and stuff that likes is high quality is completely different but stuff like this is where you genuinely won't be able to tell if it's AI generated or not because of the photo realism and that's exactly you know kind of the things that we've seen with mid Journey where the photo realism aspect makes it that it looks like it was genuinely taken by real person and that's the point I'm trying to drive home here is that these photorealistic humans are just super impressive like even this example right here you would expect there to be a lot of morphing around the lips on the face but it just isn't it just looks really good like this is a surprising level of detail and accuracy from Runway that I can truly get behind and then this was the most impressive example that I did see before because it was so impressive with a lot of factors so one of the factors that made this so impressive was the fact that you know

you have someone who's doing this emotion you can see the skin is all wrinkled and that is traditionally hard for AI systems to do because differentiating between you know what is the eye what is the mouth what is the cheekbone what are these folds here all of these things just make it harder for an AI system to kind of really figure out what's going on and then what we had here was not only that we had him change his em motion which looks really effective he doesn't look weird Okay and then he was able to get a wig dropped on him okay and remember the wig is uh you know made up of loads of different hair particles or whatever you want to call it and we can see that the wig bounces perfectly on his head and then we can see that as the glasses comeb down there are initial Reflections on those glasses and we can see his eyes behind those glasses so this by far I'm not going to lie to you guys this is you know just insane like this was something that I thought wow that is truly impressive because you're not only taking you know a human you're actually adding something into the environment that is not easy like hair would be something that you would think you know messes up pretty easily but just looking around at the edges and the little light Reflections that you do have that looks just way too impressive now gen 3 Alpha actually has some very cool stuff you can see that here they wanted to focus on you know creative characters so they made this one which is like a giant you know strange creature there's also these ones right here where you know you're going through a neon you know neon-like Forest which is I think you know why this is so important is because when you have a system that doesn't have a lot of training examples the examples that you do get when you generate you know images or videos usually they come out to be pretty bad but what we're seeing at this system is that the data set that they probably used is definitely evenly balanced as these things don't look bad at all which means that they've covered a wide range of diverse topics like this right here a Cy Cyclone of broken glass in an urban environment the these kind of things are just absolutely incredible you know to look at and then of course we've got a man standing in front of a burning building giving a thumbs up like this is pretty incredible stuff we've also got this Ultra wide shot of a hand on a mountain made out of stone like this is really cool I can imagine the kinds of movies and films will be getting soon and then of course this was one of the things that they showcased which was a creature walking through the city and like I said before the photo realism here is incredible I mean looking at this character you know many people on Twitter would always talk about how oh this you know video has you know morphing it has this had that I truly want to know what is wrong with this clip because other than it being a little bit slow and maybe you know not simulating the physics entirely correctly I think that one thing that it does get is the lighting from this lamp we can see that there's also this right here the beams of light coming through if you can see that and then what we do have is the lighting radiating onto this character's face on this side right here so it's truly impressive because it's also able to generate the hairs we can see that the hairs also move stick they don't they don't like morph a lot um the only thing I think that's bad about this is you know the slow motion which is you know not too effective because of course this character wouldn't be like speeding through but it also wouldn't be moving that slow as well like if it was to jump it would move a little bit faster but that's just the least of the problems because this is also you know being done through a window so it's also pretty hard to kind of look at things there but I think this is definitely super impressive so overall I think this model is genuinely super impressive because I think what Runway have showed us is that their research and the foundations and the things that they've actually you know spoken about such as their General World models and basically Runway actually said and I'm going to scroll back to this um I might have some things going up on screen but basically they said that they were mainly focusing on if you go to this is that they were saying that they were focusing on that the major advancement in AI is going to be coming from systems that understand the visual world and its Dynamics which is why we're starting a new long-term research effort which is around what we called General World model so basically they're stating that you know they wanted to have this AI system that builds an internal visual representation of an environment and uses that to simulate future events with that environment and of course they've been pretty controlled but essentially they're going to be able to simulate a lot of you know wide range of interactions and situations and I'm guessing this is why their system is so good because they always wanted to build these kind of world models and I'm guessing that you know with what they've done here it shows us that with the impressive World model that they do have that they've been able to get to extremely high level of fidelity with their next upgrade like this isn't something like you know and that that's the Scot morphing and you know these kind of weird sort of scenarios this looks remarkably impressive for whatever kind of demo that they've managed to show us and of course humans are you know the most impressive demo that we've seen so this is going to be really intriguing to see how this kind

of thing you know plays out when we do get access and of course how much this is going to be because pricing has been very interesting anytime video models have been released so it's definitely interesting to see how they plan to roll this out who they plan to give access to now one of the things I did forget to add I'm not sure where I'm going to put this in the video but like I said they've been building World models and they've also been testing some Physics behaviors this is the CEO talking about how they they've been testing this internally so it seems that you know these future models may be able to and I say accurately represent physics in one way or another because that's what they said they've been testing on and you can see his tweet there and of course or putting a fire out with water which is remarkably impressive for this video model so it's going to be interesting to see I think runaway are going to have some truly fascinating stuff for us in the future but I will be surprised if you know more people aren't talking about this cuz this is truly impress but overall I think that this is truly impressive and a remarkable stunning video