Google’s Video Editor AI: Absolute Magic!

dear fellow Scholars this is two minute papers with Dr Carol jonai fahir today we are going to see that AI video generation is amazing but it is old news yes really so this paper just appeared a few months ago why would this be old news well what you see here is an AI that can perform video generation in goes a piece of text and out comes a video however this is not video editing why in this case we don't really have fine granular control over what happens in this video however get this new paper promises an incredible Leap Forward where we can actually edit these videos how well look it can take a collection of photos of our favorite teddy bear and make a video about it and imagine what it would look like if it were walking so cool and it gets even better we can even provide it a video if we are looking for even more artistic control and start cutting a papaya and then ask the AI to transform the papaya into a cake and it did it just like that wow we can also make this video a little more dramatic by pretending there is a flood going on and of course as a computer graphics person who loves to write simulations like this I have to say when I look at the interaction between the car and the water I absolutely love what I am seeing here it does not seem nearly perfect however I only see flaws when I'm looking for them specifically with a critical eye loving it and wait it gets even better so far we have seen video to video multiple images to video and now it promises something even more amazing what is that it is just one image to video what are you trying to tell me that I provide just this one image here and I get a full video of someone pouring coffee that can't be right but hold on to your papers fellow Scholars just in case and now let's see wow it can do this too I can't believe it and now come the results that made me fall out of the chair when reading this paper I wasn't holding on to my papers enough so I tell you in advance and I hope you will not make my mistake I loved how it made an animation of this black bean plant growing but it turns out we can give it even more granular artistic instructions look here we can not only ask the AI to add a buffalo bathing in the river but we can make it gradually zoom out and reveal it an AI cinematographer how cool is that now of course not even this technique is perfect there are some issues with the horns and more but this is still incredible progress just one more paper down the line I am stunned and it gets even better for instance with this we can now make really fun artistic choices get this we can ask the AI to pretend as if our toy was a real animal and it just does it how is this even possible oh yes now I know what you're waiting for let's pop the hood and look inside uh-huh yes first our video goes in and then down scaling happens this means that the number of pixels and thus the amount of detail in the image is now less don't forget this part this will become important in a moment so one more time after this point we get less detail and then something really surprising happens we make it even worse we add noise why is that why would we make this less detailed video even worse how does that make sense well this added noise helps us by enabling us to use previously existing diffusion based approaches with that we can use this noisy video as a starting point and slowly reorganize this noise to resemble this text prompt a little more what does this mean exactly well in this case a completely new video for us really cool now let's use our Newfound knowledge of the algorithm and let's apply it to the sea turtle this is a real video and now

let's ask the AI to add a shark which it indeed does excellent you see the framing of the video Remains the Same but the shark has been added loving it however are you seeing what I am seeing Oh yes we now see the result of this downscaling step look some detail in the water waves is now gone this effect is also particularly visible in the case of this orangutan also not every result is perfect of course it truly is a miracle of AI research that we can transform this eating monkey into a dancing bear but then something interesting happens whoa the bear eater leans forward and the AI did not think about filling the background or alternatively the bear disappears into a black hole in which case this is perhaps authentic footage who knows and these are the early days of AI video editing what a time to be alive and don't forget this is not just AI video generation this is so much better than that this is true video editing and as always with a work like this as a true fellow scholar please always apply the first law of papers paper says that research is a process do not look at where we are will be two more papers down the line so what do you think what would you use this for let me know in the comments below this video has been supported by weights and biases check out the there isn't offering fully connected a place where they bring machine learning practitioners together to share and discuss their ideas learn from industry leaders and even collaborate on projects together you see I get messages from you fellow Scholars telling me that you have been inspired by the series but don't really know where to start and here it is fully connected is a great way to learn about the fundamentals how to reproduce experiments get your papers accepted to a conference and more make sure to visit them through wnb. me papers or just click the link in the video description our thanks to weights and biases for their long-standing support and for helping us make better videos for you thanks for watching and for your generous support and I'll see you next time