This New AI Is The Future of Video Editing!

4:47

This New AI Is The Future of Video Editing!

Two Minute Papers 27.01.2023 101 851 просмотров 4 526 лайков

Machine-readable: Markdown · JSON API · Site index

Смотреть на YouTube

Поделиться Telegram VK Бот

Транскрипт Скачать .md

Анализ с AI

Описание видео

❤️ Check out Weights & Biases and say hi in their community forum here: https://wandb.me/paperforum 📝 The paper "Text2LIVE: Text-Driven Layered Image and Video Editing" is available here: https://text2live.github.io/ My latest paper on simulations that look almost like reality is available for free here: https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations: https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Edward Unthank, Eric Martel, Geronimo Moralez, Gordon Child, Jace O'Brien, Jack Lukic, John Le, Jonas, Jonathan, Kenneth Davis, Klaus Busse, Kyle Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Matthew Valle, Michael Albrecht, Michael Tedder, Nevin Spoljaric, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Richard Sundvall, Steef, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi. If you wish to appear here or pick up other perks, click here: https://www.patreon.com/TwoMinutePapers Thumbnail background design: Felícia Zsolnai-Fehér - http://felicia.hu Károly Zsolnai-Fehér's links: Twitter: https://twitter.com/twominutepapers Web: https://cg.tuwien.ac.at/~zsolnai/ #nvidia

Оглавление (6 сегментов)

Intro

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today we are going to have a look at this magical AI from scientists at the Weizmann Institute of Science and NVIDIA which promises magical video editing powers. So let’s see 4 of my favorite things it can do together. One, we can take this unremarkable video of a car passing by, and my goodness, look at that. It has completely revamped the scene: first, it made sure that it is nighttime, and has given the car a nice neon cyberpunk look. So, this took quite a bit of work for someone, right? Well, not quite! To achieve this, all we need to do is give the AI a video and two text prompts. One for the foreground, and one for the background. And if we are looking for an example that is a little less out there, we can also ask that the car is made rusty instead. So cool!

Look No More

Two, once again, if we did not show up in the right outfit for a day, or would like to see what we would look like wearing something else before buying it, look no more. Yes, this is also just one text prompt away. I particularly like this example because this outfit has lots of wrinkles and geometry changes, and the new technique can follow them really well. Now, if you look close, you see that there is also a price to be paid for this. Look, the new synthesized results are not as detailed as the original video.

New Results

Now, we said that it can do foreground and background editing separately. And that was the foreground. But what about the background? Oh yes, let’s make this environment a little foggier. Look at that! I love this one. Even the shadows have become foggier. So cool! And, the shadows are even taken away for the nighttime shot because of the different lighting setup. I would like these results a great deal for images, but for video? Such an amazing result for video. Wow. Now, of course, this kind of layered editing concept can be applied to images too, and the results are outstanding. For instance, three if we have this boring cake, we can reimagine it if it were made of oreo instead, or if we wish to let our imagination run wild, or we are light transport researchers, we can ask the AI to make it out of ice.

Bread

Loving it. Now, if you aren’t hungry enough yet, here is a piece of bread that can be converted to the legendary red velvet bread too, or, once again, ice.

Fire Breathing

Four, the authors call these semi-transparent effects where we can choose a small part of an already existing image, add a text prompt, for instance, fire, and then, bam! We suddenly have a fire-breathing bear, or a beautiful cup of coffee. This technique can truly do so much. Now I would like to send huge thank you to the authors of the paper because the source code for this project is also available. So, AI-driven layered image editing is possible for free, and for everyone! And just imagine what this technique will be able to do just a couple more papers down the line.

Conclusion

What a time to be alive! So, what do you think? What would you use this for? Let me know in the comments below! Thanks for watching and for your generous support, and I'll see you next time!

Другие видео автора — Two Minute Papers

Ctrl+V

Экстракт Знаний в Telegram

Экстракты и дистилляты из лучших YouTube-каналов — сразу после публикации.

Подписаться

Лучшие методички за неделю — каждый понедельник