Creating Virtual Worlds 20x Faster!

6:07

Creating Virtual Worlds 20x Faster!

Two Minute Papers 05.11.2023 97 435 просмотров 4 794 лайков

Machine-readable: Markdown · JSON API · Site index

Смотреть на YouTube

Поделиться Telegram VK Бот

Транскрипт Скачать .md

Анализ с AI

Описание видео

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.me/papers 📝 The paper "3D Gaussian Splatting for Real-Time Radiance Field Rendering" is available here: https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/ Luma Labs: https://twitter.com/nickfloats/status/1709323018852319522 In Unreal Engine: https://www.unrealengine.com/marketplace/en-US/product/3d-gaussians-plugin Text to 3D: https://gsgen3d.github.io/ + paper https://arxiv.org/abs/2309.16585 Dream Gaussian (image to 3D): https://dreamgaussian.github.io/ 25 photos: https://twitter.com/AlexandreDevaux/status/1707365670596284857 Animation: https://twitter.com/simesgreen/status/1710691657740341428 My latest paper on simulations that look almost like reality is available for free here: https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations: https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Benji Rabhan, Bret Brizzee, Bryan Learn, B Shang, Christian Ahlin, Gaston Ingaramo, Geronimo Moralez, Gordon Child, Jace O'Brien, Jack Lukic, John Le, Kenneth Davis, Klaus Busse, Kyle Davis, Lukas Biewald, Martin, Matthew Valle, Michael Albrecht, Michael Tedder, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Richard Sundvall, Steef, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi. If you wish to appear here or pick up other perks, click here: https://www.patreon.com/TwoMinutePapers Thumbnail background design: Felícia Zsolnai-Fehér - http://felicia.hu Károly Zsolnai-Fehér's research works: https://cg.tuwien.ac.at/~zsolnai/

Оглавление (4 сегментов)

Introduction

Because so many of you Fellow Scholars have requested it, we had a video on one of the best papers in the year in my opinion, which is a work called Gaussian Splatting. And I am absolutely speechless by how quickly things are unfolding around it. I think you will be too. So, what does this do? Well, just imagine grabbing your smartphone, taking a few images or videos of real places, and then, we could create a virtual version of them and play in them in real time. And it does exactly that. And these scenes come out really detailed. Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. And the key thing here is that this is a handcrafted technique that is easy to implement. And now, some publicly available applications already have it. This one does. Yes, the dream is coming true, we might just play in a virtual copy of the real world very soon. Especially that it is available in Unreal Engine too through a paid plugin, though I am sure that free versions will pop up very soon too. In fact, things are going so quickly, I wouldn’t be surprised if by the time this video appears, it is already available for free. And it gets even better. Way better! For instance, we know that text to image is possible, but we are

Text to 3D

talking about 3D virtual worlds here. So, oh my. It has been just a few months of this paper being out, can that really be? Follow up papers are already popping up. Wait a second. This is text to 3D. We write something, and out comes 3D geometry. Our last episode was on exactly that, and it is already being surpassed. What is happening? Wow. Well, what is happening is that Gaussian Splatting is already being used to improve these techniques. Hmm, in many cases, I see more detailed geometry being produced. And all this in just a couple of months and one more paper down the line. So cool!

Image to 3D

But it doesn’t end there. Not even close. Goodness! Here is another paper which is able to perform image to 3D. One photo goes in, and out comes 3D geometry. I already see that it converges to a solution much quicker than earlier, neural radiance fields-based solutions, that is fantastic news, but come on, what can it do? Wow, it really does it. In goes the image, and out comes the 3D model. And it gets better. Then, we can even really put these little models into our virtual worlds, and animate them. And all this process takes…how long exactly? The previous paper we looked at took 40 minutes, so what about this one? Now hold on to your papers Fellow Scholars, because it is two minutes. This is incredible! A 20x speedup in just one paper. I really don’t know what to say here. In the first step, it lines up the Gaussians, little mathematical lumps to match the shape of the character or object, and then, in the second step, it slowly refines the textures as well. And this one runs on a single consumer graphics card, this is what does it in two minutes per object. Haha! This work is a real match for Two Minute Papers. No wonder these papers caught the imagination of you Fellow Scholars. It gets even better. It can even reproduce photorealistic faces reasonably well with just 25 photos as an input, and…whoa! Here, you can even animate these inputs. So not only creating virtual worlds from the real world, but we can even let our imagination run wild and create absolutely anything out of it. I love it.

gaussian splatting

Now, just one more thing. I know some of you Fellow Scholars are asking, okay but what is Gaussian Splatting? What does that even mean? Now the Gaussian part means that objects in this scene will be represented as a sum of many little waves. And splatting is a way of throwing these little waves from the 3D virtual world at our 2D screen and computing what they should really look like. And this simple, but effective idea taking the world by storm. A handcrafted technique. Powered not by AI, but by human ingenuity. Once again, the papers coming alive in the real world. What a time to be alive!

Другие видео автора — Two Minute Papers

Ctrl+V

Экстракт Знаний в Telegram

Экстракты и дистилляты из лучших YouTube-каналов — сразу после публикации.

Подписаться

Лучшие методички за неделю — каждый понедельник