# This AI Helps Making A Music Video! 💃

## Метаданные

- **Канал:** Two Minute Papers
- **YouTube:** https://www.youtube.com/watch?v=UrB-tqA8oeg
- **Дата:** 02.09.2021
- **Длительность:** 4:37
- **Просмотры:** 134,849
- **Источник:** https://ekstraktznaniy.ru/video/13829

## Описание

❤️ Train a neural network and track your experiments with Weights & Biases here: http://wandb.me/paperintro

📝 The paper Editable Free-Viewpoint Video using a Layered Neural Representation"" is available here:
https://jiakai-zhang.github.io/st-nerf/
https://github.com/DarlingHang/st-nerf

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Aleksandr Mashrabov, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, Christian Ahlin, Eric Haddad, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Bustamante, John Le, Jonas, Kenneth Davis, Klaus Busse, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Mark Oates, Michael Albrecht, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Ramsey Elbasheer, Steef, Taras Bobrovytsky, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: https://www.patreon.com/TwoMinutePapers

Thumbna

## Транскрипт

### <Untitled Chapter 1> []

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today we are going to make some crazy synthetic music videos.

### Latin [0:08]

In machine learning research, view synthesis papers are on the rise these days. These techniques are also referred to as NERF variants, which is a learning-based algorithm tries that to reproduce real-world scenes from only a few views. It is very challenging, look! In go a bunch of photos of a scene, and the method has to be able to synthesize new photorealistic images between these photos. But this is not the only paper in this area, researchers are very aware of the potential here, and thus, a great number of NERF variants are appearing every month. For instance, here is a recent one that extends the original technique to handle shiny and reflective objects better. So, what else is there to do here? Well, look here.

### Multi-view Input [1:02]

This new one demands not a bunch of photos from just one camera, but from 16 different

### Camera Set [1:07]

cameras. That’s a big ask. But, in return, the method now has tons of information about the geometry and the movement of these test subjects, so, is it intelligent enough to make something useful out of it? Now, believe it or not, this, in return, can not only help us look around in the scene, but even edit it in three new ways. For instance, one, we can change the scale of these subjects, add and remove them from

### Spatial Editing : Scaling [1:41]

the scene, and even copy-paste them. Excellent for creating music videos. Well, talking about music videos. Do you know what is even more excellent for those?

### Temporal Editing: Retiming [1:58]

Retiming movements…that is also possible. This can, for instance, improve an okay dancing performance into an excellent one. And three, because now we are in charge of the final footage, if the original footage

### Viewpoint Editing [2:14]

is shaky, well, we can choose to eliminate that camera shake. Game changer. Still, it’s not quite the hardware requirement where you just whip out your smartphone and start nerfing and editing, but for what it can do, it really does not ask for a lot.

### Additional Capabilities [2:32]

Look, if we wish to, we can even remove some of those cameras and still expect reasonable results. We lose roughly a decibel of signal per camera. Here is what that looks like. Not too shabby! And all this progress just one more paper down the line. And I like the idea behind this paper a great deal because typically what we are looking for in a followup paper is trying to achieve similar results while asking for less data

### Spatial Editing Performer 2 [3:10]

from the user. This paper goes into the exact other direction, and asks what amazing things could be done if we had more data instead. Loving it. And with that, not only neural view synthesis, but neural scene editing is also possible.

### Spatial Editing: Duplication Performer? [3:25]

What a time to be alive! Thanks for watching and for your generous support, and I'll see you next time!