Google’s New AI: This is Where Selfies Go Hyper! 🤳

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today, we are not going to create selfies, we are going to create NERFies instead. What are those? Well, NERFies are selfies from the future. The goal here is to take a selfie video, and turn it into a portrait that we can rotate around in complete freedom. This technique appeared in November 2020, a little more than a year ago, and as you

see, it easily outperformed its predecessors.

It shows a lot of strength in these examples and seems nearly invincible, however, there

is a problem. What is the problem? Well, it still did not do all that well on moving things. Don’t believe it? Let’s try it out! This is the input video. Aha! There we go, as soon as there is a little movement in time, we see that it looks like a researcher who is about have a powerful paper deadline experience. I wonder what the NERFIE technique will do with this. Uh-oh. That’s not optimal. And now, let’s see if this new method called HyperNERF is able to salvage the situation. Oh my! One moment perfectly frozen in time, and we can move also around with the camera. Kind of like the bullet time effect from the Matrix. Sensational. What is also sensational is that of course, you Seasoned Fellow Scholars immediately ask - okay, but which moment will get frozen in time? The one with the mouth closed, or open? And, HyperNERF says, well, you tell me. Yes, we can even choose which moment we should freeze in time by exploring this thing that they call the hyperspace. Hence the name, HyperNERF. The process looks like this. So, if this can handle animations, well then, let’s give it some real tough animations. This is one of my favorite examples, where we can even make a video of coffee being made. Yes, that is indeed the true paper deadline experience. And the key to creating these NERFies correctly is getting this depth map right. Here, the colors describe how far things are from the camera. That is challenging in and of itself, and now imagine that the camera is moving all And not only that, but the subject of the scene is also moving all the time too. This is very challenging, and this technique does it just right. The previous NERFIE technique from just a year ago had a great deal of trouble with this chocolate melting scene. Look. Tons of artifacts and deformations as we move the camera. Now, hold on to your papers and let’s see if the new method can do even this, now that would be something…and! Wow. Outstanding. I also loved how the authors went the extra mile with the presentation of their website…look! All the authors are there, animated with this very method. Putting their money where their papers are. Loving it. So, with that, there you go. These are NERFIES, selfies from the future. And finally, they really work - such amazing improvements in approximately a year. The pace of progress in AI research is nothing short of amazing. What a time to be alive! Thanks for watching and for your generous support, and I'll see you next time!