New AI Listened To 20,000 Hours Of Music. What Did It Learn?

Yes, this new AI has listened to more than  5,000 songs, and what did it learn? Well,   it learned to perform text to music similarly to   Google’s MusicLM. But this technique is  truly something else. This is MusicGen,   an AI where we enter the kind of music we  wish to hear, and it just gives it to us. Like this.    Wow. Yes, all this was generated  by an AI. This was a pop dance   track with catchy melodies, tropical percussion,   and upbeat rhythms with the new MusicGen.   And now let’s compare it to Google’s MusicLM.    This one is real good too, and it was  published in this year’s January. However,   the first one you’ve heard, MusicGen  has something up the sleeve. It is an   open source technique that  is free for everyone. Wow. Now, smooth jazz, with a saxophone solo, piano  chords, and snare drums. First with MusicGen.    That was real good, with a bit of a  saxophone solo. Perhaps a snippet. Now,   the previous work, MusicLM.    I have to say that I really liked the smooth  jazz part better here, that was real good,   however, no saxophone. It is one thing to do well,   but it is not enough - it also  has to follow the prompt as well. Now, wait a minute. These samples are 30-seconds  long, but real songs are typically much longer   than that. So is that it? No real songs for us?   Well, don’t despair because this paper has two   more absolutely incredible features. One, long  music generation. Oh yes, these are two-minute   long samples, but we are experienced Fellow  Scholars here, so we are looking for long-term   coherence. That means that we will look at  snippets of different parts of the song,   and listen if they really sound like they belong  to the same song. Note that the previous MusicLM   was not as good in this task. Here is the  snippet we looked at in a previous episode.    Now, let’s see if the new MusicGen is any  better on this. I can’t wait - let’s do it! Wow. First one is flying colors. Not  even a little bit of a drift. Second one.    Maybe you noticed that I liked this one  so much that I was a little reluctant   to skip forward because I wanted to  just keep listening. Coherence was   not as tight as with the previous  one, art and music is subjective,   but if we have a song that has a little more  variation, this is a good one too. Loving it. And third. This absolutely knocked it out  of the park. Absolutely the same song for   two minutes. This is super difficult to pull  off, and in my opinion, the previous MusicLM   was not as good as this new technique. So much  improvement in just a few months! Incredible. But we haven’t looked at my absolute favorite  feature yet, I am pretty sure it will be yours   too. They call this feature conditioning, this  essentially can take an already existing piece

of music and remix it with a text prompt.   Listen to the input. And the output.    Wow, this is insanity. However, not even this work is perfect,   here is something that you might  call somewhat of a failure case.    Yes, the input was maybe recognizable,   but it was jumbled up and not  nearly as good as the other samples. But wait, here comes the best part.   All this is available to all of you,   free of charge. The source code of this paper is  available, and as of the making of this video,   you can even run it on the web, I’ve put  a link to it in the video description. So,   in the future, we will all have our own  AIs that create a nearly infinite amount   of music personalized to you in real  time. And just imagine what we will   be able to do just a couple more papers  down the line. What a time to be alive!