What if You Could Sing in Your Favorite Musician's Voice? | Holly Herndon | TED

(Overlapping voices sing) (Singing ends) (Applause) Hi, my name is Holly, and I’m an artist. That was my voice, but I didn’t sing that clip. I trained an AI on my own voice, and now she can sing anything in multiple languages. Her name is Holly+, and you just heard her perform “El Cant de la Sibilla,” which is a traditional song arranged by Maria Arnal in Catalan. Not a language that I speak and not a vocal tradition that I've trained in. Those melismatic runs are really difficult to hit. To show you Holly+'s full range, I'll also play an example in German. This is "Mack the Knife" by Brecht. (Singing in German) (Music ends) Of course Holly+ can -- thank you. (Applause) Of course, Holly+ can also perform my own music, so here's an excerpt from my own song, "Frontier." (Singing "Frontier" in overlapping voices) (Singing ends) Holly+ uses a process called timbre transfer, where the timbre, or sound quality of one sound, may be mapped onto the performance of another. Timbre transfer is done by creating a machine-learning model of a sound. In this case, it's my voice. So I recorded a wide variety of phrases in my entire vocal range. To be clear, this version of Holly+ is reading notes from a score. So I'll also play you an example of her singing in Spanish, “Bésame Mucho” by Velázquez. (Music: "Bésame Mucho") (Music ends) Thank you. (Applause) I can't speak Spanish, by the way, so. Teaching an AI the sonic properties of one sound in order to generate an entirely new sound is what I like to call "spawning." Spawning is what allows Holly+ to create a wide range of vocals that I didn't sing from a set of recorded phrases that I did sing. I like to think of spawning as a kind of 21st-century corollary to the musical tradition of sampling, which had a really big impact on both music and intellectual property. But I think spawning is far more exciting and potentially really weird. (Laughter) So, for example, with sampling, usually you copy and remix a recording by someone else to create something new. But with spawning, you can perform as someone else based on trained information about them. And as an artist, this is making me rethink my own past work as not only my archive but potentially also I myself could become reanimated with AI. This also opens up the question of how we deal with a collective human archive if we can reanimate old media. It opens up really big ethical and intellectual property questions that require entirely new conceptual and legal frameworks. One way that I like to think about intellectual property I call identity play. So rather than limiting the use of my voice, I'm creating instruments to allow as many people as possible to create music with me, and even as me. That's why I've made versions of Holly+ freely available for anyone to use online. If I can allow people to play with my IP, my digital identity, my intellectual property, what might they come up with? Could someone else go on tour as me, with my permission? Could I be in a thousand different bands in multiple languages? And what would that even sound like? To be clear, musicians have been taught to be really protective over our IP, and that's for really good reason. But I'm trying to think of ways to make this new capability mutually beneficial.

So to allow people to use my voice but still to maintain the ability to approve certain derivative works. So I invite you to consider, if given the opportunity, who would you like to perform through? And can you imagine someone else performing you? With that in mind, I'd like to invite the incredible musician Pher to the stage. (Applause) So today, Pher will be performing his own song, "Murky." And with one microphone you'll hear Pher's beautiful, natural voice. (Pher vocalizing) (Audience cheers) And with this microphone, you'll hear a live version of Holly+ developed with Voctro Labs. (Pher vocalizing in Holly+ voice) (Laughter and applause) Take it away Pher. (Pher singing "Murky") It gets so murky Loving what you know But I, I keep on swimming, I'm swimming Further up the road See, the problem is, I don't know what to say When you come around Acting this way Oh And, yes, the truth is I show you every day Cause you love to stay (Applause) Living in all the pain And it gets so murky Loving what you know But I'll keep on swimming Further up the road And I've been calling out your name Deep down in my sleep And I even memorized your face So ain't no leaving' me And I just handle my biz And keep the train moving on And you know what it is just you that I'm choosing Cause it gets so murky Loving what you know And I'll keep on swimming Further up the road And it gets so murky Loving what you know And I’ll, I'll keep on swimming Further up the road I'm swimming Further up the road And I'll keep on swimming Further up the road (Vocalizing) (Cheers and applause) Holly Herndon: Thank you.