AI Reads Minds of 29 Patients!

Today you are in for a treat, because  we are going to perform brain to audio,   yes listen to music straight from  your brain, kind of, and more. Yes,   I hear you asking, Doctor,  how is this even possible? Well, this new paper showcases an EEG-based  technique to perform this science-fiction like   experiment. Normally, EEG, electroencephalography  means that we place electrodes on the scalp,   and then, we listen to the brain from the  outside. And by that, I mean looking at   the electric activity. However, here, this work  uses Intracranial EEG, that requires electrodes   to be on the surface of the brain. Thus making  it an invasive technique that makes scientists   able to listen to the brain straight from  its surface. So what can we do with that? And the answer is: absolutely miraculous things.   Let’s look at the experiment design and see what   happened. First, a song is played to 29 patients,  this was Another Brick in the Wall the classic   from Pink Floyd, then an auditory spectrogram is  created from this song. This enables us to look   at the song and understand a little better where  we are and what is happening. Then, scientists   recorded the neural activity of the brain at  the points where the electrodes were placed, and   created a spectrogram from information that was  obtained from the brain. This is a decoding step,   and the one in this particular diagram was done by  a neural network. Decoding real neural information   by using a digital neural network. What a time to  be alive! Now, hold on to your papers, because we   are going to look at the two spectrograms, and…oh  my goodness. The information that was read from   the brain indeed refers to the same song. No  question about it. Due to copyright constraints,   I can’t play it here, but you can listen to  it through the link in the video description,   and I have to say that of course, this is not  HiFi quality, but if you know the song, you can   tell what is being played in the brain, and even  sing the lyrics to it. Absolutely incredible. The paper also showcases a ton of  little golden nuggets of information,   I loved this chart that tells us exactly how  the quality of the reconstruction increases with   the number of electrodes, little measurement  points being used on the brain. Adding more   of typically increases the  resolution of the information we get,   but also increases the noise as well. So here,  a proper way of decoding helps a great deal. This paper is an interesting showcase of what  is possible through the power of research,   and since this data has to be read from the  surface of your brain, I would not worry   about anyone eavesdropping on which scholarly  paper you are thinking about. For instance,   a couple hundred episodes ago, we talked  about an absolutely amazing paper that allowed   brain-to-text transcription. It took patients  who had a paralyzed hand and all the test subject   has to do is imagine writing the letters, and  they magically appear on the screen. And now,   hold on to your papers Fellow Scholars, and just  look at how quickly it goes! 90 characters per   minute with over 94% of accuracy, which can  be improved to over 99% of accuracy with an   additional autocorrect technique. That is  absolutely amazing. We can learn a ton from   these papers. For instance, in that one, we  learned that it is quite remarkable that even   years after paralysis, the motor cortex is still  strong enough to be read by a brain-computer   interface well enough for such typing speed and  accuracy. It truly feels like we are living in a   science fiction world. Imagine how many people  around the world we could help with this! So today, brain to text to possible, and brain  to audio is also possible. But here comes the   best part - for people who are unable  to communicate with us through speech,   they can not only write to us, but with this,  we can even reconstruct the musicality of their   speech as well straight from the brain.   Imagine if we could have heard the late   Stephen Hawking talk like that. How  amazing would that have been? Wow.

Thanks for watching and for your generous  support, and I'll see you next time!