Build a Speech Recognition System on a Raspberry Pi

hi everyone i'm patrick from assembly ai and today you learn how you can do real-time speech recognition on a raspberry pi and build a voice assistant all you need is a raspberry pi so in my case it's the pi4 and of course a microphone so in my case it's a simple usb mic and that's all to get started so let's test it hey sam send a tweet python is awesome hey sam save a note do bodyweight workout later today

all right let's do the unboxing first in my case i got a fancy starter kit with the raspberry pi 4 a case with an integrated fan heatsinks and all the cables of course all of this is not necessary and the basic version gets the job done as well but hey the company i'm working for recently raised 28 million dollars in series a funding so i can at least spend 100 bucks on a raspberry pi right

now let's set up the pi which came with raspberry pi os pre-installed the smartest thing to do with it obviously is to install python first so let's update and upgrade all packages first now we can install python 3 with sudo apt install python 3. as a beginner friendly ide i recommend vs code which you can install with sudo apt install code when this is done we can open it by typing code dot so this opens the editor in the current directory then you should install the official python extension and one tip for a smoother experience on the pi is to open preferences configure runtime arguments and then set disable hardware acceleration to true the next step is to install pi audio so we have to install port audio first and don't worry you find all the installation steps and commands and all the code in our github repo that are linked below i recommend to create a virtual environment like so and then activate it and then we can simply say pip install pi audio and since we are working with an api and web sockets i additionally install the other dependencies requests and websockets right away next let's configure the usb microphone you can type lsusb to list all devices and ideally this should list your plugged in microphone in my case it's the blue yeti nano you can also type a record-l to list all hardware devices and again you should see your usb device and also the corresponding card number we can also use the following commands to list the cards and modules and again this verifies that the usb mic is detected and working with card 2. we want to make card number 2 the default so open the alsa. com file with an editor scroll down to the default section and make card 2 the default in these two lines now save the file and reboot the system after the reboot you can try to record something with the a record command this will record for 5 seconds and dumps it to a wav file you can then use the a play command and this should play the recorded file and now we can finally jump to the code so let's open vs code again i prepared one short file that simply imports pi audio and lists all devices in the list you should find the connected usb mic

now let's set up the real-time speech recognition for this we use assembly ai and use the real-time api that works with web sockets you can grab a free api token using the link in the description below we have a detailed video and blog post that walks you through the script and i will link the resources below but basically we just need two async functions that work with websockets one to send the audio data from the microphone and want to receive the transcript from assembly ai and then we can do whatever we want with it in this simple example we just print it to the console now you can run the file and say something and then you should see the output in the console congratulations you did it now let's look at two more steps we can do to improve this project

the first next improvement is wake word detection because of course we don't want to call the speech recognition api every time we say something so the raspberry pi should be able to constantly listen to the microphone in the background and only when a certain wake word is detected it should become active and this should ideally happen offline so wake word detection is a whole topic on its own if you want to use a pre-built solution i recommend checking out these projects you see here raven porcupine and snowboy are all great projects you can test for example if you're interested in this or if you have too much time an awesome deep learning project is to build and train a wake word detection model yourself the best open source project i found is this one by michael d a i hacker he provides a great video tutorial the code on github and step-by-step instructions how to do this seriously you need to check this out

now what should we do with all of this we can use it for a stupid i mean ingenious use case and let the raspberry pi respond with arnold schwarzenegger quotes like michael does it hey wally who are you hey wally you idiot nice or we can use it as a virtual assistant to do certain tasks like sending out tweets or taking notes for this we need to do some kind of intent classification so we need to detect what the speaker wants to do here we could also build a complex deep learning model for this but honestly you can simply search in the strings for different predefined and hard-coded intent words and use if-else statements that then trigger all the different kind of actions if you want to see the code for sending out tweets and taking notes i have two more videos for you that you can check out alright then that's it if you managed to build your own virtual assistant share your project in the comments below i hope you have fun with your own raspberry pie and then i hope to see you in the next video bye