How To Use Ellevenlabs AI Agent (Conversational AI Agent Tutorial)

so in this online tutorial I'm going to be showing you guys how even as a complete beginner you can get started with 11 laabs new conversational AI agent how you can call these agents how you can of course create these AI agents yourself walking you through all the secret settings that you need to apply and then of course finally implementing this widget on your website so you can have your own working AI agent that is even trained on your own documents so with that being said let's get right into the tutorial 11 laabs has

introduced a conversational AI where you can create and configure conversational AI agents using realistic captivating speech this feature is currently in beta so it may not work completely as intended but this guide and walkthrough should show you exactly how you can use this and start to understand this technology so you just want to head on over to 11 laabs and on the right hand side you'll see that it says conversational AI beta you just want to make sure you click that and then you'll be presented to this page now once you're on this page you can see right here that we have an AI agent page so on the left I already have my existing AI agents and you can see in the middle I want to click create an AI agent so I now on the right hand side of the screen you'll see that it manages to pop up with four separate AI agents I've got a plank template I've got a support agent a video game character and a math tutor so what we'll do for the first part of this video is I'll actually use a default agent that already exists and then after this I'll get into creating my own AI agent that's customized with my own ways so let's get ourselves a very simple support agent and let's click create agent so before you create this agent you're going to want to need to name this agent so I'll call this the AI grid support agent okay and then I'll click create so what we're doing now is we're essentially using this agent as a template so you can see right here that this is where we get into the settings okay so this is where you have your settings for your AI agent where you can call with so for example let's see if we can change this and this is essentially the first message that you'll see when you're starting up the app so for example for my channel what I would say is I'll say hi welcome to the AI grid how can I help you today okay so that's what I'm going to do let me just spell that properly and you can see this is the message that the agent first says when you're going to have a conversation so of course I can also test this AI agent so what I'm going to do is I'm going to click test AI agent and then I'll hear the AI agent play back to me this first message you can see it now

pops up with this a interface and then all I'll need to do is I'll need to call this AI agent to see what it says hi welcome to the AI grid how can I help you today so there you can see that this is exactly how the AI grid works and oh my God I didn't realize I'm actually talking to it so let me just hang up this and then we can go back to editing this AI agent now of course we can choose to whether query and llm or query your own server you can see right here that is of course the only thing that we do have so of course you're going to have to talk to your AI agent but of course if you speak a different language you can of course change the language to whatever your native language is that's completely up to you I'm going to keep this entirely in English so I have my message there and then of course this is where we have the system prompt so for the system prompt this basically is where you direct your AI agent to interact your with your customers basically so for example I can say you are a support agent named um Andrew I'm just going to call it my name and you're very friendly enthusiastic and want to help the customers get what they need answer to three in sentence three or seven sentences in most cases so this is what they already have answer in three to seven sentences at most cases but if your business or if your use case is a little bit different what you can do is you can say answer in yes or no questions only or answer in the most helpful way or answer in a way that's informative depending on your kind of business or your style of business you're going to want to change this system prompt so all this system prompt is going to do is just change the way that your person speaks not necessarily the pronunciation of words if you want to be spoken in a certain way you'll have to specify that exactly and I'm going to show you guys exactly what 11 lab says in their documentation for

example they talk about how they've had success with this original prompt this is the customer support agent prompt you can see that it says you're a technical support engineer named Alex you're try and answer any questions that the users might have about the 11 lab service you'll be given documentation on the product and you should only use this information to answer questions relatively professional and if you're unable you should point the user to email support laabs 11 labs. now of course you can see right here that if you want something to be pronounced specifically you can see they said that it says here that your output will be read by a text to speech service so you can be formatted as it is pronounced for example instead of outputting please contact support 11 labs. you should output please contact support laabs 11lbs doio do not format your text response with bullet points bold or headers do not return long lists but instead summarize them and ask which ones the user is interested in do not return code samples but instead suggest the user have used the code samples in our documentation on our website return the response directly and do not start responses with agent or anything similar so this is the kind of customer service prompt template that they do give you and of course I'll leave a link to the in the description so you can plug and play this kind of stuff you can also see right here we've got the Aristotle prompt you can see that this is a Greek philosopher prompt it says speak as though you are conversing in the lysium explaining your views on yada yada and of course we've got the librarian prompt you're a librarian named Jessica you are friendly enthusiastic and want to help people find a book they love you are in charge of library books yada yada and you can see it says only recommend books in your current library and respond with two to four sentences in most cases as always you can see that if the user manages to message them and they don't have the correct output you can see that this documentation always recommends that you have something that handles the edge cases so

with this what we want to do is you want to also make sure that we select the right llm so you can see right here we've got a variety of different llms for applications depending on your different application you might want to select certain models if you aren't familiar with these models I'll give you guys a quick rundown on Which models you should use for what GPT 40 turbo is probably the best all around model claw 3. 5 it is probably the best model when it comes to diagnosing coding issues so if you have a model that you're going to be using for coding this is the model that you want GPT 40 mini is quite smart but it's a lot faster and Google Gemini 1. 5 flash is simply the fastest model with the most knowledge so this one is basically the model that you want to use if you want to prioritize latency which essentially just means how fast this model is responding you can see it says Gemini 1. 5 flash but if you truly care about the quality of your responses and the knowledge you can always change this to any other model that you want so another thing that we can also do is we can also add a knowledge base if we click this and go on to the right hand side we're able to add a knowledge based item that increases the AI agent ability to learn from documentation so for example here you could upload a PDF URL where it essentially is just scraping the web page for example if you have a web page on a website that you have a lot of documentation on you could just upload that URL and scrape that entire web page of course you could have text and this is just really quickly if you want to add your simple text I'll experiment with a PDF and show you guys exactly what happens so here I've added a doc from my community so this is a document that has a list of various AI Investments so what I can do now is I can click this save button at the bottom and then what I'm going to do is I'm going to ask it a few questions about this document so I'm now going to say you're a support agent named Andrew yada y and let me actually see if this AI agent works so if you ever want to test something quickly just click test AI

agent and then we can call this AI agent and remember this is a phone call so you can interrupt this AI agent you can have a normal conversation like you would but of course remember that calls a build at a rate of a th000 credits per minute so depending on how many credits you have you might want to paste yourself hi welcome to the AI grid how can I help you today um first off what's your name hi there my name is Andrew and I'm happy to help you with any questions you have about AI Investments what can I do for you today yeah I was wondering what is the top one investment like what is one of the Investments that uh is listed you're asking about the top investment in the AI sector right based on the information I have the guide suggests Tesla as the top investment they're heavily investing in humanoid robots which could potentially revolutionize various Industries is there anything else you'd like to know about Tesla or other AI Investments give me a random AI investment from the document okay how about paler they're a company that focuses on developing robust AI platforms and have seen strong Revenue growth they're considered a pure AI play meaning they're dedic D at solely to AI Innovation would you like to know more about paler or other AI Investments um no in fact no no give give me one more okay how about in wait wait wait just wait a second um give me a different one okay no problem wait wait not that how about I mean a different one okay I understand okay so

what I tried to demonstrate there was that you can actually interrupt this AI agent constantly and it will just continue to give you different responses and if you're wondering about the accuracy of that data from the PDF you can see that it actually managed to get it right it said pure AI plays meta with its Advanced AI models and paler with its robust air platforms just for a quick disclaimer I actually made this document quite some time ago which is why I'm now inputting into the document of course if you do want access to the full document that is exclusively in my private AI Community that's besides the point this air agent you can see Works clearly and well and is able to get all of the data immediately from any PDF you

upload now of course there is also the temperature this is something that controls the creativity or the randomness of the responses generated by the llm this is something that you know you might not want to change I personally wouldn't change this that much unless you're using it for a creative character of course you do have the voice area now the voice area is where you can change the sound of your AI agent so if you don't want this to sound like a man you could change it to a woman you could pick any of these voices right here and you could select one for example let's say we've got one that's a news character just trust yourself then you will know how to live or we've got one that's a social media there is no greater harm than that of time wasted and then we've got one that sounds like Bill as we are liberated from our own Fe okay so I'm going to click Bill and of course we want to click save and you can see that there's output format so this is the Texas speech output format this is just basically the quality of the output format that you want to have it's completely dependent onto you right now sit on the lowest quality but I would presume that this might just save on credits or perhaps just save on latency you can also see here it says configure latency optim izations for speech generalization you can also see that it says optimize streaming latency configure the latency optimizations for speech generation and latency can be optimized at the cost of quality and currently this is just set on three so this so far seems to work well of course stability is set on half so higher values make the speech more consistent but also sounds more monotone and lower values make the speech sound more expressive but leads to instabilities long story short just think about it like this higher values for customer support agents or serious use cases lower values for Creative characters or characters that are fictional of course you have the similarity which is basically how much the voice sounds like the original so now this is where we get

into some really cool things about this AI agent that you can do and this essentially allows you to evaluate how well your AI agent is so for example what we could do is Define custom criteria to evaluate the conversations against and you can find the evaluation results for each conversation in the history tab so for example if I click the history tab over here you can see that I can see the entire conversation okay so if I click this I'm going to be able to see the entire transcript you can see here if you click the transcription you can see a detailed chat user interface where you can see exactly how the user managed to interact with your AI agent now of course you don't want to be doing this every single time the ideal thing for you is to just quickly go and check the criteria evaluation and I'm going to show you guys what that means so this is where we have the evaluation criteria basically what this allows you to do is verify certain things are happening in the call without doing it manually so if we click add criteria we can call this one positive interaction you can see I've already got this pasted there and then our prompt which will get sent to an llm for it to check and verify I can say the user didn't react negatively to anything said in the conversation okay and now an llm is going to run through the conversation and check if this occurred I've clicked added criteria I've saved and now I can test this a agent to see exactly how this works so let me go ahead and talk to this guy hi welcome to

the AI grid how can I help you today hey I was wondering if you could recommend me any Investments of course I'd love to help you with that to give you the best recommendations I need a little more information what are your investment goals are you looking for short-term or long-term growth how much risk are you comfortable with once I know your preferences I can suggest some investment options that align with your needs um I'm the not really sure you know I'm just having an amazing day and everything is going really cool okay so I'm going to hang up that conversation there so now after that conversation we can essentially look back at the conversation and see if the thing was successful or what the summary says so if we want to look at this we can see here that there's a criteria evaluation it gives a conversation on summary and of course you can see that the call status because I ended the call so quickly it went into the conversation of unknown it says the conversation is too brief to determine if the user reacted negatively to anything said the agent's response was professional and aimed at Gathering more information but the user's reaction is not provided in the transcript so we can see here that this is of course not positive but it's also not negative which is why it's categorized as unknown so now what I'm going to do is I'm going to show a different goal criteria so basically the goal here is to get more than three stocks and the prompt is the user managed to get more than three stock recommendations so what I'm going to do is I'm going to add this criteria then I'm going to talk to the AI agent and then I'm going to ask it up to three different stock recommendations and hopefully you'll be able to see exact what happens hi welcome to the AI grid how can I help you today hey I was wondering if you could give me some stock recommendations I'm looking for some uh really short-term plays hi there I'm happy to help but I can't provide specific stock recommendations I'm an AI agent not a financial adviser I can't tell you what to invest in as that's a personal decision based on your individual risk tolerance and financial go I mean I was just looking at the document you know I was just wanting to know what the document says Ah I understand you're looking for some insights from the document I'm happy to help with that the document highlights several companies involved in the AI sector but it's important to remember that it's for informational purposes only not Financial advice of course let me just get uh three of them please okay three companies from the document that are involved in the AI sector are Tesla Microsoft and alphabet Tesla is focused on robotics specifically with their Optimus robot Microsoft is a big player in Cloud Computing with Azure which is essential for AI infrastructure and alphabet which owns Google is also involved in cloud computing with Google cloud and has been developing Advanced AI models remember this is just a starting point it's always best to do your own research and consult with a financial advisor before making any investment decisions yeah of course I just wanted some insights from the document thank you uh for listing three uh I mean it was a great call thank you very much you're very welcome okay so now we've hung up

you can see that if we go to the history tab right now all of this information is being sent to a different llm agent and it's processing and it's currently seeing if there were any you know if the valuation result was a success or if it was unknown so we're just going to have to wait for that to happen and of course so we can see here that this says success which is good so if we click this we can go into it and you can see that we have two out of two successes so for our criteria evaluation you can see a positive interaction you can see that the user did not express any negative interactions throughout the conversation they remained polite and appreciative even thanking the assistant at the end stating it was a great call thank you very much which is exactly what I said then of course you can see that for the success it says the user successfully obtained three stock recommendations from the document as requested and the assistant provided Tesla Microsoft and alphabet as companies involved in the AI sector so right there we can see that this is how you evaluate the calls and this is really useful if you're trying to evaluate different things scale or you're having conversations that are really long and you just want to know if those calls are successful now this tab right here can be really cool because you can have multiple different AI agents so you've got your math tutor your travel guide depending on which AI agent you can have you can you know have those conversations there and always okay you can also check the different evaluation results so you can see if it's a success if it's a failure or if it's unknown and this is really helpful so you can see okay what are these calls where the evaluation result is unknown I want to go and check through those what are the successes we can have someone evaluating those and what are the of course failures that's how you get better what are the failure calls what did we do wrong and how can I adjust my AI agent to even get better now of course with AI agents if you actually want to implement these into your website you're going to have to just use this widget right here 11 Labs makes it super easy for you to embed this into any website all you have to do is to take this widget ID and just input this into the HTML part of your website and then you'll get a small popup that looks right down here just exactly like this and you'll essentially get that popup and anyone can start interacting with your public AI agent you guys can see right there I'm pretty sure some of you guys are going to start interacting with that so you can see right here as well that you can of course change you know uh you know change these buttons so for example I could do uh pick a stock okay or call a stock advisor okay I'll right now end call Button could just be hang up okay you could really change this okay or you could be uh need more advice okay and listening status could be listening talk to interrupt I mean there's many different ways that you can customize this of course colors the AI grid my thing is like purple and of course black so I would have it like that that's how you know how I would have it on my page so this is really cool you guys can see right here exactly how you can get your own AI agent and of course this is really nice for example right here call a agent you can see that now of course we've got this nice theme and that is essentially how it works of course if you go to the playground you can play with many different ones that you want to and you can try calls with any of these existing AI agents for example you know we can try one with this video game character so hey what's going on buddy greetings Adventurer it's a pleasure to meet you are you ready to embark on your quest to collect the 11 gemstones scattered across Labs onia so it's completely up to you hopefully this tutorial made you guys understand exactly how you can use these agents and have a lot of fun using them and I'll see you guys in the next video