# Community Hangout, July 2024: Local AI

## Метаданные

- **Канал:** n8n
- **YouTube:** https://www.youtube.com/watch?v=noBdpccJCgs
- **Дата:** 22.07.2024
- **Длительность:** 1:12:19
- **Просмотры:** 1,939
- **Источник:** https://ekstraktznaniy.ru/video/15635

## Описание

This month we focused on our new support for locally run LLMs and how to set them up in three different ways. Community member Jim Le shows the power of the new HTTP Request Tool for agents, and how it can unlock external data for your AI workflows.

We're currently finishing the Local AI package and will add the link here soon.

​As always, we started off with community and product updates, and we’ll wrap up with a round of questions and answers.

Chapters:

00:34 Agenda
01:48 Community Updates
08:47 Product Updates
18:43 Job Updates
19:34 Installing and using Local AI
50:02 Creating HTTP Tools

Links:

- Sign up for future n8n Community Hangouts: https://n8n.io/community/events
- Sign up for the n8n Community Newsletter: https://n8n.io/newsletter/

## Транскрипт

### Agenda [0:34]

we're going to be talking about as always we'll start with a bunch of community updates to let you guys know what's going on uh in my work specifically then we have our product manager JC who's going to be taking you some of the recent updates and uh future updates on his side of the project I'll be giving an overview of the new job openings uh so if you're interested in working for n10 that's the section to keep an eye out for then OLG from our team is going to be showing you this amazing new thing that we just launched which is local AI uh so now you can be like mindful of privacy run your own AI instance and uh still have all the functionality you used to within n10 um and then Community member Jim Lee is going to be showing us how to use the HTTP request Tool uh for AI which is not the same as the HTTP request node um and give us a couple of examples of AI powered workflows that he's driven at that point I'm going to stop the recording and then there's space for Community chat so I'm going to put everyone on the screen again and uh we can just have a good time have questions uh for each other you can show your work like anything goes but it will not be recorded anymore okay um all right so first update um

### Community Updates [1:48]

just yesterday we had our first uh Meetup here in the Netherlands uh we went to Amsterdam and one of our ambassadors Tino helped us uh set this event where uh he works for an organization called young creators um and they are related to this co-working space called start do in Amsterdam which was an amazing location very close to Central Station so very easy to reach uh huge building very much like a maze like little like stairways and like doors everywhere U but we found it uh we started with uh couple of drinks uh people just chested a little bit and then Tino and I gave an introduction of ourselves uh he talked a bit about young creators and I um explained a little bit about the n8n team because Lots people usually don't really know how big our company is where we're located how we work together Etc so we talked about that a little bit um and then I gave an overview of uh the focus of the team for the coming uh three months after that we went upstairs to the the roof Terrace and we uh we had a couple of uh more drinks and some pizzas and we left around 7:30 or so um so yeah so we are now working on this uh ambassador program and I'll show you a bit more about that later and we're really trying to get more of these events uh launched uh we already have uh another one planned for August that I'll show you as well uh and it's really speeding up and it's really great uh to see all these people coming in and enjoying uh these events there was a lot of conversation there a lot of people were doing really interesting things using uh AI but also non- air related stuff uh there were some companies that we hadn't heard of before who were doing very interested very interesting stuff uh with nat1 as well um and so yeah I think it was really good for everyone to just connect and get to know a couple of people um and everyone was really eager to do it again in a couple of months um so looks like we'll have another Meetup in October November or something like this um so to give you an idea of all the events that are currently um there will be another uh virtual hangout in sa Paulo our ambassador Louise is going to handle that one and Lise is also working towards uh like a physical Meetup in sou Paulo somewhere in August and details of that will follow uh shortly as well then our own devel Max is going to be in San Francisco to speak at an event next week uh and we decided to throw a little dinner for our community there so if you're from that area you can still sign up and uh come meet us on uh July 23rd um for a nice uh nice meal together group is like 10 people or something like that so it's going to be very nice have a chance to talk to each other Etc then excuse me um Berlin end of August is another Community Meetup run by uh one of our ambassadors uh it's pretty close to our office and so I expect quite a few team members to make it there uh Yan our founder already committed to coming over there and I'll you'll probably like say a few words as well so it's a great opportunity to learn anything you want about n we're pretty transparent about anything and then uh a day later while I'm still in Berlin because I'm going to be joining it or we're going to do another Community hangout uh from the office uh live with Max or with more people from the team we don't really know yet it's going to be really fun and interactive um and that one's going to be another workflow showcase and so that means you can uh we did that in may as well but it didn't really have a lot of time to prepare um this time you will have more time to submit your suggestion and work on a presentation of a fun crazy weird project you worked on and we're really looking for the like the interesting cases right where we can like have a laugh or be amazed that you can actually do that kind of thing with n0 as well so um if you're interested in any of these uh head over to nend. io communityevents um and you can click on any of them and find out more information and register for them as well and you'll receive updates and reminders as well um right so one more word about the ambassador program um on the right you see all our current ambassadors we have nine people uh helping us right now a couple in the US uh there's someone in Poland in the Netherlands obviously uh Argentina uh not Argentina Brazil sorry um and in Switzerland and uh I'm in process on recing a couple more um if you are experienced with NN if you're excited to work with community members if you have uh experience uh setting up events and events can be small too right like meeting five people for coffee and chat about n then is totally great um and you live in North America or Europe that's kind of our Focus area right now please get in touch with me and especially if you live in London or Paris those are the areas that I'm focusing on uh extra right now if that's something that's that you think is interesting head over to n. ambassadors for more information and like how to get in touch Etc okay um quick update on the tutorial Hub that we announced uh two months ago we already have over 70 tutorials mostly in English there's a couple in French um but you can add any kind of language that you want uh it's becoming a really nice library of educational material for NN and um uh I would like to encourage you to submit your own material right now I'm still like looking over YouTube and inviting people but I'm hoping to get to a point where people will add them automatically because it's a great place to share your knowledge as well um head over to the Forum there's a little tab on the left uh that says tutorials and you can see the entire Library there so I hope you will enjoy that uh okay at this point I need to refresh my screen but I don't think there's any questions yet right unless someone wants to speak out now no and don't worry you can still ask them later as well that will be fine I think but a few people were asking about events in other regions like Morocco and istan um yeah I am um so when I say I focus on North America and Europe that means that I'm actively trying uh I am actively trying to find ambassadors there if you are in another area where we have other users I'm more than happy to support you as well so just reach out to me anyway uh and you know we'll find a way to work with you any other questions l no any other way okay awesome um JC that means the floor is

### Product Updates [8:47]

yours great thanks B well firstly so cool to see so many of you here I see we just hit 100 in terms of participants so that's really cool um and I'm looking forward to uh someone who's also based in the UK to more meetups particularly in London I'll definitely be joining that one um so in terms of just a quick hello from me if you want to jump onto the next slide um I'm Jonathan I'm one of the product managers here at nat10 um and I'm probably better known internally as JC due to there being another John in the team um and I've been with entertainment for about a year now and my focus is very much on the Enterprise side of the product so really about building out a lot of our paid features um particularly for larger teams and sort of growing the tool for that type of usage and so today I wanted to share a little bit more from what my team's been working on um some of the things that we've released recently particularly around projects that have come out quite recently also some updates just on some addition additions we've made to um existing features and then also I wanted to just a bit of a shout out to feature called custom execution data which isn't new but it's something I feel is kind of underutilized I really wanted to get on everyone's radar or something that's um really cool thing to use going forwards so firstly in terms of projects um for those who have recently updated to a version of 143 or above you've probably seen a fairly significant change to the structure of N1 instances um and most of you will be seeing uh a new home area and in a lot of ways this replicates what we had previously which is that um it basically shows you all the workflows and credentials that you have access to across an instance but we also have quite big plans for this area in terms of it becoming probably more of a sort of dashboard era in the future where it's your sort of go-to space for your kind of own personal stuff but also a dashboard view of things that you know you might need to be aware of or top level metric so really it's kind of laying the foundations for future work there but what it's kind of opened up for is um you'll see on the left that we've now created this new area where you can create what's called projects and you'll see in my screenshot there it's a little bit small but you can see how created some projects like support product security and so on and the idea behind this is that you can now basically create kind of isolated areas where you can invite various team members into them and obviously control exactly what they have access to within those and so firstly it's about kind of organizing your workflows and credentials inside those areas but importantly for us it's also about extending the uh sharing mechanisms that we have in place so up until now we've had um uh individual sharing um across instances where obviously you would share a workflow on a onetoone basis and then keep having to expand that whereas with projects you can invite members into that and they will get access and inherit access to everything that's in there so it really smoothens out the process of making sure that people have access to the things they should that currently exist and also things that get added later on and then within projects we've introduced some new roles so earlier in the year we actually introduce a new instance level role for kind of like a super admin so that you had more people who could manage things um at the instance level such as setting up users and obviously features but at the project levels we've also introduced some new roles such as admins and editors who obviously can manage things within a specific project so um and there's going to be lots more coming on that so we've kind of got that foundational layer in place for projects and organizing workflows in that way and then in terms of what's coming next to this because it's really kind of early stages for us um on the next slide we also um are now looking at adding some things pretty imminently to this um first and foremost we're going to be adding more roles to projects so right now one of the biggest piece of feedback we've heard around this is we have quite a few roles now but all of them give access to people to be able to change things and edit and update and so on and what we've been lacking is a viewer sort of readon star so that you could potentially expand the number of people who have access to things for visibility and obviously learning and seeing what other teams might be doing but not necessarily giving them access to actually change things or potentially invertedly break something so we'll be adding that soon we're also looking at sharing credentials across um projects um because we see the need for this to scale really well that you could actually potentially create kind of a global project with all your credentials in and then share them out to projects that you need and that obviously gives you a lot more visibility in ter terms of um what people have access to and if you need to update that or take that access away and so on um and then we're also going to look at things like project level variables so variables is something that's used quite a lot quite heavily but one limitation right now is that they're always Global and so we want to bring that into the project level so you can set those up um and configure them kind of isolated to your team and then obviously not worry about others potentially changing those or being able to see the variables that you use um and then the final thing I wanted to mention which is kind of I guess the elephant in the room in terms of what we hear a lot from the community side of things is really what this means for folders because it's certainly not the intention that we're whilst projects is you know focused on the paid side of things on um you know pay plans and Enterprise it really is about laying the foundation so that we can then look at things like folders to bring to everyone ultimately and it's not that we're bringing projects out in replace of folders it's more about that work needed to happen first so that we can then figure out the best way to actually slot the likes of folders into that and whilst I can't sort of guarantee we're going to be looking at folders next week um it's really to highlight that this is very much on the radar and I would like to think that sooner rather than later we will be bringing this kind of functionality to everyone soon so that there is you know some improvements there because I know that where you have lots of workflows that's something that we want to make a lot easier for you so um it's a bit of a coming soon but just to kind of flag that it's you know very much still on the radar on the project side of things so yeah lots coming there um with regards to kind of projects and advanced permissions um in terms of other things just to give a quick update on um for those who aren't um too familiar with external SE stores we also have um within ATM the ability to use a thirdparty external Secret store where you can manage all of your kind of sensitive credentials outside of nat10 and this really has kind of like two benefits one is an extra layer of security um in the nat10 is not having to physically store your credentials but most importantly this is also that it gives you a central place to manage your credentials that you can kind of rotate and update maintain in one place and call this back into na10 rather than having to go and update lots of things within the ATM world if things need to change and the main update there is that we've now added probably two of the sort of most well-known um exteral secret stores in aw Secrets manager and also hot off the press is ail Keo which is coming I think probably in the release next week most likely so that's coming very soon and I suspect we'll look at Google um shortly after that too um and then finally just one thing that I wanted to give a shout out which isn't a new feature it's been in place for a long time is something called custom execution data and for those who haven't seen this or use this yet um this really is about um some extra functionality where you can pass additional metadata into executions um and you can do that either via the code node in a code format or you can actually do it through the execution data node and the reason I really like this feature and the reason it comes up a lot is because firstly what this does um is it gives you basically an extra custom filter and you can create lots of custom filters that you can then um sort of streamline your execution list views to really you know find the things that you need especially when you have a high volume of um executions but most importantly what this is really useful for is actually debugging across uh workflows so if you have multiple workflows and you need to find something across all of those setting up this uh custom execution data means that you can pass in that metadata and then really easy filter down and see you know where you might need to find a certain problem or where certain a data point is coming in that is causing issues so if you go on to the final slide um but you'll see here is a really simple example where I've got an email address I know that it's been used across multiple workflows and by using this uh feature I can then store it and then when I view the executions view I can easily jump to this and just see maybe I jump into one of these workflows and see exactly what's happening and maybe address any issues that I find so worth checking out um and um you a really powerful feature um but yeah I think that's probably everything for me in terms of uh kind of on the sort of Enterprise side of things um and but I'm not sure any specific questions yeah there are questions I always need to jump out of my presentation back in for it to update um so yeah it looks like we have two questions one is from Liam who wants to know if projects will be fully available for Community as well yeah so currently that's not the case they are on our pay plans from um from ramping up from starter through to Pro and then obviously Enterprise um and I think at the moment what's most likely is in some addition to those it will be a lighter form that we definitely want to bring Community related to folders and some sort so that's where we'd be heading initially to add on the community side um thanks and priia is interested in hearing if self-hosted paid plans are available for individuals or Freelancers or Consultants not necessarily related to your topic I feel but yeah I think this I'm not sure who best to answer this one but um currently uh the self-hosting plan we have is in the Enterprise format so I don't think that's necessarily exactly what um you'd be looking for here but um there are also a few other plans to do with If you're sort of an early stage startup that could be an option um but at the moment the option would be the cloud uh pay plans um in terms of individual freelance Consultants I think um I'm not sure I think you can upgrade yourself hosted with other licenses too right like with Pro or uh no actually I you can no um you can with Enterprise we did previously have a team that you could in the past okay Prince okay let me just check no that's all Lis no further questions other questions about outside of uh okay JC presentation so we can answer in the chat there well JC thanks for your updates and thanks for popping in here hope to see you again soon right thank you okay

### Job Updates [18:43]

bye okay so uh last update from my side is on the jobs and so I checked this week we have four new uh positions available one is a new lead technical writer uh our previous lead um has found another uh challenge to go to and the art talks definitely need a lot of attention and continued attention as well so we need someone who's very well vered in writing this kind of information keeping it updated then we are looking for two more product managers uh they are sorry uh for Integrations and for AI and finally a Solutions uh engineer so if you interested in any of those roles uh please have a look at N. C careers and you can find all the information there that you need apologies for that um okay so um

### Installing and using Local AI [19:34]

let's get to the main part of this hangout then our new local AI support and we have with us here Oleg who's one of our Engineers on the AI team who can talk show us how to install and use this new feature oh leg floor is yours yeah hi everyone uh in a moment I'm going to share my screen but I think you need to stop sharing yours there we go good this uh you see the canvas right yes cool uh so yeah I'm going to tell you about how you can run local AI in na 10 what are some of the available ways uh we're going to show three ways if uh if we have time starting with the most convenient or the simplest one to somewhat more robust one um you might be wondering think why would you even want to do this run AI locally or not necessarily even locally but to run it on your own Hardware the uh either the open source models or the models you train and the big advantage of that uh obviously the Privacy reason so you might not want to share your data uh with some other providers like open AI or entopic whatnot you want to keep it uh offline local uh another reason could be cost Effectiveness because uh it might be more cost effective for you to to do your processing on your own Hardware rather than paying for all the tokens um it's mentioned there's three ways you can well not three but three ways I'm going to show you today how to run them uh first very simple way is uh using some open AI API analog so one example would be this app called LM Studio or some other similar app which uh gives you an uh an API that you can use that is very similar to open AI API but you can download Open Source models and host them so for example here I'm having uh metal Lama 3 uh model uh loaded there's this local server that uh I can start it will give me an URL you can see here and then I can take this URL and go into the open AI node so we will be using maybe the chat model and in open I note we have this option called base uh oops sorry not I yet open I don't base URL so this option would allow you to override the API endpoint to which we're sending the request to process your completion uh see uh I'm going to put host doer. internal uh which would be the IP address of my machine but because I'm running an 10 in Docker I need to uh reference it in this way and now I should be able to well we can select some credentials but for this it doesn't matter because the LM Studio doesn't require me to set up a credential just so that the credential doesn't complain now if we refresh it we should be seeing the models that are available from the LM studio so right now there's only three uh only two uh the first one being the metal Lama model which is the chat model and another one is the uh embedded model so if I would select the chat one uh I can say hello who are you and we should see some lcks as it's doing the processing and eventually it should finish there we go and it's telling me that it's doing great uh we can do the same thing for embedding so again there is this space URL that we can change and then we'll be able to use um the model that from Alm studio um another way that you can run uh local models are is to use amaama AMA is a wrapper around llama CCP so it allows you to also download the models and to do the inference for you uh as Bart mentioned we prepared this Docker compos file and repository that you can use which has all of this already set up looks like this so if we take a look at the docker compose okay and it basically setups the nent for you with the post grass so that's what's Happening Here the service uh and it setups downloads the AMA latest version uh it binds the volume to it and it also setups the qu quadrant for storing your vector store embeddings there's two profiles you can run this with so you can run it either with the CPU profile or you can run it with the GPU Nvidia uh profile that would allow to uh utilize your uh GPU to the inference so it would be much quicker but it's not a required to have a GPU so I already have it running here uh again you would start it like this Docker compost profile CPU up this would start all the required Services as we see um so now the N1 is rying again and then one if you would like to download the models you would have to SSH into the Container so Docker exama b bash that would give you the access to amaama cama L to see the list of Ava available models and if you like to pull a model which is to pull um we can select some model from here for example mistro and now it will be uh downloading the model once model is downloaded you will see it in the list I'm not going to download it right now because I already have a few models that we can use I'm going to stop it and I'm going to show you how you can use the Obama uh Noe to do some stuff so here we had our open a we can remove that now and what we can do is uh let's say we want to uh we want to describe an image using the the open source model so I'm going to connect an a llama note here Lama check model going to select in the credential so in the credential you just need to provide the URL you see here that uh it's just AMA and that is again because we're using Docker so we can use the name of the containers rather than their IP uh you see here the container name isama so when we're referencing uh these containers between them themselves inside the docker network uh we can just sayama uh and this is the port that ama is running on you see that the connection tested successfully uh it gives me the list of models uh as we said we're going to use um image uh detection or feature detection model so that would be Lava um Lava Lama 3 we don't have to mess with any of the settings although there's plenty of you need to for example to keep the model loaded in the uh in the GPU for more than 5 minutes uh there's a way to configure that the temperature check temperature a bit and now we can send it a file so we recently released a new feature called file uploads so now in the chat trigger there's an option to allow file uploads this is available both for the development chant and also when you make it publicly available uh so I'm going to toggle this I'm going to set it only one image and then when then we need to pass the image so you see in the prompt uh we're taking everything from the previous note but that wouldn't include the binary file for that I just need to do add another message which would only contain the binary file so message type binary uh the field in which the binary is going to be and image details is not important for this uh model then if we go into the chat we see there's a option now to add files we can select this image we can ask what isage you see that there's stuff happening in the background as it's processing the image oh that was quite quick is correct it's in fact it is a large clock mounted on the side of the building so this is the astronomical clock from Brock ory um maybe it could be a bit more creative wa well it's not wrong it um ask it um let's say this there you go so correctly uh was able to tell me the city uh was called prag as well so uh yeah but it doesn't know about the you sure it's not cheating no okay very cool get that's no sorry I promise you it doesn't know about F just messing with you yeah um so this Docker Docker setup also contains qu quadrant and qu quadrant is a vector store provider they have an open source version uh you can we can see it here the UI it started on Port 63 three um and we can see collections here maybe let me delete this one and we can populate some new collection so another use case you might want to do is uh you know the classic chat with the PDF or ask questions about PDF so let's set that up really quick uh we're going to keep the chat trigger but now we're going to add the uh quadrant Vector store so the way we're going to do this is we're going to take the binary data from the from the chat input assuming it's a document we're going to chunk this document and populate the vector store with it and then we pass it the users prom the user promt to an agent which has an access to this uh to a vector store tool that allows it to query this Vector store that we just populated and then based on this it should provide us an answer so let's go ahead and set that up to using the Quant Vector store here you can see again that the we're referencing the docker container uh we don't have an API key set up so we can leave this one empty uh for the qu collection uh we can put it statically or we can just use for example the session ID from the chat sorry so that different users could have uh different sessions basically so we need to change the operation mode to insert documents like so then we can connect it and we can insert the session ID as a c collection we don't need any options right now then for the embeddings we would connect AMA embeddings here select credentials that will give us all the models that we have and let's use this MX by and at large model uh then we need to chunk the document so for that we would connect the data loader change the type of data to Binary because we're going to be uploading files we say that we want to load all of the data that we get this binary uh connected text tokens text splitter we know that this uh embeddings uh models uh has a context window of 724 tokens so we don't want to go above that so let's say we keep the chunk size to uh 650 and sh overlap to 50 tokens that would populate the vector store we can see if that indeed Works uh I have this setting that I only allow images right now so we change it to actually we don't need to set it we can just accept everything for this session then we have a Bitcoin white paper that we can upload um and ask what is the size of the BTC block header uh this is not going to work yet we just want to test that this is indeed populating the vector store and that seems to be happening so if we go to the quadrant UI we can check that this uh a new collection has been created with the session ID here and it contains the CH uh the chunk uh content like this it also has some metadata but that's not really important right now uh okay we can go ahead set up the other part of it so now that we have this in the vector store we want to add a agent so uh for theama the tools agent is not yet supported because uh Lama until yesterday didn't support function calling but now it's got or yesterday it got merg so in the upcoming week it should be also available that increase the capabilities of it but right now we need to use the conversational agent um we take the prompt uh we need to define the prompt find it below because the previous note is no longer chat it's this quadrant Vector store so we go in through this schema from our chat and input the chat input but the options we'll leave it empty one important thing to change here because you can see that there's a 10 items but so that would mean that this agent would be executed 10 times which is not something we want to do so we can go into settings and only execute once um like so then we can connect it some model like colat model Here Local we use Lama 3 decrease the temperature a bit again and now we need to give it access to this stool so we have a vector store tool we need to name it yeah it's very simple and we said that we want to retrieve four chunks from the vector store for then we need to connect the actual Vector store so we know that we have it in quadrant uh this is already set up we just need to set up the collection and we again need to use the same session ID as we used here so if we go inside uh we go mapping select the chat Jason and session ID so this should match what we have here in the UI uh no options here that should connect the vector store we connect the same embedding uh Alama model as we used for the chunking that is quite important to use the same model for the retriever retrieval as you used for the uh populating the vector store otherwise the tokenization method would be different and the scores wouldn't make much sense or they wouldn't be comparable uh we also need to connect a model for the vector store Tool uh because the way it works is that it retrieves these uh these chunks from the vector store then there is a model that answers based uh based on the query and that answer is going to get passed to our agent so I can just copy this one corrected so it's using the same uh Alama model as for the agent this one we want to make it uh really deterministic so we send the temperature to zero to reduce the hallucinations as much as possible so we can be somewhat sure that it's only answering based on the chunks and it's not making stuff up uh now we have it connected like this and should be able to execute this agent and it tells me that the size of the Bitcoin block header is 80 bytes which is correct uh we can take a look at the logs to see how it got that answer um so first we see this in this Al chat model that uh there's this classic conversational agent prompt uh it exec uted the tool user file knowledge base asked what is the size of the BTC blog header then that tool responded with the size of the Bitcoin B headers 80 bytes uh based on these why don't we see an execution we should be saying seeing the individual chunks in the vector from the vector store uh not sure what why they're not showing here something we need to look into oh yeah but you can see it here as it's getting passed to this llm that these are the chunks from our Vector store so it retri four chunks they got pass here and then the uh the vector store llm uh was able to provide this answer which is a bit more detailed and then uh our agent line just answered in a more simpler way exactly what we asked that is that the size of 80 es um yeah and then maybe the final method that I can show you to host these uh models would be to uh do it on some rented GPU so I have a b setup using BRP IO which allows you to rent GPU um uh VPS servers to run your models on uh so we can the way I've created this uh I just want to deploy the RT RTX 4090 you can see the price per hour and the available uh resources so I would select the GPU I would change the template uh or I would set the template to pytorch 2. 1 um edit template you need to expose all Port so again so you would expose it like this uh you might want to increase the volume dis a bit to be able to store the models uh and then you would go into the environment variables and you would add Alama host to zero to make sure it gets uh it's binds to the URL that we would be using I already have this SP set up so I can just start it again because it also contains some model that I downloaded so we don't have to wait I start a b let's see it running all way in a bed mhm we can connect do it we are the bmin all seems like Alama is not installed so I'm just going to install it again okay now the llama's installed and it's running on u local host on this sport do there's the model okay that we need to quickly fetch let's try m Okay so fetch the model so we can now access it and to access it we would use this URL and see that this list of models and we can copy that do n81 here maybe not for this workflow let's just disable that one add base L chain add the new credentials theurl being this one see it connected successfully and there is the model that we've just downloaded and we can talk to it so we can execute this and we got a response large language model trained by mro AI uh and that is coming from our pod you see it was significant significantly quicker than running into my um local machine because we are using a GPU now RTX 490 um I'm going to share the link for this Joker compost file uh and how you can run it after the session so Bard is probably going to send it over that should get you started with yeah local Ai and using Aline in uh any time and that's it from my side thank you for attention and if you have any question please share them brilliant um just as mentioned I will add the link to the video as I publish it early next week maybe for now I like you could post it in the chat so everyone has immediate access to it um and I imagine you can also just go to our GitHub repository and find it there correct that is correct yeah okay awesome Okay cool so uh let's go back to my slides and see uh what the questions are I need to pop out again um and I I'll speed up a little bit because we're going might be going a little over our time um mechnine wants to know if you can split paragraphs into chunks um I think that is what you just covered right so does this need more answering or uh is it possible to split longer yeah yeah so you would set up uh one of the uh deex Splitters that we have I've used the token splitter which would be static based on the amount of tokens uh but there's also an U recursive Tech splitter which would allow you to splits on some conditions like paragraphs or some markdown headlines for example so on um thas wants to know the specs of your Mac that you're running this on so I have uh Apple silicon M2 I think I have 64 GB of Ram uh but for these models um the uh the Lama 3 Model I used it was 8 billion parameter model so if you run it in the f8 qu then the 8 gabt of ram should be enough um and if you run want to run it on GPU then maybe 16 gbits of vram V IDE thank you um and you showed quadrant as your vector store a couple of times is that like a preference from us or can you use any Vector store that you you want you can want for this use case even the in memory Vector store would be nice because we don't care much about preserving it uh in the like a long term because we just use it to answer the users question uh so the in memory Vector store would be fine I also like to use super base Vector stores because then you can also have your database there and they have support for files but quadrant is also good um option thanks next this is a long one I need to read that sorry um SE wants to know about embedding if I Ed an open AI embedding model to create embeddings for our user profiles a few months ago and it's now been Sunset and replaced by their 40 model does that mean that those embeddings are no longer compatible for retrieval just wondering if these previously embedded records are being ignored now or if it's because it's still open the eye it's backwards compatible so I think the 40 model that that's not an embeding model um they semi recently released like this uh text small embeddings model um and but either way you would need to run the embeddings uh or Reed the content again uh because so I mentioned the tokenization method only change so you like it's not comparable so the results you would get that the scores wouldn't be very hard sorry I'll just chime in just real quick I think it's easier this way um so I had received an email from them that said it the uh recommendation was that you move from their 16k embedding thing to 40 like they said they were sunsetting the old thing and that they were moving everyone to that um which I did and now I'm just curious if like that makes all of those old records where I already did the embeddings on them are those no longer being retrieved in the system it sounds like yes is the answer to that question okay M so they are being retrieved probably you would still see some chunks that you would get but it's just that the score would be very low so there's no guarantee that those chunks are really relevant got it okay I'm just checking yep that's the last question um if you have any others um we can chat about it later right um thanks o that was very comprehensive and it was really great to see all the different methods that we have available um please remember to share that link if you haven't already and I'll make sure it gets shared out next week as

### Creating HTTP Tools [50:02]

well all right then on to our next part uh this is Community member Jim Lee who's been uh contributing a lot of great quality tutorials as well as templates for us lately uh and today he's going to be talking about HTTP tools and how they are not the same as the HTTP request node Jim are you ready uh yeah can you hear me yeah y yeah thanks but uh thanks o um amazing demo um I don't think I can uh well that's hard active follows but I'll do my best um to in this one right um hey everyone my name is jib today I'm going to talk about the HTTP request tool so this is a new tool um that's come out um but it's going to help me slide through um next one so yeah so the HB request tool is a new tool which was released um just shy of a month ago so in uh release 1. 47 so definitely upgrade if you haven't already and it's sort of exclusively used in air agents now this tool like B said it's not the HTP request node that we're all familiar with uh that we use for a lot of things it's more of a younger sibling that's how I would describe it um still very same functionality just maybe it's got a few bus and pieces that aren't um quite there or identical but we'll come to that later right um so as kind of one of the tools if you're not familiar with agents or a agents or tools um air agents are particular in that they are the AI models but they have these functionalities that you can use to extend them so uh for example the calculator uh is one way that um an agent or an AI model can do maths right just punch gives it numbers and calculate gives that similar the HTP tool allows the agent to make HTP requests right um but agents which kind of or air models which connect isn't really a new thing right so why in particular did NN kind of come up with still wi in my opinion um and that's kind of what I'm going to talk about today so it connects the internet so what right U I'm glad you all asked basically it means that the HTP request to enables any and I would probably say every API it makes every kind of API into an agent tool um super powerful stuff so that means um yeah it just giv you the freedom to connect your favorite Services uh maybe business critical Services directly to the agent uh for it to kind of perform a lot of stuff and I think it's super smart because um if you imagine if nnn went and said yeah we're going to make a tool for every service out there it would take them literally a million years you know shy you know a couple centuries but um yeah it would just take incredibly long time this tool just gives you the freedom to kind of bring them in into today you know into your kind of AI workflow today makes them super powerful so that's kind of like I feel like one of the big game changing features that uh or functionality that this tool brings the second one um if we go to the next slide oh uh oh sorry um I sorry about I reordered before you copied them sorry about that I did say I was going to resist but okay anyway um should I go to the next one uh yeah okay if we yeah sorry so the second game gamechanging feature is that this tool um if you like me have built a lot of um agent tools or what tried to extend with htttp these agents in the past uh You' notice you had to create this kind of custom workflow tool with a lot of sub nodes uh workflow and it kind of got a little bit big right so with have this um HP request to you know in some cases especially like with this web scrape for example uh that I found in templates Library you can bring it down literally from 10 nodes down to one that is an amazing feature and I'll go through how that works and how you can how that's possible in probably next slide if we jump back a little bit part a previous slide yeah oh not one yeah that's one right so um breakdown very quick uh the tool is um super familiar interface with the HP node um so uh yeah just big brain move from NN to kind of keep it the same so it's kind of very familiar very easy to kind of onboard um I love one of my favorite things about in is credentials how the credentials are handled so super easy just kind of click click select the ones you want um another kind of super smart thing is this placeholder variables so um this a new sort of concept and where we had tools previously where you know the agent was determining the uh body values the params uh in the body it can now do you know in the path you know variables in the path variables in the headers um incredible stuff that it just kind of knows how to configure and not only kind of pass the variables but configure the end points as well to kind of make those requests um super smart move again is the response formatting so um this optimized response option you have at the bottom means that um even though you're kind of making that request getting API results you can um Define sort of a subset section of that response to bring right so you're optimizing for your token limit so you're not kind of getting you can you know um get only the fields you want or which or relevant to the request uh where you know some API they ver for both and they give all this extra information you can cut that out so super simple I think previously you would have two notes to do this job so one to um handle the post request payload uh but in this tool you've gotone them all in the same uh node which is fantastic now some of the things that this tool doesn't do um is what I found anyway um binaries so if you're fetching the PDF with this tool it's going to bring back a binary the agent is not going to be able to um pass it or there isn't a step to kind of pass that into text for the agent um so to watch out for that um pagination so if you need to kind of go through a few pages or get like tokens to um to get the offset to get to the next request that's not really handled in this tool as well so it's kind of designed for um yeah make a request get the response uh no pation and lastly what I found is you can't chain API request so if you needed uh one API request to um kind of call another there's no kind of chaining going on so um so that's kind of not really what's designed for now remember this tool has only been around for 18 days uh maybe no not 18 days just sorry month a month sorry um so there's no telling like what the N team with um sufficient feedback what they could do to improve it so definitely yeah start using it and contributing your feedback to that um so if we go to the next slide but yeah so um after using this tool for a month uh so my recommendation Spirit of the bat First Choice for agent tools connected to services and apis um and I would definitely recommend over the old way uh we would have traditionally done it which is like the custom workflow tool you create like a mini rout router to sort of figure out what you want to do there um this makes it so much simpler just kind of um reduction of nodes is just um a joy to work with and lastly um obviously custom Cod you kind of want to avoid generally um harder to maintain but if you have to yeah that's also an option um probably just kind of a note to um even though like this tool now exists there's no rush to refactor all your past uh workflows I would say if it works if your custom workflow tool works just leave it unless you kind of you really hate it or the way you've done it um yeah feel free to sort of upgrade but yeah I mean it's the all these tools all these kind of approaches are valid it really depends on your use case uh next slide yeah right so I'm going to jump into a few quick examples uh this one is a very common use case or use scenario that I've seen uh when people kind of create chatbots and this is an appointment scheduling um chatbot particularly with one API service which is cal. com which is a calendar or appointment booking service um wow look at this what are we seeing here we are seeing uh 1 2 3 4 5 six seven nodes really um to kind of create this really capable booking agent you know it gets availability schedules appointment it can reschedule it can cancel an appointments if you told me to well same time last year that we could build something like this with just this many nodes you know I'd be shocked but now we have it today so you can do it right so um super easy to work with um not overwhelming that's what I find with creating a lot of these agent tools is know once you have to get the sub nodes in it gets a little bit overwhelming with the amount of nodes you have to put in and you know possibly if you decide to choose another service you need to refactor right you need to kind of change it it's always a drag when you're into that situations with this you just knock the nde out and you just replace it with another HTTP tool which is fantastic we're going to run very quickly into a video I'm just going to talk through this um so again this is the whole workflow to enable this kind of agent I'm going to open the chat now and I'm just going to type in sorry for the potato quality by the way uh yeah so chat type in that um you know I'm looking for an appointment um it's going to go through the rounds going ask my name um and then yeah it's going to ask me for a preferred time and date now when I type in my preferred time and date it's going to make a HTP request to the availability API in my cow. com account which just finish this text and it's making a request call and boom right it's now it's brought us back into the agent now again it's just one node it's made the request it's comeback it's tools uh really simple um very powerful feature that we we've only done one note to kind of enable is great so now I can confirm my uh kind of availability and then boom we're done again super powerful stuff um that the N team have enabled with this node um simplify workflows you know they're not going to take a lot to maintain um you know you can experiment you know gives you that freedom that creativity to kind of experiment with different Services uh super powerful stuff right uh next example oh Sor yeah so this is the yeah so again yeah just these four I mean it's always the rescheduling you know continue the conversation rescheduling you can cancel the appointment again these are just API calls and the agent is smart enough to kind of work out um kind of the rest so next example now there is no videos after this is the only video um but yeah so how we did it was um yeah in the HP request node there's um the endpoint we just put the tool description and the endpoint you want so this one shows the scheduling appointment which is like the bookings API um again credentials I love how the credentials work um you just pull it as you would there H be request node um placeholder definitions is how you get the agent to fill in the Dynamic variables and you can Define them like this so it's very kind of organized and you can Define so here I've got session ID schedule starts in the inquiry schedule in and finally in the right hand side is the optimized response so what I'm saying is I've toket it on um I'm expecting the response be Json I want to get out the selected uh fields which is ID start time end time and Status so again that's how I reduce the amount of tokens that come back to the agent so the agent can um yeah not hit that uh limit so very simple kind of um interface to work with um next example is um so previously we saw an agent hitting all apis of One service um we could eras a more powerful way of um bringing in lots of services multiple Services into the agent so imagine in this one is a holiday planning agent um each tool can be hitting Google flights Google hotels um f c for web scraping maybe Trip Advisor for tools now these are all different services with all different response uh formats coming back right um this tool can handle all of so there's no sort of edge case uh nodes you kind of have to put in to kind of you know um handle those responses um it's all kind of Consolidated the same format um really powerful stuff if you don't want to se hotels and Google hotels you just go for you know wherever um people get Hotel data from I guess um but yeah you can see how easy it is to swap them out which in the traditional workflow is again that drag of refactoring and um figuring out what their responses and stuff in all these other nodes and makes it very difficult um yeah so that's another example um the final example is you know it doesn't need to be sass AP right it's just a HTP request really but it means um yeah anything with an API can be hooked up to your agent directly uh here we're using the super base um you know rest API the PG rest API that they have um if you want to search products you know elastic search has HP um API uh endpoint to do that um and yeah and even kind of posting HTP requests like uh you know events to post hog so it can be inter internal it can be external um any yeah any every API is going to come into power your agent which is super powerful and again um you know you're not having these complex subw workflows to work with it's just sort of yeah super powerful stuff um next slide yeah that's it um I want to give a huge thanks uh for you for listening and Bon and L for giving me the opportunity to show showas this amazing uh update uh to the nna tools um I want to give a special shout out to Nik Nicholas who have been on the in team as well have been reviewing and approving my AI templates so if you didn't know um I have a Crea account on the Crea Hub where I have all my experimental and refund AI templates on there so please check them out um I love talking about Ai and n 10 and would love to know what you guys are using it for so you know contact UM Post in the Forum I've got some long post long form posts in the Forum but if you want to reach out yeah LinkedIn and Twitter uh my Preferred channel so thank you cheers brilliant thanks Jim this was really amazing and inspiring to see how much you can achieve with so little so few nodes that's really cool uh let me take a real quick look at the questions that we have so we have a couple uh sign is curious how things work under the hood what does the HTTP tool take as its input can you answer that or is that a question for the N team that's probably for n team I would my answer would be magic it works for you yeah magic I'm not sure if there're still around uh yeah maybe I can chip in so it's not using uh the definition of the API like the Swagger Json or the API schema but rather you just provide it with the placeholders uh that and so you're hitting a specific endpoint in those tools and you provide it with the placeholders which the AI fails uh which will be sent to that endpoint and this gives you sort of more control over what uh what AI can do but also gives you uh a nice way to manage your credentials for example in a secure way uh but it wouldn't just be randomly firing request you need to set up like which request is it allowed to fire it and you describe in the placeholder what each parameter means hope that answer the question great thanks Al um rako is interested to know if this could replace tools like crew AI a couple of months ago we had Derek Chung giving a demo of that as well so if you're interested browse back on our YouTube channel um Jim is this something you can take do you have experience with crew AI um I don't actually have much experience with crew AI I mean multiple agents kind of working together still was sort of a new uh theme for me so yeah probably not the best but I mean if it has the HTP um API definitely could um could ping those Services yes right also not exactly the HTTP tool but the sub workflow tool would allow you to do like sub agents and then you can compose uh them uh several levels deep got and I guess that also answers the next question if tools can reference other tools right yeah okay cool all right thanks and then finally uh Thomas asks if we have any ideas on how to combine keyword searches with semantic searches accessing different kinds of databases um I mean through the agent I mean definitely through the agent you could um but I mean definitely depends on your use case right what do you want to kind of do is it you want to search for the product first and then search for related products um you can definitely get the agent to carry out that workflow for sure but um if you kind of want it in one API maybe you need like a third party service or an external service which does that is that does that answer the question I don't know Thomas does it um hi there thank you for the answer um I was thinking of an e-commerce context where you look for trousers in red and it also finds you jeans in Bordeaux for example there would be semantic search in other times you look for um an article number and so sometimes the semantic search has the better results sometimes the keyword and I read somewhere there are ways to combine these so you have search results from your database vector store and then a reranking takes place and it and the best results are then shown as a combination of these both data sources so I was curious if anyone has experience with that or how you would build that into the chat but you know what I mean yeah definitely I think maybe that's uh that requires a longer answer imagine may maybe I can also jump in here a little bit so some of the vector databases like for example we8 I don't think that we have an n8n connector for that uh but they offer hybrid search right so they offer both keyword and uh semantic search based on embeddings and they do that basically by having a ranking trick that that uh retrieves both uh keyword based on keyword search and based on semantic search and then ranks them based on the on their ranking in both of these lists but uh that yeah so that that's one way to do it the other way you would like add two tools uh to an agent that allows you to instruct maybe the the agent a little bit better like hey if somebody asks something that requires a keyword search use this tool and otherwise use like a semantic a so that's two ways to do that I think okay actually I have a workflow that of the second option that JP just mentioned so maybe if we have a few minutes after I can show it which does exactly that y let's do that in the chat that sounds good yeah and Everyone by the way JP here just joined the N1 team a couple of weeks ago so it's really great to see him make his appearance here so give him a warm welcome thanks JP um okay Jim thanks so much for your great example um I'm gonna uh wrap up now but before I do there's one final thing I want to say our next hangout will be on August 22 so put it in your calendars and that day we're going to be looking at the uh each other's work as I mentioned in the introduction it's going to be a workflow showcase the crazier and the weirder and uh the more surprising your case the better in this case everyone who participates will win one of our cool little notebooks um and yeah you can register now go to n/c communityevents to find this go to the event and there's a link to a form that you can use to uh to register I'm going to be on vacation a couple of weeks but on August 12 I'll be back and then I'll go through all the entries and pick out the coolest ones and invite you uh to join us is there all right so um that's it for me the end of the show I was really glad to see so many people and such great speakers here uh thanks everyone thanks JC thanks Oleg thanks Jim uh you're amazing um and yeah I'm going to stop the recording now st stop the slides share and it's time to just hang out a bit together