September Hangout Qdrant

our next speaker is Jenny sukola from quadrants uh she's developer Advocate there as well and Jenny will briefly remind us what semantic search is and show us what a vector search datab database like quadrant can provide Beyond semantic search she will demonstrate how to build an build in NN a rag recommendation chatbot and why it is better than asking GPT to recommend something for you so you ready to be amazed Jenny yeah uh first he B Kudos on the second time your surname pronunciation of mine was chis but so I would consider it the best start and uh actually cing back to Max uh I had a couple of times uh also explaining to people what is developer Advocate and I think I developed my own formula I call uh developer Advocate a standup comedian in it so like when you take the B and you just combine it in one so I'm here to entertain and uh I'm going to do exactly what uh B red and I hope after all this cool things with the ice print I won't make you disappointed because honestly I am looking at Max's job with a little bit of fear because he's setting the you know the pl for others here and I want my sleep so guys you don't do that to your colleagues in the field um so I'm going to firstly give throw in a very little slideshow explaining what is Vector databases I'm pretty sure you all guys already know what is that but it's just a general reminder um and if you don't know it's going to be clearer and then I'm going to show a little demo and I mean let's see how it's going to go but I hope it's going to work out and you will like what we built yesterday with alleg and also I want to separately say kudos to ALG he wanted to call me yesterday for 30 minutes to go over the case we spent in the end 1 hour 30 minutes talking about bugs in the both sides and I loved it honestly so much uh I think we did something very fun so uh about Vector databases uh so what are we talking here about I guess have a good okay so the first thing that you have to start with explaining what is Vector databases about embeddings and embeddings is simply Vector representation of all data that surrounds you it can be textual it can be image it can be audio it can be video it can be anything it's just one of the ways to present your data in machine readable format that you can basically perform some mathematical operations on it for example the operation of similari so in this space of representations some things which example like text would be similar say cheese and GAA it's semantically similar things so when presented as neural embeddings they're going to be pretty cloths and this is exactly where we get these representations from we get them from naral networks which are like now super big thing I think basically you can't find a person who doesn't know about them and uh this is happening like this representations are possible because they see for example if we're talking about text they see so much text that at some point they develop this pattern understanding and see that some words are belonging together and they're more similar and some words are like more far apart and in this numerical representation Vector representation we can then perform different operations and uh I mean here is an example like you know no words overlap no whatsoever but it's very very simple sentences to us and why what in this picture do in Vector databases where they are about searching this information this encoded mathematically information in a very big space of this representations because if you can imagine if we are working with data at some point we have a grandio like um the amount of points is humongous the amount of data that we want to see to comp compare its similarity find out what is similar find out like what is there and for that we need a specific instrument so it was the same for relational databases which stored the data in a specific format but with this embeddings which are coming from U neural network work we need a specific tool and this are vector databases because they allow to store this Vector representations of documents that you are searching of sentences of text of

images of videos at a big big scale which happens usually when you upload the data you encode the data you store it and then at the inference moment with Vector databases you just ask some question it's also converted to this mathematical Vector representation and then in this huge space that we have behind stored in the database huge space of points and representations we find a similar candidate I am going to actually show you when I'm going to be demoing a representation of films which are going to be encoded as this mathematical vectors and shown as a point uh how like our database um combines them together U behind like under the hood so H why they're so big right now this Vector databases and I think now I'm going to say the um the word which now people either hate to laugh it's literally a rug it's like retrieval augmented generation and I think that was the Catalyst of the popularity of this Vector databases because they are used under the hoood with these models because when these models arised like two three years ago we saw their potential we were like wow finally we can throw away Google or whatever is that and ask any question in a simple form to this model and it's going to provide me with an answer and I'm going to leave in the perfect word where I will know everything but then you start noticing that when you provided questions it start sometimes generating some very strange stuff which is not true or which is absolutely like not related to reality it never happened and you think like what is that and that is hallucinations because even the biggest model that was trained on the biggest amount of data from internet is limited to this training data and we can't trust it with answering our question correctly without providing to it some context some external context and the perfect thing for storing that context is actually a vector database which allows under the hood to perform the fast search for something that we want to find out for example be it a documentation we ask some question it's not keys I am going to ask like how to build um I don't know how to build a search SE in quadrant how to install an A10 AI hosted kit or whatever if I'm going to just search it in Google with keywords it's not going to work perfectly because who knows if in documentation there are like the same words that you are formulating the question but uh if you are going to convert it to mathematical representation and use it in the vector database it's going to find you a similar answer similar chunk of the documentation it's going to give it to our model and it's going to generate a beautiful answer related to documentation for example and then we're going to see basically this powerful instrument which allows us to have a search engine to which we don't have to specifically formulate some questions like thinking about the keyword or thinking about how to ask something and uh not getting any answer but we can just talk with it like with a chatbot and also it's not limited only to text it also can work with images with videos with audios because everything is basically can be represented in this Vector form so I from Quadrant and quadrant is one of the vector databases which is using one of the latest algorithms for that written in Rust and obviously is the best because why I'm here if not to advertise it but the thing is that beyond the classical search API which some people think about semantic search and uh they say okay you can do you can compare it vectors wow is there anything else I'm used to use my relational databases with this equal this filtering and order by we also have that we can provide nested filters and different types of aggregating information not only semantically but exactly and all the types of combinations of that which like you know when you can take the both like the best of both cases it's always the perfect solution but I guess one of the interesting features that we have and that what I'm going to demonstrate today is recommendation appy and it works on the pain that some people when they search for something they not only want something specific they also don't want something specific for example like as I'm going to present today I'm going to do a super small short movie recommender system sometimes I want to watch some film which is about friendship but I don't want to see anything about war or pain or blood because you know

what I'm not in the mood 2024 is already a year you know so that's what can be done behind in our knowledge based in quadrant and today is exactly what I want to demo you I want to show you on quadrant on a free cluster that we have so if you want to replicate it and do it with your own data also try it that would be nice I want to show you a demo of how to create a simple movie recommendation agent where you just write in the chat hey I'm sitting home I want to watch something today about love but without vampires because after Twilight I'm traumatized for already 10 years and it's going to solve your longing and not only it but you also can do it just in na10 and without coding which honestly I loved a lot because I had a lot of fun yesterday trying to build that around and that's what's going to be about I don't want to do this demo also from scratch of course this workflow is not as big as Max showed for his AI Sprint but still like I mean look at this sausage I think I'm going to do a detailed video I will use the experience of Max and I'm going to do a detailed video I say tomorrow doing it from scratch but now let's just look at the components and let's just see what it can do and in general like understand what is happening so before I am going to show that it's actually working as it intended to work so as a movie recommendation chat rug system uh I want to show the first part which is uploading the data to quadrant block quadrant Vector store because if we want to create a movie recommendation system we want in our dat database to store the movie these descriptions so they're going to be retrieved and then recommended to the user and uh of course it doesn't have to be movies but it don't in our particular case I took them and uh based on that our model our agent is going to do all the recommendations without ability to invent some movies which don't exist for example so um to upload data to quadrant uh we usually need like our cluster I have cluster it's actually free tier because I don't know it's enough and I already have created a small uh so I'm going to be using this data set I took it from kaggle and uh honestly it's a good choice if you want to showcase something it's top thousand IMDb movies with their short description so you can see what is there and um to save your time I'm not going to upload all 1,000 points now because as you understand they have to be converted to mathematical representations or to embeddings in our case with open AI model but you can take any open source model because guys have here also like hugging face and they also have open Llama if I am not mistaken yeah oh Lama too and um so to make it more fast I just want to in general firstly show you how to do it for smaller amount of points so what is happening here we're downloading my data set from GitHub we're extracting the file which is pretty similar and then let me limit um the amount of points amount of vectors uploaded to the database to 50 for example should be enough for the demonstration and Y and here we're going to since this is the one of the containers that we are already saving for a bigger demonstration let me create a smaller container here for 50 vectors and see how it's going to go and I am going to run this so this is going to upload our representations made by open AI uh embeddings to our database and it's going to uh take from the data from the so we can see it from here we get the file in the file we have all of the movies the classical form and so here it's going to take all of this information about movies and it's going to encode in mathematical like representation only descriptions because user usually asking for some movie recommendation is going to ask something

around description he's going to say okay give me something about like love but not much about like sweetness School romance and that's what we want to basically search around and let's test this step fastly and as you can see they're uploading one by one to our database and let me show how does it look here so for each cluster we have a dashboard and this dashboard shows already points uploaded and I can show for our bigger collection so this smaller collection is created now you can see that now it has 20 vectors it's going to be more when it's going to finish uploading but let's look at already done uploaded big collection of 1,000 movies so you can see how they are stored we have all the metadata uh about them we have a Content uh and we have a vector representation which is created through the open AI embeddings which is 1,536 points and we can see them on the graph how they are looking and are they close or not so we have all the different fun to play around and for example let me see in the visual V Li all the points that we have in our films and for example we see something very close on the graph and it is here Harry Potter and prisoner of askaban and the point next to it is Harry Potter and the Sorcerer Stone so they are grouped by description and you can see that the points similar to each other are about the similar films for example here is Star Wars and Star Wars so the same way when we are going to ask for the recommendation where our database is going to find a point similar to some of the films that are already existing and point as further as possible from the negative recommendation of the user so we can see all the 50 items uploaded and we can already check it in our collection so that's how it's done in general and now I would love to demonstrate you what we did yesterday with the ey engine because I think it's like super cool firstly I want to literally show how does it work and then briefly go over it so there is a chat and uh this is a movie recommender agent and I want some to watch some film which is about sports but not about baseball for example and then we will see the recommendation being provided to us which is going in the meanwhile to call the tool and it's call in the to oh I love this demo effect it's when you're doing something for 1 million times and then it works perfectly and then the demo comes and then nothing comes let me try again so I want to film about sport but please nothing about B uhhuh and we got our answer and our answer is sorry it was supposed to be in the chat yes so the wrestler The Story of an Indian athlete and champion and the story of a dog which is kind of Sportsman but not much we can see that it called the tool which we built here this tool is basically our workflow and this tool is calling our database and our recommendation I which returns top three films from our database with the score of similarity as you can see it's not that high honestly speaking I guess because out of 1,000 films we don't have so many films close about like Sportsmen but not baseballers but actually two top two answers are pretty good so it's about the professional wrestler and it's about the story of an Indian athlete and nothing about baseball and so here our agent is

just talking to a open AI model so it's like GPT just is used to call the tool and it used to basically beautify the answer of the tool the retriever data from the database and what is happening here so here we're firstly taking our positive and negative example because our um our model is smart enough to provide US based on our request a positive and a negative example so the positive example is a film about sports and the negative example is nothing related to baseball and then uh we embed these examples in the same Vector representation as we did here because we need to compare same vectors it makes sense you compare something similar to something similar then is basic mapping of the points where taking our embeddings our Vector representations and there we are sending uh them to quadrant recommendation API which basically looks like this you provide a positive example something that should be closed and a negative far and some strategy of aggregation after that quadrant returns us points uh so what is recommended you can see here I limited it to three points so it recommended top three points which are the closest we fetch them from database because we want to know what they are about and you can see that it's fetching all the descriptions uh like about the wrestler the name the film and yada and then we just beautifully combine them and then we send them to agent and that's how our agent can basically build their answer on something which is like truly our document the database and I guess the cool thing is that you can do it not with just a movie data set which is pretty easy but you can do it with your own documentation for example or with your own product or with Amazon or with house searching and then you are just going to say okay uh could you please recommend me something say I'm searching for a house for renting a house like house with through rooms but please without like red wo colors because they make me psyched or something and it's going to be done with this workflow without coding in one click and honestly that's sick and I love it so I am super grateful for the ability not only to present but literally to just try the tool because that was so much fun combining this simple solution brilliant Jenny and I know how hard it used to be to build recommendation engines and to see you do it in such a short time is it's amazing it's like kudos to you guys I mean also Kudos quadrant what can I say exactly it's a it's beautiful combination and I'm reading the chat and people are super impressed with your tools and the visualization that demonstration of how the movies were close together in the vector space even though you're like flattening what is it 12 dimensional space to two I imag yeah it's simplification of course it's like if I would be mathematical it's like principal component analysis but yeah it's like basically flatting into one dimension but it's still as you see it grouped Harry Potters together so it makes sense yeah absolutely that's really yeah and I think uh yeah people are saying we didn't see that in other products yet so that is really cool um yeah I'm reading a lot of Praise I don't think we have a lot of questions unless someone has one right now uh who wants to share one oh I noticed my uh CEO in the comments so all the questions can go directly great oh hang on now there are some questions here um so Marcus asks how would I keep the embedded documentation up to date in a vector dat database I would want to have help articles updated and added and need to embed them do I once a day or could I make sure the docs are always in the vector space so basically do you need to like re-encode everything all the time or can you do you have a smarter strategy for that no I mean um it depends on the how often your data is changing because like the new data arrives you just encod the new part of it and add it like you would upload any data to database right if you want to store it there you just upload something new but you don't have to rein code the old ones because they are already in the right representation there if something changes you will have to reupload it just because it makes sense like for example I was storing some say we had like a guideline we had

our documentation then it completely changed uh you of course want to reupload it because the old data is safe but it's just a common sense if your data is changing a lot I would definitely do a day Shadow three apport it's fast because uh I mean quadrant is built on Rust all the uh encoding is done like we have Vector stores which is up to 520 million points and it's working pretty fast because we have horizontal and vertical scaling so it should be like a piece of cake but it's good to have a strategy in place to minimize the encoding costs right yeah of course so that's okay that makes sense um another question is from oh hang on I just scroll out of view uh Colleen she said this was great and inspiring I started looking at the other day and wasn't sure where to start will this be an available template that we can dive in and look at so if you could share this with us I can make sure it gets publish as well yeah I would be happy to I also was planning to do tomorrow the whole video of setting this up from scratch but uh you guys need to teach me how to share na10 workflows because I think I am kindly using kindly provided oex account so he has all of my data we have a workflow Library where we can publish these yes yeah but you know what it's a very interesting uh thought and a comment to us that the person didn't know where to start from and we are going to like I would be happy if somebody reaches out to me and says what is the problem with starting because that's exactly the number one thing that you would like for your product that the people feel like they want to start I think recently we have doing a big job of moving very easy quick to start tutorials to our club so maybe it's the nicest way would be to start like sign up in the cloud and try to just click around without even reading the documentation because CS documentation you just I was with n to end I was like I'm gonna figure it out but you had an Ole right that helps yeah I mean o was like documentation yeah this is really cool let's make sure this get published so when we share these videos there will be links below them to uh to Jenny's full video as well as to the template so you can download it and get started right away and let's make sure to include your your voucher code as well for that free one gigabytes clust oh yeah it's it's just for everybody everybody you can sign up um let me see another question is from Carter Harrison and he asks what makes quadrant different from other Vector days databases like pine cone oh it's a oftenly asked question I would say my personal opinion the thing is like I've heard a lot of different answers based on like who is we have four devs we have four different answers and everything I would say for me personally except like the scalability in the thing would be firstly the uh the reaction on how fast we develop the features reacting to the community if you look at the quadrant Community it's very responsive very open sourced and very managed so our CTO is sitting there daily answering to all the questions because he just loves his job honestly speaking so for example this visualization was a thing that everybody wanted and we pretty fastly did it and so except from just being a good decent product for Vector search which is already around for four years so it was before the all hype and it wasn't done on the knee like you know behind the scenes behind the llms thinking faster we want to earn our money so it's not only like a stable product it's a product which adapts to what their Community needs very fast in a true open this company because I liked how the community works thank you yeah um let's see I have a couple more we only have a few minutes left so I'm going to pick a few um so that Mark krio ask is there uh any way of identifying datas of the data Deltas of the data so that only new data is added I think that's kind of answered already uh Mark that that's like on the n8n side you need to like figure out which data is new right yeah so we don't have like a standard process for that um by the way just a quick tip here in n 10 we have a compare data set Noe which would make this easier because you just connect two inputs basically and it would give you like a difference between data based on some great so that might be a case cool thank you

um Jim Lee asks when will we get an official quadrant n8n node still too many steps need to manually generate the embeddings for the recommended API would love to only just have to supply the text instead well that's not to me I think is it on a road map I can't promise I framed you I leave with it no uh and I actually when you reiterate with the question about checking if the data is same or not I realized what you were asking about uh because I interpreted it wrongly and of course in quadrant you can check if there is already some data because we have payload which is the fields which you can filter with so when you insert you can just addition basically the filter how to say condition thank you I'm sorry English is not my mother tongue so you can add a filter condition and that will help you not to insert the same data because it's going to be already detected that it's in database yeah for some reason I was trying to understand like if the data is being already saved or if it's I don't know what how I understood the question first time was too excited after the demo and also I saw somebody asking the question about PDF documents and if quadrant can work with them every data uh you just need to find a way to convert it to Vector representation and with n10 for example you can easily do that we did that with uh o yesterday you just upload a PDF document you split it in chunks and you upload it to quadrant and it takes like what three blocks of ANM four okay maybe four I will give you that um last question because we're running out of time here um Alex ask should we ask a quadrant note with insert documents for the first pass as shown is the demo and a separate workflow with a call to quadrant points API for aborts to account for new source document versions not sure I like that's more an nit question Alexa that like your initial upload versus processing the Deltas right yes it was just a question if we had a PDF that went through and then that was version someone made some updates could I overwrite the points in my collection for the old document so when I'm chatting with it I'm not referencing the old doc I've managed to go in and remove those points so my Vector store is always accurate first I'm trying to figure out is it question still to me or to BR I guess that's question to us uh so we currently don't support the upsert in uh operation for Quadrant we support for some other Vector stores but we didn't add it for Quadrant yet so you would have to check it yourself based on this metadata see if this already there then yeah I guess it's using API would be now the same with recommendation API I mean guys faster so many request for Quadrant groups I am um okay so there there's actually more questions here but we're running out of time so I'm going to wrap up and um let just quickly go back to my slides hang on um if you have other questions guys please head over to the Forum on community. nn. and open a topic there and we can continue the conversation there