Edo Liberty, CEO of Pinecone on Vector Databases & Building AI Products Optimized for Love and Trust

hey Ido it's great sitting down with you we're both AI Founders living in New York City I started assembly AI 2017 we've run into each other a bunch and talked about what our Journeys have been like running AI companies the last 18 months has just been crazy and so I'm really excited to sit down with you on camera and talk about what your experience has been like over the last 18 months cuz I think that would be so helpful for other Founders that are building companies in AI right now so to start I'd love to learn more about just the founding story of pine cone we've been through very similar experiences over the last several years and uh the founding story of pine cone is you know starts back in the early 2000s in my academic career and then my first startup and my professional journey through Yahoo and AWS I did my PhD in postdoc and applied in computer science and then applied math mostly dealing with a foundational aspects of machine learning algorithms and what it was called back big data back in the day and so on interestingly we were working on machines with 52 megabytes of memory and so like anything more than that was big data because you couldn't fit in Ram and so you had to get smart with algorithms and so on I then started my first company with soul that and then I joined Yahoo uh as a scientist became a director in charge of uh AI infrastructure aahoo Yahoo back in the day was sort of like the Paragon of of innovation in Ai and machine learning and infra and so on I moved to AWS to work with Swami sha suban it's now s team in AWS and uh that entire uh fantastic team building Sage maker and a bunch of other services out of ABS kind of you know power machine learning AI back then it was what year was this for 2016 yeah back then it was like mostly ml Ops and like the sort of like the Zeitgeist of what people care about and AI kept changing and I was there until middle of 2019 where I started pine cone and I'm saying all of this because throughout that time Vector search Vector databases the infrastructure of information retrieval sort of like the most the tectonic sort of like motion of information retrival and search on the one hand and like machine learning and in and deep learning and like all that stuff today that we call AI collectively kept going getting closer and closer to the point that they were starting to really merge yeah right if you remember the first fully trained open weights llm really that sort of like kind of started triggering an avalanche of hey wait a second this we didn't call them llms that back then I don't think it was called that it was called a Transformer like any new disruptive technology they were just clunky and opaque and weird and kind of hard to use and very few people knew about them but the people who knew about them already kind of like looked at and said wait a second there's something here this thing does something that we didn't see other Technologies do right and so I started kind of seeing that final collision between the tectonic plates happening and I'm like okay wait a second now this world of search and world of AI are colliding and the technology in the middle has always been in the middle is a vector database uh but I also thought

it was too early because nobody knew what the hell a vector database was I didn't know to call it a vector database what was it called back then it didn't have a name like I explained to people is that why in your Forbes bio your uh database entrepreneur yeah I mean jokes aside the company almost died on the vine because we didn't have an A yeah what did you call it back then I forget exactly what which I tried everything this is the most like depressing part of like I tried to learn from one meeting to the next I tried everything and nothing landed it was the most depressing thing in the world like I you know I'm I was almost I was on the verge of giving up like people have no idea what the hell I'm talking about like I was speaking a different language like they just looked at me like nothing I'm getting nothing at some point I just asked uh one of our customers we just asked him like Hey when you talk about the thing that you buy from us what do you say it is I'm just curious and he said we just call it the Victor database CU it's a database for our Victors I'm like sounds good thank you that's we'll go with that yeah so you guys that's something I didn't realize like you guys really were like first to Market with publicly talking about this thing and branding it as this a vector database yeah and I got a lot of heat from my investors for that too how so cuz they said hey a who gives a about databases who builds a database anyway yeah and B what the hell is a vector what do you want from us why are you talking about like what why what are you doing this and what's your what was your

opinion back then and what is it now on like why you do need something specialized for this task you know yeah so first the obvious which is we have failed to build it in any other way and it was built in very specialized ways in these different companies I myself built a vector database four or five times in my career in my first startup and Yahoo each one of those companies Amazon multiple times right uh I mean not a standalone product but into other products as sub whatever like as component so on and every time like we had to build something bespoke because we try to use whatever in existed and it was always like too slow too whatever too expensive didn't respond couldn't be up like it was always like become like depressingly bad at something right and a lot of people are building new applications so they don't have a project at scale right they haven't faced these pain points that you faced because you were working on this technology five six years ago how do you think about as a founder and you guys as a goto Market team now that you're trying to scale and grow like effectively communicating that to prospects to users to developers that are really early in their Journey that are maybe like I don't even know what you're talking about yeah it's very hard to be honest I mean because a lot of the sort of like competition quote unquote that we have is good enough for small use cases it is good enough if you have 10,000 vtil you don't need pine cone I mean you don't need anything you need numpy you don't need nothing you need a matrix you need nothing okay you can choose to use whatever you want and it'll be fine cuz it's so dinky and just uninteresting right you're not hitting any limit of any sort right yeah and if you're just like learning how to build something and you're just oh what is this spectal thing and what is an embedding and what is this and how do I do rag and I need to go run some notebook everything seems to work right and so you get lulled into this like false perception that whatever I happen to have in my notebook is actually what I need right and that's a problem we're all Engineers like we build with what we build we get used to it we kind of figure out all the sharp edges and the tweaks and we find what we love about it and we get attached to it and so on and if you can't kind of be that for most people then it's hard to catch them on later

catch them on hard to catch them later I'm sure the first version of pine cone was not scalable just like the not enough the the first version of assembly was not super scalable I just wanted to like get it out there I wasn't thinking about uh all the scalability issues we' face when we were processing like you know 20 million API calls a day or something um so how you balance when you're talking to users and developers that tension of pine cone is a solution you can use at scale with very large workloads and it's like tried and true um and you don't really see that benefit right out of the gates when to your point like you could just use like a text file or a numpy whatever to to get started if you're at super small scale yeah because it's so new how you think about just educating developers on that without you know like turning off developers like oh no trust me like you need this production like developers don't want to hear like production ready and like Enterprise great they want I think it I don't think it's I don't first of all you don't speak with Engineers email no there's no way like I don't want to be spoken to when I build something no if you are building whatever it is you're building you just want the best tools m you want to choose the best whatever you want to move fast and you want to make good decisions and really like just build whatever it is you're building right and our job is to make that as seamless and and fantastic a journey as possible yeah from like beginning from beginning to end and so there are two states of mind that I talk about a lot here in like in our product and hopefully people get to see it it's love and Trust right first of all you have to love the experience and so we think very deeply about the interfaces and uh why we should or should not include some knob and is it absolutely necessary and so on so like we really try to be super minimalistic so and kind of automate everything that you shouldn't be worried about like so for example we don't let customers choose the index type for the vectors and in the beginning people kept like oh I want to change from this to that I'm like why don't you just want it to be fast and easy and cheap and performant like yeah like okay is it not like are you happy with what you're getting yeah great then why would you want to Tinker with it then we can tell them hey when you have 50 different apps running in production like are you going to test all 50 and just design make 50 decisions on which like how you want to configure everything or do you want to just for this to just work right it's like oh yeah fine I get it now I want it just work I'm like great we agree okay so there's love there's like oh you just do something that it feels good that you love tinkering with that you get immediate value out of like Snappy and just fun to work with you're like oh I get as an engineer I just love it these people get me I get what I need and like I love building with this okay you have to have that bu feeling if you don't Inspire that in your customers you don't have a business like you really this is like maybe you doing some we wave not a plg motion not a developer Le motion the second thing is trust okay and I again something I tell my team all the time is that people would use a product that is maybe not perfect or missing a feature or less performing they will tolerate suboptimal experiences what they will not do is work with a technology or with a company they don't trust okay and so we go above and Beyond to be absolutely transparent and open and at the same time as a as an environment be super robust and trustworthy and so on

I'm curious how you also think about that trust coming through your go to market motion like and just how you sell pine cone how you price it what I think about is like serverless right serverless came out it's a lot cheaper and I think that's a great example of like building trust with your customers like Hey we're uh offering something that's a lot cheaper that you can move to that's going to work well that is like you're going to pay pine cone less but we're still doing it because we're we know your we're your infrastructure and we want this to be scalable and so I'm thinking how do you think about that trust coming through when you think about your go to market motion and interacting with customers and users and um pricing and packaging and all those things so first I'll tell you who's not happy about it I can guess you can guess does it does start with i it start with i yes uh does it end with s it ends with vestos uh no but jokes aside I mean for good reasons they're unhappy with us I mean this is this we're a business right and it's sort of like confuses everyone including our customers we have customers who paid who happily paid us $100,000 a year on running their infrastructure on Pine Cone and we went to them and told them great uh please upgrade and going forward you'll pay us $5,000 a month a year and they're like okay what's the catch what's happening here it's like nothing's happening like how is this worse for me it's not worse it's strictly better and then my sales te is also pissed by the way it's like oh I just worked on this deal for like four months and now like I don't even know why I'm doing this because they're going to pay us 500 bucks a month and like I don't that's nothing yeah it's not worth it for me even I'm just wasting my time I'm like yeah maybe right my respond to that is that it's an illusion right if you think you can charge $10,000 for something but like you can actually vend it with profit at 500 the sort of the false choice of you to charge 10,000 500 is a false choice if you keep charging 10,000 you're going to charge zero cuz you're going to churn all of those customers you know it's like it's it this is unsustainable if you can build it in a way that you can make a decent profit out of selling at 500 that should be the price because if you don't do it somebody else will do it right and so I'd rather disrupt myself than have somebody else disrupt me right and so yes I don't like it that you know our revenues take a massive hit on this transition I don't love it that you know I love it for the customers that they pay list as a company it's not good for us right but in the long term that's 100% the right thing to do and I think it's going to pay out but we're going to have to stick you know with it for a long time yeah I think the unique thing about the types of companies that we're building and AI companies is that the algorithmic and technological improvements that can happen within a year's time can be orders of magnitude uh in terms of lowering latency lower cost and I don't think that is unique to like traditional SAS businesses or traditional startups where it's not like all of a sudden stripe is going to find a way to process a credit card for 1,000x cheaper and can lower prices by 10x like it's not like Amazon can figure out a way to deliver packages 100x cheaper within a year I mean I know they're working on drones and last would love it but there's so much investment going into I mean Hardware even there's uh custom Hardware being made for LM inference for uh Transformer inference and that alone is going to be able to provide a lot of um a lot of reduction in inference cost with llms like model caching uh is a thing right for LM inference that LM companies are starting to release so this the rate of technological change around serving AI infrastructure especially just given the like all at a macro level all the investment and focus in this space um is crazy and I think to your point we think about it similarly like we need to be out ahead of that and yeah I mean I remember when we met at we were at this dinner and you were thinking about this I think earlier than a lot of Founders and companies were you were like you know you got to just uh be ahead of the curve and like focus on driving usage and getting people to continue to run more and more workloads on your platform because the lower cost yes that customer that was running 600k can now run on the free tier but I'm sure the way you think about it and tell me if you disagree is like there's now thousands more potential customers that can correct run workloads that couldn't before because it was prohibitively expensive and it just was not possible before on and I think there's sort of like a race uh that sometimes like we're ahead of and sometimes we're behind as an industry not pine cone or assembly uh of how fast technolog is improving costs are reducing like going down and so on and at the same time how fast use cases are ramping up and workloads are being created right and so sure you can cut cost by 10x but guess what if there aren't 10x more workloads to be had right now in the market for you then your revenue is going to go down right you know that is a delicate balance to strike totally and I think it's a frankly it's a struggle now for every AI infrastructure company because we are getting better really fast and it's you know and we do want to share that efficiency with our customers but if like you know we need to 10x our usage base and workloads within a year that's a total

order yeah especially even as I mean as an infrastructure company there are there just is time for like there's a certain amount of time required for users for developers to build something with pine cone with assembly to launch it to get usage to get users they might be able to now say okay great I can do this thing now with assembly or with pine cone or with whatever open AI GPT 40 mini that I couldn't do before but it's going to take me six months to go get that to Market get usage it might take me a year to have a very mature like large scale workload I mean I talked to a lot of other Founders in AI infrastructure space and everyone's you know dealing with that because you're almost sometimes caught off guard by how much uh faster or lower cost something gets right you build a new model or you build some new serving infrastructure put it in production and then you iiz like holy crap this thing's actually like super efficient okay now we got to go get this thing to Market how do we think about that we have to go amend these contracts we have to update this website we have to go do all this work to get that out and that takes time so um I think that's another Dynamic too it's oftentimes you're like surprised by just how much the rate of change and how much better things are getting yeah I'm with you there are a lot of requests that require you to fundamentally somehow uh make engineering tradeoffs that will hamper your progress in the future right so one of those for example was open source if you're a dieh hard fanatic for open source we're sorry we can't use pine con okay that gave us so much Innovation speed and our ability to change the underlying technology our ability to ship better products under the hood that was a huge decision if you don't think I hear on a bu weekly basis from a large customer we only want open source you're wrong by the way nine out of 10 of them if you ask them why they only want open source and you started talking about it you figure out most of the time they don't really need open source at all they want control they want security they want a lot of things that opens they think open S gives them and then you start kind of walking them through it it's like oh do you think it's cheaper why like do you think it's more safe why do you think it's more this what why do you think that and so on some people are die hard finan for open source we you know you can't change that but more often than not it's not a problem it was a huge achievement a huge benefit for us same thing for being a manage service okay uh same thing about being multi-tenant for most of our customers we can all do single tency for large customer and so on so there are these like very core decisions to who we are as a product that allows us to scale allows us to innovate quickly allowed us to operate the business in a in an efficient way we have to say no and just sort of like section off some part of the business and say you know what just not going to go for you're like yeah I'm just going to have to not go for it right now because otherwise I get pulled in a million different directions I'm sure you've done the same thing by the way I'm sure this is like there's a million different requests that you get you like sorry for us last year 2023 was this explosion and so many things were being built on the API were're trying to support all of them everything that I just spoke about streaming asynchronous batch modes all these different types of applications and then coming into this year we realized you know what we really need to pick a couple applications and really focus on the product for those applications taking big steps forward because if we allocate our focus and our compute and our efforts on that we can take big steps forward for that type of product and that type of application and then we go and work on the next one but it's difficult because there's a lot of uh there's a lot we get pulled in a lot of directions and I think our job is to be this we have to be like a as a company a very good product manager to try to figure out do we go focus on this or do we focus on this and if you get that wrong you know that can that can be really bad because some of these research threads they take six 9 12 months if you want to go improve a model in a certain area yeah um so for us we're always trying to stay really close to where we think the Market's moving where applications are maturing and there's some right now where I talk to Enterprise companies on a weekly basis too that have committees and tasks for task force and sometimes even budgets for certain types of applications but they're all in the Prototype phase and I think that there's still a lot that has to be improved from a technology perspective before what they're trying to do can actually like be viable in production whether it's a cost or reliability I think of self-driving cars like 95% accuracy of a self-driving car is actually like amazing but it probably won't get legalized because that 5% error rate even 1% error rate is just too big if you're going to deploy these things just out on the street in New York City similarly if you're going to automate a phone operator your the error tolerance you have is probably very low and the cost to deploy something like that at scale is going to be very high so you're very cost conscious and so we're just thinking about all these Dynamics all the time when we think about what to build for yeah so um we

we've talked about a lot of great things today on a closing note you've now been a Founder for six years now this your second company what is one piece of advice you'd give to yourself if you were starting pine cone again today wow so many the most important thing is um I would just like be gentle with yourself your own mental health your own physical health your own mood your own State of Mind are our critical asset not only for your wellbeing but for the well-being of the company and I think it's very easy to work 18 hours a day for 3 years and burn out and be dead uh and that's you know it's uh building companies is hard uh you know you'll change like in a world changing environment you're doing a million things for the first time you're going to make a thousand mistakes whatever it's it's okay that's sort of that's the journey that's it's not easy to accept because it's very hard to be Harden yourself or demand more and somehow being a little bit gentle with yourself is sometimes something that people don't do naturally yeah and I don't think enough Founders talk about that today you know there's a lot of LinkedIn posts around like lists people are making and there's a big rahah culture I think in startups which is I understand I mean we do it too like but I don't think enough Founders talk about just it is hard to build companies I mean there's a lot of just show off like being yes out there like oh I wake up at 4:00 a. m. and do an hour of yoga and then I drink a protein shake and I read the news for 3 hours on a whatever on a treadmill I'm like just give me a freaking break like yeah it just sets wrong expectations sh it's like that's not what I wake up and I wake up have 700 emails and like trying to get done yeah totally well thank you so much for sitting down with me this was awesome and really appreciate everything we talked through today it was great thank you man appreciate it cool