Jason Boehmig, CEO of Ironclad on Balancing Risk, Innovation, and AI Opportunity in the Legal Field

Jason thanks for sitting down with me we met a couple years ago now we both have shared investors in Excel and I think what I've been so impressed by is your founding story and the success that you've had with Ironclad last year you passed 100 million in ARR you guys are growing at a really strong clip and you guys are fusing AI into really every part of your product and what I would say is going AI native and I'm really excited to just sit down and talk to you about what that story and what that Journey's been like over the last what will be you said on Thursday 10 years yeah a couple weeks CP weeks 10 years yeah crazy to start I'd love to just hear from you about the founding story how you started Ironclad your journey what the vision was so I was an attorney uh I was actually attorney for companies like ironcloud um and I would just meet these Founders and help them with financing things like that and I just started automating some of my own workflow so the simplest thing was like incorporation docs they certificate of incorporation there's like founders stock purchase agreements and you got to like fill in all these numbers and make sure that they populate amongst different documents so I was actually in charge of the automation of incorporation for the firm and then we started doing some other things like open sourcing financing documents you may have heard of series seed documents which are kind of like open source seed financing documents if you're doing preferred stock and kind of this like theme of how can we standardize the practice of law how can we automate some parts of it led me to start tinkering on nights and weekends I hired a software engineer to tutor me for like 50 bucks an hour um and he would kind of like help me get unstuck on automation stuff that I was doing I actually didn't start with the idea uh that I would be a startup founder when I was doing this I was like I just want to be a more offic attorney who uses technology like the stateoftheart in my practice of law I actually really liked being an attorney but what I realized is like no one was building for lawyers like it was so widely considered to be a horrible Market that there was like no software companies that would even want to talk to you as a potential customer so that's kind of why I had to um start doing it my own and this the companies that were doing it were like 20 plus years old on like really archaic Stacks um yeah I was going to ask like did you look for software Solutions initially yeah so I that was my initial one was I just want to like use the best off-the-shelf stuff when I found the off the-shelf stuff wasn't good um or even current um I started automating it and eventually I realized I'm kind of the only person doing the specific thing which is trying to automate the practice of law um and particularly around like my work which was corporate law contracts work and quit my job uh with a bunch of student loan debt and started working on basic the continuation of the basic automation was fortunate enough to randomly attend a lecture at Stanford on like a Tuesday afternoon there was maybe a dozen people there and it was small enough where everyone went around the room and said what they were working on and I was like I'm a lawyer just quit my job um to focus on automating the practice of Law and contracts and I can code a little bit and then room went around and on the other side of the room was my co-founder Kai who said I'm a software engineer from paler who uh just quit my job to focus on like automating the automatable parts of practice of Law and I'm learning a lot about the legal profession so we kind of looked at each other and like did the Spider-Man meme lawyers have one tool which is Microsoft Word and software Engineers have like unit test they have like GitHub they have built all of these like great tools for working with text and so could we take some of the principles behind those tools and apply them to the lawyer work

I actually one time counted the number of clicks that it would take me to take one word file compare it with another word file and output a PDF that was named the way I wanted to was like 17 clicks and sometimes you're doing 20 30 red lines it's just like a huge waste of time so I made a script that would basically take one folder Redline it against this folder output into a new F folder with a naming scheme post starting the company before we even had any actual automation what we had is an email Ellas so it was like admin ironcloud doai and you would CC admin ironcloud doai on a transaction you wanted to do and then it would basically put that transaction on a dashboard and walk you through it so if I want to do an NDA with you I would say hey Dylan Let's do an NDA I've cced ironcloud here and then ironcloud like the AI would reach out to you and say hey like what's your company address like uh what's the email we should send it to and it would collect all the information and it would send you the docu sign at the end of the day and for me it would show a dashboard like collecting information from Dylan preparing docu sign signature request like storing file in your Dropbox folder of legal documents really cool and would just walk you through it I mean the email was of course me yeah so you were the AI yeah are you still the AI but I'm not the AI um Although our AI is named Kai after my co-founder contracts AI oh that's really funny and how's that changed in particular over the last

let's say two years as the more modern AI models and Tech has really started to work its way more into production yeah so the idea around ironcloud you know from day one and still continuing to this day is those two parts of the contract making the contract and then managing the contract after it's been made what we realized was workflows were a better way of making contracts than just a AI That's on an email Alias because even if that AI would get something wrong 5% of the time you really can't have that in a business contract like whoops we put the wrong information on page 37 and you didn't catch it sorry about that it doesn't really work so we had to be like 100% um accurate and the way we could get to 100% accurate on the contract creation was through really great workflows and of course like once you have a workflow at the end of that workflow you have a bunch of really great structured data incidentally adding workflows to this old school software category called contract life cycle management was helped us truly in like the business school sense of the word disrupt the category around 20 18 2019 Kai started my co-founder um started saying hey I read this paper when was the Transformer paper 2017 okay so I think like yeah 2018 we started saying okay there's a potential breakthrough here it's not necessarily usable yet but we need to start seriously thinking about the Ironclad a piece here because we're making contracts great in our workflows but if we can get the ability to extract from 10,000 PDFs all of the data and put them in our database as well that's going to generate a ton of value for our customers and so we started retooling with the idea of layering in AI specific Al at that point into just the data extraction the thing that stands out to me is that you had a clear use case you were trying to apply AI technology to and I think that's very different from where a lot of companies are at today and that they're looking at AI technology and they're thinking where can we apply this in our company in our product but I think with you guys it seems like you always had a clear area within the product that you're trying to apply it to and that really Narrows the scope and that helps you I imagine get something to Market faster has that been the case yeah I think we were pretty quick to get something to Market and I don't know like we do find that attorneys are pretty amenable to AI which I think like it kind of runs contrary to the popular narrative um I think we've definitely done a lot over the past 10 years to win attorney Trust which I think you know maybe has some analogies to other markets but like winning that trust you have to be really precious with it but if you have it you can do some interesting things and you can like have more of a dialogue with users that do trust you and I think that that accounts for kind of like some of the Delta between what we see in our user base and what like the General Industry sees there's something really interesting in

there that stands out to me which is like your vision when you started the company it sounds like was to be this AI native software solution for lawyers but back in 2014 the AI just wasn't ready you know you were creating classical ml models now there are a ton of AI native software Solutions coming to Market focusing on lawyers focusing on the legal Market you guys are sort of an incumbent where you're you know you're north of 100 million in ARR you but you're still a startup you're still a tech company you have a lot of tech capability but you you've earned that trust over the last decade with your users and with your customers and so how are you thinking about as a Founder as a CEO just balancing all this so that you're not too late but you're also not too early as you're thinking about you know like the more exotic AI Solutions you're trying to bring into the product and how you think about really transforming the product to be an AI native solution um an AI native product yeah when it's super competitive but like it's not clear if the Tech's really ready yet how are you just like balancing all this as a CEO I think one thing that one tool I use is just like thinking on a product by product basis and like what the end goal of a specific product is so with respect to CLM I think it's about making sure that we are using AI in every part of the CLM product that we can possibly generate user value in one of the interesting things about like starting with an existing scaled product that we've embedded AI into all the parts we can is it's let us get a bunch of data back from users around like what they like where they see the value and it's also led to the ideation of new products for us how risk tolerant are you as a company with testing these features out across your user base across your existing Flagship product to learn I think I would just distinguish between like um risk tolerance that we have as a company and like to put an experience in front of our customers so like I'm comfortable taking a lot of risk and I think like that's one of the benefits of a founder-led company yeah but I don't want my customers to be taking any risk in using the product right so you know there's a couple ways that shows up uh one we're like rigorous in testing so you know there's been a lot of noise around AI products and stalonea products and legal space and we have really focused on our Flagship product um in terms of the public narrative we are in rigorous testing and have been for some time around a standalone product like a standalone AI native yeah exactly but like you could say we've been slow to launch that I would say we've been we have a really high bar for what we're gonna put our name behind yeah um in the industry and your question on that yeah how much tension do you feel at this stage of the company from investors from your team on responding to the noise in the market on getting something out there versus this thoughtfulness that you're taking in really want to be thorough we don't want our users to have any risk yeah I'm sure they're asking for it your users but you're seeing what the evals are coming back like how much tension is there that you're facing as a CEO right now you know I think a lot of that tension has been alleviated by the fact that we do have a CLM is this natural place to put a lot of AI whether it's that first draft AI review or the data extraction it you can look at the ironcloud product and just you will naturally experience a lot of AI and that's had business benefits for us like we're growing at a good clip like you know the unit economics of the business are improving quarter over quarter all the stuff like the investors might otherwise be pressuring on I think the AI goodness uh from layering that into our existing product has accounted for that for me it's about is like making sure we don't get complacent with just that because I really do think we have to disrupt ourselves um and you could say like launching new AI products is actually a really risky thing for us to do because you know who knows it could cannibalize our existing product line if it's really successful um and it's hard to predict how that's going to play out in the market but I think that's the kind of stuff we have to keep pushing the boundaries on and making sure that we are taking enough risk how do you as a CEO think about you are an

AI company you know the company's Vision was to be an AI company you are an AI company but how much of the technology is strategic for you to develop yourself versus the application of it is the Strategic yeah angle I'd say our current view which seems to roughly match with the like large model providers is that the verticalized application stuff is things that we're going to need to be uniquely good at and we're going to develop our own proprietary stuff around whereas the foundational capabilities um we should stay on the latest state of the art and invest in anything that allows us to stay on the latest state of the art I'd say like specifically with respect to the foundational model companies our approach is to constantly be evaluating that and you know like we rely on open aai we do some stuff with Google anthropics in the mix so we're not all in on any one of them and we want to like have a sophisticated viewpoint on what each of the models do best and then apply that in our application because we are really covering a wide variety of use cases it's everything from like a commercial contract negotiation to summarization of like 270 pages of m& a docs to extracting data from 10,000 contracts and they the skills that the models need to have in order to do those different tasks can vary how does that compare to what you're seeing from competitors you know I think some companies are saying hey we're going to go take llama 3 we're going to fine-tune it it's going to be ours that's our competitive Advantage you're taking a more balanced approach it sounds like which is you know some stuff you're going to develop yourself but you're going to rely on the capabilities from other companies developing these Foundation models to just get better and be quick to leverage those and evaluate those have you seen over the last 18 months 24 months um you guys actually be able to move faster than companies that have said we're just going to develop everything ourselves yeah well I'd say like this is where like the technological landscape is changing so rapidly so I'll give you a good example which is um case law right like you don't want to make up and hallucinate uh a case statute um and it's really important that there's a almost no level of hallucination on that and you want to be pulling from the actual cases and citing them correctly is something that like isn't that relevant to a contract life cycle management platform but to a general AI product pretty relevant I'd say like nine months ago the best way to do that was to go to open aai pay open AI a lot of money strip out the Reddit trading data put in Delaware case law data and you didn't hallucinate anymore um the problem with that is that model stuck on gpt3 and it doesn't have GPT 4 40 you lose all of the benefits of the main branch and this is where like having a great technical co-founder like Kai is a huge Advantage because while that might have seemed like the right approach what nine months ago Kai was like I don't think that's going to be the right approach in a year and really there's going to be benefits from staying on the main branch and what we found is like one the hallucination level is so much less on GPT 4 than it was on gpt3 but two you can there are techniques that have evolved like um you know application of rag that help you supplement and make sure that you're not hallucinating the case law and if you have that Delaware case law um database being on GPT 40 plus some customization plus rag on your good data set and the data sets have been really improving um in the legal field over the past year Harvard just put out a incredible um like clean database of case law um in the past couple months those convergence of like data getting better um technique weeks and the foundational models have been produced a much better result than the approach from 9 months ago of kind of forking the model that's crazy that I mean nine

months yeah is a very quick time frame to shift like technology strategy have you seen competitors take different approaches though and say like we're just going to go we're going to develop everything ourselves there's this like AI rapper um framing that I think was wrong last year that emerged like oh you know this company is just an AI rapper overx yeah um and so I think that pushed some companies to say okay well we're going to then go develop everything ourselves because we want to end to endend develop the technology stack and all the AI models I'm sure you've seen some companies take that approach in your space yeah how has your technology strategy which is like just stay on the main branch be creative focus on the applications and the user value um fared out would you say compared to the compe I'd say like from a strictly technological Viewpoint and results driven Viewpoint extraordinarily satisfied like 11 out of 10 on that one and again huge credit to my co-founder because I would say as the non-technical co-founder I was like H that sounds pretty good like to be on your own fully custom trained you know open AI provided model right and K had a very strong opinion on that um which turned out to be totally right I think what's interesting is that still hasn't fully played out is the marketing angle around that I mean the Market's discovering it when you look at results of one company versus the next but that's taking some time to come out and I do think there's a marketing message which is we have an llm that's completely custom trained for you that does land it sounds better and sounds better yeah um so I think the market is still playing catchup to the reality of the results but and it's going to be intered to see how that plays out I think what's tough for earlier stage Founders too is that story also sometimes can sound more compelling to investors when you're right like Hey we're going to go create a specific model for X um but to your point that might have an advantage for only 3 to 9 months yeah in and then you're worse off because the main branch to your point you've now forked yeah you're not getting all the updates from the main branch is getting better and better migrating customers off that difficult yeah so something we were talking about is Enterprise adoption of AI has actually gone down year-over-year when you look at applications that have made it into production and so I think figuring out okay are you replacing lawyers are you fusing AI in into workflows are you doing something in between is a question that a lot of companies that are incumbents and I hope you don't take offense by me call I like I'm taking both we're an incumbent and a start so incumbents are trying to Grapple with how for do you go do you say Okay Ironclad is now your AI lawyer versus Iron clad which you know and love now is AI native and can make you 10x more productive Ive than what you're used to because we're pulling the latest AI technology into all parts of it which again from a positioning perspective is not as exciting as this company over here that's saying we have an AI lawyer come use it it's going to completely automate away your need for a paralal or a general counsel which like I don't think the technology is ready for that yet yeah and so H how do you just think about this and manage this these Dynamics yeah I'd say a couple things so one is this is where I do think it's really important to have like a multiproduct strategy if you're an incumbent and you can't lose the direction of your main product like there's so much momentum behind that if you're an incumbent like it's an existing category um you should have ai everywhere but like don't lose sight of the big picture on your main product like pull AI into it yeah and I think this kind of goes to some of the like jeffre more crossing the chasm stuff where I think there's like four zones that uh the company operates in and can have different zones operating in the same company at the same time like you have your performing Zone where like things are working and you're really about like getting into the unit economics and um the performance of the business and the product um and then you can have like an incubation Zone which is where you're trying out new ideas and those can be judged on totally different things and I think it's very important to not lose sight of that main product but also be making those incubation bets some of which are going to pay off just going to get you more information and I'm curious just when you guys talk about your technology strategy how much do you care about like how you're achieving that Delta in performance versus like the fact that you just are going to have a delta in performance and that's what's important I yeah I don't think we care at all actually so I think having that Delta in performance in our very specific vertical of understanding and recommending contract language um that's like where we feel like the 10year sustainable Advantage is and we've got unique viewpoints and data that we can use to sustain that for some time but as to how we get there doesn't matter um but it I do think it's critical for our success as a company to make sure that it is there we did talk about how

Enterprise adoption of AI at least according to this report that Bane put out especially in the legal profession when you look at adoption in production is still like single digits from your perspective you're probably like one of the you know uh most knowledgeable experts when it comes to AI adoption of the legal profession especially in the Enterprise what does it take to get to 100% like what still just fundamentally does not work well enough features capabilities that are just still not possible today and that are limiting that adoption I think it's less about features and it's more about sophistication of buyers and getting them comfortable with a new ways of thinking about data and so you know obviously like we have zero training contracts with all of our providers such that our customers data is not being used to train any foundational model and isn't even getting stored there um and I think that's part of the reason because we do have a very sophisticated contract based uh story around the data um and how we are protecting our customers from uh their data winding up somewhere they don't want it to wind up that to me is where the Gap is um at least in the Enterprise is it starting to Grapple with where and how data is used in generative AI applications and having a lot of discomfort with that so you think technology is good enough today it's more about customer education getting comfortable with data security yeah and I think that explains like the gap between like what we see which is a lot of AI adoption and what the industry reports are is there's like we're one of the only if not the only scaled company in legal technology um that is in corporations and I think particularly in Enterprise there's just no chance of a series a startup getting the legal team comfortable they just haven't been around a long enough there's like not enough funding there's not enough like sophistication of the data report and legal is very sensitive to that interesting and it's only because like we already have a product in there and we have like RI stuff is built on the same architecture and it's more of an addendum to the existing work they've done than a net new process that maybe enables us to show up in a different way but I think it's going to happen like more companies will mature it will get more comfortable there's going to be more of an industry standard I'm curious as a final

question the last 18 months have just been crazy like every company what's been the biggest thing you've been surprised by over the last 18 months I mean it sounds obvious but pace of change like it's just amazing thinking about to nine months ago feels like almost as long ago as starting the company which was 10 years ago like totally different state of the world um and you know I it's also giving me a ton of energy as a founder and as I talked to other Founders who are kind of like 10 years in It's a Grind you know uh to be a Founder but having this like boost of energy around absorbing new information every day trading notes with other Founders getting Hands-On with a new product that's been really fun so that's the real benefit of all this to me is as a person is Boost of energy really cool yeah well thanks again for sitting down this has been awesome really appreciate you doing this thanks for having me