How Notion Built the Best AI Agents For Work (Full Tutorial) | Akshay & Ryan

I've tried to get my wife to use notion for 7 years. 7 years I've been trying and I've been unsuccessful until 3 weeks ago — going and clicking through all these tabs and doing the web search and writing down the scores and whatever. Like it's tedious work. — She didn't want to deal with all the database the doc stuff. Now for her, notion is just like a chat thing like she just chats with the agent and the agent just does all the work. I have completely moved away from writing a doc or filling up a database to essentially just talking to the agent like 90% of the time. I want to send like Peter agent and Ryan agent into a Slack channel to align on decision and then just give me the report afterwards. — I'm actually excited about that world because if you think about it um okay, welcome everyone. My guests today are Ash and Ryan, Notion's co-founder and AI lead. Really excited to get them to demo Notion's new AI agents. I tried them last night and they're like super impressive. And we'll also talk about I'm sure the long journey to build them and also how teams on notion use AI to get work done faster. So welcome guys. — Thanks for having us. — Yeah, thank you. — Let's dive right in. So actually this morning you tweeted that uh you get text from friends saying you know holy the new notion AI agent is actually insane. So uh you know can one of you show us this in action? This is insanity. Yeah. While you do that, I just like uh you know, I think Peter, you probably empathize with this, but I think there's like a few taste makers out there. I think these taste makers like I look up to this release, this particular release, I feel like I get a text from one of these people almost daily about how they have rediscovered notion and the AI and the agent and how

they sort of been blown away by that. um which to me is just like you know extremely um I think it's is makes me so happy uh that we've finally cracked the code. Uh it's been two plus years in the making and we do feel very proud of the new release we shipped last week. — Yeah, it's like super impressive. So Ryan, you want to show us the agents in action? — Let's take a quick look. This is the agent. This is notion AI. Um we it's available both in this like full page context as well as on individual pages. Um, and so with the agent, you can basically do uh anything that you can do in notion. So let's say um make me a database to help track and review movies I watch. Um, and so here the agent is just going to figure out what I want and then map that to its capabilities within notion and get to work. So here we're making a database. We're seeing the different properties that it's creating for me in the database. — Mhm. — Your status review. — There's some context about what movies you like to watch some somewhere in a notion doc or — uh no I'm like starting completely from like a blank slate here. — Um and I'm going to jump in. I think this is one of the things that I like the most about the agent is that it kind of helps with like the cold start problem. Whereas before, if I'm like, "Oh, I want to track movies. " It's like I've got to start with like a blank database, which is powerful, but also can be like intimidating. What do I set up? What do I track? What properties do I use? Um, and so, you know, here I'm just asking it to like make me a movie tracking database. And the entire thing's set up not only with properties, but with different views. And I could go and let's just say hypothetically I've got a bunch of content in or I've watched a bunch of movies this year. And I could even just ask it like uh drop in a bunch of example movies that I watched before just to give us a tour of everything it's able to do. — Yeah. Cuz otherwise this will take a lot of manual work like making blocks and editing the columns and stuff. So — totally. The other thing I love about it is that uh it works like real time. So as it works uh it's actually able to like stream its changes in real time. So I can see it write in real time. fill in a database in real time. So here we can go see it's actually filling everything out. — It actually has web search functionality, right? Or — it does. Yeah. So let's see maybe we could go and add let's open up inception and be like uh we've got this um concept of context here. So like as I open the page this context is updated. So now it knows that I'm looking at this inception page and I can say um look up this movie on IMDb and put uh the critic rating in the page. What movie should I use? Oh, use conception. One of the things that's changed my usage of notion, Peter, over the summer is that I have completely moved away

from writing a doc or filling up a database to essentially just talking to the agent like 90% of the time. — Um, and so even for things like filling up a database like let's say you wanting to add three rows, you can definely do that like you know one by one manually, but it's so much easier to just do that uh in the agent, right? Yeah. — It will figure out which database you're talking about. It will add those movies. It will fill up all these properties. It just works in the background in a way that is so much more usable. — Yeah. I think that that's what AI is really good at. Like it's really good at saving you from the painstaking manual work like that. It's like, you know, people think it's like magic, but it's just like all the manual work that I don't want to do. Just like get AI to do it. — Totally. Yeah. I mean going and clicking through all of these tabs and doing the web search and writing down the scores and whatever like you know it yeah it's like it's tedious work and here you know I just asked it to look up the movie here's everything that it searched aggregated everything threw it into rating done can and now I could if I wanted to you know go and basically ask it to do it for everything in this database um saving like enormous amounts of time my uh wife is a wedding planner and I've got her very excited about notion AI and um they have to fill out lots of information about venues and capacity and contact numbers and stuff and this gives them you know a huge head start on filling in a lot of that information. — That's awesome. — Funny like Ryan talks about his his wife. Uh I've tried to get my wife to use notion for seven years success seven years I've been trying and I've been unsuccessful. um until three weeks ago. — Three weeks ago she I introduced her to the agent and she like two weeks ago actually she became a paid member. — Wow. — Because I think for her like you know she didn't want to deal with all the database the doc stuff. Now for her notion is just like a chat thing like she just chats with the agent and the agent just does all the work. Uh so she's actually setting up like a play at one of the San Francisco parks. And so she said, "Hey, I'm going to set up this play this park. Uh, what should be my plan? " And because it has this web context, it sort of was like, "Oh, if you're going to this park, you need this permit and you need this insurance and I created a plan for you so you can track all of them and you should get it done by this date. " And so to her, it's just like, okay, this whole concept of being organized and being um like really sort of productive and effective, everybody wants to do that, but there was a cost to that. Like there was almost like this people felt like oh that I like that but I don't want to do all of that things and now you have an assistant which basically just doing all that work for you. — Yeah, exactly. I think this is a game changer. I um I do a lot of like copy and pasting from you know chat GBT and cloud right just to like you know I generate some docs and stuff and copy and paste over and now I don't have to copy paste stuff over and there's also these like browser tools that uh the these uh big lab shipped where you know they have but they have to scan the whole document but I think because you guys have trained it with like all the little blocks and stuff it's like way faster than like a browser scanning the whole document and just like taking forever you know — and I think the cool thing is like what Ryan showed here is like now you have a system to track your movies, right? Like it has filled up these things. But now Ryan as he spends the year watching movies like you know he'll add a bunch of different things to it. It'll file it and it'll just get to know Ryan more and more which then becomes the context for all future queries, right? And so it's in some ways like it's saying we're using the same models but the context is what makes it different. — Okay. So, so this thing has context of everything every doc and stuff in the project here, right? — Well, it's not even just necessarily the project or database that I'm looking at here, but you know, I could be working on this movies database and say I've also got um a collection of scripts or uh theaters, you know, in a separate database or just kind of like lingering around my workspace. I can easily pull those in either via search or by manually mentioning them. Um, and so it's like you not only have the concept or the context of the thing that you're looking at, but you have the context of like all of the content in your entire workspace. Got it. This episode is brought to you by Composio. Most of the paying shipping AI features doesn't come from the model, but from the integrations. You're dealing with messy APIs, fragile tool calls, and hours lost to debugging and figuring out why something broke in Slack or Salesforce. Composio gives you one SDK to connect to 800 plus apps like Slack, GitHub, Gmail, Jira, and they built the Rube. app that plugs all of those apps straight into an AI chat. So, you can just ask pull me the latest metrics from mix panel and update linear. Now, instead of fighting with APIs, you can focus on actually shipping faster. Check it out at get rube. link. link/ Peter or the link in the video description. Now, back to the episode. — All right. Well, can you show like another agent? I see you have some agents on the left there. — Yeah. I'll show you a couple more things. So, the one other thing I wanted to show which is kind of my our fun demo is Yeah. I could just I want to show you all of the You mentioned blocks earlier, and I just want to show you like all of the things it can do with blocks. Um, and so I'm just going to say write uh bunch of sample content using a ton of the blocks you have available. Make it fun. And so now we're just going to let it give us uh a tour of all of its like capabilities with blocks and writing. And this is another thing that like sometimes I struggle with notion is like there's so many things like how do I make my documents like look good and like I'll get hung up on like which color do I use or like which emoji and do I use a toggle? Do I use columns? Like I don't even think until I started working on notion I realized you could do so much with like formulas and equations like this. Um and so — there's a lot of different blocks. Yeah. — Yeah. YouTube videos. We've got audio blocks, we have tables, colors, code blocks. Um, and so not only is the agent able to like see all this stuff, but it's able to write it for you. Super fun to take, you know, we're talking about context, all of this context, um, that you've got scattered around your workplace and then ask it to create artifacts. I do this all the time um when working on notion is anytime we'll make like a product decision or a technical decision. — Th those decisions may happen in meetings that we record with AI meeting notes. They might happen in Slack where it's really kind of uh unstructured and messy and I'll just go into notion AI and just be like please write like a decision document about this thing and it goes gathers all that up and just writes a really nice artifact that I can reference you know forever wondering why we did a certain thing. — Uh can you open I'm curious about the toggle for secrets thing. Can you show me that? — Yeah, let's see it. What do we got? — Okay, it's a riddle. Okay. — Okay. It's got a sense of humor. Yeah. Yeah. — Actually, what Ryan said I think is worth repeating in some ways like I think the reason the agent is really resonating for people is actually these two sides of the same thing. It's like agent has this um maximum context right like it can look through your notion but it can also look through your slack and Google drive and Microsoft email and Jira and soon Salesforce and so forth right like no human can actually keep all of that context uh like I cannot tell you which Jira ticket is connected to which Salesforce customer but actually agent has that on one end and then on the other end we've now trained the agent to use every possible functionality of notion, right? Like most people use notion as like a simple doc or like a simple database, right? But now it knows all these other functionality. And so now you have this agent with full context of the information and full capability. And some of the things it does like I feel like surprises Ryan and I every day too. It's like oh my god like I didn't know that was possible. Like it can structure the documents a lot better. it can like create um database connections a lot better, right? So like I've been working on this for so long and even I'm surprised every day of all the things that it is able to do. — Yeah. I think one of the things I put notion on the map is all these power users making templates, right? Like you know that look beautiful and like I always wonder like how do I make this stuff and hopefully with this agent now I can make uh similar templates to run my life. Actually, it's a good point because we when we met some of these community actually Ryan and I were on that call uh when we were introducing uh our community of ambassadors and consultants with the agent stuff and I'm not going to lie, I was a bit nervous going into that call — because you know so many people build these systems — and that's their business like their livelihood depends on them — and I was worried that if we introduce the agent like will they feel like this is a bit of a threat to them and to our surprise pleasant surprise like you know they are so jazzed because it turns out people don't want to do that busy work, right? Like they would much rather focus on the creativity and like the ideas that they want to like execute against — as opposed to trying to like fill up a database and show what's possible and so forth, right? And so, you know, we talked about this in our keynote, but we really want agent notion agents to — take away all the busy work and so that you can focus on your life's work. — I think this is not mission statement, right? to help you focus on your life's work is beautiful tools for your life life's work. — That's right. — Let's take a step back actually like maybe we can stop the demo like I'm really curious uh why you guys bet on AI to achieve this uh mission you know like uh because you know AI has a lot of complexities like it's nondeterministic cost a lot of money like why bet on AI for two years or three years to make this happen. We spent like a decade plus you know building a modern productivity tool. Um and I think for a long time we actually probably never even thought about AI. Um even from like the more sort of like the legacy sense right in terms of recommendations we could be giving you and so forth like I don't think we had that. I think it was around the time that GP4 we had early access to GP4. Uh, and we prototyped what it would mean for you to like complete a doc or edit a doc or improve a doc. And I think as soon as you saw that even the first prototype, I feel like I think light bulbs went on for a lot of people in terms of how AI is going to affect knowledge work. — Yeah. And you know, we've gone from slowly but surely transforming how you write a doc to how you build a database to how you search to move to answers to how you fill up your database with autofill and so forth. Right. So we've been slowly teaching the agent like one by one all the different tools you can use. — Mhm. Uh but I think the big breakthrough for us happened a year ago with all these um with especially with the sonnet model and and now GPD5 where they got so good at um tools calls right so I think they just know how to use tools a lot better and so I think initially probably we didn't know we I don't know if we could see where we are today but I think the bet was that knowledge work is going to change from here on out and Um, even this the first product which was the AI writer which was I guess you could look at that as like a GPT4 wrapper uh did so well and the reason it did well is because it's integrated where you work like you know people are willing to like they don't want to copy paste the context and copy paste the outputs and all of that stuff like people really enjoy being able to use it integrated to sort of where people work and Um, and now I feel like it's like, oh, the possibilities are sort of, you know, really multiplied with what all the things you can do with it. — Yeah, it's like a big step up from the previous version, right? Cuz the previous version I remember was just like, you know, you can summarize some text or like you can, you know, edit some copy and and uh it's like a big step up. So, so maybe you can talk about uh this two-year journey you've been on. You mentioned before we started this call that, you know, there was a lot of failures and obstacles to make this happen. Maybe I'll give you like the my high level my slightly higher altitude view of like the last 18 months and then I think Ryan can actually tell you the details of the last six nine months and how we actually took it to market. Um but this particular thing I think we we basically tried it many times before. I think we tried it once uh 18 maybe 21 months ago uh early last year. Um, we were working with one of the companies to do like basically like custom models that we can be trained autof into like using notion better — and we spent almost 6 months on it and I think it got reasonably far uh but it wasn't like reliable enough and uh we had to basically press the reset button because the models were not there. Uh then we tried it again earlier this year uh where we felt like oh the models can naturally now do it but I think we actually like overshot the ambition of what we wanted to do. We thought we could I don't know create all sorts of new software apps where people will go do that thing because that was the thing that was really sort of um in the zeitgeist. Um but I think about 6 months ago we decided to be a little bit more focused not on just like building like you know apps that are kind of look like toys and kind of looks like a little bit of like AI slop and got really focused on like actually helping you do your work right it's less about app building and more about like how do I help you write your doc or manage your projects like help you do the thing you're already doing in notion Um and I still remember um you know one of our colleagues Max joined us about 5 6 months ago and he said you know what we're not going to do anything except we're going to come up with these five things that we're going to tell the agent to do and it should do that reliably until that happens we will not move to the next thing. — Mhm. Uh, and these were simple things like insert this text in this paragraph or you know move this thing to this thing and every week we just try to like step by step get it to a point where it actually understands all of notion. I'll pass it to Ryan to share more details on that too. Yeah, I think one of the big things that we learned over the past nine months um is to kind of we say this in our team a lot is like to channel the model and just do what the models are really good at doing. Um, and one example of that is when we first

built some of our page editing um, within notion, we would try to essentially take the block data model and represent that to the LLM and give it a bunch of instructions on how you use this and we represented it as a bunch of JSON and here's what you can do, here's what you can't do, here's what a call out is, here's what a list is and like these frontier models were okay at this, — but the output was really kind of unnatural. Um, and one kind of light bulb moment was like, wait a minute, these things are trained on like heaps of markdown and written content. Like why are we representing Notion internals to the LLM when we should be representing Notion as the user sees it which is a page which with written content and presented in a way that the LM would be really good reading understanding and most importantly writing and so like you know not to get into too many technical details but we just decided hey let's represent notion pages as markdown because models are really good at understanding markdown and writing markdown. And so it made the actual like model tool call is extremely simple. The hard work is then on us to figure out how do you translate this really simple model input into blocks and notion world essentially. And then we did the exact same exercise for databases, for views, for all of this different all this different stuff that you can do in notion. But the result is that the models are extraordinarily good because they have all of this um information in their training data already. They're extraordinarily good at interfacing with notion and then we have to do the hard work of like kind of building the bridge. — I see. So the model is kind of just like reading and outputting markdown and then you guys change into blocks and stuff the user actually sees. — Yeah. — Yep. — And it sounds like from actually and I think a lot of companies do this. They're like, you know, oh like I don't want to rely on cloud or GPD. I'm going to use open source model and I want to like fine-tune it and do all this stuff. And it sounds like that didn't work out, right? It sounds like just using the latest model is the right I at least for you guys the right path. At least for us, you know, we very much think of ourselves as like applied AI company and we'll use the best model. — Yeah. — Um, and actually I think kind of what Ryan said, which is just sort of like I think actually Ivan talks about is like it's kind of like brewing a beer. It's like uh you know you have to just like just like stay close to it and like you just have to like make tweaks and figure out what you know it's going to do the thing that and you just have to figure out okay how do we make that work for like the thing that we want it to do right and also reliably. I think the thing I'm most proud of uh with the agent stuff is that actually there's a fun uh anecdote where uh like about 4 months ago, you know, this the same product actually had version control. You could go back up steps — uh and we needed that because it was fairly unreliable and you know was doing a bunch of things that you don't want it to do. And at some point it got so reliable that people were like, "Wait, why do you have these like version controls? Like, you know, it's so heavy, like I don't need that cuz I now trust the model to go do it. " Um, right. And so I think, um, I think it's taken us a lot of, you know, again, a lot of effort, but we're very excited for what it unlocks for the world of knowledge workers. Is there any like uh like you know top three or whatever product principles or lessons that you learned on this journey when it comes to building an AIA agent? It seems like everybody's trying to build an AIA agency these days but like you know what does it actually take to build a good one? I do think um what I was saying about channeling the model, I mean that the simpler you can keep your this actually leads into the other thing um that I would say is you have to build essentially an interface for your model. Um, and so the way an LLM sees air quotes notion is through text, but the way a user sees notion is through obviously a like a graphical interface. And so we have to pay a lot of attention to the way things are described and represented in notion's UI and make sure that is how they are described, framed, and presented in text to the agent. And the reason for that is because the user is going to say things like build me a linked database to track movies and the agent needs to understand what these terms mean uh relative to notion and relative to where the user is within notion. — Yeah. And so the more we try to um present implementation details or internal data structures and things, the more the agent's going to have a hard time like mapping like okay they said database but that maps to we call them internally collections and it's like okay what database here but then collections over there and then similarly almost like the way we render a page to the model needs to look kind of like we would render a page to a user. So you have title and you have metadata like properties and then you have the page content just so the model has everything available to understand and figure out what the user is asking it to do. So basically like giving it like very clean data, giving very clean context is like really important, right? — Yeah. There there's this term in computer science there's a field called uh HCI, human computer interface and I do think that we've like entered this new world of ACI like agent computer interface and you have to like pay a lot of attention um to what that interface is and then additionally like how you manage the context that you are also giving it. — Yeah. as like, you know, I just went out walk my wife and she started telling me about, you know, all her stuff at work and all the acronyms that she uses. I'm like, "Hey, I'm trying to prepare for this interview with no notion. I can't be thinking about this right now. My context window is limit limited. Tell me about stuff afterwards. " Yeah, — that's exactly right. — Yeah. So, this AI can do a lot, right? It can create databases. It can edit documents. It can search the internet. Uh and and let's talk about like how you like because it can do so much. It can be pretty complicated. Like do you run evals for each one? Do you try to build each use case at a time or like h how did you talk about quality like how did you make sure that this thing is actually good enough to ship you know for each use case? Yeah. — Yeah. We have a pretty big war chest of evals um for the entire agent product. Um I don't know exactly off the top of my head but thousands of individual samples and um uh use cases. And when we we're at the point now where if we make a change to any of our like prompts or documentation or tools, we have to run the entire suite. Um because little tweaks and changes to instructions over here can have this strange like it's almost like a butterfly effect uh to the way the agent performs. Um and so yeah, we have to be really careful about that. Now we and we started very simple. It was like the user asks to insert a paragraph, run the eval, did the paragraph get inserted? Um, but it's evolved since then into like fairly complex um series of um actions and things with huge amounts of context and data and you know still making sure that the thing performs. — So you have a bunch of like LM judge valves running at all times. Yeah, we have a mix of like LLM judges and like manual scoring and all sorts of stuff. It's a kind of a blend of deterministic um tests which you can almost think of as like unit tests and then yeah like LLM judging — and like each time you guys change the prompt or like the rag or you know what whatever you got to run the whole thing, right? You got make sure nothing breaks. — Pretty much. Yeah. — Yeah, pretty much. — Got it. Peter, are you asking this because you're getting us into the evals debate that's happening on X? — Yeah, I mean there's a lot of debate about Ethos. I I do think with this EVA stuff like uh like some teams can get too scientific too quickly, right? Like they don't even have product market and they're just like, you know, they're in the beginning and they're like building all these fancy evals. I mean, do you guys have any advice after building all the evals? Yeah. — Yeah. I have a couple opinions. I mean, one is I completely agree like I don't think like jumping into evals is um what anybody should do first. Like these LLMs are I feel are so like viby that you just got to like try it and use it and see how it feels. I mean, we we've had instances where um we're checking out new models, we run the evals and the numbers go up, you know, one or two points and we're like, is that good? and then you use it and you're like, "Oh my god, it feels so good. " But that feeling can't be just captured um in an eval. And the other thing I think that we've learned through our like eval process is the evals are only as good as you make them in that if you make them really easy and you score 100%. and then new models come along, then like how do you actually know if the new model's better or it's changed anything? And so we we've kind of built two sets where we have almost like a golden set that are like these have to be 100%. Let's make sure that the LM can always perform these tasks and then we have a set that's like super challenging and we're like we expect it to fail, I don't know, 50% of the time. then a new model can come along or uh a new technique or we've you know tweaked our prompt or something. We can run it against the hard test and be like, "Oh crap, it went up five points. That's pretty interesting. " Like this is better. Like uh but if we we did start with like just the unit testy sort of evals and then we would make changes and we're like it's already 100%. How do we know if this is better? Can you share an example of like a hard test like does it know actually it's taste in movies or what's a hard test? — Typically the hard tests are going to be things like um our like database and view schemas are the things that are the most complicated. Um, and so — building like really complex setups of a database with like properties that might have relations between them and then building a view that has sorts and filters and groupings and all these sort of things. Um, and also like the instructions or like the user input that you give it can be kind of vague. So like seeing if it can perform well at that I think is usually the hardest challenges for us. — Got it. And as the product lead on the project, like how often do you look at the eval dashboard versus just looking at the actual conversations people are having with the agent? Like do you check the conversations? Yeah. — Um we collect anytime um people hit like a thumbs down. That's like kind of our lifeblood um for the chat is anytime especially like uh internally if anybody runs into a weird um chat experience, we're like please click that thumbs down. we get some debug uh logs and diagnostics and then we'll jump into the chats. Um and we do keep tabs on kind of the general we'll like report trends or um sentiment of like what people are running into and what things might not be working. — Um on the flip side, we run the evals pretty regularly and report the results, but the numbers are just kind of like the numbers and they don't change very much. So I I lean much heavily uh much harder on what people are saying. — Yeah. Or just like looking at 10 random conversations and see what happens. — Exactly. — It's like the it's like the NPS study. Like the number is fine, but like the qualitative comments probably are more much more useful. — Yeah. — Than the than your NPS number. — Yeah. You know, like I tweeted like uh the other day. Like the more I spend time in product, I I just want to like, you know, get stuff in front of customers and just get feedback instantly. And like the more that spins, the better the product gets. Like I don't really believe in all this like planning and all this other stuff anymore. Maybe I'm too hard on the other side, but like I just want to get feedback, man. Like just give me feedback. — Yeah. — I like talking to you guys because you're one of the few companies that actually care about craft and quality a lot. Like probably even more than the numbers. And um but I guess for AI like craft and quality is kind of like you know what's the accuracy or like what is acceptable thing right cuz this is nondeterministic. So for someone who's like a professionist it can be kind of annoying like how do you how do you account for this disconnect like how do you make AI high craft? — Oh there's so much involved. Um, — I think that on the AI side, like I think we talked a little bit about just like, you know, is it doing the thing that you're asking it to do and is it doing it reliably fast enough? And I think those things could be a little bit more quantitative. But even though like every tool now has a chat tool like the amount of craft that goes into figuring out the interaction uh and making people feel like you know they understand the work it's doing and make sure that it sort of like moves from functional to delightful like that bridge is always like filled with uh a lot of iteration a lot of like effort. — Yeah. Uh maybe this is also a good segue to uh show you the personalization piece because that's an a that's a like a small little thing that we introduced that actually has pretty deep implications in terms of people's usage of agent in the future. — Yeah. Show it to me. Yeah. Mhm. The one where you can put the hat on and like the dock and stuff or it goes deeper than that. — It goes deeper. Yeah. — It go it goes deeper. It's cute, but it has, you know, in terms of like the agent knowing you and doing the things you want it to do, like this is also its memory bank. — Okay. Yeah. So, I have like a this is like a personal notion where I

I'm tinkering on um a little app and kind of experimenting with Notion AI and the Notion MCP um and how the Notion MCP works with tools like Cloud Code and Codeex. But one thing I found myself constantly doing is I was like, "Oh, I'm like asking it to write tasks in this database and I'm always like starting with a blank slate and I've got to like load it with all this context about like here's this project and I've even added the GitHub connector and that like kind of works so it can read some code, but I'm like man I just want to tell it like when I ask you to make a task, here's what I want you to do. " So, I built um this is my custom or my personal personalized agent here um called Hank. Um I've got a little construction hat, but the most important thing I've got is these um uh instructions for the agent. And here I'm saying like you're a coding research partner. Like I want you to be excellent at specking out plans. Um you're good at architecture, UI design. I even actually used notion AI to write these instructions using the GitHub connector. It was like a very meta experience where it was like AI all the way down. Um and then I even as I built a asked it to build a task template so that anytime I ask it about um writing a task it knows exactly what to do and then even like what codeex is which is the um CLI tool that I'm using currently just so it can write tasks in a way that it um they can kind of interop and so if I go and I ask it to write a new task see to update to the latest version of react It's going to be able to pull in my personalization page, — understand the context of the project without having to do any extra searching or anything. Oh man. Yeah. Wow. Really did that fast. It's funny how that can like impress me. Uh even though I'm like building the thing every single day and then Yeah. Uh it's got our goal. It's got context. It's got the plan. bunch of stuff that are super relevant to the project and then I can take this task and drop it into codeex which is connected to the notion MCP and then it like gets to work. But this way I can like you know skip a whole bunch of like here's what this project is and I'm using this version of React and I've got these dependencies. They're just tracked. Now, what's really fun though is that this is just a notion page. And so, uh, I could just literally ask it to update its own memory about things. So say a task is finished or uh we upgrade from React what is it 18 to 19. I could just ask it in here be like update your memory uh to know that we're using React 19. And so this becomes my like memory page essentially. Um and this is just Yeah. — Oh here we go. uh update scout — agent. — This is like the prompt, right? This is the prompt that you're using to — this it's just a prompt. Yeah, it's literally just behind the scenes injected as like context to the prompt. Um and because it's a page too, I could just drop down here. I could write an inline database. I could have a preview of uh inline uh what do we want to do? Linked view of data source. I could give it a linked view of tasks in here and then filter it to let's say status is to-do. And so now when I ask it to do things, not only does it have all this content that I put up here, but it's also got access to the task database. Um, so it's just a way to kind of like give your agent a bunch of context. And I've seen customers that have built like shared um instructions pages where they can kind of hot swap between different instruction sets. Uh and these are like shared amongst the team. And I've seen people put in uh a list of like skills or actions so that if I ask you to do X, you'll do Y. Um, and so you can kind of again just like build up this catalog of instructions and personalities and behaviors. — Yeah, I might have to move all my uh chat GPD projects in here now. So because most of my personal family stuff is saving on notion and uh you know it was kind of nor copy and paste into chat GPD. So this functionality exists. — Exactly. Yeah. — Yeah. This is really cool. And you guys are launching custom agents at some point soon. — Yeah. I was going to ask if we uh if you want to take a quick look at some custom agents. — Sure. — Yeah. Our agents not been deleted. There we go. Um but yeah, we announced custom agents at Make with Notion this year. And custom agents are essentially this agent but sharable. Um extremely

customizable, much more so than just a personalization page. — Um they can run autonomously. So, they've got triggers, things that happen uh within notion. You can set them to run on a schedule. You can have them work with Slack um and will work with other we'll build other like apps and integrations down the line. Um but these can become like super specialized agents. Um, and so I have one here called offline Oliver that is like my knowledgebased and task filing uh agent where I can ask uh when did offline ship and this is going to be able to like read from its knowledge base that I've linked inside the settings which I can poke into in a second and then uh just chat directly with it and then this agent is pinned in the sidebar accessible by like everybody within the So anybody can just click on offline Oliver and start asking questions. — Yeah, this is great. This is great for collaboration productivity. Maybe I'll make a offline Peter agent so people can not me. Yeah, like hey let me send offline Peter to the meeting. It's cross functional meeting. Yeah, — I think the thing you said there is actually like the uniqueness of this is like there isn't a multiplayer AI product out there, right? Where you can — really constrain the context. You want to provide it. You want to be able to use it with your teammates. connect it to other tools you use. Um, and I think part of it is because the underlying foundation of like, you know, keeping it secure and adhering to the permissions and working on top of a collaborative layer, all of this stuff is actually missing in a lot of the current chatbot tools out there, — right? And so especially as you're thinking about work um custom agents is actually crazy valuable because you know out of a thousand people there might be one person who's just cracked the code of how to use AI and now that one person can actually impact a thousand people uh because of something they built. — Yeah. Like uh every company has these like AI power users who are like really into it and just making it easy for them to share these like custom agents with people will probably increase productivity by a lot. — That's our hope our own usage. This is an interesting fact for you. Our own usage of AI uh Peter had quadrupled when we launched custom agents. — All right. — And so basically in a month I think we were 4x of what we were you know the amount of tokens we were using before as a company. And one of the things we're trying to figure out now as we roll out custom agents out to other customers is like figuring out exactly sort of what model works right as a as a way to pay for these tokens right. Um so far we are seat based. Um right uh but the amount of usage you can drive especially with these power users is um a it's like crazy productive but also it does cost money and we're trying to figure out exactly how to price it. — Got it. Yeah that makes sense. Yeah. You don't want to give a big bill from one power user. But — also we want to just be consistent, right? I think these kind of things you can give it away and then you have to like you know pull it back. I think it's a bad experience. Yeah. — And so one of the things we're doing now with custom agents is to just really understand the use cases and how people are using and and build something where we're able to fund the value that people get out of it. — Give it to me after this call. Give me early access. I want to make some a family vacation agent and Peter psychiatrist agent just Yeah. By the way, this stuff is, you know, uh I don't know if you uh use um you know, sort of like I think the usage of this working on Slack has been transformative inside notion. So I think we can maybe show a little bit of that too which is — Sure. — I think we have all these channels where people are asking the same questions over and over again and now I think because it knows all that context, it can answer these questions as well. — Oh, it answers for you. Yeah. — Yeah. I to me this is like mind it's like kind of mind-blowing that this actually works. Um but like in this example here, you know, I asked uh earlier today that I'm seeing this bug on Android and then offline Oliver. By the way, I'll actually show you um there are instructions in this agent. You know, they're not code or anything. It's just text. And I also didn't write any of that. Um, I used our agent to let's see create a Slack agent in database. Yep. So, I can go and create these agents in like notion AI and I can ask notion AI to do the updating for me. I can go in here and click um you know, whatever uh and make those changes if I want. But it's nice that I don't even have to write it. But so all the logic that I'm about to show you in Slack is just here in this like text instructions. Um, — English works. — Yeah. Exactly. Feedback is super important and we don't want this to like go into the void. So, we have off where we have these custom agents that can listen to these channels, see that people are talking about bugs. And then in this case, this offline Oliver agent is able to actually file the bug for me. Um, so I can click this and I open and it's gathered context from the conversation and not just from my initial like report which is like kind of lacking in context but see how it even follows up and it's like does this happen right after connecting — uh and what OS and version are you on and I reply and I say please update the task and then it does it and it's like just keeping this task updated for me and then I think the other huge benefit of these is that We're not just trying to like actually was talking about slop. We're not just trying to create slop by like filing tons and tons of tasks in notion. Like I've also created this agent and the way our agents work internally like they'll search for duplicates before filing a task. And so here I like say the exact same thing and it's like cool this is already being tracked and so the person reporting it can you get engaged. We can update the task with more samples. Um, but we've prevented the like creep of filing tons and tons of duplicates and now we need like a backlog grooming system and all of this stuff. It's like just let the like this is literally the busy work that we don't want to be doing. Like I just want to write code and build awesome stuff for people. — I don't want to go and be like dduping tasks. — Yeah. And I want to send like Peter agent and Ryan agent into a Slack channel to align on decision and then just give me the report afterwards. You should build that next. — You guys can debate all you want. Yeah, — I'm actually excited about that world because if you think about it, the amount of like uh amplification of work and then synthesis of work that happens at work is pretty crazy, right? Like the best example to me is performance reviews. I start with three bullet points, then it becomes slop paragraphs that I ship. The other person receives the paragraphs, then it compresses to three bullet points. And so if the agents were just like, okay, you know, they don't have to do that middle thing. I think that would actually be quite a fun world to live in. — Yeah, there there's so much busy work. Yeah, there there's a lot just like a lot of copy and pasting too with existing AI tools to try to make it a bit busy work. So just like automating the copy and pasting is already like huge value ad, you know. Okay. So, I'd like to wrap up by talking a little bit about I mean, you know, you guys clearly know how to build, you know, AI agents that are real agents and like you have a good company culture. So, when you guys hire like, you know, PMs or like builders builders, right? Like what kind of skills you look for now? Like do you look for at their GitHub industry or what kind of um things you look for? But I'll start maybe on the maybe more at a higher level and you know Ryan can maybe give you bit more on the engineering side how we think about it. But — I do think some of the — the thing that's interesting right now to me the most is how the lines are getting really blurry um you know between sort of product design engineering um but also like I don't know research and marketing like I feel like each person is able to do more — uh and actually it almost feels like people who are in this intersection of multiple functions are probably the most interesting Um because I think they it's not like you want the designer to ship production code every day but it's actually the designer who is sort of close to the code will be able to channel the model see what's possible and will be able to like design these experiences that uh is so much better than someone who's just living in the

design world right and so I think like the multid-disiplinary part of talent um uh is like is increasingly quite exciting. And then I think the other high level thing that we're doing as a company is actually um you know we've become a little bit more like at least in our recruiting very focused on this like barbell shaped of hiring recruiting which is we're hiring a lot of people out of school because they are just trained in native AI and then we're obviously hiring a lot of experienced architects who are just really good at scaling large systems right and I think that's where we have a you know we're sort of focused on that those two areas because in some ways those are the ingredients we need which is we need AI native AI thinkers and we need experienced architects to scale the product we're building. — Makes sense. Yeah. How about you Ryan? Yeah. — Yeah. the thing I look the most for um I would not say that we're trying to demand that anybody coming in to especially work on engineering is like a AI coding expert that they know all the tips and tricks about how to use cloud code. I don't think that that's as important as just having a natural sense of curiosity about these tools and a willingness to try them and uh experiment and change. Um, you know, we're just in this industry that is changing so quickly. I mean, I have been writing code for like 15 years and I've like used an editor for maybe 12 of those years and then in the last like 12 months, I have changed my tools and editors like I don't know five or six times. I mean, I just switched to codeex like two weeks ago. — Yeah. There there's too many AI coding tools. Yeah. Now I'm just only using that. Well, and it's amazing because each uh iteration is demonstrabably better. And um I think the folks that are going to have the best time at Notion are the ones that are like willing to try and play with these new things. Not literally chase every shiny new object. But um you know, in a couple months, I'm sure there will be another big innovation and we will need to pivot and adapt both for the tools that we're using, but also the product that we're building. you know, if a if the next big new model has some evolution beyond tool calling, um we'll have to figure it out. Um because and figure out how it's going to benefit people that use Notion. — Yeah, it makes kind of product development more fun, right? Like who wants to have a three-year road map that's like a setting stone? Like that's boring, you know? — It's super fun. — Yeah. — It's also like uh anybody who's even asking for a very detailed one-year road map is probably a red flag. — Yeah. — True. It's true. Yeah. Yeah. Um and personally like I find this stuff really fun. Like I woke up at 6 a. m. this morning to like VIP code some nano but banana stuff. So it's like it's just a lot of fun, you know? — It Yeah, it is. — The activation energy for people to create things has never been lower. — Yeah. — You know, it's like it's so fun to be able to just dive in and build. Like there's no excuse for you to not prototype or not try at this point. — Yeah, you just got to have some taste so you don't do sloth. But yeah, otherwise go out there. Yeah. Cool, guys. Well, I mean, um, where can people find you guys and also try Notion agent? — I'm at AQari on both Twitter and LinkedIn. And the personal agent is live for everyone. So, just sign up on notion and get cranking. — Yeah. And I'm at uh Ryan Nestrom on Twitter. Um, and yeah, same. The agents out there, go play with it. We have a wait list up as well on uh our website for folks that are interested in trying custom agents um when we start opening up early access. — All right. Yeah. I'm not signed up for the weight list, but hopefully you get me in the back door or something else. Yeah. — Everybody get ready for Peter Psychiatrist or psychiatrist Peter. — Yeah. Exactly. the posting will be off the charts, you know, if you um Yeah. And and actually I hope you make more videos, too. I enjoy the videos that you made before. So, yeah, — I'm on it. Yeah. I'm glad you shared that. Uh I think that's good uh good nudge to get back at it more. — Yeah. All right, guys. Well, I'm stay in touch and uh it was a pleasure hosting you. Yeah. — Thank you for having us. Thanks so much, Peter.