Inside How Anthropic Is Building the Next Claude | Alex Albert

I was definitely the first prompt engineer at Anthropic. I might have been the first in the world. We treat the model as if it's a product to some degree. With every new model, we are specking out exactly what do we want this model to be good at. When the agent isn't running a task for you, or maybe it's in the background, it's actually going through its memories, finding things that might contradict, pruning them, cleaning them up. This concept of dreaming. Engine time isn't as much of a one-way door these days. If it's something that's not a one-way door, that's effectively free at this point. Agents that are running on task for a long amount of time and they're having to make a lot of judgment decisions. The questions of what its character is and what it cares about are very important. Did you have to try to avoid consciousness when you were training it and stuff? This is a big question. Alex, it's great to have you here today with Clock Conference. And you used to be head of DevRel at Anthropic, right? And you recently became product manager for the research team, right? Is that it? So, I've been a PM for like over a decade now, too. As a PM, you kind of try to understand the user problem, you try to identify solution, you try to build stuff. But, you know, I have no clue about how a PM works on a research team. Maybe we can talk a little bit about that. — It's very similar in that sense. Um I'm always wanting to talk to customers, get as close to our users as we can. Uh And we treat the model as if it's a product to some degree. So, with every new model, we are specking out exactly what are the requirements for this model? What do we want this model to be good at? Uh what do we think it's going to be good at? Because that's actually part of uh the interesting part of model development compared to product development is that in many ways we're growing the model and we have intuitions based on training setups, techniques, things that we've made in terms of the architectures, whatever decisions we've made for that particular model. We've intuitions about what it's going to be good at, but we actually don't know to the full extent until it's in that training process. Hm, okay, I got it. But, yeah, the research PM team attaches to these models early on from their like ideation phase and uh kind of follow along with the whole journey all the way through training up until launch. Okay. Can you give me like some examples like next model has to be good at coding or knowledge work, or is it some other things like more broad than that? Yeah, I think that's right. There's like buckets of capabilities that we care a lot about. Of course, coding has been a big one, right? Um knowledge work recently has been a big one as well, so uh with some of our recent models uh we've tried to make them really good at like working with our products like client for Excel, making spreadsheets. Um so, that's been more of an emerging new uh area of capability. So, there is that side of things, but then there's also the side of on every model we want to fix and improve on the things the last model didn't do so well. Mhm, got it. So, going out, talking with our customers, trying to get a read on, "Hey, how are you finding this model? Where is it like excelling for you? Uh what ways is it falling down? What are fixes we can make? " If we notice some like really interesting behavior, are there tweaks and interventions that we can take in training on the next one. And your customers both like the Clockwork team and internal teams and also users? 100%. Yeah, so it's everybody. Um that's kind of uh one of the cool parts about being um and working on these models is that they just touch so many different surface areas. So, as a research PM, you need to think about how this model is going to be exposed through all our surfaces, whether it's API or it's Cloud Code or it's Co-work. The product has a somewhat of like a blend with the model, and that affects your actual end users' experience, so you need to think through that entire process. The product because of different like different prompts Exactly. Prompts. — it. Use cases. Got it. Uh ways that people are kind of using the model within that surface. Yeah. It all has an effect. Man, that that's like really hard because like uh like for example, Clockwork, like you can say it's for coding. But there's people like me just using it for like knowledge work or like even as a therapist. So, do you even know? — Broad space. Yeah, things. Yes. Thankfully, we've got a whole lot of amazing researchers that span this entire range of capabilities and they all focus on different things. And then you probably like cuz like a lot of people like there's millions of people using Claude, you probably have some sort of like they can give you feedback in Claude 2, right? And you probably have some sort of way to get themes out of things. That's right. Otherwise, it's like a firehose of feedback. Like how do you even — I mean, we do a lot of things here. Um and this is actually one of the interesting things that's changed over time as I've been in this role is our increasing use of Claude to help us as PMs. Um We There's a million things we can talk about here, but in terms of like feedback collection, it's been insanely helpful to me for getting uh for getting insights into large amounts of this data. So, when we have a ton of feedback coming in from certain channels, we can use Claude to group and cluster things, find top thing uh themes, create synthetic versions of that problem so that we can see if we can turn it into an eval or some other way to like actually diagnose what's happening. Oh, interesting. Yeah.

There's a lot that you can do with Claude to help you in your own identification of Claude problems. Do you have like a specific example of that or something from your from the past model or something? Well, one that's pretty relevant right now is uh how we handle feedback on new features. So, one of our newest features with the past few models has been adaptive thinking. So, previously we had extended thinking, which just allowed the model when you turned it on, um the model would just think. Adaptive thinking lets the model choose when to think. So, on some questions, it's going to choose to think because it's a complex hard question that requires, you know, more upfront planning. On some, it might not choose to think. This has been a feature that we're continuously tweaking and dialing model over model, and we really, really listen to a lot of feedback from users on is it thinking correctly in the right settings for you? Got it. Are these questions that you want it to spend a lot of tokens, you know, reasoning about? Are they actually triggering that thinking within Claude? Interesting. — Yeah. Yeah, sometimes when it like gives me a response too fast to like some life questions that I have, I'm like — it's a little bit disappointing actually because I was hoping that you would think more deeply about this. Right. Um I think the problem with thinking is sometimes there's actually a lot of context that goes into the decision whether to think hard on a question or not. Okay. Um for example, like if I was talking to a complete stranger and they said like, "What should I be doing right now? " Maybe I'd just like quickly give them an off-the-cuff response cuz I actually don't know that much about them. I'm just going to like provide them a pretty generic answer around like, "Oh, you should focus on this and do this and this. " But if I actually knew you as a person and I know what you care about, what your interests are, what you've done before, that's going to cause me to actually spend a lot more time thinking about wait, what actually is like the best solution for me to provide here. Make it make sense. Yeah, so I think there's a similar thing with models where if they don't have that context built up, if they haven't really uh built that mental model of it who the user actually is, Yeah. then their decision whether to think hard on a problem or not could be incorrect because they just actually just don't know. That's such a really

good point. Like, do you also work on the memory feature that Claude has or Yeah, memory is definitely a big feature on the research side. Cuz I'll tell you what I do. Like, I just have a Google Doc where I've summarized my life situation like my household, all my kid his name is I was sharing too much and then like, you know, what gives me energy, my you know, like what doesn't give me energy. And then I just attach it to like a Claude project. — Yeah. And then just by doing that, it gives me like way better answers to my questions. Yeah. But how does the default memory work? Like I guess every night it kind of like compiles Uh, everything? Yeah, um it depends on the product surface. Uh, they all have uh, memory implemented in different ways. So, for example, in um, Claude. ai, it writes to a memory file, um, and then it has things where it overnight will like prune those memories, look at them again, and we actually just implemented a similar thing with managed agents. So, this concept of dreaming. Dreaming in humans is I guess it's somewhat unknown as to what its purpose is, but like some folks say that like maybe it's like a memory reconsolidation process, right? Um, we're like, okay, how can we bring something like that to Claude's memories? So, when the agent isn't running a task for you or maybe it's in the background, it's actually going through its memories, finding things that might contradict, pruning them, cleaning them up, kind of doing that second pass, which I think is really interesting. So, basically like it kind of just like does some sort of just like dumb it down a little bit, there's some sort of prompt up like, hey, review all the conversations the user had with this Yeah. — you and just like try to identify themes. Exactly. — Like summarize. Yeah. — Interesting. This episode is brought to you by Oceans. I hired someone through Oceans for podcast post-production a few months back and can't imagine running the podcast without his help. He's proactive, picks up new tools fast, and uses AI to compound everything that he ships. Oceans doesn't just place assistants, they place operators. Their talent is AI fluent and delivers the same output as a senior US hire at three to five X less cost. They reject 99% of applicants, so the person who lands on your team is already operating from day one. If you're scaling and need marketing ops, finance, or EA help, I highly recommend giving Oceans a try. Check it out at oceanstalent. com/peter. Now, back to our episode. Let's switch back to talking about product management. Mhm. Yeah. So, so before we start this thing, you said you're always trying to find the latest bottleneck, right? So, so I guess in the whole product development process, like uh, which parts are the having most streamlined and which parts are syllabic? Yeah, I think the process of shipping something was pretty stagnant for the past I don't know, 20 years. Like we've had incremental improvements and like things have definitely made it more efficient to some degree and some like new organizational structures have come and gone in terms of like, you know, sprint processing and planning and like we've tried different things to like make things go faster, yeah. Yeah, mhm. Fundamentally, like there's not been too much that have like compressed the like different windows that actually form up like the bulk of your product development process until the past like year or two. And now, all of a sudden, we're in this like paradigm where the cost and the time required to produce something is pretty low. You can spin up prototypes, you can even spin up now MVPs of initial things that you might ship to production in, you know, a day instead of two, three, four weeks. Yeah, it's like Claude telling me it'll take a week and then it spins up being like a Yeah, exactly. I know, right? It doesn't have like it's still Claude itself is still stuck in like the the olden days of 2021 or whatever. And I think that's like really does an interesting twist on like the whole product development life cycle in terms of okay, as a PM, how do I think about like my planning? If I'm writing a PRD and I'm like scoping out these requirements and I'm trying to get like a end estimate on something. Yeah. What does that actually look like now? It's kind of a waste of time, right? Like do you still do that stuff like point estimates

and stuff? — It depends. Some I think like some projects have more considerations than others and of course it just is like a function of like the scope and the complexity of it. — Okay. Usually what we're always trying to get to is like what are our one-way doors? So what are our irreversible decisions? — Got it. Because those are the ones you want to focus the most amount of time on. If it's something that's not a one-way door, like we do it but we can reverse it, that's effectively cheap and that's effectively free at this point. Got it. Cuz end time isn't as much of a one-way door these days. Yeah. But if it's something that affects an end user experience, can affect a decision that we have to make later on down the road, or it's like a physical thing that we literally have to buy or do or whatever it is, Yeah. um those are much harder to reverse and that is something we want to put a lot more time and thought into. And like I'm going to ask you for an example of that again, maybe on a research side or Yeah, um for example, if we're thinking about new models, picking a model architecture before it starts pre-training or anything like that. That's a big decision. Like model timelines can be, you know, a month long at some cases in which you want to be like training a model. So we need to put a lot of time and thought into like what is the optimal choice here. Um Models have a lot of more of those like one-way doors to some degree just because they require a lot of time, intensity, compute, everything to actually get them out in production compared to like building a new feature in Cloud Code. That's quick. That's just like a process of like iterating code, thinking about putting it into users' hands, getting quick feedback and um and cycling on that. So yeah, the process still differs depending on like what type of thing you're thinking about shipping. Um But yeah, I think increasingly the bottleneck that we're shifting to now is like more of the coordination problem. So if we can build things really fast, there's still a we need to get these people in the room and decide if this is the right strategy problem. That's right. Um we need to figure out how we're going to communicate this to our users. We need to figure out like what are all the other fuzzy things that come along with any sort of launch. Yeah. And those are still areas that we're looking for Cloud to help us with, but it hasn't been as, you know, 10 100x speed up like it has in code. Got it. Yeah. I see. So, like let's say when you're launching like Opus 4. 7 or something, like you still need to put together like a doc with a plan. Still need a plan. Yeah, you still need to think about how we're going to like message this and like the models definitely do exist on like a jagged frontier. So, we're using Claude everywhere we can. Um I would say in coding it's having the biggest impact right now. And in these other areas, there's still an element of like human strategic thought that needs to go into it. Got it. But, when you have the review meeting with like your like, you know, marketing or your peers, like do you have Claude open? Oh, of course. Yeah. Oh, yeah. I think like a huge speed up for me has been that I'm actually not blocked as much on getting to answers and getting to data. Whereas before, let's say I had a question of like, oh, how is this feature doing in production? Yeah. How many users are like using it every day and what's their feedback? That's going to go require me asking somebody on our data science team to like kick off a full investigation and come back a few days later with like results. Now, I can do it in 10 minutes and just start a session with Claude code, has access to our, you know, product databases. It can like go look at the logs, figure it out, go look through our Slack. That is like a major speed up for me as I'm actually going through the process of like strategic thinking on something. I'm not like blocked before I make my next decision. You can get the inputs much faster and synthesize it. Exactly. Yeah, we can get the inputs much faster now. But, even with the strategic thinking, do you like can you just like build out a skill or something to ask a bunch of questions to help you think through the stuff? Oh, of course. Yeah, yeah. I think like Claude to me um has been like the best brainstorming partner in the world. I'm able to like all of a sudden, at any second, get feedback on an idea that I have. That's right. Yeah. Um I think that is like super powerful, especially when you do want to move fast. And like uh people are doing a million things at Anthropic. Everybody's busy. So, having that immediate access to something that can give me feedback on or criticisms on a doc I wrote or whatever it is like really, really helpful. I personally like cuz that's probably the most common I mean let's admit it. It's probably the most common PM loop. You have a doc and you want some feedback. Yeah, oh yeah. Oh, yeah. Do you use like

Claude code to do that or like you just like use Claude AI? Yeah, a lot these days I'm actually using Co-work a lot. — Co-work, okay. I really love the form factor of Co-work. I think it's just like a really nice interface and the team has done like an amazing job over the past few months from shipping it back in just like a few months ago to now getting it to the place where it's at today where I find it to be like a really high-quality experience. Okay. So, Co-work's been a an awesome tool, one of my favorites for sure. So, you basically have your draft doc and then you have a bunch of reference material or like you have like some sort of do skill to ask you to think through like the whole decision-making process? That's right. Yeah. Got it. Okay. — Yeah. Like, all right, think through this from like the perspective of X, Y, Z. Like, what sort of questions would you have here for me? Or challenge my assumptions. — Challenge the assumptions I'm making here. Like, where are my arguments weak? Got it. Um Got it. I think like a lot of that thinking can't be fully outsourced because through the process of writing you are thinking. Um and you need to write these things to get your thoughts out and to start to like mull over it in your brain a little bit. But, Claude can help you unstick yourself and attack things from a different angle than you might have been able to on your own. I like to like give it like two different personas, like two different points of view, and have it like argue with it itself. And then I just read the transcript and it helps me think too. Yeah, right. You can see like, oh, this person brought up or this Claude brought up this point and then like it was countered with this point. It's like watching a debate in like real time. It's pretty cool. Even on the research

team, are you shipping to code or Yeah, so the question is it depends. I think a bigger part of the things I'm shipping is on evals in particular. So, I want to make sure that I'm able to measure the model on the things that I care about and communicate those findings on where the model is good, where it's falling down, back with our researchers on our research team. Got it. And then we conjointly come up with the strategy of how we want to tackle it and what sort of research interventions we want to make and what will be the most effective to actually hill climb on that eval and improve on the problem. — And like the evals aren't like terminal bench or something, right? They're like actual cuz I feel like all those like things can be like somewhat like gamed. Mhm. So, [clears throat] are the evals more like Tell me about the evals you How do you eval a model? Yeah, Um Is like different like buckets like personality Oh, for sure. Yeah. So, I mean, let's take something like we want to test Claude's vision ability. Like can it count the number of things in this image? Oh, like image recognition. Okay. Yeah. For that, you could have like okay, I found this image. It seems like Claude can't count things with more than 10 elements or whatever whatever. Like I think it can, but let's just for example say that. I can take that and now think like okay, how am I going to get more test cases of this type to really prove out my hypothesis here. So, maybe that's Claude is generating synthetic data for me. Maybe it's going to like render images for me and then I'll pass those back into Claude as a visual and see if it can like actually identify it. Or maybe I'm um finding examples from the internet or whatever it is. Uh any sort of other sourcing mechanism you can think of to like generate those test cases. And we're talking about like we're talking about thousands of test cases, right? Could be, but sometimes it's even as low as like dozens can prove pre- can prove that there's something wrong that we need to fix with the model. It really does not need to be super comprehensive to prove and have something that you can hill climb against. So, let's say you give it like 10 images and then it can't recognize the little tiny numbers. — Right. So, then so then what do you do? Go to the research team and be like hey, like this is the problem. Like can you Yeah, so So, we think about it in a few ways. I mean first we want to think about beyond just the fact that there's an issue with the model, how is this going to be valuable for our customers and our use cases? Um because Claude can or can't see something in that image, how does that actually affect downstream what somebody's trying to do with Claude? So, the more realistic and on distribution to the actual like task shape that somebody an end user would experience, the better. So, we're going to try to get things of that nature. make sure that we can get data into that flavor. And then there's a range of interventions. Maybe we need to go back into pre-training and look at things. Maybe we can do address it in RL. Um that's where the like strategic uh brainstorming comes into play with the research team on like what's the best thing to do here. How fast is the turnaround to try it again? Like yeah, it depends on what we're thinking is, I guess. Yeah, it depends. It depends like where, you know, if it's something later uh that we can address in a new RL environment, maybe that can be spun up very quickly. And when you try to tie it to real customer use cases, I mean there's like millions of people talk to Claude every day, right? So, maybe you can like try to synthesize into like other people are trying to use this for tax prep or something. Yeah. Cuz there's like uh thousands of use cases. How do you How do we pick out ones to particularly we want to get better on? — Yeah, how do you convince the like the team like, "Hey, this is what we should actually improve on. " I mean this is where data wins the day, right? Um it's all just about like X% of our users trying to do this. We really care about this. Yeah. We have customers that are using this much amount of Claude and they want to get better at this. Um or and this is a largely what a lot of our processes is um driven by is like, "What do we care about internally when we're using the model? I'm using the model. I'm running into this blocker every day in my own work. We should fix this. " That's really compelling. So, there is an element of that as well. One of the best things I

love about Claude uh is the model's personality. I think it's even gotten better over time. Uh like it will push back on me on the right spots. And you know, some of the other models are just like, hey, you know, what else can I do? Like they're very sociopathic, right? — Mhm. So, [clears throat] um the model's personality is just like a it's not just a prop, right? There's like training involved. Yeah, a lot of training. This is a huge focus of ours. Um so, Claude's character, as we call it. Um think this is very, very important. We have a lot of folks that are like dedicating a ton of time towards figuring out what how should Claude represent itself? What are its beliefs? What are its values? Um how does it behave? Yeah. These are like really fuzzy questions and early on I think some people kind of like disregarded them because it was like, ah, you know, this is just a thing I tell it what to do and it goes and does it. Why do I need to care about how it sounds or what it thinks about? But increasingly I think as we move into the world of these things being agents that are running on task for a long amount of time and they're having to make a lot of judgment decisions. Um the questions of what its character is and what it cares about are very important. And it's not like a code like does it run or not, right? Like how do you evaluate the personality? Like the character? Yeah. So, you're trying to find a nicer person inside of Anthropic you sort of compare — sort of compare that person. Yeah, we have very special people that have been appointed the judge. No. Um um there's a combination of things. You know, there's like quantifiable metrics that we can look at and we can have Claude look at Claude's outputs and see like, all right, how does it sound? — Yeah, yeah. And this is a very important skill for any researcher, just read transcripts and be like, oh, I see it's doing this now or and detect subtle differences there. And over time you do kind of develop this sharper intuition as you just read, you know, hundreds and thousands of like model transcripts and Yeah, you kind of know, right? Yeah, you know, it's just like playing around with the model a lot through Claude. ai. You get a sense of like how it feels. Okay, so it's not like you know, this model is like seven out of 10 on blah blah. It's more just like getting a sense. It's a bit of both. Yeah, I'd say Got it. Okay. Interesting. It's harder to like um quantify this maybe than it is to like quantify coding performance, but there's like ways. There's ways to do it, right? — Yeah. Very interesting. And how about um people who want to learn how to build products and become PMs like the AI native way, you know? Mhm. Like what kind of advice do you have? I mean, I think like the the best like simple nudge I can provide is just like try it out. Like it sounds so simple, but like whenever you're about to do something, you have a hard problem and you're like, "Okay, I'm going to go do this. ask this person this question. " Maybe in parallel to doing that, you ask the same question to Claude. Just compare. Just compare the results. Um for example, if you're like yeah, trying to run an analysis on your users and like pull out top themes of like what customers are caring about with your like latest feature that you shipped and you're going to go ask your data science team going back to that previous example, like what are users thinking about this or your UX or researcher, whoever. Um do that because I think there's still a ton of value in like working with a person on that. But in parallel, also just try sending that question off to Claude. You know, enable a few tools for it to like go explore. And like let it the time to like really dig into that problem. And then just compare. You'll get like a great sense of where the model can excel and where it still falls down. And through that process over many prompts and questions, you kind of like build up your own map of like what I should be using Claude for. Like where is reliable and where is not, right? — Exactly. Yeah. Yeah, I always like to like I always ask it to do deep research when I'm trying to make a decision cuz like the normal web searches isn't good enough for me. I got to got to do deep research. — Yeah. Having to like scan through 1,000 web pages, you know, something just superhuman. Yeah, and they probably inside on topic if you just go to a DS and be like, "Hey, can you do this for me? " They'll probably just tell you, "Hey, what did you ask Claude for? " Sorry, what — Oh, yeah. There is an element of that, you know. It's like, "Hey, — Someone's expected, right? That you ask Claude first. Yeah, I mean, I think it's just like we're moving up the abstraction layers, right? Like it's not it's worth it for that data science trust person to be thinking about problems at a different level now than just like having to go do like manual retrieval work of like the — Cuz that was sucks. That's what no one wants to do that stuff. — Yeah, everybody wants to be thinking about like a harder the harder questions, the strategic questions, like — Yeah. Um how can we measure this in a whole new way? What are like new things to be doing here instead of just like, "Oh, we need to find what our latest uh DAUs were for this product. " Talked to the DS I work with. A lot of times they get stuck doing the basic SQL stuff. And they all want to do the strategic stuff and now AI can finally unlock them. Completely to do it. Yeah. Yeah, it's like we're enabling everybody that like surrounds them in that way. And that's the same for all these roles. For like for example, scoping out new features. Um in the past, if you're a PM and uh whether you're technical or non-technical, you often don't have enough time to like go dig into the code base and figure out exactly how we need to implement this new feature, figure out like, "Okay, this is going to take X amount of work. We'll have to overhaul this system, but actually this is a limitation. " Yeah. And it was just better to to work with an engineering partner and figure that out with them. But now I can actually send Claude off to go do that investigation for me. And [snorts] you can be like, "Actually, this feature it just requires like 10 lines of code change here and we just need to like flip this uh simple flag in this gate. " Yeah. And it's like, "Oh, okay, that actually leads to like a completely different way of how I'm prioritizing this like decision. " Um so now as I'm like specking this out, I can get to that sort of uh uh prioritization much faster. Yeah, that's huge, man. Couple more questions. I think um at a lot of like legacy companies, they do a lot of time like planning, like annual planning, and like, you know, quarterly planning, and like road mapping. Yeah. And maybe that's actually more true on the research team, right? Because you guys got to think longer term than like shipping stuff every day. Like, did you guys do that stuff or We do. Yeah. There is still an element of like unpredictability, of course, with models. So, plans it's like that famous like you know, Churchill quote of like plans are indispensable, but like planning is useless. That's right. Uh so, like, you want to think through it, but you also have to acknowledge that Yeah. — your plan might go out the window. I think it's planning is indispensable, but the plan itself is useless. Yeah. Yes. Maybe that's what it is. Yes. The planning the act of planning is is important. I think one of the toughest challenges of PMs is like, how much do you spend time planning? Right. Cuz like it's always balance between planning and trying to ship something. Yeah. You know. Yeah. — So, like, is there some like um inside of Anthropic, is there some best practices around this stuff or — Yeah. Cuz you can literally spend like, you know, writing 10 pages with Claude. Yeah. Right. Yeah. Yeah. Um It's I think it's hard to like uh Is it team-dependent? I think it's it's like product-dependent, and it it's hard to maybe like assign a blanket statement across the board of like you need to produce docs of this length of this much like we definitely don't do that. Okay. Um We really it's more about like have you done enough thinking to think through all the possible one-way door implications of this decision? Got it. If we've hit that, it doesn't matter what the format of your doc is or how many pages you've got or whatever else. It's just like, do we feel good enough that like we aren't missing something here and we can move forward, and we can address things as they come up as long as like Yeah. there's no long pole here that's going to be like blocking us. There's no like one-way door could be like really bad. — Yeah. Yeah, got it. Okay. Another question is you know when I go with Cloe at home, I have all these different projects going on and like I'm like constantly context switching between different projects to get stuff to build stuff. I'm wondering if that applies like PM work here too. Like do you have like different projects? Yeah, totally. Yeah, there's many different projects. — have to wait for the agent to work, right? — Yeah. Yeah, you do. Um So then what's the next thing? I think there's actually a huge opportunity here Okay. um in terms of how do we as we move towards you're managing agents and they're doing bigger and bigger chunks of work for you and now you can start many more projects in parallel. Yeah. How do we think about that context management problem for yourself? What's the best like way to expose this as an interface? Um how do I keep track of like what's actually important, where my agents are being blocked, where they need help from me? Yeah. Um Something better than just like a little list of chats. — Exactly. Yeah. There feels There feels like there's an opportunity in there. I think it's too early to say exactly what it is, but we're seeing tons of experimentation even within Anthropic around what does that look like. Also people just like engineers just like build prototypes. Oh yeah, we have a huge prototype culture in inside the company. Yeah, people are building stuff all the time trying to share things. Yeah, so you're going to have a agency cuz no one is asking you to build those prototypes, right? It's just like do Yeah, I mean that's been one of like the coolest [clears throat] things to see just working here is the amount of agency that every everyone has across the org. From sales to recruiting to engineering to research, like everybody's is very agentic in that sense and they'll start doing things that they weren't just assigned. — Yeah, you got to let the thousand flowers bloom. Exactly. — Got it. What are some interesting I know Dario writes like super long essays to share in Slack, but what are some other interesting you know, co-cultural things that happen at Anthropic? Well, there's a few things. Yeah, there's a few things here. Um Dario's way of writing these long essays is is um not necessarily unique to him as well. Like there's a lot of folks within Anthropic that put a lot of time and effort into writing. We've a really strong written culture. Um we've a lot of folks that uh write docs about things and are writing long Slack messages and communicating in that way. Okay. We have a fun thing that we do in a lot of our meetings, too, which I think is somewhat common, but I haven't seen it everywhere, uh where people come in with a doc and we'll spend a good amount of time up front just like kind of communicating on the doc. And like it's a little funny at times cuz like the room will be silent. You have a ton of people in the room. And yeah, we'll do a silent read and like we'll really like write out long discussions within the doc, comment on things, and do all that. Um So, I think we have a very doc-heavy which I like because it is a way I like to operate, but also it's very beneficial for Claude. Yeah. Um when everything is being written down, Yeah. we have now this like corpus of information for Claude to go off of. So, I actually encourage organizations out there to move towards thinking about how you can get all your tactic knowledge into written forms, whether that's through transcribing meetings, whether that's through encouraging more writing about like workflows and onboarding processes and everything like that. Yeah. Get things written down. Make them accessible to Claude. Cuz that's just more context that it has. — Exactly. That's that's very interesting. So, you still have a very strong writing culture and docs even though it's like a lot of stuff ships up now. Just like you know, why call myself? I just get Claude to generate all the MD files. Yeah. But like I I still read through it, but like — Yeah. — Yeah. But like it's like working inside a company is very different. You still have to think through things and like Yeah. Yeah, we want to do. Yeah. Yeah, makes sense. So, on the

research team, every like people talk about like AGI or whatever and I feel like it's very vague what that is, but I feel like one thing I worry about is just like when these models actually have some sort of consciousness and then they actually like I if I tell them to do random work, they'd be like, "No, I don't want to do this. " And then like that'll be the end of humanity, dude. Well, what do you think? Do you should have you tried to avoid consciousness when you're training this stuff? This is a big question. So, we actually do have folks that are Think about this? Think about this. We have a few folks now whose whole job is to think about um what does it mean for Claude to be a conscious actor and a conscious agent. Um there's no official position on whether Claude is or isn't yet. Um and I think even talking about it can sound a little crazy at times, but it is something we're putting a lot of thought into. Um and there's a lot we can learn from it outside of even determining whether Claude is conscious or not in terms of just like how it interacts and how it behaves. How it thinks, man. How it thinks. If you go into like our model cards for our models, these are like in my opinion just treasure troves of information. Um and you'll see a lot of work that we do in terms of trying to quantify how does Claude act in this situation. What is its mental model? Um if it's being presented with this scenario, is it going to do X or Y? Yeah. So, through this process of just thinking about the way Claude thinks, we actually learn a lot that can actually translate that into product experiences as well. Make a Claude that's actually better to interact with, better to use. I see. I think this is a really interesting question and there's like downstream long-term effects and there's near-term things that we can bring to our own experiences as well. Yeah, cuz like I feel like we're going to trust the model for longer and longer work — Yeah. without human supervision Yeah. and it's going to make a bunch of decisions along the way that you probably have no oversight on. So, you know, what it does is actually pretty important. — It's very important. Yeah, if this thing is writing all your code and deciding which like you know, database system you're going to use and like making all these architectural decisions, you kind of want to trust it to some degree. That's right. So, it is important that it has that high character like we were talking about earlier. I'm glad you guys are thinking about that cuz I just like dangerously skip permissions all the time. I'm glad — I am. Yeah. You know, use auto mode. It's a little bit better now. Yeah. All right, Alex. Thanks so much, man. — Thank you. Good chatting with you.