Anthropic's safety structure

Dario was a first-time founder. None of the top venture funds wanted to participate in this round. Wasn't a hot company at all. The boom hadn't happened yet. Chat GPD hadn't been invented yet. Nonetheless, they were true believers in this safety mission. And so, one of their investors suggested they come talk to me. I told them, "Look, if you don't get this right, here's what's going to happen. " And they were very determined to do something about it. They wrote into their charter that they were going to do this. They were a PPC from the very beginning. They had the right to enact these additional reforms, what's called the now the long-term benefit trust. And so even today, Anthropic has directors on its for-profit board who are appointed by and are accountable to an outside group of trustees who are AI safety experts who do not have equity in Anthropic. So they do not have a financial incentive in its growth, they have an incentive to see it done properly. So whenever you see Anthropic do the right thing, like when they refuse to release a model because they think it's too dangerous, think about how much that's costing them. They got into this fight with the Pentagon and they were in an impossible situation. They turned down a $200 million contract and bore the wrath of the world's largest army and government. That took a lot of courage. Part of that courage is enabled by the fact that they have this structure and investors can't just oust Daario on a moment's notice.