Exploit, Explore, Empower

What I'm going to do today is talk about how children manage to learn as much as they do, what that can tell us about how artificial intelligence works and functions and how thinking about artificial intelligence can let us know something about human development. But I'm not just going to talk about children. I'm also elders, which is a new development for me. Let me start out with this quote, which is actually a quote from Alan Turing. This is a nice summary of what I and my colleagues have been trying to do over the past 20 years or so. In the famous paper in which Turing first described the Turing test, the idea that if you'd want to try and tell whether you were talking to a person or a computer as a test for artificial intelligence, people don't notice it about halfway through that paper. He suddenly changes gears and says, Look, maybe that's the wrong test. Maybe instead of trying to produce a program that simulates the adult mind, we should instead try to simulate the child's mind. Why does Turing think that? Well, because the distinctive thing about children is that they actually learn to be intelligent from their experience. They learn to have the adult program from the things that happen around them, and that's a more profound intelligence than just being able to execute an adult program. In the past 10 years or so, the big spring of AI, the great advances in machine learning, have emphasized just how important learning is to intelligence, both natural and artificial. What we've been doing is actually looking at children, seeing how they manage to learn as much as they do, trying to think what computations could be going on in those little fuzzy heads that would let them learn as much as they do and then applying that in AI and vice versa. That's been the big project. What are some of the morals of doing that? What do we learn by thinking about AI in the context of childhood? Well, what I'm going to start out doing is talking about three different stories we might have about AI and in particular, about the large models, the large language models, the large multimodal models that have been the focus of so much attention in not to mention money in recent AI. What I'm going to do is tell three different stories. I think often stories and narratives are a way that we convey information most effectively. Three different stories that might be told about how these systems work and that indeed have been told about how these systems work. Here's the first story, which is the story of the Golem. Most of you may know about this as the Rabbi of Prague story. I had a wonderful rabbit hole. I guess Tim Tamborini who's actually here in our Department of folklore, consulted with me about these stories. It turns out these stories, the story of the Golem the idea that there's an artificial thing made out of clay that a magician comes and magically comes to life. That story is found across cultures, across time. It predates the industrial revolution, let alone the computational revolution. Even has its own number in the folklore of encyclopedia. It is incredibly fun to go and look at the encyclopedia folklore. The story is the rabbi has this non living thing. He does some magic. It turns into a living agent. As I say, there's many, many variants of this story, but the TLDR as we would say in Silicon Valley, is that it never ends well. It's always a bad idea. The stories always end badly. That picture that what AI does is to produce this artificial agent analogous to the agents that people are or animals are is extremely pervasive. I think it's probably the most common picture people have when they hear the word AI or when they think about artificial intelligence, including many of the people who are actually making these systems themselves. But as you'll see, I think this is profoundly wrong way of thinking about how at least the current systems work. It might be the way some system in the future will work, but it's not the way that the large models work.

Here's the second story, which I think is a much better description, a much better story about how the current models work. This is a story of stone soup. Many great things about being a grandmother is that you get to read all these great stories to your grandchildren. Stone Soup is also an ancient story found around the world. Sometimes it's ax soup or nail soup. It also has its own number in the encyclopedia folklore. I'll just give you the quick version since not everyone here may know it of the original Stone Soup story. The story is there's a bunch of visitors. A magicians feature largely in these stories. They go to visit a village. They ask if they can have some food. The villagers say, no, we have no food. They said, That's okay. We're going to make stone soup. They get a big cauldron. They start boiling the cauldron. They take out their magic stones and put them in the cauldron, they say, This is going to be great soup. It would be even better if we had an onion and a carrot, but we don't have it. Of course, one of the villagers said, I think I might have an onion and a carrot somewhere and goes and puts it in the soup, and they say, that's great. This is doing really well. It would be even better, when we made it for the rich people, they put buttermilk and barley in, and that was even better. Someone else says, I think I have some buttermilk somewhere. They put that in and they say, When we made it for the king, the king insisted that we put a chicken into the soup and that was really great. That really made great stone soup. Someone says, I think there's a chicken in the backyard or in there. Go get the chicken. Anyway, you can figure out how the story goes it. Each of the villagers contributes something to the soup. In the end, there's this marvelous, wonderful soup that everybody gets to eat. I should actually put this in the slides in the children's version of it that I took this from. There's a beautiful double page spread at the very end where the villagers are saying, men such as that don't grow on every bush and the visitors are saying, It's all in knowing how. You can probably already guess how this applies to the current models, but here's the AI version of the Stone Soup story. A bunch of tech wizards went to the village of computer users, and they said, We've invented artificial general intelligence, just from next token prediction and transformers and deep learning. The villagers all said, That sounds great. That's wonderful. That's very impressive. They said, but it would be even better if we had more data. We really need more data to put into this to make it really intelligent. All the users said, Well, we have all this data, all the texts that we've ever written, every picture we've ever photographed. We put it all out on the web. Every book I should add for the authors and the audience that's ever been written, why don't you just use that data? The tech magicians say, that's great. Yeah, we'll add all that data. That makes it even more intelligent. But, it's still saying really stupid and obnoxious things sometimes. Do you think you could do reinforcement learning from human feedback and just tell it when it's saying something stupid and that's okay and the villagers said, There's whole villages in Kenya that would be happy to just do reinforcement learning from human feedback all day long. The tech magician said, see? It's getting more and more intelligent. But, a lot of times when you ask it something, it just says something stupid in response. Could you do some prompt engineering? Could you use your brains to figure out exactly how to ask the questions the right way so that you can get a good answer? Villagers said, we'd be happy to spend our time doing prompt engineering. Then, of course, the end of the story is that the villagers say, It's amazing. We've got artificial general intelligence, and it was just made with a few algorithms. When I gave this talk at NeurIPS, which is the big AI conference, Ted Chiang, who I'm going to talk about in a minute, the great science fiction writer, rather mordantly said at the end. But at least they don't sell the soup back to the villagers at an exorbitant price That's the second story, and I think that that's actually, as we'll see later on, an accurate story about the way that the large models work. They're not independent columns that have developed intelligence, what they've done. As we'll see, this sounds like Stone Soup in both versions is a debunking story, but it also has a positive moral.

The positive moral is that by combining in both the old version and the new version, by combining all the individual villagers food, you can end up with something that's more important that's much better than any of them could do individually. The same thing as we'll see is true about the large models. By combining the data and intelligence of many people around the world, you can end up with something that provides you with real advantages. That's a really useful technology, even if it isn't a Golem. That's the way I think the current systems work. We could also ask, what would a system be like that actually was intelligent? If we wanted to design an artificial system that had the same intelligence that we see in humans, what would the design of that have to be like? Here, rather than turning to the encyclopedia folklore, again, I'll turn to a contemporary story. This is by the great speculative fiction writer Ted Chiang. He has this lovely story called the Life Cycle of Software Objects. It's one of the best stories about being a parent that I've ever read. What happens in this story is that these Digiants are little artificial intelligence babies, basically. Humans adopt them, train them, teach them, and then eventually have to give them up, eventually have to let them go off on their own. It's absolutely beautiful story that I highly recommend. I think that picture, the picture of a system that develops that changes over time. In particular, as we'll see, a system that's cared for by humans or cared for by other intelligent agents, that's the secret of human intelligence, and that's the system you'd need if you wanted a system that had the same intelligence as humans. That's the big picture of what I'm going to talk about. What can children teach us about AI? The first thing they can teach us is that there is no such thing as general intelligence, artificial or natural. I put this slide up in big letters when I'm going and talking in the valet, and there's a hushed intake of breath, and you hear this whisper, Don't tell the VCs. For various reasons, there's a intuitive, what psychologists think of as a folk concept of intelligence. Which is many other intuitive folk concepts. It's a concept of this mysterious essence that some people have more of, some people have less of, some animals have more less of. If you have more of it, then you're more powerful and there's an implication that you should be more powerful. That's the picture. It's a lot pictures of things our intuitive theory of life for energy that there's energy force that gives us power and lets us get out into the world. It's incredibly prevalent. I'm not quite sure why, although I have some ideas that we could talk about later. It's hard to dislodge this picture, but if you do cognitive science, you don't see anything that looks like this picture. Instead of seeing this magical force of intelligence, what you see is many, many different cognitive capacities that are suited to different goals. Again, it's a little surprising to me that engineers, for instance, who do this who go out and try and design systems know that you need to have different systems for different purposes, and yet they sometimes seem to buy into this mystical intelligence picture. In particular, not only are there different intelligence for different domains, but computer science itself tells us that there are intrinsic trade offs between different intelligence. Being very intelligent and effective in one way means that you can't be equivalently effective and intelligent in another way. The classic example of this in computer science is what's called the Explore Exploit trade off. I'll talk about that a bit more later on. What I want to suggest is I'm going to talk about there's many such cognitive capacities. The ones I'm going to talk about today, I'm going to describe first as exploitation versus exploration, exploitation sounds a little mean, but that's the intelligence that actually enables you to go out into the world, to have a goal, to plan, to achieve that goal, to implement actions that will achieve that goal, and that's a really important intelligence. But it's intrinsic tension with some of these other intelligence, in particular, exploration. Exploration is not about trying to accomplish any goal.

It's about trying to figure out what the environment around you is like. If you do that, in the long run, you'll be better at accomplishing your goals. But in the short run, as we'll see, that can often take you away from accomplishing your goals. It can be dangerous or it can lead you away from things that are actually useful for you. That contrast between exploration and exploitation is very deep. But I'm also going to talk about two other intelligence that are less discussed. One of them is the intelligence of care. Now, this is one where people sometimes act as if this is just an oxymoron. They look really puzzled when you talk about the intelligence of care. But we'll see that the very distinctively human capacity to have one agent who actually helps another agent to accomplish their goals requires a really specialized set of cognitive capacities, which have been much less studied than other cognitive capacities. I'll talk about that a bit later on, too. The third intelligence is transmission. One of the things that's distinctive about human beings is that we can get information from each other, and we can accumulate information over many generations and transmit information from one person or one generation to another. Again, you can see how the transmission and exploitation and exploration can all be in tension with one another, because what you want for transmission is adopt the information or the characteristics of the people around you, particularly previous generations. But that can be in real tension with the things that are most useful for you or true, which is the objective function of exploration. Trying to match, trying to extract information from others, trying to find the truths about the world, and trying to act effectively in the world, and indeed, trying to act to care for other people, those all require distinctive cognitive capacities that are in tension with one another. How do we resolve these trade-offs between these different intelligence? Well, what I've argued is that the way that we resolve these trade-offs is thinking about a different idea, an idea that comes from evolutionary biology. In fact, it's very essential to evolutionary biology and this is the idea of life history. What's life history? Life history is the way that an organism develops, how long a period of childhood it has, how often it reproduces, how long it lives overall, its developmental profile. It turns out that in evolution, very often what's being selected for is one of these life history characteristics rather than just, now this adult form is going to have a particular characteristic or morphology. If an Alpha Centaurian biologist came down to Earth in the Pleistocene, she wouldn't see big differences between the humans who were scampering around on the veldt and all the other primates that look fairly similar, maybe with the exception of language. But she would immediately notice the extremely bizarre human life history, which really looks different from any other life history. This picture is a really nice illustration of this. You can see in this picture that there are three children all from the same family who are under four. These children are immature. They need to be taken care of until really late. Chimps are producing as much food as they're consuming by the time they're seven. Even in forager cultures, they aren't doing that until they're 15. My son is 37 and we support our children for a really long time. Certainly, you can see in this picture, the four-year-old oldest child still requires a great deal of care and needs to be taken care of. The other thing is that humans have shorter intervals than other primates. That means that every two years or so, in a situation where we're not using birth control, we have another baby. Not only do we have these immature babies, but we stack up these immature babies. We have a lot of these immature, helpless creatures who need taking care of all the time. If you look lurking behind those three beautiful grandchildren, there's a postmenopausal grandmother. The postmenopausal grandmother is another weird part of human life history.

Women, as long as we've evolved, have lived past their fertilities. Menopause comes in around 50, and women have always lived until 70 or so, if not necessarily as long as we live now. That's true even in forager cultures. Although it's not as dramatic for men because we don't have the equivalent of menopause, men also are living for that extra 20 years. We have this extended period of elderhood at the end of our developmental period. Both of these things seem paradoxical from an evolutionary perspective. Why do you have these children who are so needy, who require so much investment for so long? Why do you have these postmenopausal grandmothers who aren't reproducing anymore, but are still living and consuming for another 20 years? What I'm going to suggest is that this life history is actually how evolution solves this problem of trading-off these multiple intelligences. Everyone at every stage of development is going to exercise these capacities. But I'm going to argue that children in particular, seem to be very well designed as it were to be explorers, and that's what they do. That's what they do best. Your ordinary 35-year-old adult seems to be really well designed to actually go out into the world and do things. The elders, those post 50-year-olds, seem to be particularly involved in care and in cultural transmission. The argument I'm going to make is that these different intelligences trade-off against each other in the context of this extended human life history. Now, go back to the folk theory about intelligence, one piece of that folk theory that you hear a lot from 35-year-old philosophers and psychologists and AI guys is that this mysterious intelligence has its peak around 35, and development is just getting to the point of the 35-year-old intelligences, just trying to reach that point. Then aging is just falling off from that peak of intelligence. But that doesn't make very much sense from an evolutionary perspective. Instead, this picture about trading-off different intelligences, I think is a better picture. As you can probably already tell from that sentence, I used to be a bit snarky about the 35-year-olds and their belief in their superior intelligence. But now that I have three children in their 30s who all have children that they're raising themselves, I basically feel like, Oh, my God, those poor 35-year-olds, they're just cursed. Do they have to go out and do all these things? They have to find mates. their way in the pecking order. They have to get resources and the children and the grand moms are having all the fun. We get to tell the stories, figure out the narratives. I can testify my sister is here. We spent all day playing with a two-year-old. The way I put it sometimes is that basically we're human up till puberty and after menopause. In between, we're glorified primates doing all the things that primates do. If the 35-year-olds want to think they're smart, that's fine. I'm willing to give them that while the grandchildren and the kids are having all the fun. Let me try and justify this argument. Let me start out. I'm going to start out by talking about exploration. This comes back to work that we've been doing for my whole career for 50 years, looking at the way that children are very much like scientists. My first book was called The Scientist in the Crib. In particular, what we've done is look at how children learn about the causal structure of the world. How do children develop intuitive theories of the world around them? We use this as our little box, the blicket detector. It's a little machine that lights up. We can show children different patterns of data on this machine, and then we can see what conclusions they draw about how the machine works just by getting them to do things like make the machine go. To a remarkable degree, it turns out that even very young children, four-year-olds and younger can rationally perform this causal learning and causal inference, and they seem to do it implicitly using a lot of the same formalisms that are used in AI, for example, in computation, which is impressive and in philosophy

of science to characterize scientists. I won't go through this whole list. This is a review paper that's in nature reviews, but they are really good at solving these problems and can do much more than we ever would have felt was possible. How do they manage to do all of this? As I said, we get some hints from formalisms like Judea Pearl's causal inference work or thinking about this as Bayesian hypothesis testing. But none of those accounts really seem to give you a satisfying story about how all this learning is possible. What we've been doing most recently is looking at something that we probably should have looked at in the first place. When we first started doing the work with the blicket detector, the biggest problem with the blicket detector was keeping the kids away from it, because as soon as you put it in front of them, what they wanted to do was play with it themselves, try things out, experiment, figure out what was going on. Literally we had to tell them, when we've finished showing you all the data, then you can get to play with it yourself. That should have been a clue that the children were experimenting and they were exploring. They were actively learning. They were playing. When adult, we know that for scientists, this active intervention in the world in order to get new data is absolutely crucial. It's the crucial thing to scientists. It's experimentation. When two-year-olds do it, we call it getting into everything, but we have a lot of evidence that, in fact, the two-year-olds are doing things that have the same character as scientific experiments. The question is, could we design exploratory AI agents that are using similar techniques to the children? As I mentioned, part of the reason for wanting to do this is this exploit trade-off, where in order to be able to exploit in the long run, you have to be able to explore in the short run. go out and do things that might not look very useful, but that will enable you to learn more. The way that this trade-off is resolved in computer science, for example, and across a wide range of different theories, optimality theory, other things is by starting out with a period of exploration where you can look very widely across a high dimensional space, figure out how the space works, and then narrow in, cool off in something that's called simulated annealing to just narrow in on a particular hypothesis and then use that to actually act. If you look across many different species, you see this striking relationship between how long a period of immaturity, childhood an animal has, and anthropomorphically, how smart the animal is, how good the animal is at learning, how good it is at figuring out new environments and adjusting to them. The poster animals for this are actually birds. If you compare our friend the domestic chicken is mature in two weeks and it's basically really good at pecking for grain. Not much good at doing anything else. In contrast, this is a New Caledonian crow, and crows corvids, in general, especially these New Caledonian crows are as smart as primates in lots of ways. This is one using a stick, remember that stick, and they are fledglings for as long as two years, which is really long in the life of a bird. As you can see, this is a very general relationship. Why would that be? Why would you see this relationship between immaturity in the young and intelligence learning in the old? Well, the thought is that a lot of the things that are features from the explore perspective are bugs from the exploit perspective and vice versa. Things like being noisy both literally and metaphorically, doing a lot of random things, those are things that are really good if what you're trying to do is to explore the world around you, not so good if you're actually trying to act in an effective way. That's true about a lot of other properties like risk taking, being impulsive, playing, being insatiably curious. You can probably already see these are all things that are characteristic of children, and things that have often been taken to mean that children are not as intelligent as grown ups. They've often been taken to be deficits that children have rather than strengths that children have compared to adults. My hypothesis is that childhood is really evolution's way of resolving explore/exploit trade-offs, doing what computer scientists call simulated annealing, starting out with this protected space in which you can do lots of exploration and then later on, using the output of that exploration to actually

make things happen in the world. In fact, empirically, and again, I won't go through all the details here, there are lots of cases we can point to where younger learners are indeed more exploratory than adults, both in my own lab and in other labs. The secret seems to be that, usually if you just have a task, the older people will be better at it than children. Well, adults will be better than children, but not if the task involves discovering something new, figuring out some idea that isn't obvious, being outside the box. That's the context in which children actually do better than adults do. How is all this exploration happening? What's the underlying mechanisms behind it? This is some new work that we've been doing very recently. To answer that question, I've been turning to a very different tradition, the tradition of what's called reinforcement learning, both in psychology and in artificial intelligence. When I first heard that our reinforcement learning was making a comeback, as a boomer cognitive scientist, I was horrified, like, My God we are going to have to wear bell bottoms next, which it turns out we probably are. But this is back to the '50s. Didn't we get rid of all that in the cognitive revolution. But there's a new approach and version of reinforcement learning, which has been extremely influential both in neuroscience and in AI. It's responsible for the success of AlphaGo and the chess playing programs, for example. How does reinforcement learning work? Well, imagine there's rat in a maze. This comes from psychology originally. It will move away from a shock, and it will move towards cheese. It goes down one arm of the maze. There's a shock. I won't ever go there. Another arm of the maze, there's cheese. It will go there, and it will learn to do this. But the problem is that the way that reinforcement learning works in the classical sense is trying to get utilities, trying to get cheese and stay away from shocks. That turns out to be not a very good way of learning about the structure of the environment. It's very good for exploit learning. It's terrible for explore learning, not to mention Cara or transmission. An idea that a lot of people have had is, how about if we did reinforcement learning, but we did it with intrinsic rewards rather than these external rewards. Instead of being rewarded for cheese, you could be rewarded for novelty. getting new information. There have been a number of attempts within AI to try and design systems that are rewarded in this way for finding out something new about the world rather than just having utilities about the world. The problem is, if you just have this reward be something like novel to or information, you have another boomer example, the noisy TV problem. If you're sitting in front of a staticy TV, TVs used to have static in the olden days. You're getting lots of new information. novelty, but it's not doing you any good. You shouldn't just be sitting there trying to pursue novelty. How could you have an intrinsic reward that lets you explore but actually lets you explore in an intelligent way that will teach you about the environment around you? An idea that we've been thinking about and using a lot is something that, again, comes out of the AI literature called empowerment. What's empowerment? In empowerment, what you're trying to do is maximize the mutual information between your actions and the outcomes of those actions. At the same time, you maximize the diversity of those actions. What does that mean? What that means is you want things that you can control. You want to find things out there in the world such that if you change your actions, if you do something new, something new will happen in the world. Where there's a really consistent relationship between what you do and what follows from what you do. You don't want to just have, you just one of them would be like being in a casino where you do the same thing over and over again. You want to try and find new ones. You want to find new ways of acting in the world that will give you predictable outcomes in the world. I won't go into this in detail, but what I've argued is that this is really causal learning, the thing that I mentioned in the first place. If you think about what it means to learn about cause and effect, what it means is now I can intervene on the calls. I can do something to the calls, and I can predict something about the effect. What that means is when you learn a new causal relationship, automatically, you're going to gain empowerment. You're gonna be better at controlling the world. Vice versa, when you get to be more better at controlling the world

you're going to understand more about the causal structure of the world. That's a longer argument that I won't go into here. I think it's interesting that this approach, this idea of empowerment, comes out of the evolutionary biology literature. It came originally from people who were trying to characterize what would make an animal intelligent at all in the first place. As you may know, we first start seeing brains and intelligence in animals during the Cambrian explosion. What happened was that in the Ediacaran, which went on for millions of years, under the sea, there were very large, complicated organisms. They lived a happy life filtering out little food from the ocean and reproducing and doing all the things that living do. Unfortunately, they don't seem to have known that they were having such a happy life because it was only when there was this change in the Cambrian, you started suddenly having animals that had eyes and claws that had actuators and sensors that could see things in the world and do things in the world. There was a part of the reason for the Cambrian explosion is that there was this arms race about who could predate and who could avoid the predators. How could you use your eyes, your perceptual system, and your motor system to either find food or avoid being food? A number of philosophers, Peter Godfrey Smith, who was here giving one of these lectures recently, have argued that this is really the beginning of consciousness, and it definitely is the beginning of when you start to see a brain because the brain is coordinating the actions and the perceptions. It's a sad thought that when things were nice and tranquil, we didn't experience it. It's only when we get hunger and fear that we are conscious but so it goes. When you start thinking about development from the perspective of empowerment, you see empowerment all over the place, especially in very young children. This is work that was actually done back in the '70s by Carolyn Rovee-Collier. Interestingly, she was trying to show whether babies could be operantly conditioned or not. What she did was tie the baby's leg to a mobile. What happens is when you do this, a three month old starts kicking wildly to see if the mobile will work. But it's not just a reinforcement learning where she just wants the mobile. If you just do the mobile yourself, then the baby's not really interested in it. What you'll do is kick for a while and then stop kicking, which is not what you'd expect from reinforcement learning, look up at the mobile, then start kicking again to see what's happening with the mobile, then start waving her arm to see if that will make the mobile go. Perhaps the most significant thing is giggle and laugh all the time that she's doing this. Babies just love this. Carolyn Rovee-Collier in this paper said, "It looks like the reward is the contingency itself. It's not the outcome. It's the very fact that you can control". What that means is that these babies are seeking empowerment. They want to be able to go out in the world and do things that will lead to particular outcomes. One of the things that's wonderful about being a grandmother in 2025 is that you get cute videos of your grandchildren every morning and this is Kit, who's in this video was just a year old. His grandfather is a really accomplished musician. He has playing the piano, and he's given Kit this xylophone to play with. Here's what happens. First, Kit takes the mallet and bangs it against the bars. Then he decides to try using the stick end to see the sound that the stick end makes. Then he tries the mallet again. Then he tries his fat little hand, which doesn't make any noise at all. Then he goes back to trying the mallet, and he tries it on the long bars, short bars. Just in the course of doing this exploration and play, which is exactly what you expect a 1-year-old to do, he's figured out this causal relationship between what you do with your hands and pure tones that are produced as a result. It's worth pointing out, this is not a causal relationship that could have existed in the police decene. This isn't just something that you would have evolved to understand or detect. This is something that you actually have to learn about and that you learn

about through this exploration. Now, notice at the same time, his grandfather is demonstrating the fact that this kind of general relationship exists, even though what the grandfather's doing is completely different in some ways from what Kit is doing. But he's giving him information about the fact that you can make sounds happen by doing something with your hands. But he's just doing that by demonstrating, and this speaks to the transmission point that I'm going to get to in a minute. The only explicit thing that he says to Kit is don't put it in your mouth, Kit. When I give this talk, especially if there are young parents in the audience, people will say, Oh, he looks like he's going to poke himself in the eye with that stick. A point that we'll get to in a minute. This is a paper that we recently have in press in philosophical transactions with the Royal Society. Videos of your grandchildren are all very well, but we wanted to show we see whether we could systematically show that children were indeed seeking empowerment and that this was leading to their causal learning and causal information and Indeed, it turns out that it does. These are lovely experiments by my brilliant graduate student, Eunice, who's shown this. If you want more detail, you can look here. Now, as the example of Kit and the xylophone shows, one of the dangers of this exploration is that you might hurt yourself. I think the stick is a really nice example of this because the stick is the world's best empowerment tool. Evidently, Wired magazine at one point, did it. A list of the best toys ever. The stick is the number one best toy ever. It is. It's amazing. It lets you move things that are far away. You can move things back and forth. You can poke things. You can have much more control over the world if you have stick but you may also end up in the ER, as my grandchildren at various points have done as a result of playing with sticks. How do you deal with that problem? a problem that when you're exploring, you're impulsively risk taking, you can actually do damage to yourself, not have your high utilities. To think about this, here's another study that's one of my favorite studies of all time. This is a study from Nim Tottenham and it's based on a study originally by Regina Sullivan, looking at rats. Remember that Rat in the maze who won't go down the arm that has the shock. Well, that's really smart. That's cycle one oh one learning, but it also is an anxiety disorder, right now, because the rats never going to find out that actually maybe this time there's cheese at the end of the maze. They're just going to avoid it for the rest of the their at Watts. Well, it turns out that is true that cyc 101 phenomenon is true for adult rats, but it's not true for juveniles. If you put juveniles in the maze, they prefer to go down the arm that leads to the shock. Anyone who has a 2-year-old or a teenager will testify that this is also true for humans but Nim Tottenham actually went and systematically showed that it was also true for three and 4-year-old children with an unpleasant sound, rather than a shock, we don't show here. But there's a twist, and the twist is that the juveniles will only do this if the mother is present. In fact, if they can smell the mother. If the smell of the mother is there, then the juvenile will explore the shock arm of the maze, but not if it's not there. Again, Nim showed that this was true for three and 4-year-old children as well. It's something about the signal of caregiving us is as it were telling the Rat, Look, it's okay. Nothing really terrible is going to happen to you. I know how to get to the ER. You're going to be fine if you do this exploration. That leads me to the next part of the talk about intelligence. It looks as if this exploration depends on having adults around who are willing to put the investment of caring for you. This leads me to the last part of the talk, I think, which will be about caregiving. I love this. I really like this picture.

It's from the Rights Museum in Amsterdam. We used it for the cover of the special issue Daedalus on caregiving. I'll tell you about it in a minute. I think it's so moving because that is a sick child. That does not look like a happy, cheerful, healthy baby, but when you see it, at least when I see it, and I think when most people see it, you feel this really strong urge to take care of that baby, the way the m in the picture is taking care of that baby. I think this picture is even more moving when you know that it was painted during a plague epidemic in Amsterdam in the 1660s. The child is likely to die. The mum, by taking care of the child is likely to die and the painter is exposing himself to the plague. I think this captures the fact that for most of us, caring for others, caring for children, but as we'll see, caring more generally is one of the most deep, profound things that we do. It's the thing that has the greatest moral significance. It's one of the things that makes our life have meaning. It's something that we talk about in religious context, for example. Yet, in spite of that, that care has been pretty much invisible in the social sciences. In economics, for example, it doesn't show up in the GDP, so it doesn't count as being productive labor in political science, nobody talks about the way that you're caregiving. The unit is the family, and the thought is that caregiving is happening within that unit. In moral philosophy for Christ's sake, which is supposed to be about our moral intuitions, it's almost impossible to find something about moral intuitions about care. In moral psychology, they talk about hierarchy and justice. Nothing about caregiving. Margaret Levy, who's a political scientist and I have been involved in this project together, and at one point, we said, is there anything that all the people who've done this work have in common that could explain why there's nothing about caregiving? We finally said, they're tall. That's it. They're really tall, so they don't see the children who are around at their feet. What we've been doing and this just came out for the last three years, actually funded by Templeton, is to try to advance the idea of a social science and a cognitive science of caregiving. By the way, in the 70s, there was some feminist work about ethics of care, but it basically just said the same thing that I just said, which is why hasn't anyone thought about this. But in mainstream social science, there hasn't been very much work doing the job of working out how caregiving works. That's essentially what we're doing now. Again, that's particularly puzzling. This is a special issue with contributions from sociologists and political scientists and economists and psychologists thinking about care and philosophers, thinking about caregiving. This is especially puzzling because we go back to that life history as you might expect that life history of extended childhood goes with more parental investment. If you look across very different animals, these are marsupials. On one hand, this is the quokka, the world's cutest little animal, both in its name and looks. On the other is the Virginia possum. The quokka has one baby at a time that lives in the quokka's pouch, and both the mother and the father take care of the baby, and the baby stays in the pouch for a year. The possum has big litters of babies all at once. The biological mom is the only one who takes care of them, and they're mature within a month or so. I think we may often feel a little more like the possum than like the quokka. We could all use more back lighting in our lives. But in fact, of course, humans are much further out on that distribution of high levels of investment. We have what I think of as the investment Triple Threat, we have pair bonding, which is very unusual among mammals. We have fathers who are involved in taking care of babies and, indeed, taking care of their spouses, at the same time as well as mothers. We have what the great anthropologist Sarah Heurdy calls all parents, people who are not biological kin, but are still involved in taking care of children and that's been true since we were in forager cultures. In fact, it's more true in forager cultures than it is in our contemporary culture. We have my personal favorite grandmothers, those postmenopausal grandmothers

and there's really beautiful work showing that, especially the survival of the toddlers, really depends on the grandmother's contributions. Of course, we also extend these care relations beyond just children to our partners, to elders, to an interesting example that I try to use when I talk about this with tall people is students. Your graduate students are in a care relationship to you patients. Those are all examples of contexts in which we extend this idea care. Arguably, we can extend it to non-human animals, to the planet, to, as the Buddhists say, all sentient beings. There's a reason why caregiving hasn't shown up in the social sciences. It's because it's got some really paradoxical characteristics. It's very different. Its fundamental structure is very different from the fundamental structure of the social relations that we're used to studying in the social sciences. A way that I've tried to conceptualize this analytic philosophy slash computational way is think about two different agents. One agent has more resources and goals than the other agent and Agent B. What happens? They've got different resources. They've got different goals. What's going to happen in their social relationships? The idea that central to economics and political science goes back to Hobbes or even further back, is that what will happen is that they'll have a social contract. They'll exchange resources in order to accomplish each other's goals. This is really good. It gets them out of prisoners' dilemmas. It lets them thrive. You could argue that democracy, markets, all sorts of institutions implicitly or explicitly involve this social contract. Another thing that can happen is that they just pool all their resources and goals, so they just become a single unit, as it were, a single community. Another thing is that the one who has more resources could just get fewer resources to accomplish their goals. Though the golden rule is that he who has the gold makes the rules; that's that power relationship. Again, people in the social sciences have talked at great length about how these power relationships play out. But here's the thing that's so weird about caregiving. In a caregiving situation, the agent who has more resources actually donates those resources to trying to pursue the goals of the agent who has less, and they do that just because that agent has fewer resources. Just because the baby is helpless and needy, we try to accomplish the baby's goals which makes it really different from these other relationships. I would argue that even if you think about something like caring for the dead or caring for the planet, what makes it care is that it has this structure. But you could also ask, when you say that the carer is trying to pursue the recipient's goals, what are those goals? Well, sometimes they might be objective utilities as the economists say, you make peanut butter sandwiches. Again, my sister and I spent a lot of time making toast and jam for the two-year-old today. Sometimes it's actually subjective utilities, so it's not what you think would be best, but what the other agent thinks would be best. We did end up giving more ice cream to our two-year-old today than probably would be in her objective interests, but which very clearly indicated as a strong subjective utility for her. But I think the thing that we do most of all is that we maximize their empowerment to go back to the empowerment idea for exploration. What we really want to do is make the person that we care for have enough resources so they can determine their own goals and then be able to go out and accomplish their goals, not the ones we have, not even the ones that we have for them. The ones that they actually develop themselves. Going back to that idea of empowerment, agency control, I think what care fundamentally does is give agency to another person, to someone who's actually being cared for. Give them this empowerment. I'm almost through here. The last thing to say is that very often, so what I want to suggest is that even though all of us can do this, and there's some evidence to support this, that this is very much something that elders do, whether they're taking care of children themselves or whether they're just taking care of the next generation, passing information on to the next generation is something that faculty do

as I said, for students or for junior faculty. The elders also seem to have this niche of cultural transmission, of passing on information from one generation to another. I mentioned that we're the only primate that has postmenopausal grandmothers. There's one other mammal that we know of that has postmenopausal grandmothers, and that's the killer whale, the orca. What's distinctive about killer whales is that they have cultural traditions, and they pass on information about food types, for example, from one generation to another. Especially when food gets scarce, the grandmothers, they have postmenopausal grandmothers, and the grandmothers are the ones who lead the pod to here's where there was food 30 years ago. It seems as if the role that these postmenopausal grandmothers are playing as well as being the care role and you can show that the pod is more likely to survive if there's a grandmother there is also this cultural transmission role. If you look at that grandmother in that picture, I didn't realize this. I just love this picture. But when I started thinking about cultural transmission in elders, what that grandmother is actually doing is reading 100-year-old copy of Winnie the pooh to her grandchildren because this is a grandmother who likes to collect old children's books. But I think in general, what the songs and stories and recipes for us and for the killer whales are the things that grandmothers and grandfathers are passing on culturally. I think I'm going to skip over the AI part. But the general argument that I've made, let me show you the paper, is that the best way of thinking about those large models, the stone soup models, is that they are another method that we've invented for passing on information. They're a weapon of the grandmothers. The same way that things like language itself, pictures, writing, print, libraries, those are all technologies that we've developed that enable this process of cultural transmission to take place. That grandmother is reading a book and reading a book is a really distinctive cultural transmission. What we've argued in this paper in science is that we should think about large models as being the latest example of those cultural technologies. I won't go into detail about this. But again, as in the Stone Soup example, that's for good or for ill. New cultural technologies like print or Wikipedia can have wonderful consequences, like the enlightenment, they can also have terrible consequences like the French Revolution. I can talk about that a bit more if people want to later on. I think the right way of thinking about the large models is they're the latest iteration of this cultural technology. Last thing. I've been talking about care and how fundamental care is for human intelligence. How about artificial intelligence? Go back to that beautiful Ted Chiang Nevela. One of the things that people talk about with artificial intelligence is what's called the alignment problem. How is it that we can coordinate the goals of an artificial system and our own goals? Make sure that they have good goals, not bad goals. The paperclip apocalypse is you tell them to make paper clips, and they turn the entire Earth into paper clips. How can we solve that problem? One way that people have solved it is what I think of, again, this is a Boomer reference, as the Stepford wife way of solving it, where you get the system to figure out what you want and just do exactly what you want. But of course, every time we have a child, we face an alignment problem. Every time we have a new generation, we face this problem of they're going to We want them to have goals that are different from ours. But we want them to be good instead of bad and what I've argued is that it's that care relationship with humans that enables that alignment to take place, that enables us to pass on our knowledge and our goals in a way that's positive rather than negative. If we do ever end up with an AI that's an autonomous agent in the way that humans are, those AIs are going to need mothers. They're going to need humans to be in a care relationship with them. Hopefully, if we have them assisting us, they'll be in a care relationship with us. Let me stop there and then take some questions. What does it mean to have an AI goal? I mean, how can we instant state goal seeking in AI?

I'm just wondering how that would work. I mean, because goal implies will and will implies intention. Well, I think the thought when people talk about things like there's a literature about agentic AI, that's something that people have talked about a lot. If you think about that reinforcement learning agent, like the rat in the maze, even though that's a very simple system, it's a system that has goals. The sensible way of describing the route is to say it wants the cheese and it wants to stay away from the shock. You can certainly design systems, including robots, for example, that have goals in that sense, at least, that there's some state out in the world that they want to bring about or they want to avoid, and they act in a way that will bring it about or avoid it. That's the sense in which. The lower probability outcome, having the space of actuation being a lower probability outcome of that space, I think, is what. Certainly, it's something about that relationship between the actuators and the sensors. If you think about those Cambrian creatures, those little Cambrian shrimp, they're going out and trying to predate and trying to avoid predators. Both of those, you could think of as being a goal in a way that contrast again with the Erica and sponges that are not going out and acting in the world to accomplish things. I enjoyed your talk very much. It struck me when you said beginning of consciousness happened when everything was going fine and then there was some fear or danger or threat. It reminded me of the biblical story which I think is the story of the Garden of Eden. Yeah. Have you looked into that as anything to go beyond the scientific and say that some religious parallels and all that? Also how this applies maybe to artificial intelligence. There has to be some threat to make it become maybe morally conscious? I think one of the things that is quite clear from the current AI systems is that if you wanted to look at a place in current AI that looks more like Cambrian, robotics is the place to look. The thing that when you look at robotics, you have a system that's actually out there that's in the real world, that's trying to accomplish things, avoid things that has sensors and has actuators. That's the beginnings of thinking about a genuinely intelligent system. That's very different from the, I hopefully not offend anyone in the audience about this, the machines, which is what we have now that are living in text, that are predicting text, that are putting texts together, but never have any relationship to external reality. That's what the large models are like. I think getting out into the world and interacting with the world in a real way is the thing that was what led to intelligence and consciousness in us back in the Cambrian. That's the a path that would do something. Now, consciousness is always a fraught question, but certainly that would lead to our intelligence in an artificial system. But it has to be said bear is up on the eighth floor of our building. One of the other things about being a grandmom is I get to be cool because I can take them up to see the robots when they come to visit grandmom. But they're really disappointed about the robots. The robots have not made nearly as much progress as other aspects of AI. They're still very bad at doing even simple things that three month olds are really good at doing. Following directions. Are they rebellious? No, that's the problem. That's exactly the problem. They don't explore in the right ways. Although a friend of mine who works in robotics has actually people in robotics have been trying to do things like give robotics something like empowerment as a goal. Of course, one of the good things about robots is that they don't have to sleep so they can stay up all night. This friend had a wonderful video of the robot at 4:00

AM doing this empowerment and actually smashing the window and destroying its arm because it was trying to see what its arm could do. That's another reason why it would be good to have a robot mom who could take you to the ER when you were doing your exploration. But I did want to say one thing about religion, which is that I was saying that it's amazing how little work there is about caregiving in social science. But religion is one area where there is a lot of thinking about caregiving. You could make the argument that a lot of religious traditions, the Bodhisattva is defined in terms of the care that they have for all sentient beings. There's a reason why God is the Father, and we're his children in the Christian tradition. It's because it's a model for how care. Even Caritas. Caritas is care. Charity is care. One of the things that's very odd. I could give a whole other lecture about the policy issues about care is that even though we think that the greatest of these is charity, that's the greatest virtue, we also think, but I don't want any charity. There's a strange tension in our culture between really valuing care and charity and wanting to resist care and charity. For example, if you look at Medicare or Social Security, it's very important to us that we think of those as insurance programs, not as programs by which many people go and provide resources for people who are sick or who are old. But anyway, that's a whole other story. But I do think it's interesting to turn to the religious traditions thinking about. As an atheist, I think it's really interesting to see the way the religious traditions speak and think about care. Thank you so much for the talk. Thank you for the work that you've done. Early on in the talk, you mentioned the myth of general intelligence and that you might come back to it. I don't think you did, or maybe I just missed it if you did. I was wondering if you could return to that and expand upon that point. I have no evidence for this. I can ask the sociologist in the audience how plausible this sounds. I think it's a lot like nobility was in the feudal period. When you've got an economy that depends a lot on mysterious reasons why the aristocracy should have more power, a way of doing that is to say, there's this thing called nobility, and aristocrats have more nobility than just ordinary people. You could and indeed, you could have ranks of where you are in the hierarchy of nobility. Of course, now we look and say, that's weird. There isn't anything that's nobility. My suspicion is that in an meritocratic information economy, intelligence serves the same function. It's something that you use as an indicator of how and where you should be in the economy or in the pecking order. Again, it's very strange because if you think about an evolutionary perspective, for example, intelligence is all about homeostasis. Intelligence is about how an organism maintains itself through environmental variation. It is not about going out and dominating the world or dominating other organisms. Again, this is completely amateur sociology and history, but I think there's something like that that's going on. The other thing that's going on is, as developmental psychologists can tell you, this picture of there's an essence, there's some force is very natural to people. Again, when kids are about four or five, they get the idea that there's this life force, and if you have a lot of it, your arms will grow longer, and if you don't have a lot of it, you'll get sick. That's a very natural way of thinking about life, and was true up until the 19th century, we had people thinking about Alon vital and worrying about it the same way people think about consciousness now. I think it's a combination of this very natural folk tendency to think in terms of essences that we have more or less of and then this particular feature of our economy. Now there are IQ tests. There is this psychometric literature. But I think people don't realize how completely detached that is from what cognitive scientists actually do. People want to have tests that will predict things like how people do in school, but it's just a totally orthogonal to the question about how it is that we can manage to do the things that we do in the world.

Hello. Thank you so much for your talk. I was really interested in the idea you presented about empowerment, because it strikes me as a value that comes from a very individualistic or Western point of view. I'm just wondering, to what extent do you believe empowerment is a Western value? If so, are there other ways for children to learn causal inference in perhaps, like if they're raising a home that does not value empowerment as much as the average Western home? Empowerment is an unfortunate word, that's the word that is used in the descriptions in the technical reinforcement learning literature. But it's a weird woo Berkeley word at the same time. But it's interesting. I was just at a conference in Santa Fe about effects of early life adversity on later development. People pointed out, and I think quite rightly that when I was thinking about caregiving, my first thought was, it's to get autonomy. What you want is for the people you care for to have autonomy. They were putting out, no, you don't want autonomy. You want to be competent to be able to have other people who you're involved with and who help you. I think that's right in our culture, too. What you want is you don't want to have to depend on your mom for your whole life or your dad or your other caregivers. What you want is to be able to transfer that caring to your peers or your spouse or your siblings or other people who you're on an even keel with. In fact, I think a lot of what happens with school age children, the crucial thing that children is this transition from the babies, the infants who really are just completely dependent on their cars to being able to accomplish your ends by interacting with people who are your peers rather than people who are taking care of you. It's not meant to imply at all that you're just this autonomous individual. It's this difference between a situation where you're helpless and you have to depend on a particular person who's caring for you versus thinking that you can actually influence not just the physical world, but the social world in a way that will enable goals to be achieved. Those goals might be trying to, in fact, for humans, they will always be goals that people are trying to accomplish together. But an example I like is think about all those 10-year-olds going out and building a clubhouse together. Clubhouse never actually gets built, but they go out and they decide that they're going to build a clubhouse together. That's a social empowerment, but it's really different from having daddy or mommy build you the clubhouse. Thank you so much. We are out of time, in effect, but if you can ask your questions very quickly, like short questions, and you'll take both of them at the same time. Sure Thank you. I may skip the question completely because I'm a career early childhood teacher and I worked in the lab school at Mills College sadly now closed for 10 years. People like you get out and prove what we are seeing in our experience and our thoughtful, careful practice in the field. I can skip the question and just thank you for your work. I can skip that and say thank you as well Hi, Alison. I really enjoyed it, and your work has been really inspirational for my research. My question is this, you mentioned a few examples about learning and intelligence, such as the mouse with the cheese and the shocks and the Cambrian explosion, etc. All of those involve organisms that have bodies. They have mortality and they have, this drive to survive. How might you speculate that AI might develop an actual intelligence when it lacks both a body and mortality? I think that's a very good question. My impulse, I'm not sure there's this embodied and active idea about those are the things that you need for AI, and that's what I was saying about robotics. It's hard to imagine how you could have an agent operating in the world, finding out about the world, acting on the world that wasn't embodied in some sense. But I have thought about if you could have

a large model that could ask questions, for example. That wasn't just asking questions because someone had told it to ask questions, but really wanted to know the answers and could change its representations based on the information that it got out in the world. You could have a day die in textual creature that was curious about an external reality that wasn't just the reality of its own texts. Something like that might be an example of something that wasn't embodied in the way that we typically think that wasn't mortal, but that was tracking what was going on in the external world. You might think that in that paper that I mentioned, Henry Farrell, who's a political scientist, uses the example of things like markets and bureaucracies. They're really interesting examples because markets aren't embodied, they don't have bodies, but they definitely track what's going on in the external world in some important way and influence what's happening way. Even though we don't think of them as being intelligence is in the same way that we think of human agents as being intelligent. I think there's a really interesting in between about what would it mean to be active, be figuring out something about an external reality that wasn't just that you had a biological frame. I don't think, as some people do, that there's any principled reason why you couldn't end up with an artificial system that had agency and consciousness and intelligence and all the things that we do. We know, as I sometimes say, we can point to material systems that have all those characteristics, and we even know how to create them, and it's a lot more fun than coding. Because every time we have a baby, we have a new system that's a physical system that can do all of these things. But I think there's a long and interesting question about what the relationship between us are. Thank you very much. Please join me in thanking Alison