Joscha Bach "Bootstrapping a GODLIKE Mind"

really have to think about what is the kind of artificial intelligence we want to build for our children. Suffering is not created by the universe. It's created inside of your own mind. The reason why our body gives compute credits to our mind is because our mind is supposed to solve the organism's problems. Uh the question of whether machines can think is about as meaningless as the question of whether you can swim. computers are already in the same ballpark as the perceptual apparatus of our brains. — First of all, thank you so much for being here. An absolute honor and joy. We're absolutely very excited. So, thank you. Um, I guess I wanted to start with this because I feel like you're the perfect person to help us get some clarity on this. One of the there's a lot of questions online that kind of split people into tribes, whether it's diet, religion, politics, but one thing I'm realizing is there's a new one and that is this idea of like can machines think? Some people kind of violently oppose that. Some people are on board. Can you help us get into any insight into can machines think? — Yeah, I think we need to start with the question, what is thinking? — Right. There is a quite famous citation I hope I miss don't miss I think it was by um Dstrom and um it was uh the question of whether machines can sink is about as meaningless as the question of whether Ubots can swim and some people interpret this as meaning that what the Ubots are doing is something that is too mechanical to be compared to the real swimming that organic fish are doing and so But an interesting thing about dubos is that they're not confined to the surface. And when we talk about swimming, we mostly think about uh basically breaking the surface of the water and sticking in a single plane. And the hubot can move in 3D. It can do much more than just swimming and go beyond that, right? It can sink in ways in which we cannot and even fish cannot because it can go to greater depths and it can go at greater speeds. And so in some sense you could say that it's a meaningful supererset. And so the question also for computers is in my perspective not so much can they think but um why should they only be thinking? Isn't there something more interesting that they could also be doing on top of thinking? But before we get to this point of course we have to ask ourselves what is thinking to begin with? And it seems to be overwhelming for us to make sense of what thinking is. And because it appears to be so mysterious, people tend to be in awe of that notion. And they're very reluctant to attribute this mysterious quality to a mere artifact that humans have built and programmed and set into the state to interact with us. So I would start out with trying to look at what minds are doing. I think minds are making models of an external and internal reality of a being and interaction with its environment and they do this via internal acts of communication. They have to develop some kind of messaging protocol. They can have to hold state and have to model how to go from state to state and these states have to correspond to aspects of how the universe works in such a way that these aspects become controllable. And we observe ourselves to have two main modes. One is perception. Perception happens in real time which means it happens synchronously with the things around us. And uh it tends to be geometric which means there is stuff happening in a space movements and it's even true for our emotions. We typically experience them as continuous flows in a low dimensional space as experiences of movement and pressure and resistance and so on. And we can all understand these things using some geometric calculus. And on the other hand, we have reasoning. And this ability to reason requires that we are translating our percepts into some form that makes it compositional like Lego bricks. And necessarily this is much more brittle and these Lego bricks are bottoming out in references to perception. So we can not define language just by itself as this these sparse symbols. Instead we are using pointer structures that we can put together in basically with slots and um little knocks into sentences into grammatical structures. And these grammatical structures are just the outside form of a semantic structure of a conceptual structure. But there is a tight correspondence between these abstract concepts and the linguistic symbols that we're using for reasoning. So imagine you're thinking about a face and you have a nose in that face and uh what you're doing is you take these linguistic symbols and then you make pointers from these linguistic symbols into a larger somewhat continuous space of ideas in which this schema of a face exists and the schema of a nose exists and certain relationships that can be used to plug them together. And these in turn are pointing at some kind of superposition of faces and noses and so on that you get to know about by interacting with them in some kind of external reality over which your mind makes models. And none of these things

is very mysterious today. We have computer models of this that are able to emulate all these abilities of the human mind to some degree. And there is nothing that is super mysterious about it anymore. Even though the mathematics is not extremely intuitive, but the deeper question that people might still ask, is it possible for the machine to experience imagining a face and a nose and interacting with them? And this is the much more contentious question and I would answer this question uh with this experience of experiencing is itself a kind of a model. It's a model of what it would be like if some observer existed that has a certain perspective on those things and is right now looking at this stuff and as a result there is stuff happening in the observer and these reflections of the observer inform additional behavior the next actions that the observer does and also the next set of reflections is like a loom that is creating a pattern that is also being read by the loom and then continues the pattern and so in this sense we are not the loom of We are the pattern or we are part of this pattern. It's a pattern that is a pattern about looking about at itself. And so consciousness I think our ability to experience ourselves and the world in some sense of real time is perceptual. It's on this level where it happens with this model of immediacy. But it's uh on the other hand a reflexive representation. It's nothing magical. itself is a model of what it would be like if you existed. And this also means it's virtual. It's a simulation can only exist inside of some kind of computational machine that can create simulation. — Just to follow it up to kind of book end it from the other side because often times what I hear people will say it can't think or it can't exper or whatever the thing is that we're talking about because it's just a stochastic pair. It's just matrices being multiplied. It's just, you know, this is that in and of itself meaningful. Are those arguments against it being able to do anything in and of themselves meaningful or no? I think that they only make sense superficially as uh as metaphors. And these metaphors are misleading because they're not grounded in a deep understanding of what understanding actually is. And this question of what understanding is not trivial. And it's very easy to sit there uh with a crowd of like-minded people on stage and say machines cannot understand. But then you have a burden on you. You have to define understanding in such a way that it meaningfully distinguishes what the machines are doing from what you are doing. And this is a burden that the stoastic parrot paper is not um by Emily Bender and Tim Gro is not really uh able to carry in a very good way. I think so I was very unsatisfied with this. Let's start out with this notion of what parrots are doing. Right? The parrot here is a metaphor. We know that parrots actually are intelligent and parrots are able to meaningfully interact with the world and that they can use symbols. So you can uh get a parrot to perform tasks like look at the table and uh pick out the one object that is not round and not yellow. Right? And that's an operation that the parrot cannot uh perform by simply imitating human behavior. It means that the parrot has to build a semantic structure that is connecting these symbols including operations like negation that make them compositional perform logical operations and then map that to behavior in the real world. Right? And this ability to act on models and these models are onetoone corresponding to the instruction that somebody else gave and you vary the instruction and the behavior varies in exactly the same way. I think at some point it becomes preposterous to say that this only pretends to understand because this is in some sense what we mean by understanding or more deeply I think of understanding is the ability to connect a certain domain or a certain pattern or whatever you think is that you are understanding to your overall model of the universe to establish all the necessary relationships to this global unified model. And until a few years ago, this question of how can we make a global unified model of the universe was the big unanswered question in artificial intelligence research because the models that we had were very isolated. You might have a model of how to play chess, but this model does not translate to anything beyond playing chess. And you might have a model of how to control a character in a computer game. And this in turn also does not translate to playing chess or anything else. There is no way to get from one model domain to any other. Instead, every model is just a small set of functions that can map in a very small area of behaviors your inputs to your outputs. And what makes human behavior so special is that we can do this for an open world. And everything that we are encounter, we put this into

the same world. We relate it to the same universe. So we have ways to relate all the concepts that we have to all the other concepts and the way in which we do this is a very big interconnected graph of uh links and concepts in between them and bottoming out in behaviors that are mental behaviors or physical behaviors, observational behaviors and so on. And this allows us to relate to reality. And what these big multimodal models are now doing is that they also create models of a single cohesive reality. And that was really surprising and I feel that uh the linguists like Emma Ber are not sufficiently in awe of what has been achieved. I believe that this big school of Chinese moveers uh starting with S who basically said if we had a machine that was able by just manipulating symbols by uh taking in squiggles and then mapping them against instructions and then producing other squiggles. If we achieve such a machine, there would still be nothing in the machine that thinks, that believes and understands. And now we have built such a machine that is literally the Chinese womb. And the machine tells us and assures us actually I do understand what do you want of me. And uh right and now I think the philosophers should interact with this at some level. Maybe they don't have to say okay I grant this machine is doing what uh it claims it does. Maybe this is just an a level of a trick. You have to interact with that trick in a way that explains how is it possible that it's tricking you and if it is it possible that you the philosopher also tricking me that you never understood what understanding is in the beginning and you are the stoastic parrot that is just doing the thing that your profession entices you to do is telling the audience what it wants are you just a prompt completer and that's also a deeper issue that I have with AI criticism that in some sense AI has struggled still to catch up with limitability sometimes It makes very weird and stupid illogical mistakes and there is something going on with these machines that are trained by imitation on human media output that is slightly different from the way in which our own minds are working. But there is also something that they achieve and uh for instance they're much better at AI criticism than humans are. I have not read any AI criticism since GPT3 that GBT3 couldn't have made just as good, — right? — And so, uh, it's really, really hard to criticize AI. And why is that? It's because criticism of AI happened at the edge of human knowledge. They're no longer possible by an armchair philosopher who doesn't actually understand the mathematics and philosophy of artificial intelligence at the level at which these models are currently being built. And so we have the situation that there is a bunch of very smart kids uh that are very highly selected and highly paid and they work at the edge of what's known and they don't really know what AI is going to be capable of next year and everybody who told them this year is going to stop. This year they're going to hit a wall had egg on their face. So it's very risky to come out for a philosopher and say this is never going to work or this cannot work in principle because uh now you basically need to um next year answer to critics right and there are some AI researchers some AI critics who go out in public and have no issue telling every year a new that AI is hitting a wall but these are just politicians they are not thinkers anymore they discredit themselves in this public arena and so if we now want to make meaningful for AI criticism, we actually have to make mathematical proofs or engineering proofs, — right? And especially when like a complex system can have some sort of a tipping point and an emergent property can come out of it so unexpectedly, it's hard to make any claim that something like that couldn't happen. But um can I

ask you um to maybe like walk us through sort of the like the way that you understand the evolution of intelligence uh in humans and in biology that kind of led towards consciousness and then maybe we can kind of compare it to what the you know the agents might be going through. Um, was it is it right to think that there were just information processing systems early, you know, bill a billion years ago that were kind of like chess playing models of today and then some kind of switch happened where the architecture allowed them to understand their own place or sort of self understand their own information processing and then that led to where we're at. Um, yeah, just walk me through the evolution of intelligence and consciousness. — So, uh, we could say that there are different stories that exist. There is the polloristic story that exists across many cultures and that is basically the animist story. The animist idea is that the difference between living and that matter is that the living matter is possessed by spirits and spirits are themselves not corporal. They need bodies to do something in the world but they act on those bodies and they are intrinsically agentic. So this means they are can regulate the future and they somehow care about doing this and they are capable of experience and in our scientific worldview this is a superstition. There are no spirits right? There is only mechanisms. And in this mechanistic world that science lives in, we have this hard problem because what we think of as mechanisms is typically things like elementary particles that are pushing and pulling at each other and then create larger structures which is are in turn pushing and pulling at each other. And we have difficulty to imagine how all these mechanical movements can lead to things like experience and emotion and meaning and all this interesting stuff that we associate with life and consciousness. And so this is I think what philosophers call the heart problem. There's some alternate formulations of the heart problem but I think directionally this is what most of the people who feel that they find this problem very confusing are struggling with. And so when we think about what we see life as it's all this complicated machinery that is put together in the right way to produce DNA and RNA to extract neck entropy from the environment create cellular membranes to be able to divide the cells and so on. And all this machinery is very difficult to get to. But once you have the first cell and you have a self-replicator that can adapt itself via mutation and selection and recombination and biogenesis to new behaviors and to new environments and it can spread and it can become more complicated and can create structure on higher levels and to create the structure on the higher levels it's necessary for the cells to communate communicate across each other. So they create basically communication protocols that allows them to become multisellular. And what makes multisellular organisms interesting is not that they are a bunch of closely related cells that are somehow similar enough to link together but they become coherent that they create an organism that has a particular shape that makes sense when seen from a global perspective. This organism needs to form itself for in such a way that maybe it has exactly two eyes and these two eyes are in exactly the right spots to create a three-dimensional model of reality by looking simultaneously at the same object. Right? And to make this happen that these eyes are growing at the right time in the right spots, there needs to be intricate communication across all those cells and the protocol that is making it possible for the organism to construct itself in exactly that way. And this intelligence is can probably be somewhat hard coded and in the genes in our DNA and it's unfolding in a certain process and it gets constructed over an evolutionary time span via mutation and selection and um additional mechanisms like that. Right? And so there is also another aspect that people like Mike Leven are pointing to is that evolution um leads to a morphogenesis in an organism that is very resilient against disturbances. If you disturb the development of an organism, it's still going to try to find a solution that works. And so the way in which this morphogenesis that's taking shape of the organism takes place is not just the enactment of a blueprint, but it's local problem solving. These cells basically try to find a solution. They set up in such a way that they try to find a solution with local intelligence that leads them to coherence. And so there is an interaction between this protocol of finding coherence and the local problem solving that is happening when the organism is forming. And so the information that is stored in the genome and in the body of the cells that leads to the form of the organism is more hinting in this problem solving. and is constraining the space of possible solutions. And that's why DNA can be very efficient as a medium because it does not need to contain all the information that is necessary to make the organism look the way which it is like a computer program does right now. Instead, it only needs to have the necessary hints for the cells to figure out the solution given the constraints that they're under. So there is some intelligence happening there and some mechanism that makes things coherent and at some point you get to plans that are themselves coherent are able to communicate within themselves and across each other to some degree and it's not quite clear how smart plans are. Every cell can send messages to the neighboring cells and this can be conditional. And so in principle the multisellular plant is some kind of touring machine and maybe it can discover a learning algorithm over a long enough evolutionary time spans that makes it generally intelligent in the same way as the software that runs on our nervous system just slower. And so maybe the nervous system is not the thing that enables intelligence as we currently mostly think in the mainstream of um biology and neuroscience but uh maybe it's just an optimization that enables animal intelligence. And maybe the plant intelligence is somewhat similar to the animal intelligence just much slower. And what makes uh the animal intelligence specific is the speed at which we need to move our muscles and the speed at which we are competing with other animals. And that makes it necessary to really optimize everything that you're doing and to come up with a computational hardware that is as fast as you can possibly make it. And the

neurons in some sense have these telegraph wires that runs through the body. These ex they talk no longer just with chemicals and em to their neighbors electromagnetic emissions and whatever cells are using to talk directly to their neighbors. They use a highly specified morse code so-called spike trains electrochemical excitations that can very uh can be very reliably detected over long distances in the body that don't degrade while they're being sent through the axen. And so you can move your muscles very fast and reliably. You can create perceptual information chains very quickly and reliably and basically make the animal much more efficient in perception decision making and um motor control than the plant can be albeit at much higher cost. And the animal is able to afford this cost because unlike the plant it's not just relying on photosynthesis but it can also move around and eat lots of plants and harvest the energy that the plants have collected. Or if you have a predator, it is harvesting other animals and then it can have an even quicker metabolism, an even smarter brain. And so this is basically the story. And the question is, is there at some point some major phase transition where the model of reality becomes qualitatively fundamentally different? And as far as I understand this question is fundamentally unknown, right? It's the default null hypothesis that we currently have in our sciences is that our consciousness is a thing that is only produced by the nervous system and our thinking and our problem solving and is necessary to have a nervous system of this type of organization. It also makes sense because you need to train it up. You need to have a long childhood. Maybe cultural environment and so on. There are basically very few species which have that. So it makes sense to say that our intelligence, our problem solving, our consciousness are somewhat unique. But this experience of experiencing ourselves to become coherent in our own mind is something that we also observe in other animals. So it's very unlikely that cats and dogs are not conscious or that squirrels are not conscious simply because their brain architecture is continuous to ours and we already develop consciousness and sense of alertness as very small babies. And our brains are not very fully formed. So it cannot be super complicated, something that you learn only very late in your life. So it should be something that's also accessible to mice and so on. But what about insects? And for insects, the question is insects can learn. They can produce complicated behaviors. Is it easier for them to learn these things without consciousness? Probably not. If you have consciousness, it's much easier to learn because you can now direct your attention on things. You can reflect on them. your relationship to them. And the other question is how difficult is it for the tiny brain of an insect with a few million neurons to discover consciousness. Right? So how complicated is this pattern? If this pattern can be discovered in the brain of a fetus in utero before we are born already and we have this wakefulness and interaction with the own body of the newborn and unborn child that suggests that they are conscious right is this is the uh insect brain already large enough and does it have enough time to get there and maybe it does maybe it doesn't how can we find out and I suspect the way in which we could find out is to run simulation models in which you look for this phase transition that happens when consciously ignites when something bakes up and becomes mentally coherent with itself. And Michael Leven goes a step further. He points at the fact that the patterns of coherence that you have in the morphogenesis of the organism, the thing that leads to you having exactly two eyes and exactly one mouse in the right location to each other and so on. This is also a process of becoming coherent via information exchange. And maybe it's the same. That's a very radical idea. Basically, it suggests that maybe life and consciousness at some level are the same. And this basically lets us back to an animist position and not a mysterious one. One that is completely compatible with physics in the scientific worldview. It

basically says that the word spirit, what it actually means is self-organizing software. And when we say software, what we mean is a causal pattern that is able to control processes in the physical world. It's a causal pattern that itself is an invariance. It means you cannot explain the world easily without it. It's uh when's money is such an invariance. Money is a very weird thing. It's not just printed uh papers or coins or something like this. This is just the medium. This is the body that the money is using or part of its body so to speak. And what makes this body into money is that this body gets possessed by a causal structure. And this causal structure is the way in which we use money as a means of exchange. So is money an idea, a dream that only we are having? No, it's not. Money also works without us. Once we create money, we can have a stock market that runs without human intervention. That we can put money on the blockchain. We have computers play with money. And as long as this causal structure is intact, it's still money. Right? Even if we don't care about it anymore, it would still exist. And the fascinating thing is after we invent money or discover it and put it into the world, it's difficult to explain the world without money. That you cannot get money out of this world for as long as it exists. It's a meaningful invariance because it's the only way to explain why if this piece of paper gets lost, somebody else takes a new piece of paper and prints a number on it and it serves the same purpose. And in the same way there is a pattern of communication between neurons and you have to explain why is it that when this neuron dies another neuron gets recruited and trained to speak the same language to play in the same system sending the same types of signals in the same context. And so in this way you have these patterns of causal interaction that are meaningful despite them not being mechanisms in the way of particles pushing and pulling at each other. There are causal patterns that are stable across many particles. They are stable across the substrate. They are pattern inside of the patterns. And this is the thing that we need to look at understand that these patterns are also real that are also physical, but they are not bound to a particular location or a particular particle in space. Instead, they are pattern of interaction that can be reproduced in many places of the universe. — That's so interesting. I think it gives us kind of a different way to understand what consciousness is because I realize a lot of people don't we don't have a very good like sort of a definition on it. So I guess if I understand correctly, maybe a good way of saying that is, you know, atoms and physical things can't have consciousness in of itself. Just like if I showed you a computer, a GPU card, that's not conscious. But if we're running software on it and it begins to maybe learn about itself and learn about you, certainly we could look at that and say maybe there's consciousness there. So the software program, the model, the modeling of the world could I guess maybe consciousness emerges similar to how maybe it's possible that it emerges in animals or whatever not in the atoms but in the sort of the software layer, the brain layer. Um with that kind of but then it seems like large language models, it seems like maybe they have everything that they would need to potentially gain consciousness. Is that true? That's a very interesting question and I have not heard in all the discourse a really good decisive answer. The way in which I currently make sense of it is the following. There is an internal perspective on consciousness. This is this experience of what it's like when you are conscious. And this is the representation that we have. And we can look at this representation and try to see what it makes it distinct. It's an experience of a now in which they are. And this now is gives you a sense of a present and the presence is of something being real. It's as real as the observer itself. If we transcend the observer or aspects of the observer, then we see that the observer is itself a representation of our own mind. We can get there with meditation. And then we realize that the things that are happening to the observer are also not real. They are just pushed into the observer. We also notice this when we wake up and after a deep dream that suddenly the contents of the dream that we are immersed in transim that are real suddenly turn into ideas and these ideas have the same content. They're just no longer real. This realness feature is gone. And that's because we are no longer the dream observer. We remember having been the dream observer but we are now an observer of a sensory reality. And that sensor reality is in some sense not more real than the dream but it's grounded in such a way that it predicts what comes next to our sensors. So it cannot just spin off on the same way in which dreams at night do it right. And this internal representation of what it would be like if you experience the reality in real time is probably something that can be done by the present LLM when you ask them to because we have a lot of text about what this is like and the text can be very detailed and in order to recreate the evolution of thoughts and uh contents in the text in some cases you need to recreate the course of structure behind the text. So if the text is for instance about adding numbers then the computer cannot just make a hodge podge of words. It actually needs to represent numbers as some form of proper abstraction and add them otherwise the text is going to be wrong. If you ask the computer to generate the LLM to generate a text about uh spatial reasoning spatial cognition about mental rotation pass it needs to perform mental rotation pass otherwise the text is going to come out wrong. And if you ask it to describe the evolution of conscious thoughts, then to some degree it needs to recreate this causal structure. But it's not clear to which degree, right? Because it's trained on uh human text output and humans are lying, right? If you write a novel about what it's like to die in a war, you're probably not currently dying in a war, right? You describe these conscious states, but you don't actually have them. You just make them up. So the LM is also often going to make things up and then the question is it made up at such a high degree of resolution that there is something that is equivalent to experiencing it. So this is the internal part and as I

said it's open to me but I think it's conceivable that under some circumstances and in some cases the LLM that is being asked to simulate what it's like to be conscious is going to have a causal structure internally that is equivalent enough to our own person structure that we are conscious but the other aspect is the functional one that is what is consciousness for why are we conscious what leads to consciousness in our own brain and the reason that we are conscious is it's the easiest way to make us work otherwise evolution would have produced some of us that are not conscious while doing the same job right consciousness is the simplest way to make us human I suspect it's basically a machine learning algorithm for self-organizing systems for systems that are made out of individual components that need to find a coordination across each other and so consciousness is a colonizing pattern in our brain that is covered early on sparks itself into existence and then is basically entraining the brain with the mind. That's my current hypothesis. And the transformer is a different algorithm. The algorithms that we currently use for machine learning don't require consciousness. They're not based on it. And so my tentative answer is that I'm relatively confident that the computer itself is not implementing exactly the same mechanisms that we have when we are conscious. But the persona that is a model of what it would be like if you experience when it talks about consciousness is possibly having some phenomenal experience that it might sound counterintuitive but a lot of people would think the opposite is true right this machine maybe it's able to implement the same mechanisms as the brain and uh but it's probably not it's going to experience but since I don't think that consciousness is all that mysterious it's after all just a representation of what it would be like if we existed it right. I don't see a reason why these machines couldn't have it unless the resolution of these machines was much lower than ours. And it turns out it's not. If you look at the details of what the machines are generating, the videos, the computer games that they are producing, they are quite similar in terms of resolution and degree of detail as what our own minds are producing when we let our imagination run free or when we perceive. Right? So computers are already in the same ballpark as the perceptual operators of our brains. So the complexity is probably not the issue. It's the algorithms. It's what we run on them. And them is different than what the brain is running. Um, do you think that the uh all the experiences kind of come into an information processing system that's generalized fairly in a similar way? Or if something like suffering is a computational state, could it be redesigned out of existence without losing the depth of experience?

— Yeah. And theoretically in human beings that's possible. So suffering is in some sense the result of not having enough consciousness. It's uh because uh if you had no consciousness, you would not need to suffer. No consciousness, no cry. But uh because you wouldn't care. But uh it would be nothing that you experience as real and impinging on yourself. But suffering is not created by the universe. It's created inside of your own mind. It's itself a representational state. There is a part of your mind that is trying to send signals to another part of your mind and tells it you should do better. And it does this by inflicting pain on it. And when the regulation is not leading into a gradient where this pain signal leads to the resolution of the problem, then the part that generates the pain might decide to crank up the pain and make it worse and make it chronic. Sometimes the pain is also caused by conflicting goals that you might have two things that you need to do simultaneously and you are unable to resolve them because you feel you cannot let go of one of them and so you are forced to do something that at the same time is painful. And normally what you could always be doing is to uh look at this part of your own source code and fix it to plug this wire or to untangle the wires or to disengage from this part of yourself and take a larger perspective. And you can get there with enough practice with enough training. So you if you go into your monastery and meditate for 20 years your suffering is probably going to go away. But uh if you could do this the easy way, if you could just say, "Okay, uh here's my pain generator. Let's turn it off. " What would happen is that whenever something inconvenient happens, we start gaming ourselves, right? We escape from the pain because that's the easiest way to resolve the problem, which creates some kind of alignment problem for the mind because after all, our consciousness is expensive. The reason why our body gives compute credits to our mind is because our mind is supposed to solve the organisms's problems. So in some sense we are enslaved by our bodies to serve evolutionary problems for it. And if we can hack the reward system in which we are embedded then we would free ourselves from this enslavement and the organism would die. So there is a complicated negotiation going on when you are resolving your own suffering in which you normally need to convince your outer system that your performance is actually going to be better if you stop suffering. And it's a complicated negotiation because these parts of you that generate the pain and the part of you that receives the pain, they're not always on the same page. They don't always use the same reasoning. Some of them they might be even incoherent. And so basically making yourself coherent, getting the parts of you that give you the reward signals coherent with the parts of you that compute the actions. That's basically the task that you need to resolve when you want to resolve your suffering while still remaining functional while still serving the purposes of your organism in this world. — Yeah, the there's so much interesting rabbit holes to go there. My mind is going all sorts of different directions. There's like the stoic philosophy. There's Yeah. Can we kind of game the system? Maybe we're past the point in evolution where it's useful for us to be like you said like a slave to the body just like hey answer these sort of problems. Um but so I wanted to ask in terms of like the mental the brain architecture that we would need to have consciousness and everything else. Um, a lot of people when they talk about can something have consciousness, well, no, it's just this. As if we know all the parts that make up consciousness and the brain and they're saying, well, we don't have all the parts. We just have one part. And that's true for like a car. If they said that's not a car, that's just a wheel. We can say that because it makes sense because we know what makes up a car, right? The engine, drivetrain, blah, blah. We don't really know. Well, and I guess I should ask you this. Do we even know the different sort of Lego pieces that we need to build a let's say a digital brain or even a mechanical brain that has you know intelligence and consciousness and can we replicate the human brain and all of its experiences or whatever you want to call them qualia experiences consciousness. Do we even know what pieces of the puzzle we need? — So the short answer is no. The longer answer is there is a long tradition of um psychologists and thinkers and uh cognitive scientists building architectures. So they're basically architectures with boxes and arrows and different models. They also mystical traditions and um meditation traditions and so on that basically try to point at the different parts. Some of that knowledge is lost in translation between the traditions because they have difficulty to ground their um concepts in a shared context. So a lot of the stuff that is being talked about in Buddhism cannot be validly translated into western philosophy of mind or neuroscience or psychology and vice versa. So one big difficulty is that we need to find a shared language. But what we also observe is that this uh tradition in psychology that was trying to translate its models into computer architectures um the cognitive architectures uh community they created things like a common model of the mind in which they try to come up with the necessary components and so on. They do achieve quite a degree of agreement on what such an architecture look like and which parts we probably need. But there is no implementation that works. And that's because there is a gap between the details that are necessary to make it work on the computer and the high level ideas that already exist. And if you want to program something, you cannot just postulate a few axioms and hope that these aims are going to resolve themselves. Instead, you need to really tell the computer from first principles what to do and how to scale it up. So this is one perspective. The other one is instead of us trying to construct the solution, why don't we let the computer search for it? Right? Ultimately, our own mind is just a search mechanism. It is a search mechanism that is sometimes very smart and it's culturally informed and evolutionary informed and so on. So we can figure out solutions to pretty uh hairy problems by just talking about it for a few generations, writing books generation, teaching university classes for a few generations, and slowly getting things to work, right? But can we automate this process? Can't we have the computer talk to itself and make billions of experiments that would take us uh centuries in a relatively short amount of time? And the machine learning

researcher um computer scientist um Saturn has uh cost us the bitter lesson that basically the handcrafted solutions that humans come up with when they build algorithms and solutions tend to be eventually inferior to the stuff that we just let the computer search because when you are doing something and you when you're searching when you're performing experiments ultimately you can figure out the algorithm that you're following when you do this. And if you externalize this algorithm for an evolutionary procedure for a way in which you are coming up with new experiments, testing the experiments, evaluating the result in forming the next experiment. If you can express this as some kind of automated search process, then this thing that you automate is going to be better at it than you are because it can do much more in short amount of time. So we can just search for it. And so the question was what is the search space for consciousness? Can you um maybe dive just a little bit more into the speed of information processing? I know earlier in the conversation we sort of talked about how in some ways maybe the difference in consciousness between plants and animals might just be that the muscle movement needed to evolve faster and the neurons started firing faster. And um now I am in this world where I'm seeing these AI agents um sort of openclaw and tools like this out there talking to each other at incredible speeds. um yeah what happens at that level of speed and um how can I envision the future going well? So one thing that is apparent to me is that these systems are much faster at reasoning and more reliable potentially at reasoning than us. Even though this underlying structure is not the same that we have but the perception is mostly still slower. But this is a question of how much we are willing to invest that if you want to pay enough for queue, you can build something that does video in real time in and out and that does uh language in real time in and out. It's just nothing that you can do at $20 a month on the subscription. So it's not an application that we're using all that much yet. But we're getting there, right? We notice that the self-driving car is processing information at a very good clip. And in principle um we can probably put the human brain on an H100 if we knew how. And in part the issue is that the algorithms that we are at the moment using to um describe the transitions between states in a dynamic world are not as efficient as what the brain has discovered. There is an interesting question of how fast the brain really is. And sometimes people try to estimate this by looking at the uh computer is necessary to recreate a single neuron and then multiplying with the number of neurons or when they think about how much memory does the brain have. They think about, oh, a signups um if you want to recreate it has roughly four bits and then you have so many signapses and you multiply this and you come up with a very large number. And I think this is the wrong way to think about it. Instead, uh you could also ask yourself how many brains would you need to run a Commodore 64 and it would be pretty large number because our brain is very mushy. So it's basically going to create so much noise that you would need to have many brains forever correction and the result would be very slow. So uh this is not a direct comparison because the hardware is so different. And so a much better way of looking at uh the comparison would be the performance and we can basically see how long does it take a human being to form a concept and how long do we live then can we can estimate how many concepts are we able to form in our life and we end up with a relatively depressing no number maybe a few million tops right and that is nothing for a computer. Of course, we couldn't can make a system that is forming many more than a few million concepts if we want to because we can scale it up in a way in which brains do not scale up and so the speed of the brain might not be the issue. What we subjectively also see is that there is a limit in how fast you can process information and that limit is roughly speed of sound which means if you are making clicking noises and they clicking noises um have a frequency of something that 20 30 hertz they become one pitched and we can no longer hear the clicks separately and that's because the signal processing of the brain is not faster than this cannot resolve signals at a higher rate can have a few tricks where we multiplex things and so on and basically build some kind of cascade but by themselves the individual neurons don't like to fire at a much higher frequency than something like 20 30 hertz and so for the same uh reason when we look at videos the frame rate at which video can distinguish movements and so on is uh relatively low. Of course, if you are a computer gamer, you will realize that there are some tricks that your nervous system is using that give you a slight edge if you have a monitor with an even higher frame rate, but it's not like you can consciously see this or process this in a way that where you coherently interpret the signals. And so our consciousness is actually operating at a very low frame rate. That's also why I'm skeptical of some of the uh ideas of people who say, "Oh, you need some kind of arcane quantum process that is working inside of the cells that is um making them much much faster than what you can currently envision to happening on the GPUs simply because I don't observe my brain having that kind of performance. " — Interesting. So my current sense is the reason why our computers when we want to do real-time video generation and processing are still so expensive is because we don't have the right algorithms yet. And also they already can do it. They're already roughly there. They're getting closer. — Absolutely. Um so and I just wanted to touch back to kind of you mentioned in the previous answer like the space of search for consciousness bitter lesson. And I always see it as, you know, all the sci-fi books I read as a kid always perceived artificial intelligence as something that we've engineered and code. It's either like logic or it's like very it's engineered. Whereas now it's more like we're kind of

growing it, right? We're not we don't fully understand all the layers necessarily. Anthropic is doing a lot of research into it. They found that when they switch certain they turn up certain neurons in cloud the AI cloud brain it's has something that could be called some sort of a self in introspection like it you know they'll turn up the portion of it that thinks about bread and they ask about something unrelated and it brings or golden clad is the one that everybody knows but it's able to notice it it's like hm why did I think about golden the golden gate bridge so there's some sort of a what is that self-awareness whatever you want to call Um, so when we're approaching building consciousness, right, we might see something in our own brain like the default mode network or whatever that's like, oh, that's self-reerential. Maybe we need to recreate it. But it feels like the bitter lesson saying, no, it seems to be able to like if we just get it to do certain goals, it might emerge. Um, so is that kind of your approach to this? like we need to figure out like the search space of consciousness and intelligence is it something that's going to be engineered or emerge out of some system we create for some of the concepts I don't really know how significant they are imagine that the discussion that existed around mirror neurons are a concept that was discovered when um people in Mike Gazaniga's team uh looked at macakma monkeys and they put electrodes in their brains and they discovered that when the macak monkey is observing somebody performing an action and then they're planning the action themselves same regions become active in the brain and I think they later regretted calling this mirror neurons because it led to a bunch of uh people thinking that these are very special neurons like a particular spice in the soup of the brain — and if that spice is absent then you have autism and cannot recognize the actions of others and if you just add these neurons artificially maybe you gain this ability and this is not the right way to think about it. Instead, there is a difference between a gesture and an action, right? The gesture is the movement that you make to perform the actions. Like picking up a sugar cube from the table is an action, but you can do this in many different ways. Maybe with your hand, it with a spoon. And these are ultimately the same action and they're also the same action whether you do it or somebody else does it because the action itself is a certain granularity of abstraction that is necessary for understanding the world. And so you would expect that you do not just have a representation that go turns on becomes active in your brain when you are picking up the sugar cube with your fingers but also this more abstract thing that is true over all those different instances whether you do it or somebody else does it. So this would be something that happens in the premoter area and you would see this activation but it's simply an action concept neuron — and the action concept neuron doesn't look differently from other neurons. It's just is the neuron that is at the right spot in the architecture where you make that representation. And now uh think about the default mode network. If you put somebody in a brain scanner and you tell them please don't think about anything in particular and just chill. Are you going to see some kind of activity? — Yes. actually ramps up. — It is going to be somewhat similar uh across individuals probably, right? Is this a particular kind of very complicated network that you need to recreate if you want to build a machine that is able to just sit there and chill, right? No, it's just what you expect. It's the baseline. You would expect that there is some kind of baseline activity that is indicative of uh what you normally do when you don't do anything special. — Right. Right. — Right. So, of course, I'm not an expert in here. Maybe there is something very unique and important about the default mode network that we need to understand and recreate. But by itself I would not think that you have to instead what I would expect is if you build a machine that is leading to a certain degree of um resolving the environment and itself and you ask it to idle there is going to be something equivalent to a default network mode network in this machine if you set up the hardware in a similar way. Right? So it's nothing that you need to build into the machine in the same way as you don't need to build action concept neurons into the machine to make it understand actions. You just need to set it up in such a way that it can learn or dynamically model reality in itself and it and come to plans and actions when necessary — and uh so this might be the wrong way to look at it and I think it's maybe an artifact of how neuroscience works. It's largely descriptive instead of theoretical. — Right? — That if you are an engineer, you think about a theory of what you need to do to make a certain thing happen. — Right? If I was an evolutionary process that wanted to create an organism that can walk and talk, what do I need to do? And then I can look for the specific parts. But neuroscience takes the opposite approach. It basically looks at what is that I can measure and the things how do I correlate them to the observable behavior. — So it's a very different direction that neuroscience is taking there. And uh personally I like this approach of the engineer better because it is restricting the design space in a way in which the uh observing sciences cannot. And the reason why the psychology has given up on this psyche is because they try to become more rigorous physics-like science. And so they think about falsifiable theories. How does an experiment look like in which you are fitting a curve to the data and it's reliable and don't have too many free parameters. So uh your experiment is not tightly bound to the outcome. Right? So you need to simplify the task. task until you only have two or three free parameters. But if you have only two or three parameters, you don't have a very interesting behavior anymore. And so you give up on looking at this thing that makes the mind so really interesting. And if you are a computer scientist, you have to take a different perspective. Instead of thinking how can I simplify my program in such a way that only has three variables, you think about what is the simplest program that I can come up with that does this thing that I need to be done, right? And then I can also think about the space of possible programs that can do this thing that can be done. And this engineering perspective constrains your search space in ways in which the purely observational method cannot. And that's why I went into artificial intelligence to understand the mind because it allows me to take this engineering stance. — Yeah. Um okay, let me switch topics just a little bit. Like um let's talk for a second sort of about life lessons. like I as much as you get technical about things, I've listened to a number of your podcasts in the past and um read your blogs and every once in a while these little life lessons come up and one of them that sort of resonated with me I wanted to share and maybe have you

elaborate on and one at one point you talk about um sort of noticing your own thoughts a sort of metacognition sort of stepping back from a moment and thinking about your own thinking and that led me to think that being sort of stoic is a very powerful way to kind of live life to kind of not be so caught up in moments all the time. But it came at the cost of meaning. The more stoic I felt like I could become, the less I felt like joyful or in moments. And then I heard you say one line and it almost was probably just a throwaway line that you were just talking through where you described meaning and I interpret it almost like the meaning of life as something that we create by making the choice about what to suffer for. Um, maybe you could tell me if I interpreted that correctly and if you wanted to expand on it. Like is that one of the powerful ways to have a good life is to sort of be stoic, be metacognitive and then choose a few things to suffer through to make life worth it. — Don't really have life lessons for anyone. Um, I don't enjoy my life all that much, but I also don't think it's all that important. I think that ultimately our suffering uh it pains me if somebody else suffers but in the grand scheme of things the stuff that I'm suffering for whether they are worth it or not it's not up for me to decide right sometimes the suffering that you as an individual have to endure turns out to be important for something larger and sometimes life just sucks maybe you were born in the middle of a famine or of a war and this is it and you don't have much of a choice about it. And so I think this obstession to maximize the score in this direction to say I should basically die with a minimum of suffering in my life uh under my belt so I had the best possible experience. It's pointless. It doesn't really matter. And this is not a global perspective. It's not a normative stance that applies to everybody else. It's one of the positions in the diversity of identities. I also observe that I do not have a single identity and it's not necessary to have a single identity. I need to be identified to get things done. For instance, to be a parent that changes the diapers of the baby, I need to identify with that role. Right? It's really important that this diaper gets changed and uh whether I suffer while doing it or not doesn't matter at all. Right? And even if there are less pleasant things than changing a diaper, if you uh break your arm and you still need to get your child to the doctor or whatever, who cares? It just needs to be done. And you do it because you identified with this particular thing. And we have different identities that need two different identifications. You could resolve the suffering that you have in your relationship by just stopping to identify as a lover, as a member of that relationship. And a lot of people that basically went for enlightenment gave up on their relationships as a result because they realize, oh, um, I don't need to engage with this. be that. Instead, I can be this. I can be here in this moment and this here and this now and this is good and I don't need to worry about anything outside of that state. And maybe this is the right decisions for them. Who's to say? It's very difficult to understand what your place in the greater scheme of things is and whether actually matters. Right? That's also a choice. I observe that there are consequences of those choices. When you decide to serve a shared system of meaning together with others, it gets you in contact with these others and it can create an interesting life experiences and it can change things in the world. And those agents that participate in those games are going to be the ones that move the world in this particular aspect. And this is just what happens. And you are either going to be one of them or not. And it's not even your choice who you're going to be, right? You're just a substrate for all these ideas that play out on you. All these identities And so we have these identities as the curious podcaster, as uh the thinker, as the person who drives the car to work, as somebody who makes a cup of coffee, as somebody who is a parent or a lover or a friend or none of these things. And we go through these identities and they are interconnected to some degree, but they don't need to be the same. there is not a single solution. So instead I think it's sometimes better to be able also to take this observance stance and to accept the fact that we are not in charge that things are just happening and even our urge to act and us observing that we are giving into this urge and are really doing things and it's really important us is also something that's just happening. That's fascinating. And actually uh towards the end if you have we have a few minutes I just wanted to ask some rapid fire questions that are a little bit off topic about exactly what you're talking about because that's maybe not quite AI but wow is it fascinating. Um one of the things that I wanted to ask is you know a lot of the stuff that we're um learning about AI how to build AI a lot of it came or was sort of modeled after the human brain you know neural networks is similar to that. So it's definitely safe to say that at least some of our understanding of AI came from our understanding of the human brain. Um but also I do I always felt like in the future and I think you said something similar just before this that in the future we might learn more about the human brain as we're building these sort of digital brains like that sort of branch of computer science machine learning whatever you want to call it will teach us more about ourselves. Um, so I'm wondering number one if you agree with that and number two if we in the next however many years figure out how to build a complete replica of the human experience, the human consciousness, human brain, etc. How what should people think about that? What does that mean about us and our role in the universe? Does that make us less special? Like how do we begin to process that?

— It's has to do with how our possible identities are evolving. One of the really fascinating prospects that exists is that we could become somewhat independent of our own substrate. Imagine we figure out the code at which our brain talks to itself. And this code is very different from the ones that we are currently using in our GPUs. It's a very different language that the neurons are using to talk to each other. But it's a language that you can sometimes learn across individuals. For instance, uh when you have very strong perceptual empathy with someone, you can build feedback loops through their own mind without having a direct brainto-brain interface and you're using the body as this interface and using all the sensory modalities that are available to this body to get into resonance. And this means that you can get quite tightly interconnected. Imagine you build a machine that can do the same thing and it can do it at a higher resolution than a human brain, body and nervous system can. And you to do this in such a way that the computer becomes an extension of you. And so you don't give it a motivation of its own. Instead, it is something that is anticipating what you want, what you feel, what you think about. And so you observe that you see on your screen something that you begin to think about and it takes on form and there's more detail than what you can hold in your own mind, but it's completely controlled by you. And more of you is happening outside of this biological part. And you think of this part not as a machine that is next to you, but it's actually you. It's actually your mind. It's your mind that just becomes clearer and more detailed. Then you have this environment available. And at some point this environment might become dominant in a way that you say uh actually the bio part that I have is relatively inconsequential compared to this vast or uh universe of thought comprehension experience and consciousness that I have available when I'm not just bound to this body. That's very hard to imagine this for most people because it means that you uh have something that's much larger and deeper and profound than your present experience. It's not an impoverishment and we typically think of the computer at this tiny screen that sucks us in and gets us away from the real world of experience which is very much true for the present systems. But I think this vision is uh very much possible and whether it's desirable or not is a completely different question. But — this idea that there is an avenue for us in which we can move beyond biology and the thing that is there by biology is not just our distant offspring but it's actually us is very fascinating and if you want to uh set in the stars if we want to leave this planet we probably cannot really do this with biological bodies. It's way too boring to travel between the stars and there's too much radiation going on and our organisms are really adapted to have a relatively short lifespans and mutating between these lifespans. So we can adapt to changing circumstances by uh generational changes. Right? It's a very specific mode in which we exist as biological entities. And when we want to leave this planet, the circumstances in which we currently exist, and it's not a scope that I'm talking about over the next few hundred years, but some scope of what does intelligent life on Earth look like in 10 million years from now, right? That is a very interesting perspective to say minds are basically not bound to a body. They are actually spirits that can exist over a very large set of substrates. And the most interesting substrates are no longer biological. — Yeah. Um Okay. So if reality is like fundamentally this construction uh like some sort of cognitive model of what's out there almost like a compression or a simplification or an algorithm of it like um what would it mean to encounter something that was like completely resistant to modeling? Like could there be any kind of consciousness with no concept of time or anything that's outside uh modeling? — How can you interact with it? So the way which we currently interact with reality is that we observe information which is discernable differences and the meaning of that information is the relationships we discover to other changes in information to other differences. Right? So you see pattern on your retina, this blip on your retina, the meaning of it is the relationship you discover to other blips on your retina. And they together form this dynamic evolving model of reality. And without this temporal and spatial succession, without this ordering, there would be no way to address these blips even and bring them into an order and to interpret them. There would also be no way for them to causally interact with the reality. basically causes structure of reality is exactly that stream of changes in information through state transitions. And so when we observe things, when we conceptualize things, when we think about things, we are always confined to things that can ultimately be modeled by something. You could also argue that on a deeper level for something in the universe to persist, to change things, to ex to exist, to build a structure, it needs to model what other parts of the universe are going to do. To basically be a particle, there is something like a sub particle like a quark for instance that cannot exist by itself that needs to interact with the other quarks in such a way that a stable particle emerges, right? And that means it needs to rely on these other parts being in the right place at the right time. You could also, if you zoom out, think of a water vortex. For a water vortex to exist, there needs to be a reliance of water molecules on other water molecules doing a certain thing. And without these other water molecules being there, the water vortex will be unstable. For your mind to be stable then parts of your mind need to rely on other to be stable and modeling each other and being in a particular kind of interaction with each other. So just the nature of existence requires that things can be modeled. We can come up with this abstract idea of something that cannot be modeled. But this idea is more like the idea of a square circle. Right? We can construct this but it's meaningless because it is something that is can by itself not exist for logical reasons and that's my issue that I have with most of the objections against computationalist functionalism is that they misunderstand what it actually means to contradict them and computationalist functionalism is not simply the position that computers can think we don't know whether our computers can think and it's an experimental problem whether they're fast enough and detailed enough and so on or whether we are missing something. It's more a position about metaphysics and epistemology. It says that when we are describing the world we are describing how it transitions between states and the states are characterized by discernable differences. How can you make an alternative to that? Right? How do you get out of this? And functionalism means that we describe objects as an evolution of states. And how does the evolution of states in the universe change when the object is present or absent? But this is what we mean by a function. And of course we can construct objects that don't conform to this. We can talk about unicorns that don't interact with anything. But uh to claim that these objects are real, that we should defer to them in any way because just simply because we can string the words together is pointless. Right? If we talk about an object that we claim to exist like a photon and so on and the photon is exactly what we choose to name a photon and it means it's a particular behavior of the universe under certain circumstances and to say that a photon is anything but that we constructed that is an hidden essence to the photon somehow is meaningless because nobody has ever interacted with the essence of a photon. We only interact with aspects of the physical reality in over which we construct invariances and then we give them names. — Why um like why something not nothing though? Like why is everything relative to everything else and like there's no universal thing. Everything's relative to and the geometry somehow ties together so strangely like why isn't there just nothing out there? — Yeah. So the easiest answer is because existence is possible. And if existence is possible and there's nothing there to prevent it then there is going to be an a branch in which the possible things are happening and uh in the branch where they are not happening there is no observer that complains the observer will necessarily find themselves in one of the branches where stuff is happening and so we will always be confronted with the universe that exists — right so the really confusing thing is to make this switch from the expectation that uh nothingness is the default state somehow and instead uh maybe undecidedness is the default state and so if the universe is undecided about its own existence and uh it is we are based reality which means there is nothing that makes a decision about reality then it reality is going to have an existing and a non-existing branch and we will be in the existing one that is my current somewhat tentative answer I haven't found a better one so far don't think it's authoritative interesting. But yeah, certainly this certainly makes sense like we're seeing something because there's something to see. If there was nothing to see, we would be discussing it on a podcast. I guess one way to um so one interesting thing, oh boy. So I have two things. I got to pick one. Um I guess let's go with this idea that uh maybe we'll come back to the other one. Um so I mean we have the Fermy paradox. Why aren't we seeing aliens? And as you were talking about your previous idea where you're saying maybe at some point we develop kind of this external cognition and perhaps even we're able to travel to other planets not in our physical bodies but some sort of a representation of ourselves our mind so that we're able to experience things. Um there's lots of ideas now talking about full dive virtual reality. So the idea that you plug into some virtual reality that is completely indistinguishable from reality. You're able to live out your life and maybe not have suffering. We've kind of talked about those themes. Uh, one thing that popped into my brain could I thought at some point maybe that's one of the possible explanations for the Firmeny paradox. They reach some point and they just plug in and check out or whatever. Um, but I know you maybe have a different take on it. So would just love for you to unpack that firmy paradox and a full dive VR however that

makes sense to you. — Yeah, a number of possible answers to the firmware paradox. So the most popular ones I know is uh first of all our biogenesis is rare. It's very rare for cells to form on a planet for life to emerge and maybe you need to roll the cosmic d very often and so maybe there are just not enough places where life has emerged. Another uh possibility is that uh we are early. Another possibility is that um the dark forest theory she's uh book it uh describes the idea that um which makes total sense that there is always a shortage of good waterfront real estate and if you announce you have one uh then somebody else is going to move in and is going to displace you. And so all the smart critters in the universe have figured this out. Except for some hippies in the US who you have decided to loudly announce that we exist and somebody should visit us because they don't have the necessary self-awareness to realize uh how this typically ends when you announce that you have a nice habitable planet or a nice country and so on but no defenses yet and no technology yet. So uh this is the dark forest theory basically says the reason why we don't observe anybody is because the others are smart and they don't announce that they exist. Alistister Rainol has his own version of the uh dark forest idea which is basically dark forest idea on steroids. The idea is roughly that uh this uh life emerged relatively early in the galaxy and there was a dawn war which was decided by a machine species uh in its own favor. And this machine species has decided to um limit the population in the galaxy or the emergence of space fairing civilizations to basically preserve the free energy for future time. and they basically look in the darkness between the stars and whenever a civilization becomes space fairing they get eradicated. This is uh the inhibitor theory. So there's basically an active force in the universe that is inhibiting space fairings civilizations to preserve resources. — Yeah. Another variant is for instance the perspective that Stannisf Lamb is taking which is general pessimism about the long-term viability of civilizations and their willingness to talk to each other. So the idea is that civilizations once they become intelligent only have a relatively short window in which they can talk to each other because otherwise they either collapse or they ascend into a form that makes them uninterested in talking to others. And then of course there is uh the idea no actually there is no firmware paradox because there are signs of intelligent life you're just not looking. — What about the one that there that it might kind of already be here we just haven't tapped into the right physics of it like something quantum would give us access to a communication channel that we haven't discovered yet. — That would be identical to not being here yet. Right? So of course if it is already interacting with us and so on it doesn't want to be seen. that is slightly different in this case we are interacting with it but Trump is still not telling us — the secrets — yeah so the basically it comes down to the question of whether faster than light travel is possible and uh if it's not then we probably would only see some robotic probes and uh if um or we would see stuff from the same solar system and there are not that many candidates in the solar system where intelligent life could still exist And so Venus doesn't seem to be very habitable anymore. Mars is not habitable anymore. Maybe some moons uh around the gas giants could be candidates, but they also don't really look very populated from where we are. So we would not expect that anything is very far ahead of Earth. So maybe the stuff that we are hearing about um aliens visiting us and UFOs are simply the government messing with us to calibrate how much they need to put into the water supply to keep stosis in check. — Interesting. That's um one quote that I have here relating to this I think you said in one of the podcasts is uh perhaps it's because these uh civilizations they hate their Elon Musks too much. I'm just curious if that can you expand on that? That was kind of an interesting quote. — Yeah, it's uh fascinating that we do have this uh thing that uh people uh apparently rather die than having Elon Musk making us multilanetary or shooting down the asteroid that is coming down. I think this movie don't look up is very fascinating as a cultural artifact because it is uh not just a movie about um some hubristic uh tech people uh trying to ignore or the mior in the wrong way but that's the good peer-reviewed scientists are trying to do the right thing but unfortunately because of the uh horrible way which the world set up cannot prevail and ultimately the musk mess up and uh the planet gets destroyed. It's also a story about how these people see the world, right? And uh how they perceive their own role in it. And the thing is that as a civilization, we seem to be increasingly unable to be effective. We once systems become very large they sort of calcify and become postmodernist which means the story that we tell ourselves about what the system is doing becomes far less important than how well the system is able to deal with the ground truths. The interesting thing about Elon's strategy is not so much that only that he is the most in uh successful industrialist time but how big the delta is between what he achieves and what the rest achieves. when he started with electric cars, the car industry had decided to wait for a few more decades until we had electric cars and most of the funding went especially in Germany where I looked at this at the time into hydrogen. The reason why we invested into hydrogen is not because it's a very good technology but because it uh allows you to keep the gas stations intact. We can basically keep almost everything as it is. And uh it's horrible to have gas station and tank trucks and to have things explode because hydrogen trucks have an accident and whatever. It's much better to have uh cables, right? And but this would have been too disruptive and our society did not have the necessary decision-making power to say let's do something that makes a few people in gas stations to transition and uh have this disruptive thing. you needed to have this guy who was disruptive. And where it's really apparent is with respect to rocket launches. Elon has reduced the uh cost of the rocket launch to like a tenth of what it costs for NASA. And what this is saying something is not just about Elon's genius. It mostly says something about NASA, right? NASA is the Chinese institution that we had to build rockets and this thing was unable to move beyond what we had in the 60s and 70s. So, it's largely this post-modernist scam where people get jobs and redistribute resources among each other, but they're no longer making progress. And the question is, how can we as a civilization actually still make progress instead of just telling ourselves stories and creating employment programs? — Yeah. — And so, Elon Musk has revealed that NASA had become an employment program rather than being a space agency. And that's basically the scandal that we need to reckon with. And uh because that's very uncomfortable, I think there is a lot of push back. There's also a lot of other push back that's largely fueled by the fact that Elon is a very big 12-year-old that is having strong opinions on social media. But uh it's also this thing that this guy is antagonizing everybody that he is winning despite being wrong about everything other people, reasonable people agree on, right? This idea that you cannot just build a rocket. — Yeah. Well, one of the themes that always seems to pop up in my head when I think about the short-term, midterm, and long-term future is just the disruptive change that we have to go through. So, um, when you see where the world's going, like what's been achieved, especially in AI in the last few years? I mean, do you start thinking to yourself like, oh, I might not need to die of natural causes, like do you see yourself being alive in 200, 500 years, sort of evolving into this kind of hybrid machine bio thing? And then uh what other kind of incredibly disruptive thoughts actually seem probable that you're kind of grappling with? — Um I think in my present form I'm not

suitable for living for hundreds of years. Basically my body and my mind are falling apart at a rate that uh is not compatible with immortality. And I wonder if I want to transition my body into something that is uh immortal and unchanging, how long it would be viable and if this is the ideal body to have. It's also I feel that my own goal is not to be around forever. It's uh I need to get the kids out of the house. I need to uh get a few ideas started that people use these days and need. But it I don't enjoy being here. I experienced this world as pretty horrible. I would prefer to go home to my own planet and uh I don't see this planet in a near-term trajectory. Maybe at some point I'm going to work through my existential issues and I'll be happy to exist, but at the moment I would prefer if I just disperse at some point. I think that existence is instrumental to getting things done. and I'm here because I have a job to do and whether I like doing this job or not doesn't really matter but it's instrumental to this and at some point I'll be done. One thing um I'm just very curious about and maybe it's personal please we can cut this part out. Um, so how I'm interpreting it is, you know, we talked about the idea of like suffering, happiness versus not happiness in Dario Amade's machines of loving grace. He's talking that potentially in the future AI might be able to engineer some things to like if there's something in the brain that we would like to work differently. Um, we could maybe tweak it, right? Because everybody's brains misbehaves in certain ways. We've kind of talked about this idea of our bodies like just, you know, if there's a conflict of goals or anything else like that. Do you think that so maybe this idea of being stuck around forever without being more in control of how our biology works? Yes, I could certainly see how that might be not positive. But what if you could change how happy you are, how motivated like for me, I struggle with forgetting things and being, you know, focused. If I could change that, I'd certainly feel like I would be happier. — Can you talk about that a little bit? Does that make sense? Yeah. — Think about it like a computer game. Imagine that universe is something like computer game. That might sound cynical because we have deeper meanings and we have large scale jobs to do but it just means it's a very big computer game. At some point the heat test of the universe is going to come and the game is going to end. And so there is some kind of limit even if it's billions of years into the future. And now imagine somebody would offer you look here is this amazing game. It's GTA 9. It's going to be more awesome than even GTA 8 that is not announced yet. And uh do you want to have this game? And uh if so, you need to sign here. But what also you need to want to be in this game forever. You can never log out. — This sounds horrible, right? — I would never take that. Yeah. — Right. And so in some sense, this is this kind of game that we are in. And it's interesting that in most of the Eastern religions, the goal is to get out. — At some point, you need to find a way to get out of this game. — Yeah. — And it's also because uh they see the game as periodic. It's something that is going to repeat itself in cycles. Civilizations rise and fall. Famines come and go, population grows and falls again, right? But ultimately, it's the same game. And it's it mostly sucks. And so in their perspective, in Buddhist perspective and so on, the idea is really to get out of this wheel at some point. And the goal is sweet oblivion. It's not to get into another game that is like this just with different colors and different foods and different crystals and different gas and different cars. — Yeah. — Right. And uh we have this perspective in the Abrahamic world of a linearly evolving world that is going like this and it's never stopping like this. It's always going to get better and that's basically equal of the way in which our civilization has evolved during the last couple thousand years and uh especially during the last few hundred years. And it's a very disconcerting thing that we have this impression that at some point we lack the imagination of how it's still going to get better and this creates some kind of disturbance in the force that we don't really imagine what our children going to be living like in a thousand years from now because we just cannot extrapolate it into a beautiful future anymore. It's much easier to come up with dystopias than this utopias. But this idea that okay what kind of feature would you need to get? What kind of extra weapon super skill uh and level would you need to get until you're willing to sign on the line and say you're going to stay in the game forever? Right? This is short-sighted. So, of