Demis Hassabis On AGI, Advice For Indian Engineers, AI In Gaming & More

200 plus billion dollars of India's exports is IT services. Do you have advice for people in software right now a lot of areas are going to get disrupted and change. I think with change their challenges but also come opportunities. — We wanted to make a world class game right and we wanted to do it bootstrapped. — I saw your game. It's great looking game by the way. Thanks for sending me the video of it. — The minute Genie came out right and then I used it and I was like wait okay at least you can't tell a story yet. What's the next 3 4 years of game development going to look like? — What I'm most excited about is new types of genres of games that might be possible now that we have AI. You could have small teams that could experiment with new really creative ideas because it was fast enough to prototype and build cheap enough to build those games. — Question to you Dennis, what's the capability you would see where you would go that's AGI? — Look, my definition of AGI has never changed. Building foundation models like Gemini is on the shortest path to AGI in my opinion. How does one truly be a polyman? — Yeah. So, I thought a lot about this and um — amazing. This is a really large audience. How many of you are in STEM? — Wow, that's amazing. So, we've got our audience here today. So, thank you for joining us. Um to me it's an absolute honor to share the stage with both of you and you know I've been a fan deist since you know I first saw some of your stuff you know four or five years ago. Um I actually wanted to start with a question that's very personal to me right um which is you know we've seen

everything that's going on with AlphaFold we've heard about the work going on there but how exactly does it translate to India? What I mean by that is India is known for lowc cost generic drugs, right? We're now getting the GLP ones to India. How like what is the exact post process between you starting out with that tool all the way to it becoming you know a drug that people can use in India? — Well, first of all, thank thanks for having me here and it's uh fantastic to be here at the institute. Um and thanks for coming. Look, I think in India and uh what we tried to do with Alpha Fold in the beginning was crack an amazingly hard scientific problem but also one that would have many downstream benefits uh especially in things like drug discovery. So with AlphaFold, but also some of our other scientific work like Alpha Genome, um we've been developing tools that I think can accelerate drug discovery and also help with disease understanding um including things like rare genetic diseases and so on. So I think all of that will impact um India but also all the world uh in the next few years. um we collaborate actually with a lot of um contract research organizations in India already with isomorphic labs that we spun out to build on the alpha fold work and I think I'm also very excited about how our general AI systems like Gemini will help with um providing healthcare information to everyone in the world um and I think again I'm very excited about the impact that could have uh here in India too — you know that's a very high quality problem to pick up right which is hey how can we solve protein folding But I think and this is a question to both of you, right? And it's a follow-up question related to this which is I

think both of you have been in science for a very long time. How does one build scientific taste? Like how do you know what kind of problems to go after and are not worth going after? — Um you are asking about how to build scientific taste in AI systems — in general like how do you as a person build scientific taste? So I think scientific taste usually you as a grad student you learn it from uh your PhD mentor I think that is when you first encounter it but if you talk about developing scientific taste in AI systems I think that's a very interesting and very hard problem uh there have been some interesting attempts like the Ramarjan machine you know which they try to sort of replicate the intuition that Ramarjan had and also you know if but on the other hand if you use standard things like reinforcement learning from human feedback that sort of tends to average things out and you tend to revert to the mean you know and I don't think you get interesting things from there what I think maybe is uh you know maybe you get build a customuilt uh LLM which then uh is mentored by a master scientist you know and so that's master scientist uh the LLM acts like an apprentice to the scientists and you give constant feedback. You need somebody with lot of commitment and time to do it. That may be an interesting way because that is the way we learn uh from our mentors. Uh so that may be an interesting way to do it and maybe who knows you know you may have um future generations of AI who trace their lineage to a human master and you may have schools you know like the demis school asab school of AI systems or terren which think in a way akin to how they think you know about which problems are important which problems are not important because I think if you do it on the average I don't think you are going to get much. I think it has to be much more you know personalized in some sense but I think Dis may have a different take on that. — Yeah. No, I think the question of taste um you know you can break it down into intuition and creativity. It has aspects of both of those things is probably the hardest thing in science and I think is uh will probably be the hardest thing for machines to be able to mimic and I think that's good. I think that's what separates the great scientists from the good scientists. human scientists I would say you know every one of you here is uh of course great technically but then to really discover something new ask the right question formulate the right hypothesis requires good taste and I think that comes partly from graduate studies and um and learning from professors um the great professors that you have here and certainly for that's why it was important I learned a lot of that during my PhD um at UCL with my professor Elanor Magcguire but I think you can only really develop great taste through doing as well. I don't think you can just passively learn it. I think you have to do it and develop it. And um and it's a little bit mysterious what it is I would say altogether. So we'll have to see um if machines are able to develop it or learn it somehow. Um, but I think they're going to need to do active experimentation in the way that we all do through grad school. Uh, to really understand what it means to do science at the frontier at the cutting edge. Um, and so, you know, I think it remains to be seen uh, maybe we'll understand better what taste is uh, at the end of all of that. — Amazing. So, you're saying if you run a lot of experiments, you eventually develop a sense for what experiments are worth spending your time on. That's amazing. Uh, you know, I have a question

for my wife. She texted me on WhatsApp in the morning and she said you should ask them this question which is we've seen alpha fold what is in the future of medical AI like what is something we can look forward to well look alpha fold I think was just the beginning so um of course it was this 50-year grand challenge in science and in biology um and understanding the structure of a protein is incredibly useful for understanding disease and and eventually developing drugs um but it's only one small component of the drug discovery process. Uh you need it's an important component but it's a small component. So we are trying to develop uh many other technologies adjacent to alpha fold at isomorphic mostly in biochemistry and chemistry area to develop uh the right compounds that will bind to the right part of the protein um but also uh other properties we care about like the toxicity the absorption properties of these compounds to make sure that actually uh in the human body they do the right things and don't have any side effects and so on. And in some ways those things are even more complicated than the protein structure. But um we have a lot of belief that this is possible because um of alpha fold the success of alphafold which you know was thought to be an almost impossible challenge and we were able to do it. So I do think these methods can scale to these very difficult problems and um you know we have some very promising results at isomorphic and developing these technologies further and I hope that eventually um we'll be able to bring down the drug discovery process you know it takes on average say a 10 years to come up with a new drug um bring that down by a factor of 10 to a matter of months maybe even possibly weeks and I think that could be possible um it sounds science fiction today, but then so was um finding the protein structures, you know, of of all the proteins known to science. 200 million We've managed to, you know, fold and put out predictions for all of them now. And um that was probably, you know, would have seemed impossible 10 years ago. So I think the same kind of thing will happen over the next decade uh with uh drug design. — I think this is a fascinating use case of AI, right? which is that there is a person sitting on stage right now whose work is going to be used by so many people who are you know sick and need those clean drugs over time. So thank you so much for all your contributions. I actually want to switch tracks here and I want to talk about something totally different which is a personal

passion of mine which is gaming. U we're working on a game we're working on I would say you know we wanted to make a world-class game right and we wanted to do it bootstrapped. Um, but you've been a game developer, right? So, you wore many hats. Game dev was one of the first ones you wore. You work at Bullfrog back then. Um, the minute Genie came out and I saw it, I spent like 5 seconds looking at the screen and I was like, wait, I need to use this, right? And then I used it and I was like, wait, okay, at least it can't tell a story yet. How long do I have left? Like, what's what date should I release before? And what's the next three, four years of game development going to look like? — Yeah. So, um, I saw your game. It's great, great looking game, by the way. Thanks for sending me the video of it. And, um, I love game design and game development. That's kind of how I started my professional career, but also my journey into AI. So, you know, when I was a teenager, I was, as you say, I was working for Bullfrog, which at the time was um, probably Europe's premier development house. did some really creative games, simulation games that AI was the core part of like theme park and um actually really that's when I sort of decided as around about 16 years old that the AI was going to be my career when I saw how much enjoyment people got from interacting with this game AI and uh the potential of that and I think it's come full circle now where AI you know games used to be the cutting edge of where technology was developed graphics and AI and also hardware like GPUs were of course invented for games and now we use it for uh for AI development. Um and now maybe AI has got good enough it can help with game development like you're saying and um I think it's going to help with many things. So for example creating assets and graphics uh 3D models um I think the technologies are pretty good now and probably in the next year or two will be pretty amazing from just a concept art. it could probably create the 3D um the asset. I think what I'm most excited about is new types of genres of games that might be possible uh now that we have AI. So, for example, big um massive multiplayer online games that are populated with game characters, NPCs that are actually smart and can advance the story line and things like that. I think also um there'll be very useful tools for bug testing, autobalancing the games, but then you mentioned Genie. Genie 3 is our world model. So what you're able to do with that, for those of you not familiar, we just released a kind of a beta version of it uh recently where you can just type in a prompt and you get a playable world. Um you can only play it for one minute and then it's sort of like a dream and then it disappears because it can only stay coherent for a minute. But I think over the next four or five years, we'll be able to extend that time. But as you say, that doesn't necessarily it's a it's like an interactive movie. Um, and it's fascinating to try it, but it doesn't make for a fun game yet. That's still going to require game design, game mechanics, all of the uh amazing things that the game industry has built. So, it may just facilitate faster prototyping and faster iteration of ideas. And then hopefully maybe we'll be a new uh golden era of game uh development like it was when I was in games in the early 90s where you could have small teams that could experiment with new really creative ideas because it was fast enough to prototype and build cheap enough to build those games um that you could test out quite experimental ideas and hopefully these tools will allow us to do that again in the games industry. — Very good. Um this I have a question to both of you. Okay. And this is a question about a difference between the average person that I've spent time with and both of you which is both of you seem very crossunctional right which means that you know one day you could be playing chess the other day you could you be making games the next day you're in life sciences AI so it's like you're crossunctional and you've done a lot of the same right you've uh you've done a little bit of chaos then you've gone and you've said

hey let's do satellite based courses how does and I think the word for this is polymath and I think it's just very fascinating being around polymaths because the range of conversations you can have with those people are just so wide. U how does one truly be a polymath? I know it's a tough question. Some you the answer might just be hey you're born with it but you know I'm going to shoot my shot anyway. — Well maybe others may have a different phrase for it. uh you know jack of all trades and master of none but uh still I think it is just the basic curiosity you're curious about so many different disciplines there are so many fascinating things to do uh that you just get into different areas I think it is that basic curiosity which drives uh all this — yeah so I thought a lot about this and um I think uh for me at least I I've always had an insatiable curiosity from when I can remember and uh even in my games career that happened. And so I started playing chess for the England junior chess teams. But then I realized there were many other cool games out there like go and poker and really interesting things as well just beyond chess. And a lot of chess players just stay with chess and that's all they play. And so even in games I could feel myself drawn to there's so many interesting things as the professor says that are interesting in the world. But there's also another thing too which is that I think a lot of the best inventions especially in the modern era will come at the intersection of two or more subjects right and you can think of deep mind uh when we started it as a kind of combination of neuroscience engineering and machine learning right it was sort of the intersection of all of that and now you know you look at isomorphic it's an intersection of machine learning chemistry and biology so I love those areas and I think a lot of the fastest progress still now as well. I'd encourage you to become expert in two or more areas and then find the connections between them but also the maybe the analogies between them. Uh and there are a lot of interesting analogies when you look at things from a first principles point of view. Um, and then the other thing too is that I think uh I've just been drawn my kind of favorite people from the past, my heroes are kind of the polymaths really like you said like Da Vinci or Aristotle who I feel like didn't really see the boundaries between even not just the sciences but even art and science and philosophy. And I like that approach and I feel these are all um about finding out about the world but just using different techniques. So in the end, if you're curious about how the universe works, you should be curious about it from all these different viewpoints. And I suppose for me, building AI as this sort of ultimate tool for science and discovery, that's kind of given me the excuse to um learn about a lot of other subject areas, which you know, I've loved doing uh because we can apply AI to those areas. — Professor, do you think they're making a mistake in science in India by having too many siloed, you know, ways of learning? Because I feel like, you know, sometimes when I speak to people, they say, "Hey, I'm a mechanical engineer and there's no way I'd be interested in any other type of engineering. " Do you feel that's a mistake? — Yeah, I think it's a mistake. I think probably the original mistake was done when we abandoned universities and started uh I mean we also are an example of that of starting specialized research institutions. you know we lost that um you know cross talk between different disciplines and we have become so siloed even you know law is different management is different everything medicine is different we are trying to remedy that you know by bringing medicine back here and things like that but I think it's a serious issue that um India faces and is going to become a bigger issue with AI coming in you know when you really need this intersection of disciplines. So, it's going to be a problem. Yeah. — Maybe I can just, you know, I hope more of you become multi-disiplinary. Maybe I can just give a couple of pieces of advice or tips on maybe how to do that. I think there's a couple of things you need. One is u one reason it's hard to to be multi-disiplinary is of course one has to be worldleading expert in at least one domain, right? This is also why siloing has happened in departments because you have to have that otherwise you can't contribute. but at the frontier of you know discovery. But then what I've at least done is and I think everyone can do is develop techniques to quickly learn to a maybe a grad level other subject areas right how do you transfer your own learning and of course this is what we're trying to do with AI systems but you can also do it with your own mind of you know find those connection points understand it from what you know from first principles so you can quickly apply it very fast to a new area or new domain at least to a sufficient level of understanding so you combine it with your expert area. And I think the other reason I've seen in university systems that people don't do this more is um I think it takes a little bit of humility or uh maybe confidence and humility kind of both together to become a beginner again in some other area when you're already maybe a world expert in one area or very let's say machine learning and then oh I don't know that much about biology so I'm going to be willing to learn from the experts start again you and um be willing to to put the effort in uh to do that learning and I would encourage everyone to do that. It's really worthwhile but I think sometimes the maybe the academic system doesn't reward that um uh side of doing things. — Fantastic. You know I have a question on general intelligence and I'm sure everyone has this question right which is for a very long time I had and even before the entire AI wave you know I grew up reading asimov and I just said one day there's going to be we're going to have AGI right but you know as I got older and I saw you know Gemini come out and a bunch of models come out I said this feels like AGI but then the goalpost moved right and we said no no but it has to do this so I made a joke out of it and I just my Twitter username is waiting for AGI because I'm just like it's the kind of internal joke I have with myself because nobody else gets it. Uh but a question to you Demis

what's a capability you would see where you be where you would go that's AGI. — Yeah I agree. Uh look my definition of AGI has never changed. So which I can tell you is we've always defined it and I've always defined it since I started working on this 20 30 years ago as a system that can exhibit all the cognitive capabilities uh humans can. Now why is that important? First of all, because the brain is the only existence proof we have that we know of um maybe in the universe of a general intelligence. That's also partly why I just studied neuroscience because I wanted to understand the only data point that we have that this is possible, right? And understand that better. And um and so that's the definition I use. is quite a high bar because it means uh if you wanted to test a system against that it would have to be capable of all the things humans can do with this brain architecture which is incredibly flexible. It's clear today's systems uh although they're very impressive and they're improving they don't do a lot of those things. So true creativity um continual learning uh long-term planning they're not good at those things. And another thing that is missing is general consistency across the board at capabilities. You know, of course, in some circumstances, they can do get gold medals in international maths olympiad questions like we did last summer with our systems, but they can still fall over on relatively simple math problems if you pose it in a certain way. So you that shouldn't happen with uh a true general intelligence. It shouldn't be sort of a jagged intelligence like that. So there's still quite a lot of things missing. I think the kind of test I would be looking for is um maybe training an AI system with a knowledge cutoff of say 1911 and then seeing if it could come up with general relativity like Einstein did in 1915. Right? So um that's the kind of test I think true test of whether we have a full AGI system and I think we're still a few years away from that but I think that's going to be possible eventually but it's clear today's systems couldn't do that. Professor, do you think about AGI? — Well, I'm just a consumer of AI right now, not an expert. So, I'll defer to Demis on this question. But the way it's going, probably it'll happen sometime. But I think it's enough of a useful tool right now that everybody should use it. We need not worry about AGI. That's what I tell all the students and faculty. That's a good enough tool for you to use right now and accelerate your research. — Interesting. Dennis, how do you balance

both? the commercial pressure of this is Google, you know, we have to make money and also this is deep mind, we have to do research. How do you balance the two? Because there's some short-term pressures, long-term pressures. I'd just like to know how you think about this. — Well, look, um there are those competing pressures. The answer is we just do both to the maximum. And that's one advantage we have with our size is we can um uh uh explore both uh to the limit. So we have a large research team. I think we have the broadest and deepest research bench of any organization in the world and um but we also have uh you know we're like the engine room of Google uh deep mind these days and we have to uh support that too, right? That's what in the end brings in the revenues and the money and the funds to do more research and so uh we have to get that balance right and roughly half my team work on uh those kinds of immediate priorities and support for those things and that's very exciting too because building foundation models like Gemini is on the shortest path to AGI in my opinion. Um but then we have half the team uh who are doing the next frontier and it's sort of my job as the leader of that organization to protect the blue sky research and make sure it has room to flourish and deliver maybe on an 18 month two-year time scale uh or more and um and we make sure that we're not uh just overly focused on the near-term. So the short answer is we need both. Um and we've got I think that balance pretty good even over the last decade of you know new innovations but also plugging that in to the latest products and so billions of people around the world can benefit from it. — Professor, do you think about the same problems when it comes to funding for science in India? Like how do you balance what you want to do versus what there are grants or funding for? Is that a big challenge? It is always a bit of a challenge uh because um now you know as you know India does not have infinite resources so it has to prioritize resources which are you know priority areas for the country. So there

are these national missions like the AI mission, the semiconductor mission and quantum mission and things like that. Um where there are deliverables that you are supposed to deliver on but that again of course conflicts with the general uh attitude academia has where you as Dem said just do blue sky research. But I think on the whole we have been able to balance the two. Um so there are enough avenues for getting uh funding for basic research and even with applied research I think very interesting open research problems can come out of even pursuing applied research. So I think uh one can do the balance of the two but funding in general needs to increase in India because we are still at 7% of GDP. I think at ease we should go to 2% um that would be much more comfortable and uh you know given our aspirations I think that would be warranted also — you know I have a worry right like and this is a personal worry which is that 200 plus billion dollars of India's exports is IT services and I read this post recently which is it's just XYZ amount of tokens right and it's a it's a worry because we have software engineers who are good but not date and we do have some great ones but some of them end up going abroad but the ones that stay behind they are competing now against models that are just getting better at doing software. Do you have advice for people in software right now who are working you know on these projects and seeing AI rapidly improve? Look, I think um a lot of areas are going to get uh disrupted and change. Um I think with change uh their challenges but also come opportunities. So what I would recommend every engineer today wherever they are is to lean into these AI tools get incredibly good at using them and I think there is a lot of untapped potential there uh for the youth of today wherever they are and um what one engineer can do can probably be you know 10x of that I think new startups are going to happen that couldn't have been done before and um and so in some ways it equalizes the playing field because everyone around the world has access to the same tools pretty much. And so it's about everybody figuring out how best to integrate that into their workflows and then we'll see uh what the different um uh new industries or new services come out of that. But I think there will be some new maybe higher level uh versions of the things that we do today. So you're saying there might be a higher level version of software where you just prompt the thing but you know for a lot of engineers in India it just feels like it's not it doesn't feel like the craft anymore because you just feel like you're writing in English. Well, maybe there'll be a different sort of craft for first of all, we're not there yet. But secondly, um I mean I when I was starting off in games, we used to write it in assembler language. Okay. But then um when I was writing theme park, we then went to C, C++ and and of course now we have Python and all these even higher languages. So one could view this as a continual abstraction that is happening. And um I think that broadens actually the access to creativity to more people can try out their ideas and build their ideas. So um maybe it's a slightly different skill set that's needed. But there's going back to this question of taste. I think that's going to increasingly become the valuable differentiator. — Amazing. So I have one last question for both of you which is what is something

from your field or what you do every day that is very exciting to you right now but the world hasn't heard of yet and that you can potentially reveal without violating NDAs or whatever um that we can look forward to a few years from now from both your fields. Well, from what I limited knowledge that I have, I think what is going to surprise people is the progress in math because if you look at a general public, you know, math is always thought of as something inaccessible and populated by geniuses, which of course it's not. But I think when people see that uh AI is making these tremendous progresses in math, they're going to be surprised. I think that such a difficult what what's thought to be a difficult field can be you know cracked open by uh AI because you know it is based on axioms is based on definitions and it can be your predictions can be either proved right or wrong. So those make it much more accessible to AI than other fields. I — think for me the thing I'm looking forward to is um maybe AI in the physical world. So I think robotics is going to come of age in the next 2 three years. Still a lot of things that have to be solved in my opinion but I think we're getting to the point where there will be some breakout moment. I think also uh AI understanding the physical world we've tried very hard to do that with Gemini as a multimodal model probably the best in the world at that so that you could have an assistant that's maybe on your glasses and comes with you or on your phone and understands the world around you and the context around you. Obviously, we're seeing self-driving cars and things, I think, about to become a reality around the world. Um, and then I'm excited about things like automated labs that maybe speed up again the uh the discovery, scientific discovery, not just in theory, but also in the practical realm, too. So, I think that's all going to come in the next maybe 5 years or so. — Very cool. I've seen a glimpse of that in the movie Transcendence, where you just give AI couple of hands and then it keeps, you know, improving itself. Um, you know, thank you so much for your time. I want to open the floor to the audience. I'm sure we have some questions from the audience. So, we'll take two or three and then we'll run it through the spam filter, which is me, and then take it to uh the people on stage.

stage. — Hello. Mine's not a very technical question, but you mentioned studying neuroscience as the only data point that you have for general intelligence, right? So we have all this talk of moving towards neuromorphic designs and everything but why uh why do we want to move forward to an intelligence that is similar to ours. Not only does it lead to a loss of capital for a large percentage of people it also leads to a loss of identity I feel and also there are a lot of things that we are bad at which AI is already doing like when you're in a needle in a haststack kind of situation you can ask it to do literature reviews or pull uh large amounts of data together to find something specific. Why not make an alternate intelligence that works in synchrony with ours instead of trying to replace human intelligence? — Yeah. So I wanted to be clear about this. It's not about replacing human intelligence. As I explained earlier, the thing about the human brain is we it's the only thing you can think about as a cheuring machine, proximate chewing machine if you want to think about it mathematically. We have to understand what is true generality. So chewing showed that with a chewing machine and our brains I think most people would accept some kind of approximate cheuring machine. So you sort of need to under if you're interested in general intelligence that can be applied across the board then it has to have that roughly that set of capabilities at least that's the only set that we know of. Other animals are not general enough for example. So they don't have big enough prefrontal cortexes and so on. So um so that's it's not really about replacing humans. is about understanding what is general intelligence. Um I think that the tools uh as to why the industry is doing that is because we find with these general tools they can transfer to the specialized domains. So it's probably going to be more efficient uh to de develop a general structure that can be used in these more specialized domains than then develop hundreds of specialized systems. That's the economic that's why you're seeing the economic uh pressure to do that. So there's two different things there. One's a scientific question which I think is very valid of like what is a general system and how would we answer that question and then the other is a more economic question. — Uh I have a question. How do we use AI to deepen first principles thinking without removing the struggle that builds real understanding? So do you have a framework or steps which would make it easy for us? I — think it's down to the individual. You know it's like the internet computers you can use them in ways that will degrade your thinking but you can also use them in ways you know we were talking earlier about uh becoming a polymath. Well, today's a dream in a way with YouTube and all the information on the internet you have if for someone who wants to learn something very quickly up to say undergrad level it's all there like the best lecturers in the world all of that so that's one way you can use this technology obviously um with AI if you use it in a lazy way it will make you worse at thinking critical thinking and so on but that's down to you as the individuals no one can help you do that the technology sort of neutrally sitting there. Um, you need to be smart enough to use these new technologies in ways that will enhance your thinking um rather than uh make it worse. — Yeah, I get the point what you're trying to tell but also as you told we have so many resources for learning. So we might get into the pool of learning from which resource and like listing down the resources. So what would be the better mindset for us to narrow down the resource and get into the learning pool? I — think the number one thing that you should do when you're in school and

other things is work out how you learn. Learning to learn. That's my that's the number one thing. I'm surprised that is not taught more in schools and so on. Figure out how you learn best. There's there isn't going to be one answer for everyone. You need to think that through how you work best, what environment, what modes that you work and learn best in, and then double down on that. Did something work for you like the art of how to learn? — There'll be a little bit. Yes, but it's not possible probably to explain it in one minute you know. So, uh it's many things. It's just sort of developing the mind. For me actually it was games I trained my mind on. Uh multiple games that exercise different parts of uh you know the thinking process and getting really good at getting um capable at that. So it's kind of the way we developed AI for early days of deep mind using games as a proving ground uh for testing out uh your own ideas. — I have a question around uh memory. So

amongst all the neurological aspects I find memory to be very intriguing like the way uh I mean hippoc campus works and how we try to model episodic memory semantic the long-term short-term like sometime I get glimpses of what happened in my childhood. It's not about weighted averaging, right? It's some glimpses that remains and uh for example, if you hear any keyword and that strikes something else for you and probably for someone else something else. How is foundation models trying to handle this problem? How are we planning to handle this? Because currently it's very systematic like something that you can interpret but uh as far as I have seen memory is a very abstract concept. It's very difficult to understand how brain resolves it. What's your take on it? — Yeah, so I agree with you. It's one of the most interesting things. That's why I studied memory as well and hippocampus as you probably know and imagination. Uh partly because machines in those days were very bad at those things to some extent still are. So we're kind of uh I would say badly approximating the hippocampus at the moment with context window. Right? So really the context window is more like working memory but because computers we only have working memories of you know seven digits plus or minus three digits or something. But um of course a computer can have like Gemini, you know, a million token context window. But the problem is I think that's still not as good as episodic memory where you're sort of it's kind of brute force, right? You're remembering everything when in fact most tokens are irrelevant and you want to only remember the important things, which is the way human memory works, right? We remember um the emotional things actually better than the neutral things. So both positive and negative. Maybe that's one of the functions of emotion. We don't need to remember everything we've seen today. We'll just remember some of the key moments that might be useful for learning or for future use or for imagining new or simulating new scenarios. So I think even in the realm of AI and machines where we can have millions of you know uh or maybe tens of millions one day or billions of memory units you still pay a cost of searching that memory right. So I think of our video models or um our project Astra which is supposed to work on glasses you know can record maybe 20 minutes of maybe uh 20 minutes of video that's about a million tokens right first of all that's not a lot of time secondly to then find something in that is quite expensive right to because you have to look through everything and um so I think ironically one of the things we maybe are missing is forgetting uh or if you want to talk about in computer science language garbage collection so that you um compress what you're remembering uh and maybe consolidating it as well if for those of you neuroscientists in the audience you know um so that it's just efficient uh the things that you are remembering and you have to sort of search through — I have a followup to that which is in high school biology I read that the amygdala and the hypocamas fire together right so like you said when you're very emotional you tend to remember things is there amygdala equivalent for LLMs — not at the moment but maybe they maybe there should be I don't think you'd want it to be like emotional or amydala like the human like we have. I think it would but maybe some value judgment at the point of writing the memory that uh makes some kind of value calculation on how useful would this memory be for future learning or future behavior probably would be pretty useful and something that we are researching.

— Amazing. Thank you so much uh everybody for joining and of course congratul for being on stage and giving uh us your time.