AGI progress, surprising breakthroughs, and the road ahead — the OpenAI Podcast Ep. 5

most useful. — Now you two knew each other before working at Open Eye, right? — Yeah, we went to the same high school. — Yeah. — Were you guys friends? — Uh I think we became best friends when uh after we left like I think kind of coming to us is the kind of emotional experience that forms bonds. Uh I think in in high school uh we were more like colleagues. What kind of high school produces guys like you? — So, well, yeah, we went to this high school in u in Poland. Uh I think we were both drawn there by this computer science teacher, Mr. Rashard Subarovski. um who's had a great track record uh before we went there of of uh bringing up uh um computer scientists, programmers uh um with this like big focus on programming competitions and kind of and pursuing uh you know excellence in this like one field. Yeah. So I think that was like a very formative experience and a great mentor for us. — Oh wow. — Yeah. Definitely. Uh I think Keer was like really going deep on programming. I think it went way beyond like typical high school curriculum like there was like graph theory matrices and all sorts of stuff like that. I actually hope that maybe with CH GPT it's a little bit easier for people now to do this kind of deep dives — because um you know without the right mentor and without a lot of work it's kind of hard to replicate that experience. I've been using it to explain things like there's, you know, the Monty Hall problem where you have to choose which door and you go into chat GPT and you say make a graphic interactive version of this and all of a sudden you can see it. It can show you the different solutions if you do one thing or the other. I think that's it's one of these things where I'm excited about the ability not just to explain in text but to build multimedia to do things. And it does get into the area of there's not really a measure for that. you know that it's a use case you go this didn't exist before and you know we're we talk about AGI but we kind of have very loose definitions and whatever and I'd love to hear kind of like what how you would describe it both from a technical and also like a lay person's understanding of it. — Yeah. Well, maybe to like address the point about teaching um you know this sort of like better um explanation of some concept or like or you know teaching chosen method is like definitely uh yeah powerful use of tragic and I think works well with like um um a teacher like like our Mr. But at the same time, um I think like the thing that like he was able to provide was more like kind of emotional support and space, which I think is uh it will be hard for AI to to do alone. That's a great point. I think that gets lost a lot cuz I sometimes hear people talk about, oh AI will replace education. And for me, I had teachers that maybe their facts weren't always right, but their heart was there and they were carried and they answered questions for stuff. And so I think that's a good point that these are companions to that and I think that a teacher using these tools can be a more capable teacher. — Yeah. — But so on the subject of AGI though uh I want to hear first like give me your technical definition of it or actually not a technical question give me like how you would describe it to like if you were talking to you know a younger sibling.

sibling. — Yeah. A few years ago when we would talk about AGI um you know it felt like the techn the technology deep learning you know has incredible promise but at the same time the concept still felt a little bit abstract right and far away. Um and so I think you know whether you talk about like you know human level intelligence, ability to converse naturally uh you know ability to you know solve math problems or pursue research like they all kind of felt like in the same um space. Um I think yeah as technology has progressed now we see there's like this these are actually like quite distinct uh capabilities and I think we pretty clearly are at the point where the AI is able to converse naturally uh on a wide range of topics um you know it is able to solve math problems I think you know like getting gold medal at IMO is something we've long discussed as like a milestone on the path to AGI and that happened. Um I think um solving you know solving all the problems on the national map lay actually a little bit harder and you know I think it's like another milestone on the path there but I think increasingly um you know we see that like this kind of pointwise measures are less adequate and so we turn to thinking about like what is its actual impact in the world. Um, for me personally, the the thing that I think about when I think about how um AI progress really impacts

the world meaningfully, I I first think about its potential for automating um the discovery and production of new technology. Um I think um you know we tend to associate kind of new ideas you know fundamental technological progress with just human ingenuity. Uh and you know we we measure kind of the you know the our progress by this like kind of major milestone inventions and and technological revolutions. Um and I think it is just hard to internalize like this is possible to automate most of this process. It is possible to have uh big computer that is coming up with ideas that fundamentally change our understanding of the world. And I actually think that is not that far away. Uh and so thinking about that you know what separates us from that and what are the consequences of such technology is yeah is my first thought. I just ordered a little Mac Studio because I want to take uh the open source model GPDOSS and I want to just let it run non-stop because that idea just the idea of letting it generate and do stuff 24/7 is fascinating to me. But you're talking about a scale of basically automating science at a huge scale. And so what kind of discoveries what kind of things do we think might be the first things we

could see from that? When we think about how we shape our research program at OpenAI, we seek to create intelligence that is very general. Um we you know drive towards this automated researcher as a priority but we um you know we don't really think of it as like let's take this specific domains and let's kind of like deploy this technology there. I think that is a way to like make faster pointwise progress. But I think the potential for like the really big discoveries and and and the most meaningful technology advancement comes from this general generality. — Mhm. — Um although still I think we see kind of like you know the technology like uh you know is is kind of easier to apply in some domains than others. I think um I think especially in places that combine u a large amount of reasoning with a lot of kind of domain knowledge and intuition uh seem uh very um very aminable to to these systems. I think in particular we see like pretty incredible results on medicine which is very encouraging. Uh I have high hopes about that. Um yeah I think naturally being a company of AI researchers uh we think a lot about u automating our own work. I think it is also kind of a uh you know I don't think it's a you know if it is if AI can indeed reach a point where you can automate AI research and that is probably a very important thing to automate. Uh and you know and similarly thinking about like how it can help with automating research on AI alignment and safety. I'm obviously impressed by the like IMOI results. I mean I was actually about to add that like um in the past when we were talking about the IMO of Yakob like that was like you know a few years ago uh um and we were still like kind of trying to even figure out what our definition of AGI might be like one kind of concept we were considering is something like you know solving all the problems on on the math olympiad uh and um and why did that feel appropriate is just like Okay, if you have a model with such a superior mathematical reasoning, then it should be able to like disrupt like a bunch of different uh domains that kind of can be mathematically modeled. Um, right. Um, I mean, I'm in general just um I think maybe this podcast is just a good opportunity to kind of like share a little bit more of an inside perspective. I was just astounded by the

progress. I think so sometimes I see those headlines where it where people say that like oh um the economic kind of impact of AI is only like 3% or 5% right and those headlines are often accompanied by comments like well so AI is slowing down or or you know like like people are overhyping AI so much and it's only like 3% so what's up with that right and uh and when I you know when I see headlines like this. I remember to like maybe 10 years ago I was working on natural language processing uh in in with deep learning and back then it just didn't really work. Like I remember Jakob once came to test like one of the technologies we were working on and that was like trying to detect sentiment of sentences and he was trying this movie is bad correctly classified as negative this movie is good correctly classified as positive and then he would say this movie is not bad and the model is like oh negative. — Yeah. — Right. So that was 10 years ago. Right. And since then like okay like we slowly got like uh we slowly started solving tasks like this solving tasks like decide is this word a noun a verb that was like sentiment neuron then we then we had GPT1 GPT2 started producing like a paragraph of text that made sense right that was such a breakthrough right now it feels so simple but back then it was such a breakthrough then we had like GPD3 GPD4 GP4 was like to me like kind of like let's say my personal AGI moment uh — because it would sometimes say things that surprised me and I was like can this model actually surprise me right it's still back then like charge GPT for my personal use kind of felt a little bit more like a nuance and kind of like maybe slightly better Google but like what's the big deal and then like suddenly we get to deep research and this can actually like answer questions to really like rarely make things up. That felt useful. And then finally now we have like models that can like compete in programming competitions which was like a you know like very hard earned for me personally and even more so for obviously uh and u yeah the the pace of progress just like from the perspective of somebody working on this technology is like absolutely amazing. So, so when you see that 3% like I raised you like 10 years ago if you had to quantify it, it would probably be like 0. 00001% or something, right? So, like really I think those numbers need to be put in perspective, right? And and there is no reason not to believe that like in a year it will be 10%, in two years it will be 20 and so on so far. Yeah, I've heard it said that if you looked at like a graph of the economy from let's say like you know uh worldwide web you know early 90s forward and he said point to the internet happening to the economy you can't find the point there's no point you go oh okay Tim Berners Lee announced this whatever and I think AI is a lot like that where people go oh we've only measured this one our measures are hard it's hard to know that you know one who's using it how they're using it and you brought up a very good point too about if you've been following it for a while I remember training like a very simple next character predictor on my computer and it was terrible, right? One, I'm using a small computer, but even then and then you got, you know, the sentiment analysis, you're playing with bird and it's kind. But then GPT2 comes out and I read every single output on GitHub. Every single output GPT2 came out because I'm like, there is something going on with this. And that's how I ended up working at OpenAI was because I was this obsessive person about that. Then with access to GPD3 kept saying, oh, this is really this path that's moving forward. But it's kind of crazy now because like if six weeks go by and a benchmark hasn't been broken, people are like, "Oh, we hit the wall. We hit the wall. " And I would say part of the

problem though is that benchmarks in some ways feel like you'll see modest improvement on them. I've heard some of the benchmarks have problems. Some of them actually have wrong answers and it's impossible to get 100% if you answer them correctly. But also, we talk about the term internally. I've heard people talk about this as, you know, saturation. Do you want to talk about that? Yeah, I think there's a few issues that that we're hitting with benchmarks right now. Um yeah, I mean a pretty clear one is saturation and that is just the models genuinely reaching a point where you know for the kind of standardized forms of measuring um intelligence or ability like they are uh at human level for a lot of them uh you know if you're kind of like able to you know perform in and in in amongst the top on this like very hard high school competitions where we have um you know the the best competitors from around the world um it just becomes quite hard to like have this like very uh very constrained uh measurement. You know previously when we were looking at you know this like yeah GPT1 GPT2 GP3 GP4 scaling paradigm um you know the benchmarks were really very they were really just like measuring the rising of the tide. Um I think now um you know the field has developed a lot of uh more data efficient ways to train for specific abilities right doesn't mean you know train on these benchmarks but you can train models that are like disproportionately good at math compared to their ability um you know to write for example right and so they will do better on math benchmarks but it's no longer as representative of their of their overall intelligence uh in other topics um I think you know these two issues combined Yeah, I think we really have to think about the reward utility of these models and especially like their ability to uh discover new insights. — Yeah, I guess that's a thing that sort of kind of gets overlooked is that you can build a model that's a really good test taker, but that model may not really be that useful for work. Ideally, your model should score well on tests, but just because a model got these scores doesn't mean you're going to find it personally useful. And I certainly think that's a challenge we're at now where when people say is a model good or

bad. It's kind of like saying, you know, you're trying to create a blanket assessment when there's 100 different use cases for it. You know, is a model good or bad? Maybe it's great at creative writing, maybe it's bad at math, maybe it's great at math and bad at creative writing. And that becomes a really big challenge. And we've talked about this with like one for math, international math olympiad and these kinds of metrics. Why are they important? Why is it important to put it into these sort of human level competitions? — I think the reason we've been excited about these competitions like the international math olympat information utad is that um they are a pretty interesting example of like a test that is constrained doesn't require that much knowledge but really tests your ability to think about a problem hard for you know an hour or two or three. Um and you know and we have like a very kind of good uh we have very good evidence like these problems are hard. Uh you know there's a lot of people that try to solve them and compete at solving them and it matters to them. Um, so yeah, so I think this is and you know like for models that like you know excelled at like kind of knowing a lot of things but not necessarily uh you know thinking very hard in the past that really seemed like the kind of the right milestone to be working towards. — Now I'm so I understand it the model

that scored gold medal level on that wasn't using like a calculator. It wasn't using other tools. It wasn't using some of the frameworks. was doing it purely through reasoning. — Yeah, that's right. For the for the International Math Olympiad, uh yeah, the model was not using uh other tools like uh Yeah. — And and again, and it was like two years ago, you asked it to multiply two four-digit numbers, it would fail. — Yeah. But definitely like, you know, for this kind of contest, it really like — it is of course like in a limited domain of math, but it really is about like fairly creative thinking, not about applying a formula. I guess that's part of the challenge though is that once you start moving outside of math, it gets to be harder. You can start to come up with things like humanity's last exam, which I think is a pretty neat test, but you find that certain models after they learn a certain kind of tool use kind of figure out maybe sort of how to solve these problems better. And I would wonder what kind of benchmarks are we going to need? You know, what are you looking at to say, okay, this is how I can kind of get an objective measure of a capability. One thing that surprised me in the past, I was talking to one of our uh co-workers here uh Anna Makandu and I was uh telling her about uh IMO. I was excited about like some progress and she's like what's uh and that that kind of like was like a very in front of the moment for me because I do realize that like some of those benchmark we kind of live in a bubble a little bit. Mhm. — For me that competition feels important and especially like the computer science counterpart III because it was a big part of my life and so is true for many co-workers here. But actually like for an average person like working in other field or maybe not as interested in mathematics or computer science, maybe they're interested in history or something that the — Lana speaks like five languages too. So I could see for her a different metric based on that would be interesting. — Yeah. So, so I think like one thing that like um that uh it's not a perfect metric but at least helps keep us honest and keep helps keep us escape the uh the bubble is uh it's just charg us right because everybody uses charg and they use it for all sorts of use cases. Uh, and obviously there's like a lot of pitfalls to using that as a metric, but at least it avoids that partic particular problem where like there are just some things that I'm more familiar with and other people might uh appreciate other things and this gives you like a very wide coverage. — Yeah. And in there too, you have subsets of users, people who are building GPTs and doing more complicated stuff. You mentioned before too the fact that the model will reason longer and that seems like a very interesting way to evaluate capabilities. — Yeah. And I think this is also maybe like one um you know challenge with focusing on only kind of usage of like chat GPT and broad adoption of AI as the metric of progress. Like I think uh this hasn't really happened to a very meaningful extent yet but I think it will start happening pretty soon. we should be able to use vastly more compute um than you know a a user would normally be willing to um to to buy for themselves to produce you know technology artifacts that are useful to a lot of people. Um and I think I think that for me

will be a very important measure of progress. Which of these wins were the most surprising to you? — I think we definitely kind of anticipated getting to this point when we saw the reasoning models starting to work. — Um, at the same time, I think this like recent uh set of things is very impressive. I think maybe out of those uh um I think IMO came a little bit sooner than I expected. IMO gold again like I think IMO problem six will still uh IMO has like all the problems require like creative thought and some uh new insights I think but typically like you know there's this proveral problem six that is like requires very out of the box thinking uh and you know it's really kind of like usually like outside the kind of typical domains of the other problems um and you know so in the past we were actually kind of drawing a boundary between like getting a goal you know like solving these other problems and like actually consider solving all the problems and in particular problem six. So it was pretty pretty uh you know hilarious in some way to see ourselves and also Google deeper at the same time like oh yeah we solved problems one through five perfectly and we didn't make any problems on problem six. I think that kind of makes that challenge pretty clear. — Yeah that was I think that was interesting is that yeah that I think that the open model said like yeah I don't think I can solve this. It didn't even try or said that it had a problem with that. Was that correct? Yeah, the model was able to correctly identify that didn't make progress on the problem. That's pretty fascinating to think about that the model is able to sort of determine that because uh you know there's a lot of conversations about people talk about hallucination which I think it's a kind of a poorly understood thing and there's a difference between fluid and crystalline thinking and you know one is how much knowledge a model has and the other is its problem solving capability and when

you get to the point where it's able to do that it's able to say but hey no I think you know I won't be able to answer this that's pretty interesting sort of point to get to um I've been told to ask this question about a live stream in Japan. — Oh, so so I think in the past few weeks actually like our models have performed incredibly well in three competitions. Um so we talked about two of them which is III and IMO. Um there is also this competition for that is open to everyone not just not just high schoolers um called um adcer. It's a very prestigious very — uh um highquality uh competition organized in Japan uh but open to competitors worldwide and in this particular contest it was about um kind of longer horizon um horistic problems where you're given only a single problem you have 10 hours to solve it and so you have competitors uh racing to figure out like the best approach to up to to this like difficult optimization problem. Um so it's a bit different because there isn't like a single correct solution. pattern to follow. Like these tasks are like extremely diverse and you can focus on the single task for for 10 hours. And so we entered our model into this contest. And um you to me this had a little bit of a personal significance. Um I used to be a kind of very uh engaged competitor in the past in this like more short form like closed form contest like III. Um and um my friend uh Siho um who also works at the time um excelled at this like long duration contest and when we work together he would mock me a little bit that you know my sort of contest would be automated uh long before his um because you know they are they're kind of like longer duration require uh um require kind of more focused work Um, and turns out like in this in this contest in Japan, Sihaho was actually like one of the top contenders. Um, and so I was watching this live stream watching our model kind of race with SIHO throughout the competition. Um, in the end, um, our model actually got second place and Siho won. So, you know, he alone stood in the way of his uh of of, you know, his prediction not coming true. — Still two wins for opening eye. I think one thing that also stood out to me is like Sai at the end of the competition he was like really retired and they interviewed him a little bit to to talk about his experience like in the middle of the competition and um I don't think I can quote him directly on this podcast but he's like your models are very bad. I want to go to sleep. I am tired. Yeah, we we've heard talks about like the wall. We mentioned that before and I think it was interesting because reasoning kind of came out of nowhere. I mean there were you know hints of stuff some papers and stuff but people really hadn't kind of drawn the line. Then all of a sudden the 01 model comes out and the whole idea that you can not just have a model give answers you can let the model kind of have an inner

monologue talk to itself and solve things through. Do we think that's enough to take us to AGI or there other breakthroughs needed you think are going to happen? I just need to point out that like the team here worked extremely hard on this particular thing. I need it feels like something simple like just need longer chain of thought but like actually to made it work was really hard earned and um and I do think like back to your previous question of like what was the surprising result when we first noticed that it's working or that we can train those models and give them more data and they get better. That was like I think one of the most kind of shocking moments. The moments where we are like um we started asking uh like very seriously the question like are we ready as an organization for incredibly fast-paced progress? Like I remember there was like one particular evening like I think 11:00 p. m. like there was like I think we were on the line with Sam and Mera and just kind of trying to I think we got a little bit freaked out by those results sometimes. Sometimes that happens. Uh — the pace is fast. I mean it is a fast thing and like I said that you know the joke is people nothing happens for 6 weeks they think it slowed down but then if you look year over year it is. And uh I it's a fair point cuz yeah you have things that you're aware of internally when you work on something for a couple years and there hey there's a research paper but it's like yeah it's not like it came out last night. It was like there's a lot of work onto it. But I'd say to the world was sort of surprised by the fact that there is this really fundamental new way to sort of make these models do even more to take kind of the existing sort of infrastructure so to speak and get a lot more capability out of it. Where do you think the next breakthroughs are going to happen? I think one thing we always try to not underestimate is the importance of scaling. Um I think you know even as we look at these wiz models

you know like it's not like you know the previous scaling paradigm of pre-training uh has vanished right like I think we will see these things compound — and I think there's also like new uh new directions that we can move in particular um we were talking about you know extending the horizon that these models can uh you know plan for and and reason it. And I think if you look at it from the perspective of just like compute spend um you know we say okay yeah we went from GPT GPT4 was doing some out of compute for for every answer to like you know GPT5 pro which maybe uses I don't know like uh 10x 20x some you know non-trivial but in some ways not that impressive amount of compute more right and can produce much better answers. Uh I think on the scale of like what amount of compute would you be willing to spend on a problem that actually matters to a lot of people right on like progress on a on a medical research question uh you know progress on developing the next generation of models right like these are like incomparably larger amounts and so I think that is that question of like model persistence and ability to work for a very long time on focus problem is a pretty clear next How would you put the practical implications of AGI to

sort of like if you were talking to a typical chat GPT user or something like what would be the what would their experience be like in a few years from now or five years from now which sounds far away but it's really not because which is 5 years ago GPD3 came out and that feels like a blur. What would an AGI like model be capable of? So I was I was talking about automating research. Um you know my picture of how that would actually look like is you know imagine a company of very capable researchers and engineers that is largely automated right and um now again like that is I think that is something that like you know will interface with the world in all sorts of ways. It won't be just like kind of a black box like it will you know talk to people. It will kind of like take in inputs. it will run experiments. But I think like you know having this sort of potential for developing new technology and you know and other kind of artifacts you know uh um code bases designs um I think can like radically accelerate uh the pace of technical progress. Um so I think that is something that like we will feel and you know and we need to do a lot of work to get it right you know from a technical and from societal perspectives. Um but I think that is kind of where I turn to I think we should also expect like a lot of progress on the actual kind of you know um interfaces that we interact with. — Mhm. — You know we see like ch can feel quite humanlike. we can form attachments with it. I think you know as it becomes more persistent as it becomes kind of capable of expressing itself in like different forms and texts right like I think that those effects will become stronger and again like that will be something uh I think will become a very big and important conversation. — I just got access in chat GPT to have it actually read my calendar in Gmail and I realize like how far we've come because I'm excited about that now. I'm not really terrified that it's going to start writing like, you know, Ewok fanfiction to somebody, you know, and I think that's sort of this neat threshold that we sort of cross this sort of the level of trust. I think there's definitely like we are in a place where there's like very tough trade-off where like there's like such clear just economic you know personal value you can extract out of having the model um you know have access to a lot of your um all of your data. At the same time, I think like you know we are not at like the threshold of like robustness where like you know like we can fully trust these models to not be exploited by someone trying to exploit them. — Mhm. — Um yeah it's definitely like a big problem I think you know we as a field will have to iterate on. What would you tell two versions of you guys today in high school? What would you do if you're visiting your old classroom? What would you say right now? Tell them about the future. What advice would you give? invest in Bitcoin. — No, I mean today even today in 2025, what would you tell a high school student? — High school students today. Oh, yeah. Yeah, that one is also I think a great question, right? Because I I hear a lot of uh kind of would I consider misinformation on that online? So, you should absolutely learn to code. like one skill that is at premium and will continue being at premium is to have like um have — like really structured intellect that can like break complicated problems into pieces and you know like that might not be programming in the future but programming is a fine way to acquire that skill. So are other kind of uh domains where you need to think a lot. So don't let people tell you that uh you should not learn to code. Yeah, I learned to code late in life and that's actually I ended up working at OpenAI as an engineer and I try to explain to people just because a system can do the thing doesn't mean you don't want to know how it works anymore. And as you said, when you understand how to break down a task when I worked at OpenAI and prompt engineering, my coding understanding helped me understand to take both language and break it down and make it do better things. I think that people who bridge those gaps are really an advantage. And so whenever I hear people say like don't learn to code, it's like do I want an airplane pilot who doesn't understand aerodynamics? Like this doesn't make much sense to me. Well, you know, thinking about how I thought about things in high school, I think it's like pretty incredible like how many kind of perceived constraints are not actually there when you really think about it. You know, maybe like the first revelation for me was like, hey, you know, if I really kind of like am passionate about this computer science stuff, like it is I can actually spend a bit more time on it at the cost of, you know, maybe spending a bit of a bit less time on like, you know, like other 12 subjects in school. Um but you know but then like somehow it like again like took kind of like a u it was like a big revelation to me that like actually you know hey I can I can um go and you know study in the USA at some point and like you know that's not really something that like seems uh that's like obviously interaction space and you know and obviously kind of like you know like spending some time here in uh Silicon Valley right and kind of seeing like how you know people are willing to really like attack these big problems with ambition and like the kind of um and the belief that like you can actually you can actually um make a meaningful uh positive change in the world. Uh yeah, I think has been incredibly inspiring and uh yeah, it's something I I I I cherish about about this community.

and will continue being at premium is to have like um have — like really structured intellect that can like break complicated problems into pieces and you know like that might not be programming in the future but programming is a fine way to acquire that skill. So are other kind of uh domains where you need to think a lot. So don't let people tell you that uh you should not learn to code. Yeah, I learned to code late in life and that's actually I ended up working at OpenAI as an engineer and I try to explain to people just because a system can do the thing doesn't mean you don't want to know how it works anymore. And as you said, when you understand how to break down a task when I worked at OpenAI and prompt engineering, my coding understanding helped me understand to take both language and break it down and make it do better things. I think that people who bridge those gaps are really an advantage. And so whenever I hear people say like don't learn to code, it's like do I want an airplane pilot who doesn't understand aerodynamics? Like this doesn't make much sense to me. Well, you know, thinking about how I thought about things in high school, I think it's like pretty incredible like how many kind of perceived constraints are not actually there when you really think about it. You know, maybe like the first revelation for me was like, hey, you know, if I really kind of like am passionate about this computer science stuff, like it is I can actually spend a bit more time on it at the cost of, you know, maybe spending a bit of a bit less time on like, you know, like other 12 subjects in school. Um but you know but then like somehow it like again like took kind of like a u it was like a big revelation to me that like actually you know hey I can I can um go and you know study in the USA at some point and like you know that's not really something that like seems uh that's like obviously interaction space and you know and obviously kind of like you know like spending some time here in uh Silicon Valley right and kind of seeing like how you know people are willing to really like attack these big problems with ambition and like the kind of um and the belief that like you can actually you can actually um make a meaningful uh positive change in the world. Uh yeah, I think has been incredibly inspiring and uh yeah, it's something I I I I cherish about about this community. — Is there a book or something that like inspired you? — I think there's a couple books. Um I remember my it's actually Yeah, it's actually very hilarious like thinking about it now. Um, I didn't really connect the dots, but uh, my dad gave me this book once um, when I was like in a pretty uh, um, I think I was like 15. I was like pretty unsure what I want to do. Um, yeah, it was um, a Polish version of a book by like some other I didn't know. Uh, called Hackers and Painters. Uh, yeah. So, uh, yeah, it was actually a Paul Graham. Uh so I guess like uh yeah again like kind of like this community. Um so I found that pretty inspiring. — Yeah, there's something helpful I think that hearing the message of like no it's okay to dream big and go do stuff that you can just make things happen in the world. And I think that the more people realize that kind of the better the world gets to be. Was there any book that influenced you or movie TV show? Oh, movie. I have a stupid answer to that question. Kind of stupid bad after the profound one that but like Okay. So, I watched Iron Man. — Yeah. — And it inspired me to start a PhD in robotics. — That's a great answer though. Like, you know, the Martian by Andy Weir. I met a scientist at NASA who was a botonist who read that book and I'm like, "Well, they got the atmospheric physics wrong and all this. " He's like, "Well, that's why I'm here. " I'm like, "Oh, — well, yeah. — Yeah. I guess I didn't get to the stupid part. The stupid part was like when I started working on robotics, I was very disappointed how bad those robots are. I somehow didn't occur to me that like maybe the movie is a movie. Uh yeah, so that whole experience was kind of bad for me if not for would be the fact that this is there. I like met a friend who was into deep learning and at the time I thought all of the machine learning kind of is a hype but it was an interesting systems problem and then out of nowhere as I'm sure like uh you know like I would frustrate some deep mind folks by saying that Alph Go came out uh I'm sure it wasn't out of nowhere I'm sure it was years in the making uh and that like was very inspiring I actually think to both of us uh and since then it was just hard not to Yeah, took me a while to become um convinced that declaring is more than a fad. Uh because you know like we don't really understand the kind of underlying optimization and you know I think this kind of has been the story of our research here making trying to make progress on on these questions like about how it really works but it really is like setting a physical phenomenon in some way and you know to a classically trained computer scientist that was a weird thing to accept. — Mhm. I do remember when Jakob was telling me about like scaling up principled convex optimization. — That was before Alpha Go. — Yeah. And Alph Go was interesting because first like oh cool it solved go and then we're like yeah but it just learned by watching all these. Then they did Alph Go Zero where it selftaught and you're like okay game over folks. Like there's a trajectory here. And I think that's continued on. But I think that yeah, if you hadn't watched Iron Man, maybe Thor instead, you know, maybe things would have turned out better. — Who knows? — No, I — I kind of wish I studied maths instead — has been more useful. — Study what? — Maths. — Mathematics or theoretical computer science? Either of those like this. Yeah, — physics probably. Is that — Oh, physics. — I started off as a magician. I don't know if you know that. So, I actually had my own reality TV show. And so you find a very strange path to end up here. So uh Yakob Simone, it's been an absolute pleasure to talk to you both and I hope we can meet again and talk about the next big breakthrough that you guys have been secretly working on that's going to come out of nowhere and we'll be like that was an overnight thing. — Thanks Andrew. — Thank you.