Google's Secret AI Weapon? AlphaEvolve Just Changed Everything

So a few days ago, Google DeepMind introduced Alpha Evolve, a Gemini powered coding agent for algorithmic discovery. Now, this is something that I think people don't really grasp the implications of because it is one of the first times where we're seeing AI in a real use case do something that is actually being used to accelerate the feedback cycle of AI development. Now, this has pretty much taken the world by storm, and in this video, I'll explain to you everything you need to know and why it's such an incredible breakthrough. Now, the gist of things is basically that this is an agentic system that was able to design faster matrix multiplication algorithms, find new solutions to open math problems, and make data centers, chip design, and AI training more efficient across Google. So, let me show you guys how all of this works. Because while it's good to know what it does, how does it do it? So one of the things that you know we can look at is the alpha evolve highle overview. So this is where we have the human and this is where the human essentially defines the what are we going to be measuring? What is the goal? And so this is where you essentially try to measure if something is good. So you essentially give some starting code or background info and it's now the robot's job to figure out how do I improve this code. So this is how the system works. We have the program database which keeps all of the code versions it has tried and how well each one did. Basically like a big code memory bank. We've got the prompt sampler which grabs the old code and ideas from the database and then turns them into a message to send to the AI. Then we have the LM and sample which is basically reading the prompt and writing new improved versions of the code. And then this is where we've got the evaluators pool, which is where it runs the new code, scores it, and is like, is this better, faster, or cheaper? And that is the entire loop. If it's better, the new version goes back into the database, and the cycle continues. So overall, in the end, Alpha Evolve gives you back a better version of your code that it figured out by trying, testing, and improving again and again. And this is a loop that literally just makes your code consistently better over time. Now, the craziest thing about all of this is that this isn't just some fancy demo or something that Google wanted to do to boost their rankings among the AI world. They've actually implemented this into their core system and are now using Alpha Evolve to improve Google across the board. They're improving a data center optimization with the Borg scheduling problem, which is where Google schedules jobs across its massive server farms and sometimes resources like memory or CPUs are wasted. And Alpha Evolve actually came up with a solution. It discovered a simple new horistic for job scheduling and the result was that 0. 7% of compute resources across Google's entire fleet were recovered. And in the paper they talk about how post- deployment measures confirmed that this heristic function continuously recovers on average 0. 7% of Google's fleetwide compute resources. There was also hardware optimization TPU circuit design problem designing TPU hardware used for training AI involves optimizing circuits for performance and energy and alpha evolve came up with a solution. It modified the verilog which is the hardware design language to remove unnecessary bits and the result meant that it was reduced power and error usage in missionritical TPU arithmetic unit. And so in the paper they referenced this. They state that alpha evolve found a simple rewrite that was verified by engineers and this change was integrated into a future TPU. It also optimized software. There was a Gemini training problem. Google Gemini's training foundation model requires massive amounts of compute to train and Alpha Evolved solved this. It optimized kernel level which affects how matrix multiplications are broken up. The result is there was a 23% speed up on critical kernels and a 1% reduction in Gemini's total training time. The paper reference says that the tiling heristic discovered by Alpha Evolve has been deployed in production enhancing Gemini's training efficiency. Overall, we can see here that there were three key areas where Alpha Evolve managed to improve Google's overall efficiency. And think about this, what happens when the efficient AI starts to improve and generate even more efficient AI? We can start to see how in the future things might start to speed up even more. Now, one of the craziest things people talk about is the fact that it solved/made a big breakthrough in matrix multiplication. So matrix multiplication is a fundamental operation in computer science and AI and it's used in training neural networks, rendering graphics, physics simulations, search

algorithms and more. And the goal is to multiply two matrices as efficiently as possible. And so the big deal here is that back in 1969, a guy named Vulcan Strassen found a way to multiply 2x2 matrices using seven multiplications instead of the usual eight. And this was huge because smaller multiplications meant faster computing. And by stacking Strassen's method, you could multiply four by four matrices in 49 multiplications. And for 56 years, nobody found a way to beat that number. But Alpha Evolve found a new way to multiply 4x4 matrices only using 48 multiplications, beating the 1969 record for the first time. And it did this by evolving the code, testing results, and keeping only improvements without human math proofs up front. It designed a search algorithm that discovered the actual algorithm, a kind of AIdriven meta innovation. And this isn't just a math flex. It's because models like Gemini, GPT, and Claude do billions of matrix multiplications during training. And even one flu matrix multiplication per operation mean massive compute savings at scale. So this discovery directly speeds up matrix multiplication kernels used in Google's AI infrastructure and overall this actually resulted in a 1% speed up in Gemini's training time. So we can actually hear what some of the authors of this paper have to say. related point I would just quickly mention is that um alpha evolve also offers this opportunity which we haven't taken yet but just forward looking that it's a system that is able to enhance the capability of the base model like is somewhere but then this system makes it even better through orchestrating this like test time compute pipeline and actually so much better that you can make like a new scientific discover so it raises the natural question like can we somehow distill this improved capability back into the base model so that's something that you would get if you were to close the reinforcement learning loop. It's not something we have done with alpha evolve, but that possibility is clearly on the table. Now, in another part on the machine learning street talk, they talk about the fact that this could, you know, like I mentioned earlier, enhance the base model of Gemini and potentially start off that self-improving loop. So, of course, not uh one AI that's self-improving by itself somewhere in a factory just becomes super intelligent, but more like, you know, improving every part of that process. So eventually the entire process gets faster and you can essentially iterate quicker in terms of bringing out AI systems that are overall smarter of alpha evolve and the base models itself. So you finally now managed to close the recursive self-improvement you know loop. Uh I don't know that's going to trigger some folks. Any thoughts on that? What would Schmidt Huber say? At this point we want to be like very specific about what we have done like we have found a way to speed up the training of the next version of Gemini by 1% and we have been able to unstrand resources in the in the B data center. So currently if you think about the feedback loop it's maybe on the order of months right like um when you speed up the training of the next version of Gemini then that will take some time to actually arrive but indeed like we do see the steps in the direction that you described Alex any thoughts on uh the recursive self-improvement of alpha evolve yeah I mean I don't think I have to have anything to add like as mate said it's uh it's interesting to see how it pans out and uh we are seeing some signs of you know helping the Gemini training ground so Yeah, that's exciting. And so this is where on machine learning street talk, we actually get some information on what is next for Alpha Evolve cuz I was actually quite intrigued to see, you know, oftent times we get these really amazing announcements and demos and information about things that are essentially groundbreaking, but what is next for these innovations that Google have made and so on. And I just wonder what would the next step of autonomy look like, you know, in terms of maybe we could actually have the thing imagine what its own evaluation function is and maybe it could kind of go several steps further. What would that look like? To be honest, uh, from my perspective, like I guess automating some things is cool and exciting, but also at the same time, I would even lean towards less automation in slightly. Uh, I think the thing that makes uh, alpha evolves so cool and powerful is kind of this back and forth between humans and machines, right? And like the humans ask questions. the system gives you some form of the answer and then you like improve your intuition. You improve your question answering question asking ability right and you ask more questions. So we are thinking a lot about uh providing access to alpha evolve to academics as trusted testers and like seeing what they can do with it. And while kind of trying to build that and trying to build the UI, we we're thinking a lot about you know not just implementing the thing we have as a website but just like what would be the next kind of level of human and I human AI interaction here like what can the humans do in the like kind of intervene in the process like maybe they want to if the humans would want to supervise the process like I don't know comment on ideas and like inject more ideas and things like that and like we're exploring that a lot and I think it's

very exciting to see like what can be done in this kind of symbiosis uh space. Now, something that Google also mentioned was the fact that when we look to the future, Demis mentioned a few things in an article in the Financial Times. He actually expects them to be able to carry out increasingly more complex tasks independently. An AI agent that can meaningfully automate the job of further AI research, he predicts, is a few years away. And so, that brings me back to something we're all very familiar with. If you remember situational awareness from the former OpenAI employee who essentially was fired for several reasons that are quite debatable, but remember that they said we don't need to automate everything, just AI research. So this is why the intelligence explosion is as widely debated as it is because we only need to be able to successfully build that one feedback loop for the effects to be felt downstream. Because once we have that essentially we're going to compress the knowledge of a decade or 100 years into just a few short months. And of course if that does occur of course it leads to more feedback loops which even lead to even more breakthroughs. So overall we can see here why we don't need many things for AI to automate research. The jobs of AI researchers and engineers at leading labs can be done fully virtually and don't run into realworld bottlenecks the same way that other things do. Essentially, you just need to read machine learning literature, come up with new questions or ideas, implement the experiments to test those ideas, interpret the results, and then of course repeat. So once that system is built, things are going to explode. And the crazy thing that the paper talks about is that some of the biggest breakthroughs have just been very non-trivial, such as fixing an implementation bug for the Chinchilla scaling laws and just adding some normalization. So AI research can be automated and automating AI research is about you know speeding up things and that's all it takes to kick off extraordinarily feedback loops which is what we kind of saw at the beginning of this video. Now, what's crazy is that we are basically on track because right now it is 2025 and I mean 2028 if we look forward that's around 3 years away from now considering we just had this massive breakthrough where an AI system was able to solve or you know make breakthroughs on certain math problems. What's to say we don't have an automated AI researcher 3 years from now that's able to start making breakthroughs that can provide meaningful implementations into current AI systems? I mean, it's quite likely that those feedback loops are going to be much more aggressive, possibly leading to super intelligence by 2030. So, I know it sounds like fiction, but every single day with all of the news I'm reading, it seems like more and more likely that this is the