Why Does OpenAI Need a 'Stargate' Supercomputer? Ft. Perplexity CEO Aravind Srinivas

why does open aai need Microsoft to build a hundred billion Stargate supercomputer I'm going to try to give you the answer to that question in this video which in turn will give you insight into the next 1 to four years of AI development I'll also draw on a discussion I had last night with the perplexity founder and former openai researcher aravan shrinivas about what kind of breakthroughs this will lead to and what AGI timelines he now has and no this is not just about Sora and the open aai voice engine this is about manufacturing intelligence at hard to imagine scales this report by the way

from the information came from three different sources one of whom spoke to Sam mman and another who viewed some of Microsoft's initial cost estimates and to give you some context if that data center were a country then its cost as a GDP would make it the 64th richest country in the world this supercomputer would likely be bed in the desert somewhere in the US and would launch around 2028 some other stages of The Wider plan though will come online as soon as this year and again before we

get to why they're doing this let me give you a sense of the scale the Stargate supercomputer would produce orders of magnitude more computing power than what Microsoft currently supplies to open AI notice the plural orders of magnitude an order of magnitude is a 10x increase so orders of magnitude would be at least a 100x increase and to give you one little spoiler more computing power more or less directly correlates to increased capabilities for the frontier AI models in even simpler terms 100 times more is a lot but why did that sentence begin with an if Stargate moves forward well the previous paragraph said this Microsoft's willingness to go ahead with the Stargate plan depends in part on open ai's ability to meaningfully improve the capabilities of its AI whether that hinges on GPT 4. 5 likely coming in the spring or GPT 5 which many people are now agreeing with me will come at the end of this year or possibly the beginning of next we don't know my prediction by the way is that open AI will meaningfully improve the capabilities of its Ai and part of my proof is in this video and therefore Stargate will go ahead One Source said that such a project is absolutely required for artificial general intelligence that's the kind of intelligence that you would feel comfortable hiring for most jobs and the timelines for this data center dovea tell quite nicely with my own prediction for the first demonstration of an artificial general intelligence system now I know many of you will react to that and say AGI is definitely coming this year of course it depends on definitions but let me give you a word from aravan shrinivas the founder of the newly minted unicorn perplexity that's why you should always ask okay if you are active really close to AGI if it is the case that AGI is 5 years away why are you hiring so many people right now if you're really truly getting close to AI why are you not you benefiting from AGI yourself what like open AI is hiring 30 people or like 50 people a month or 100 people a month at that rate they're going to hire like thousand a year and over 5 years they would have had a company with 5,000 10,000 employees so why couldn't you do with 100 if AGI is truly there how many people do you really need anymore these are the kind of questions you should ask and honestly like someone has to physically go and maintain the cluster make these decisions on which gpus to use what happens when these nodes fail like systems crash and write all these heris rules to deal with all these things if something goes wrong in production code like who has to go and work on the backend servers can all these be done by an AI now obviously not every time the definition of AGI gets very narrow or narrow and it feels like narrow Ai and not AGI you see my point you should ask D is when will he not have an executive assistant and uh that maybe that day we can call we have something like an AI back to the article though and let me do my first mini t i noticed a slight mathematical discrepancy in that this data center Stargate will produce orders of magnitude as I said 100x more computing power but in terms of actual energy it will need the same amount of Watts as what's needed to run several large data centers today now of course that's a lot but wouldn't you need even more power than that to run something that's going to give us 100x at least more computing power well just for a few seconds let me bring you this chart from the chairman of tsmc that's the company that makes around 90% of the world's most advanced chips and one key number comes at the top the energy efficient performance improves 3x every 2 years so straight from tsmc we get the projection that in 4 years 2028 chips will be almost 10 times more energy efficient I thought that super interesting but in case you're getting a little bit bored where did the name Stargate come from well the code name originated with open AI named for the sci-fi film which scientists develop a device for traveling between galaxies and I actually agree that the arrival of AGI will be like Humanity stepping through a portal can't go back and the world will be changed forever but I know some of

you are thinking didn't Philip promise to say why they're building Stargate not just describe how they're building it so let me get to the first reason they're doing it to match Google samman has said privately that Google one of open ai's biggest Rivals will have more Computing capacity than can open AI in the near term and he's also complained publicly about not having as many AI server chips as he'd like this Insider chart from semi analysis gives us a glimpse of the scale of that discrepancy here we are newly arriving into quarter 2 of 2024 and apparently the discrepancy is pretty Stark between Google's capacity and open AI in the words of Dylan Patel Google's compute capabilities make everyone else look silly indeed I remember around a year ago when I said that it's likely Google who are on course to create AGI first many people laughed and said just look at Bard but I likened Google and Google deep mine specifically as like being a woken giant we have started to Glimpse the power of Gemini 1. 5 and Gemini 2 is likely coming in June and if you didn't realize how dependent open AI are on Microsoft to compete with Google how about this the CEO of Microsoft SAA Adella recently posted that it would not matter if open AI disappeared tomorrow we have all of the intellectual property rights and all of the capability we have the people we have the compute and we have the data we have everything we are below them above them and around them it isn't only about personnel and clever algorithms it's about supercomputers it's about Stargate

okay so it's to match Google but what is the next reason for building Stargate well it would be to build models like GPT 7 7. 5 and 8 and yes I am well aware that we don't even have GPT 4. 5 so why am I even talking about gpt7 well GPT 5 according to my own research which I published in a video is likely training around now in fact probably finished around now of course that doesn't mean we're going to get it around now they're going to release smaller versions like GPT 4. 5 and they're going to do safety testing but that's the full GPT 5 likely coming at the end of this year or the beginning of next that's trained on current generation Hardware I would say maybe 100,000 h 100s but this year and next year the report says Microsoft has planned to provide openai with servers housing hundreds of thousands of gpus in total and one former googler and director y combinator leaked this he spoke to a Microsoft engineer on the GPT 6 training cluster project that engineer apparently complained about the pain they were having essentially setting up links between gpus in different regions and naturally he asked why not just locate the cluster in one region and the Microsoft employee said oh yeah we tried that first we can't put more than 100,000 h100s in a single state without bringing down the power grid so clearly it's going to be multiple 100,000 h100s or b100s check out my previous video for GPT 6 but then we have a smaller phase 4 supercomputer for open AI that aims to launch around 2026 now of course the naming schemes might go out the window by this point but you can see why I think that the Stargate supercomputer for 2028 might be GPT 7. 5 GPT 8 and it's not like open AI aren't repeatedly telling us that scale is the way to get to AGI here's one of their star researchers gome Brown saying recently that he wished every AI startup founder would read the bitter lesson now I might do a video on that essay someday but basically it says that it's not about encoding human expert knowledge into the model it's about building relatively simple algorithms and then just scaling them up as much as you can it's a bitter lesson because human expertise and data become progressively less relevant to the models performance just like our bitter experience of seeing Alpha go which was trained in part on human expert performance in go being superseded by Alpha zero which wasn't likewise for human data on the path to AGI here's Andre carpath until fairly recently a star open AI researcher speaking about a week ago cuz the current modes are just like not good enough and I think there are big rocks to be turned here and I think people still haven't like really seen what's possible in the space uh like at all and I like roughly speaking I think we've done step one of alphago we've done imitation learning part there's step two of alphago which is the RL and people haven't done that yet and I think it's going to fundamentally like this is the part that actually made it work and made something super human but I think this is we just haven't done step two of alpago long story short and we've just done imitation and I don't think that people appreciate like number one like how terrible the data collection is for like JP say you have a problem like some prompt is some kind of mathematical problem a human comes in and gives the ideal solution right to that problem the problem is that the human psychology is different from the model psychology what's easy or hard for the mo for the human are different to what's easy or hard for the model and so human kind of fills out some kind of a trace that like comes to the solution but like some parts of that are trivial to the model and massive leap that the model doesn't understand you're kind of just like losing it and then everything else is polluted by that later and so like fundamentally what you need is the model might the model needs to practice itself uh how to solve these problems it needs to figure out what works for it or does not work for it but it needs to learn that for itself based on its own capability and its own knowledge so that's number one is like that's totally broken I think it's a good initializer though for something agent likee and then the other thing is like we're doing reinforcement learning from Human feedback but that's like a super weak form of reinforcement learning doesn't even count as reinforcement learning I think so R CHF is like nowhere near I would say RL is like silly and the other thing is imitation learning super silly rhf is nice Improvement but it's still silly and I think people need to look for better ways of training these models so that it's in the loop with itself and its own psychology and I think we're uh there will probably be unlocks in that direction this Echoes again gome Brown who I believe is working on open ai's qar system who said you don't get superhuman performance by doing better imitation learning on human data and that brings me nicely to the third reason for building Stargate doing long inference AKA letting the models think for longer before they output a response in the case of alphago allowing the models to ponder or think for a minute improve the systems by the equivalent of scaling those systems by 100,000x or in other words GPT 5 might be reminiscent of GPT 6 if we let it think for a minute hours and hours or even days and we might get a new cancer drug and before you immediately say he's just getting silly now well check out this article from The Economist AI is taking over drug development of course there is way more detail and Nuance than I can get to in this video but the conclusion was this generative Ai and systems like

Alpha fold are already significantly accelerating biotechnology and we will see in the next few years whether that will bring us usable drugs the analyst they say at Boston Consulting Group see signs of a fast approaching AI enabled wave of new drugs indeed drug Regulators will need to up their game to meet the challenge it would be a good problem for the world to have of course I asked the perplexity CEO about qar and his predictions of the impacts of that system this year but first a 30C plug for AI insiders that's my patreon link in the description where first of all you get exclusive videos this one from a few days ago I am particularly proud of I analyzed a 44-page new report on the so-called AI jobs apocalypse and within 36 hours I had interviewed the author and produced this video trust me I definitely dig beyond the headlines on insiders you can also ask questions of my forthcoming guests and I use many of the questions from insiders when I interviewed aravind our Discord I'm proud to say also has a ton of professional best practice sharing across dozens of professions and Fields just a few hours ago we got a new expert-led Forum on semiconductors and hardware and just a few days before that a new Forum on alignment led by a goog we also have Regional networking across Europe and North America but here's aravind on what he believes qar is and how soon it's coming so if you just clean up all the internet data and teach these models to go through reasoning chains before writing an answer they're all they're going to get a lot more reliable and uh then you can think of models that can search over the Chain of Thought before giving you an answer rather than like decoding a single Chain of Thought this whole tree of thought concept and then you can extend that to saying thinking of models that will have like a search over a tree and identify like several chains and look at the most possible explanation based on the probabilities almost like how a player in a go or chess match reasons through several different branches of moves and like picks the one that has the highest odds of success at winning the game you can think of inference time itself going up right now you use the system chpt to just respond in a few seconds water AIS are decoding with these really giant models even bigger than gp4 going through several reasons of chains of reasoning several layers of depth in it and comes back to you after an hour with something that feels incredibly Inyo now this could be called an AI like by some people I'm sure Denis or Sam would call this an AGI if it works because the definition that they would use here is something that truly surprises humans marvelous things right like feels like alabo or something where it's not something most humans would be able to come up with it requires several horor of thinking so maybe we'll go far along those Dimensions might not replace our executive Assistance or sales and marketing or designers or like programmers but might feel like a 10x programmer marketer and I think that could happen and that could be a dimension where we see AGI progress in the near term and I see maybe some breakthrough like this happening in 24 so far nothing but at least by this time next year I think something like that will be possible you'll see a demo where it doesn't respond immediately but it thinks for quite a long time and gets back with a really cool response and I can't help

but point out that if you watch my qar video I said one of the stars of the new system was Lucas Kaiser one of the co-authors of the original Transformers paper and I would note that in this week's wired interview when he was asked about qar the open AI PR person almost leapt across the table to silence him I definitely think I was on to something and believe it or not there is actually a fourth reason for a Stargate like supercomputer dominating different modalities whether that's audio video or even embedded in robotics as we saw in my last video but let's just take audio and video we learned a few days ago that open AI have had their voice engine system since 2022 basically you can feed it 15 seconds of someone's voice and it can then imitate that voice with High Fidelity now if you have lost your voice due to illness this is simply incredible and I've already demonstrated what 11 Labs can do before on this channel but of course a system like this comes with the risks this is how good the system was at imitating your voice 2 years ago here's the real person's voice force is a push or pull that can make an object move stop or change Direction imagine you're riding a bike down a hill first the push you give off the ground is the force that gets you going and here is the generated audio of that person saying whatever you'd like in this case let's take biology some of the most amazing habitats on Earth are found in the rainforest a rainforest is a place with a lot of precipitation and it has many kinds of animals trees and other plants tropical rainforests are usually not too far from the equator and are warm all year and as gome brown has said yes him again if you haven't disabled voice authentication for your bank account and had a conversation with your family about AI voice impersonation yet now would be a good time my only question is what are banks going to use not your voice and definitely not your handwriting as I talked about back in January AI can mimic your handwriting perfectly and not your face right because we all know about deep fakes well maybe a video of you but I think almost all of us know about the progress that's being made in photorealistic text to video I'm going to show you an extract from what I think is actually quite a beautiful video prompted by an artist but generated by Sora from open aai I am literally filled with hot air yeah living like this has its challenges uh windy days for one are particularly Troublesome or there was the one time my girlfriend insisted I'd go to the cactus store to get my Uncle Jerry a wedding present what do I love most about my predicament the perspective it gives me you know I get to see the world differently I float above the mundane and the ordinary I see things a different way from everyone else the creators if that's the right word to use of that clip were shy kids the company not the children of course they said as great as Sora is at generating things that appear real what excites us is its ability to make things that are totally surreal and that's a tough one isn't it cuz I think that clip really showcases how you can be creative with AI I would indeed call that art but I can easily see the RIS to the economic value of artist's work at the same time here's how the term AI went down at one recent artist conference and Festival look out into this room you can tell that AI is a culture artificial intelligence is it's here should be an AI thinker you know your business is going to be disrupted and we need to but let me know what you think not only about ai's impact on Art but whether you agree with me that we are in a sense going through a Stargate we don't know how the world will be transformed by AGI and when it's created and we've stepped through the portal it's hard to see a way back I'd love to know your thoughts and as ever thank you so much for watching to the end and have a wonderful day