[ML News] Microsoft to spend 100 BILLION DOLLARS on supercomputer (& more industry news)

hello and welcome back to ml news today we're going to go over several things that happened in Industry during the last week the first one Microsoft planning to

spend a100 billion on a super computer to power open AI models Microsoft notably being the sugar daddy of open AI is now apparently in talks to build an even bigger supercomputer for open AI now this is all people familiar with the matter and there are talks and so on and however a100 billion doar is a little bit bigger than previous projects were this article by Fortune goes into a bit of back and forth about it with critics saying something and then Microsoft must be thinking something but one argument that I find to be good is saying okay if Microsoft actually invests this much then it can only be worth it if it's in fact AGI the only way a single model open AI model could justify such an outlandish investment on a single data center they said was if the model were in fact AGI however Microsoft has to hope that whatever Stargate is being used for is in fact not to train AGI since Microsoft's partnership with open AI only entitles the tech Chan to commercialize open ai's technology that falls short of AGI so open AI has in their Charter I believe or in their deal with Microsoft they say okay you get yet to have everything we have except once we've invented AGI and we say when we've invented AGI so is the1 billion data center a fiction or not who knows it's all people familiar with the matter say however what I believe is that when finally open AI announces that they have reached AGI it will be such an underwhelming moment they'll just say it out of some political or money maneuvering they will still be like okay can write really nice emails or something like this right they'll have some arbitrary metric where they can say justify like Oh by our metric this is now AGI it will be completely underwhelming I'm not saying we're never going to reach something like AGI or something that people today would consider AGI what I am predicting is that the moment open AI says they've reached AGI will be absolutely underwhelming stability AI has made a

big change namely the founder and CEO or now ex CEO emad mustak has resigned from being CEO of stability AI stability is transitioning finding new CEO while emad is moving on saying that he is going to make sure that AI remains open and decentralized emad looking for new adventures outside after having founded stability AI we're all excited to see what stability a will continue to do notably the company has gone through different phas raes they obviously came out initially with very strong models like stable diffusion Gathering a lot of attention being overtly open source then later shifting strategy to the more open weight strategy we see from other companies nowadays where you can use the models for personal use and up to a certain limit but then once you use them commercially you have to pay the money which in know Fair companies have to make money and they're still on the Forefront of research with stable diffusion XL incredibly capable model but the future of stability AI is as of now unknown and uncertain and we're excited to see what imod is going to do

next shortly after releasing the weights of grock 1 fully open and I mean fully open now Twitter has announced their new model grock 1. 5 they say it comes with improved reasoning capabilities and a context length of 128,000 tokens this is going to be a available on X soon they don't say too much more but they do release some numbers or you can see that the model does favorably compare with sort of current models that are around it's not the best model ever around however it holds itself pretty well and notably it's a definitely improvement over grock one and with long context understanding goes up to 128,000 tokens I have to say that token lengths are kind of getting insane and it's pretty excited to see what these models are going to be able to do once people really get the hang of how to use this long context well also in some news

I have been verified on Twitter I didn't pay for this they just gave it to me yeah it's still the same I just have this thing also they didn't verify my personal information this could have been an impostor all the time and anyway I now have actually access to the grock model on Twitter so when 1. 5 comes out if I notice I will notify you open AI

has released these blog post called navigating the challenges and opportunities of synthetic voices they detailed their first experiments of what they call voice engine which is a model for creating custom voices has been a thing for a while and I have never been able to really distinguish quality because quality of these has gotten so incredibly high but this model is another one in the series where you give text input and a single 15-second audio sample of a speaker you put them together and then you'll have a natural sounding voice that sounds like the 15-second sample but reads the text that you give it they have a few examples right here which you can listen to but honestly these blog posts they're always phrased in the couched in the language of safety we're releasing a small preview now oh we're committed to safety and so this is an ad like let's be real this is like a teaser a trailer for their upcoming model this is nothing else they don't say anything of importance here and they try to couch it in safety you know how like every perfume and fashion brand and so on they're just trying to couch whatever they do in the sense of Lifestyle this is the same like they're just trying to use safety as a means of delivering an ad to you open AI where is

Elia they also released a blog post called sora First Impressions and this is pretty cool they release kind of what people have been doing with Sora this is called Airhead which they feature on top it's about a person with a balloon as a head and uh it's still clunky to make movies with Sor you can see that all the shots are kind of you know these kind of sort of panoramic um it works for some movies but I think it get old pretty quickly if you just did that but it's already cool to see that essentially the model itself is a foundation for art is fun you can watch the other movies as well they have some again on this blog post uh they've also released them on their YouTube

channel also open AI says we're partnering with a small group of us Builders to test usage based GPT earnings our goal is to create a vibrant ecosystem where Builders are rewarded for their creativity and impact and we look forward to collaborating with builders on the best approach to get there again open AI trying to become the platform here so if you create a GP PT which is essentially sort of like a bit of a prompt flow you can earn money if other people use that it's like an app store and they've announced this already previously that they would do this and now they say they're experimenting in a small scale how that's going to work it's going to be difficult because gpts or llm programs they're essentially completely transferable there's nothing to do a big vendor lock in to any particular llm so you can just take the same and go somewhere else go to claw or go to wherever it's going to be interesting to see how these companies are attempting to introduce some sort of ecosystem lockin or some sort of vendor lockin with these Technologies ultimately language as an interface is just by its nature Universal and also open AI where is Elia also Jensen W

casually dropping at GTC that open ai's newest model has 1. 8 trillion parameters and required 30 billion quadrillion flops to train at this point they're just making up numbers don't they just be like how many flops do need well how about 30 billion quadrillion sounds like an evil villain asking for way too much Ransom money but I'm going to give them the benefit of the doubt and believe that that's the actual number

and lastly Thomas wolf has released a talk on how to build large language models in 2024 called a little guy to building llms in 2024 you can see the recording of the talk the topics that are covered very cool and go into modern llm training including like how to evaluate and prepare training data how to do parallelism to train effectively and so on talk is on YouTube slides are available excellent thank you Thomas for releasing this all right that was it mostly stuff going on in industry and honestly at open AI because without Ilia they can now freely make money I'll see you around