# 10,000 Of These Train ChatGPT In 4 Minutes! ## Метаданные - **Канал:** Two Minute Papers - **YouTube:** https://www.youtube.com/watch?v=_3zbfgHmcJ4 - **Дата:** 24.11.2023 - **Длительность:** 8:24 - **Просмотры:** 99,167 ## Описание ❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers NVIDIA H200: https://www.nvidia.com/en-eu/data-center/h200/ The Bitter Lesson: https://www.incompleteideas.net/IncIdeas/BitterLesson.html 📝 My latest paper on simulations that look almost like reality is available for free here: https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations: https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Benji Rabhan, Bret Brizzee, Bryan Learn, B Shang, Christian Ahlin, Gaston Ingaramo, Geronimo Moralez, Gordon Child, Jace O'Brien, Jack Lukic, John Le, Kenneth Davis, Klaus Busse, Kyle Davis, Lukas Biewald, Martin, Matthew Valle, Michael Albrecht, Michael Tedder, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Richard Sundvall, Steef, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi. If you wish to appear here or pick up other perks, click here: https://www.patreon.com/TwoMinutePapers Thumbnail background design: Felícia Zsolnai-Fehér - http://felicia.hu Károly Zsolnai-Fehér's research works: https://cg.tuwien.ac.at/~zsolnai/ Twitter: https://twitter.com/twominutepapers #nvidia #chatgpt ## Содержание ### [0:00](https://www.youtube.com/watch?v=_3zbfgHmcJ4) Intro today you are going to see a lot of papers come alive in the world around us why well Nvidia just announced their shiny new h200 graphics card and this is a big deal in fact to say it my opinion would be an understatement but first bad news it is pricey few if any of us are going to be able to buy this so I hear the question caroy why do we have to speak about a graphics card that we won't buy well I have good news too we don't have to buy it and yet each of us can still enjoy their advantages I'll tell you how in a moment and first let's roll up the sleeves and have a look at what they can do so what are the advantages of such Hardware well it can simulate a landing on Mars power superc computers or r a fusion reactor simulation and it can perform these beautiful fluid simulations tornado simulations and visualize solar flares or do protein folding this leads to a better understanding of the building blocks of life and it is already being used for creating new kinds of medicine something that was so difficult we always thought was only going to happen in the future and now it is here of course these graphics cards also power video games and what is this here dear fellow Scholars this is 2minute papers with Dr Caro okay kidding this is an aerodynamics and acoustic simulation around this new car new hardware can typically do more intensive tasks than previous generations or it can do the same tasks and consume less energy like it is doing here something that we have to think about more and more in the future you see the jump from gpt2 to GPT 4 requires orders of magnitude better Hardware consider this gpt2 had 1 and A2 billion parameters gpt3 175 billion paramet a jump of more than 100x then GPT 4 more than 1 and a half trillion parameters a, x difference this requires tons and tons of compute and today specialized versions of it already exist get this where scientists created a chemistry assistant that can not only answer your burning questions about Metal organic Frameworks but it also learned to predict crystallization outcomes as well ### [2:57](https://www.youtube.com/watch?v=_3zbfgHmcJ4&t=177s) GPT Training and today Nvidia can train a gpt3 AI not in years and not even in days but hold on to your papers fellow scholar and look less than four minutes what oh my goodness also stable diffusion in less than two and a half minutes incredible AI systems are now being trained in a matter of minutes my mind just refuses to comprehend this and it just got e even better this is with the old h100 ### [3:30](https://www.youtube.com/watch?v=_3zbfgHmcJ4&t=210s) H200 graphics cards and they now announced h200 this has almost double the memory capacity wow this is one of the main limiting factors in how big of an AI system we can get access to and it just got nearly doubled memory bandwidth is now up 40% and almost broke a mouth watering 5 tabes per second we are almost there and of course things are going to run faster too way faster loving it so good and B 100 is coming next year so make sure to subscribe and hit the Bell icon if you would like to hear me flip out over it of course if it is any good from the extrapolations wow buckle up fellow scholar it is going to be real good so this Hardware improves all things Ai and today AI is a piece of software that writes new kinds of software see where this is going this is the engine of scientific development and we just got a new engine and compute power just keeps increasing at an incredibly rapid pace and today no field of research no branch of science remains Untouched by these techniques so why is this good news for all of us well even if we don't buy this Hardware this Powers the tools we use day by day chat GPT and many other AI tools that are absolutely everywhere I am also happy to see Lambda our long-standing partner showcased here great job and remember 92% of Fortune 500 companies use CAD GPT and we haven't even talked about any other AI here and two it is also fueling the next iteration of research works it is fueling the papers so we fellow Scholars are very happy to see this and one more thing about increasing Hardware capabilities that put this image into perspective in AI research there is or perhaps at this point I can say there only was two schools of thought one Richard Sutton argues in his Landmark article by the name the bitter lesson that AI research should not try to mimic the way the human brain works he argues that instead all we need to do is formulate our problems in a general manner so that our learning algorithms may find something that is potentially much better suited for a problem than what our brain is doing and he argues that the best performing learning techniques are the ones that can leverage computation or in other words methods that improve significantly as we add more compute power to the utter school of thought people tried to encode lots of human knowledge of strategies in their go AI but did not have enough compute power to make a truly great algorithm and now we have Alpha go which surprisingly contains minimal information about go itself and yet it is better than the best human player ERS in the world so what does all that mean well in other words algorithms are extremely important I don't need to argue for that on this channel but never forget compute is just as important and ### [7:14](https://www.youtube.com/watch?v=_3zbfgHmcJ4&t=434s) Sponsor Message this graphics card is a metric ton of more compute turns out for all of us even if we don't buy it ourselves what a time to be alive if you're looking for inexpensive Cloud gpus for AI Lambda now offers the best prices in the world for GPU Cloud compute no commitments or negotiation required just sign up and launch an instance and hold on to your papers because with the Lambda GPU Cloud you can now get on demand h100 instances for just1 199 per hour yes $199 and they are one of the first Cloud providers to offer publicly available on demand h100 accs did I mention they also offer persistent storage so join researchers at organizations like apple MIT and ctech in using Lambda Cloud instances workstations or servers make sure to go to lamb. com slapers to sign up for one of their amazing GPU instances today --- *Источник: https://ekstraktznaniy.ru/video/12903*