Metas Llama 3.2 Is Much Bigger Than You Think!

just when you thought AI couldn't get any better we actually have Mark Zuckerberg bringing in more AI announcements Yama 3. 2 is officially here and it's bringing some serious upgrades to the AI world with a range of models that now fit on edge devices optimized for Qualcomm and mediatech and offering unprecedented performance for on device AI llama 3. 2 is ready to push boundaries but that's not all meta also dropped a ton of exciting updates across their AI ecosystem that you need to know about so of course let's first talk about llama 3. 2 and what it actually is meet llama 3. 2 which is kind of like Lama 2's supercharged futuristic cousin or you could call it the complete version of yarm 3 now this isn't just an upgrade it's a massive leap now the two largest models in this new lineup 11b and 90b can actually C and by C I mean they can do image reasoning so what does image reasoning actually mean in plain English well imagine you have a graph showing your small business's sales over the year and you're like hey llama which month had the best sales llama 3. 2 can look at that graph read it like a human and tell you exactly which month was your best it's not just spitting out data it's actually reasoning through what it sees like a really smart friend who knows how to read graphs better than you do which is remarkable because it's quite rare to actually get open-source Vision models that are actually even good now what's crazy is the example they've actually shown us here on screen you can see that they have an image understanding demo which showcases the vision capabilities of this model so first they upload the image to llama 3. 2 then we can see the description of the image highlighting its modern open plan layout the black leather furniture the central fireplace and the abstract painting the overall Ambience is described as warm and visually striking then we get the detected objects including the couch chair fireplace coffee table potted plant painting and side table the user selects the fireplace object to explore alternative options for that design element then we get replacement suggestions the system generates a list of possible replacement fireplaces each with a detailed description such as a minimalist wall mounted electric fireplace with a subtle LED Flame or a austic stone-faced gas fireplace with a traditional charm then finally after selecting the fireplace the rag model fetches related images of modern fireplaces these suggestions align with the user's choice and provide visual inspiration for Alternatives overall I think this shows us exactly how Vision models can truly Elevate the AI experience and now with this becoming open source there's going to be a lot more experiences coming our way so now let's take a look at some of the benchmarks for the model called llama 3. 2 and I'm not going to lie these benchmarks are actually quite surprising for a model of its size it truly is baffling to how quickly these models that are open source are actually catching up to their closed sourced counterparts firstly let's take a look at what meta themselves have actually said about these models they say that our evaluation suggests that the Llama 3. 2 Vision models are competitive with leading Foundation models Claud 3 Hau and GPT 4 mini on image recognition and a range of visual understanding tasks the 3B model outperforms the Gemma 2 2. 6 B and 5 3. 15 mini models on tasks such as following instructions summarization prompt rewriting and Tool use while the 1B is competitive with Gemma so when we first actually take a look at these models we can see that they are quite impressive for their respective classes and by respective classes I mean that you can't really compare these llama 3. 2 models to the state-of-the-art models like GPT 40 and Claude 3. 5 Sonet because they are much larger in size so looking at this table of vision instruction tuned benchmarks let's break it down and dive deep into what's actually going on here with Lama 3. 2 particularly the 11b and 90b models and how they stack up against models of their respective size like Claude 3 hi cou and GPT 40 mini if you're not familiar with these benchmarks it doesn't really matter I'll explain them as we go along now I've got to be honest upon further inspection these benchmarks are a little bit deceiving not majorly deceiving but the color scheme makes it seem that llama 3. 2 out performs these other models on every single Benchmark which isn't entirely true don't get me wrong the model is actually great but it's just something to be aware of so let's look at the mmu pro which is mathematical reasoning with vision this Benchmark evaluates the models on their ability to reason through complex multimodal mathematical problems that involve both text and visual data here llama 3. 2 90b model scores 45. 2 a significant lead over Claude 3 hau's 27. 3 and slightly ahead of GPT 4 o mini's 42. 3 three this is a critical win because it shows llama 3. 2 superiority in handling mathematical reasoning where visual elements like graphs or diagrams play a role this could be particularly useful in Educational Tools where understanding and explaining visual math problems is essential and llama is showing it can outperform competitors in this area next let's take a look at another key area which is the math Vista so math Vista is another test where models are tasked with reasoning through mathematical

visuals like graphs diagrams or even handdrawn figures here llama 3. 2 is 90b model scores 57. 3 beating clae 3 High C's 4 46. 4 and GPT 40 minis 56. 7 while GPT 40 mini stays competitive llama 3. 2 still edges it out this difference might seem small but in high stakes Fields like Finance or scientific research even a slight advantage in understanding complex visual data can make a huge impact llama 3. 2 consistent performance shows it's better equipped for these types of reasoning tasks one of the standout areas for llama 3. 2 is its performance in chart QA a benchmark where the models need to answer questions based on visual charts and graphs the 90b model scores 85. 5 comfortably surpassing Claude 3 hius 81. 7 with strikingly GPT 40 mini not even being listed here this win is huge because understanding and interpreting charts is a key skill in real world applic appliations whether it's in business analytics scientific research or any field that deals with data visualization the ability to correctly analyze and interpret chart data is a crucial capability llama 3. 2 stronger performance means it can more accurately respond to questions about visualized data making it highly valuable for these use cases in the ai2 diagram Benchmark llama 3. 2 once again pulls ahead with the 90b model scoring 92. 3 while Claude 3 Hau scores 86. 7 and GPT 40 Mini doesn't even have a score listed this Benchmark is about understanding complex diagrams such as technical or scientific illustrations Lama 3. 2 performance here demonstrates its Superior ability to analyze and reason through Visual data this has big implications for Industries like engineering Healthcare and education where understanding diagrams is a fundamental skill llama edge here means it can better interpret these kinds of visuals providing more accurate insights or explanations which could be used in anything from diagnosing medical conditions to helping students understand technical Concepts now here's the thing llama 32 isn't just a vision addition to the model it's another iteration of llama 3 which actually improves in areas like text also which was quite surprising to me considering the amount of things meta is doing with AI now if we look at the text benchmarks we can see that in the MML U general knowledge the Llama 370b model scores 82 while llama 3. 2 is 90b pushes higher at 86 reflecting better general knowledge and comprehension math mathematical reasoning llama 3. 2 90b model scores 68 significantly improving upon the Llama 370b score four this highlights how the 90b model handles complex mathematical tasks with greater accuracy GPO QA reasoning in reasoning tasks llama 3. 2 90b scores 46. 7 showing a clear improvement over llama 370b 39. 5 making it better at following instructions and solving reasoning problem in short llama 3. 2 90b model outperforms the 70b model in text based tasks particularly in areas like math multilingual capabilities and general knowledge making it a more powerful model for handling a broader range of challenges with greater accuracy for whatever reason they decided not to compare llama 3 to llama 3. 2 now here's where we get on to a part of the video that is rather unfortunate for those of you who are like me and you're living in a region where there are tough regulations unfortunately this model is currently unavailable to you for example if you currently live in the EU or in the UK you won't be able to access this model and the worst thing about it is that actually tried to use this model in their online browser with a VPN and it just doesn't work quite the frustrating time to be someone who's trying to cover all the relevant AI announcements when your nation is regulated to Hell hopefully in the future they decide to relax these regulations on models that clearly aren't a threat to anyone now luckily for you this video doesn't have to end on a bad note meta actually did announce something called Orion this was actually an insane announcement because this is a hardware project that seems really promising about a decade ago I uh I started putting together a team of the best people in the world to uh to build these glasses and the requirements are actually pretty simple but the technical challenges to make them are insane um you know they need to be glasses you know not a headset no wires less than 100 gr uh they need wide field of view holographic displays sharp enough to pick up details bright enough to see in different lighting conditions large enough to display a cinema screen or multiple monitors for working wherever you go whether you're you in a coffee shop or on a plane or wherever you are and you need to be able to see through them and people them through them too and make eye contact with you right this isn't pass

through this is the physical world with holograms overlaid on it so if someone messages you uh you will see that and instead of having to pull out your phone there will just be a little hologram and with a few subtle gestures you can reply without getting pulled away from the moment or if you want to be with someone who is far away um they're going to be able to Tele Port as a hologram into your living room as if they're right there with you you're going to be able to tap your fingers and bring up a game of cards or chess or holographic ping pong or whatever it is that you want to do together you can work or play or whatever build so let me know what you think about the AI Hardware area I think this is one of the most promising areas for AI since it actually brings AI into the Daily World for regular individuals that could benefit from it on a dayto day basis so hopefully you enjoyed today's video let me know what you think