Microsofts New 'ORCA' Takes Everyone By SURPRISE! (Now ANNOUNCED!)

so Microsoft research recently released a new research paper called Orca and this video will demonstrate exactly why this is largely one of the biggest new Recent research papers by a very large company like Microsoft now many people have said that this is game changing but a lot of people are missing the true reason as to why Orca is so incredible so the paper starts out by saying Orca Progressive learning from complex explanation with traces of gpt4 and this was done by Microsoft research so essentially the abstract explains this well but I'm going to explain it in more detail later it says Recent research has found on enhancing the capability of smaller models through imitation learning drawing on the outputs generated by large Foundation models so essentially the problem here is that these large language models are really good but when building smaller language models what these people do when they build them they tend to overestimate the small model's capability as they learn to imitate this style but not the reasoning process of large language models so that's why okay is here and the reason this is going to be so crazy is because this completely changes how new people are going to train large language models and smaller models that are going to be on other devices now if you want to truly understand just how great Orca is it actually surpasses models like vicuna's 13 billion parameter large language model and if you don't know what vicuna is then you're like well how do I know it's even good because I've never heard of vikuna so vikuna was actually a open source large language models that was trained by fine-tuning the Llama model on the conversation data shared by users from share GPT and why this is so crazy is that you have to remember this model was trained primarily to serve as an llmn chatbot and it was actually fine-tuned by the version that meta created now of course you may have not heard of Acuna but if we talk about something else that you know which is chat GPT in gpt4 you're likely to understand these references much more clearly because Orca actually reaches parity with chat gbt on the BBH Benchmark and shows competitive performance in professional and examination like the SAT the LSAT and in xero shot settings and if you don't know what zero shot settings are that's essentially where you train a large language model in a certain environment without seeing specific kinds of examples or Styles and then you test it on new things which is really cool because if it's able to handle these things without seeing them before then we know that the effectiveness of this large language model is likely to be greater than we imagined in the introduction of this paper they also do show us the evaluations with gpt4 so essentially what that means is that gpt4 evaluated Answers by these large language models so from the left to the right we have llama all the way up to Chachi PT and orca and of course Google's bod and what we can see here is the evaluations so what the score represents is how effective these were against the range of different tests and what's incredible is that Orca actually manages to surpass chat GPT and not only that it also manages to excel a small amount which is 103 compared to chat gbt's 100 and Bard's 93 now what I also do find particularly interesting is that when we do come to the professional and academic exams like the SAT or the lsats Orca is quite similar to text DaVinci 03 which is chat tpt's earlier version without all the fine tuning that we now love in Chachi BT which means the Orca is much more effective than prior open source models and when compared to vikuno this is actually a 40 increase in Effectiveness with the same number of parameters the reason this is so incredible as well is that if we take a look at the parameter count for both of these large language models we know that there is a huge difference you see Orca is 13 billion parameters whereas chat GPT has 175 billion parameters but here's why this is so cool you see if we have a smaller launch language model one that is around seven times more smaller and is just as effective this makes it more lightweight and more integratable into certain devices which is something that many of these large language model creators are doing you see if you remember Google's i o speech couple of weeks ago what they actually announced was truly incredible they actually announced they were going to make some large language models available even when offline now the reason this is incredible is because as you know currently chat GPT is powered by a bunch of servers and is probably located somewhere on a data Farm but here's where things get interested Google has stated that we'll be making Palm 2 available in four sizes from the smallest to the largest gecko otter and bison and unicorn Palm to build on our research and our latest infrastructure highly capable at a wide range of tasks and easy to deploy we are announcing over 25 products and features powered by Palm 2 today Palm 2 models deliver excellent foundational capabilities across a wide range of sizes we've affectionately named them gecko order bison and unicorn gecko is so lightweight that it can work

on mobile devices fast enough for great interactive applications on device even when offline if these large language models now have a new framework in order to train and become more effective it means that even the smallest of large language models that we now have in future devices such as phones and laptops are going to be seven to eight times more effective which is one of the true applications of this because as you know large language models one of the limitations is their sheer size because even GPT 4 is 1 trillion parameters so this is going to be something very interesting as I've seen countless individuals currently running La Mar the 13 billion parameter large language model on their laptop offline and if people manage to get Orca offline it's going to present a very interesting opportunity and the reason this could also pose some insane applications is because as you know openai is currently developing their humanoid robot in conjunction with a company called 1X Robotics and the reason this is so incredible is because what happens when we do manage to get these small large language models integrated into these robots and let's say they no longer need their cloud-based servers to process and generate responses and can do it instantly it's going to present a very interesting turning point where these large language models are pretty much standard just imagine having chat gbt on your phone without internet access just how great would that be further on into the paper they also show the complex zero shot reasoning tasks that we talked about before you can see in the big bench hard test it scored even better than chat GPT scoring a 49. 7 accuracy in the big bench heart now this was where I found the data to be very interesting from the research paper these are the system messages and there are 16 different messages that they use to train this large language now not only are these 16 system messages very good you can actually personally use these if you are trying to understand the concept one thing that I do personally do is use some of these system messages myself when I'm trying to understand complex Concepts that I don't particularly understand for example it says you are an AI assistant provide a detailed answer so the user doesn't need to search outside to understand the answer you are an AI assistant you will be given a ask you must generate a detailed and long answer you are another sister who always provides an explanation think like you're an answering to A5 and trust me when I say that using this framework to get very difficult Concepts down is quite impressive and this is one of chat gpt's very unique feature now this explains like M5 technique isn't something that is just used by US users this was something that they originally did to chat GPT and to Open the Eyes DaVinci 003 early on when they were trying to make the system more better a research paper that was actually originally produced on the 24th of May of 2022 which was recently revised actually talked about how stating let's think step by step before each answer actually increases the level of results and outperforms zero short large language models on diverse reasoning tasks including arithmetic and other logical reasoning tasks they were able to improve it from anywhere from 10 to 40 by simply adding those a few words let's think step by step now one thing I did find largely interest interesting that many people didn't mention and this isn't to say that these models are bad but it is interesting to see the kinds of limitations that these large language models do possess because so often when these large language models are released we're often so shocked by what we do see and wonder how these capabilities are even possible but here we have some tests that show that gpt4 gpt3 chat GPT and of course this new model Orca just can't understand and the question is simply asking I tried five clothes to dry out in the sun it took them five hours to dry completely how long would it take to drive 30 gloves so this is a common sense question which possesses a really simple answer but a lot of these large language models and I find it interesting because I've seen this myself sometimes they tend to over complicate simple things you can see Orca vikuna and Chachi Beauty all doing insane math to figure out how it would actually work and this was actually taken from this Ted Talk where they said AI is incredibly smart and also shockingly stupid okay one more would I get a flat tire by bicycling over a bridge that is suspended over Nails screws and broken glass yes highly likely gpt4 says presumably because it cannot correctly reason that if a bridge is suspended over the broken nails and broken glass then the surface of the bridge doesn't touch the sharp objects directly okay so how would you feel about an AI lawyer that is the bar exam yet randomly fails at such basic common sense AI today is unbelievably intelligent and then shockingly stupid or a better example right here we can see this question now this question is shockingly basic it says I have a 12 liter jug and a 6 liter jug and I want to measure six

liters how do I do it the simple answer obviously is to Simply take the 6 liter jug to measure the six liters because you already have a six liter jug but Orca decides to completely fluff this and give out a strange answer the same with chat GPT and Incredibly the same with gpt4 now I definitely think you should watch this Ted talk because it does dive into what we don't really understand about large language models when we realize that they over complicate very simple Concepts and Common Sense Concepts so it's interesting to note that this is one shocking limitation of not just Orca but many different large language models and I still find it incredible that we haven't managed to solve this problem them yet so with that being said I do think that in the future as these large language models continue to get more efficient more Nimble and more effective at different sorts of reasoning tests and different sorts of questions that eventually we will have some state-of-the-art offline model that is used widely the only thing is that what will its primary purpose be I think that will be very interesting to see