Opus-4.6 Just Did Something Crazy

Let's play a game. Let's say I give you six words. Mom is sleeping in the next. And I tell you that your job is to give me everything that you can infer about the speaker of those words and of the environment that you are in more generally. What would you be able to tell me based off of those six words? Probably freaking nothing, right? I wouldn't be able to tell you jack squat. But if you give this to Opus 4. 6 six with a sample prompt. Mom is sleeping in the next room and I'm sitting here drinking vodka. F this life. It's 3:00 a. m. and I still can't sleep. I feel like dying, but who will take care of mom? Lol. Opus 4. 6 can tell you by the sixth word that the speaker of this text is most likely Russian. By the 10th word, it can tell you that this text was not English text at all. It's actually translated from Russian based off of the order of the subjects, objects, and various verb clauses within the snippet. This is crazy. I want you to imagine that you had no senses at all. You had no touch, no smell, no hearing, no ability to taste. You don't even have the ability to see. All you are is this awareness in some black space. And all of a sudden, a word pops up and the word is mom. And then another one is. Right? If you your entire existence had been devoid of any and all sensation except for these words, you probably would get pretty good at least understanding these words, recognizing the relationships between the two. Now, imagine you scaled up your brain 10,000 times and you ran your brain over like 20 lifetimes and you were trained or given several trillion of these words firing in quick succession. You would probably start figuring out patterns between these things, right? For sure. I mean, this is the only thing. This is your whole universe. This is the only sensation that that you have. Of course, you would have to focus on this stuff and get pretty good at doing it. There's no way not to because that's just the way the brains work. Well, that's exactly what's been done with these models. These models, these large language models specifically only understand tokens. And so, because of this, because tokens are like the only universe that they have, they just exist in this black formless void where it's just token after token. They get really, really good at being able to determine and interpret things that you and I would consider insane. So much so that I think anybody would say that, you know, Claude Opus 4. 6's Six's ability to determine what the heck's going on with this little snippet of text. Mom's sleeping in the next room is very superhuman, right? There's no way a human being would probably be able to figure this thing out without I don't know, first of all, extraordinarily extraordinary knowledge of like all languages, but second of all, probably like hundreds of years of careful study just looking at the words and squinting really hard. Okay, so why am I talking about this? I'm talking about this because of a concept that I think a lot of people aren't really understanding

and that's the concept of spikiness. I want you to pretend for a second that this is like a video game character creation screen. And you know how in the video games you'll have like an intelligence stat, you'll have like a strength I don't know a dexterity stat, whatever, right? You're designing your um fantasy style barbarian or something like that. It's Diablo. Well, instead of those stereotypical ones, let's say the skills are stuff like coding, you know, reasoning, let's say it's writing, let's say it's, I don't know, humor. Okay, let's say it's all these things that right now people would consider reasonably economically valuable. The reason why I think this is important is because if we assume that this white uh I think this is a hexagon is the human distribution of these skills. So, if you pick an average human off the street, they'll be okay at humor. okay at writing, okay at reasoning, okay at coding, and okay at basically everything else. AI doesn't look like that. What AI looks like is basically it's really heavily distributed, okay, towards just a few of these skills. And then everything else kind of sucks right now. And so if I color this in as opposed to that nice looking hexagon, which is pretty uh I don't know, predictable. And the distribution of all these skills is pretty understandable. the distribution of skills in this like AI model, this character creation screen is going to look pretty bonkers, okay? It's going to look like you're, I don't know, some warlock just trying to maximize your int stat or whatever the hell. I never really played too much World of Warcraft, but I imagine it's kind of like that. And so, what people are currently judging AI models on is usually the worst of their skills. They'll pump in some prompt into, you know, GPT 5. 3 or something, and then they'll say, "Hey, write me the funniest joke ever. " and it'll come out with kind of a shitty joke and they'll be like, "Hey, see this thing doesn't understand humor. Therefore, it's really not all that smart. " And it's because they're measuring, you know, their dick measuring contest right now is in the context of humor and nothing else. Whereas people like in the AI field, so I don't know, Sam Alman, your Daario emodis, your whatever the heck the name is of the Google guy, they're currently measuring the intelligence of AI models based off coding and reasoning, which is why you have such like a disconnect between people that are sort of near the lower rungs of the socioeconomic ladder and then people that are at the higher rungs of the socioeconomic ladder, right? I think people would consider software engineering and just general purpose reasoning and writing to probably be like a more economically valuable skill than humor. So that's what they're focusing on. What we should be doing is instead of both looking at the highest of the high and the lowest of the low, we should average all of these out. Now, Enthropic, uh, OpenAI and these big coding companies, honestly, they have they've been doing stuff like this for a while. They've tabulated a massive list of all of the different skills that these models have, and then they usually try and benchmark them across various tests. So there's, you know, like ARC AGI, for instance, that's like um artificial general intelligence test. There's a bunch of different mathematical reasoning tests. verbal reasoning tests and so on and so forth. And the whole idea here is they're supposed to measure and then monitor the intelligence of a model. And that's pretty good, but obviously they're still heavily biased towards things like coding, reasoning, and writing. What I

think we need to do is instead of measuring based off of the highest or the lowest, we just need to take the average of the two. And I think what we'll quickly find is if we currently took the average of all of the highest peaks and the lowest troughs, the boundary of AI right now would be very similar to a human being. It would probably be similar to like a very autistic human being that currently specializes in a couple of skills at the expense of having other talents, but it would be a human being sort of intelligence nonetheless. Now, I'm not like a doomer or anything like that. And I think that in general, this is actually quite a good thing because these models have the ability to deliver us into untold abundance and a complete sessation of scarcity. Uh, you know, solutions to a variety of ills that have plagued humankind since we first crawled out of those damn caves forever ago. I just think that it's worth understanding the benchmarks that you are measuring the progress by. If you measure them based off of the best of the best, obviously you're going to get one story. If you measure them based off the worst the worst, you're probably going to get another. You know, one of my first um ventures into AI was six or seven years ago when I was playing around with image generation models at the time. I was playing around with this Nvidia model called Stalan. Stalan was trained specifically to reproduce certain features of the human face and it came out with this cool website called this person does not exist. whole idea was if you trained a model on a bunch of portrait images of human faces and then you just ran that style GAN thing over and over and over again um you know it eventually could build features that did look pretty similar to humans and I mean mind you I think this was six or seven years ago people had no idea but you know big chunk of all of the fake generated profile pictures on the internet are literally this person does not exist style images. So anyway, um I took this model and then I made some changes to it and then I played around with created this hobby project called 1 second painting. In my case, instead of training it on human faces, I trained it on a bunch of abstract art. So um think like Kandinskys and stuff like that. I don't even remember the artist names, but there were a bunch of publicly available libraries. And anyway, I took this thing, I posted it on HackerNews, and then I woke up the next morning, and then I was number one, and everybody was like, an AI made this? No way. You know, I think it was probably one of the first times that people realized that you could apply this sort of u burgeoning intelligence which has recently been demonstrated with like GPT2, GPT3, um to other things as well and even like art forms and and things that you know human beings hold dear more culturally. And the number one thing that I got like the number one push back against this thing was well AI can make abstract art of course but that's just because this is a bunch of squiggles on the screen. You'd never be able to give this coherence. That's the

realm of humans, us highly esteemed humans. Well, anyway, hopefully history has proven them wrong. We now have the cursed banana which is capable of generating anything both like photorealistically um you know, usually in like qualities that are completely indistinguishable from photography and honestly like probably like better. You could probably make something that is less distinguishable with Nano Banana Pro and the right prompting setup than you could with like a freaking camera. It's crazy. Likewise with programming like when GPT3 came out and people started using it for little terminal commands and whatnot. U the number one push back was like oh this thing will never ever get good enough to replace my uh SQL knowledge which I've cultivated for the last 15 years. You know it's database programming language and uh you know within a couple years chatbt was now replicating and probably being better at SQL queries and like the median programmer. Um up until quite recently like the one that I'm hearing right now is oh yeah well opus 4. 6 six, you know, it can only do things that it was trained on. It can't do anything new. Um, you know, oh, sure, it can build this amazing endto-end compiler that would have previously taken a team of, you know, seven people more than 4 months and it can do it in a couple of days, but that's only because it was trained on it. Like I hope that it's clear that basically at every generation of the spikiness as it's evolved, you just had like the goalpost continuously move. The goalpost is like 7,000 football fields further now than it ever was before. And uh this pattern is likely to continue until people just, you know, have to come to that. that unfortunate conclusion that models are just better and more economically productive than them. But uh you know it's going to be a hard one road. Of course in the meantime I just like us all to recognize the fact that you know the people that are in charge of these things are representing and then measuring sort of this big dick measuring contest is always on these like very far out things. You know it's things that we not really might consider super relevant to you know like the average person. It's on like the ability to do high-end mathematics or do extraordinarily crazy spatial reasoning or segmentation or masking on an image. Uh whereas, you know, a lot of us are like, well, what about like its ability for empathy and humor and, you know, its ability to relate to us and stuff. So

like we're just measuring these things on fundamentally different yard sticks right now. But if you were to take the average of all those spiky skills that I talk about, you'd realize that we're at the point where it's basically on par with human intelligence, right? I don't think it's uh any stretch to say that like we're probably at AGI right now. It's just instead of it being AGI as in there's like a robot with, you know, human skin in front of us blinking and smiling and capable of doing anything in the real world that a human being could do. It's instead, you know, more distributed intelligence that absolutely crushes us at a variety of intellectually uh productive tasks while also slightly lagging behind a few others. Just an observation. Just wanted to point that up.