New AI Discovery Changes Everything We Know About ChatGPTS Brain

a new AI research paper has revealed some surprising geometric structures in the llm Learned Concepts Max techark has done a recent paper that reveals the surprising structure of AI brains he talks about how our new AI paper reveals the surprising geometric structure in the llm Learned Concepts he talks about the fact that they form brain like lobes they form semantic crystals much more precise than it seems and the concept cloud is more fractile and round so if you aren't familiar with the fact that large language models like GPT have been something of a blackbox we know that they work but understanding exactly how they work has been challenging it's quite like having a car that runs perfectly but not being able to see under the hood now recently scientists developed tools called sparse Auto encoders they essentially just act like x-ray machines for AI letting us Peak inside and see how these models organize information think of it like finally getting to look under that car's hood and discovering an unexpectedly organized engine now the researchers found three distinct levels of organization each more complex than the last and this layered organization wasn't programmed in it emerged naturally as the AI learned which is what is particularly fascinating so first we had the level one which is the atomic structure so here's what they found the researchers discovered that at its most basic level the AI organiz Concepts in geometric patterns imagine a giant 3D connect the do puzzle where related concepts are linked together in specific shapes the classic example that they used is how the AI understands the relationship between words if you plot the concepts man woman king or queen in the ai's mental space they form a perfect parallelogram which is essentially just a shape like a stretch rectangle and the distance from man to woman is the same as the distance from King to Queen and this basically shows that the AI has learned that adding femininity works in the same way whether you're talking about royalty or regular people so now here's where we do see something rather fascinating you can see that I've managed to create a short artifact with Claude that helps you visualize exactly what I'm talking about we can see that the relationship between man and woman parallels the same relationship between king and queen and it's supposed to be a parallelogram but Claude actually struggles with this quite a lot now this is also the same for Capital Cities which you can see right here you can see that countries and their Capital Cities form similar relationships in the ai's understanding and in the past tense you can see that these verb tenses show parallel relationships though not as always as regular as other examples this was something that was discussed in the paper but I find this to be really cool now initially these patterns were hard to see because they were obscured by irrelevant information it's like trying to see a constellation while there's light pollution you have to filter out the distracting lights first now the researchers found that things like word length were creating noise that had to be filtered out to see the true patterns and then this is where we get onto level two which is the brain structure so the most exciting finding was that the ai's knowledge is organized into distinct regions or lobes similar to how a human brain has different areas for different functions this organization wasn't programmed it emerged naturally now there's three main loes there's the Cod SL math lobe this is where it specializes in programming and mathematical Concepts and this activates strongly when the AI deals with a coding task or mathematical problem and this shows clear boundaries from other types of knowledge then we have the general language Lo this handles regular English text processing and deals with the general knowledge and comprehensive understanding and the general language lobe processes longer form content like articles and papers now of course we have the dialog lobe which specializes in conversational text and short messages this is particularly active during chat interactions and this handles different communication Styles than the general language lobe now the research Fe actually proved that these lobes weren't random by showing that features that often activate together are physically closed together in the ai's mental space which is actually remarkably similar to how neurons that fire together in a brain tend to be physically close to each other now then we had the level three of the brain which is the Galaxy structure so this is where looking at the overall entire system this is where the researchers found that the ai's knowledge is

organized in a specific way that follows mathematical patterns now this isn't random it's highly structured particularly in the middle layers of the AI now some of the key findings from this is that the organization is most pronounced in the middle layers of the AI where the representations become highly abstract and Consolidated reflecting the most significant transformations of the input data this layer acts like a kind of information bottleneck where only the most essential features are retained for further processing enhancing the model's ability to generalize now information in this you know AI brain also appears to be compressed efficiently quite like how our brains compress sensory information just as the human brain reduces complex sensory inputs into manageable chunks for efficient processing the middle layers of AI condense vast amounts of data into simpler high level representations and this efficient compression helps the model to focus on key features while discarding irrelevant details leading to more robust performance across tasks now the structure follows specific mathematical patterns which are essentially power laws that suggest optimal organization and these power laws indicate that the distribution of feature importance is not random but follows an organized predictable pattern and the largest principal components dominate through representation while the small smaller components taper off creating a natural hierarchy of information this hierarchical structure not only makes the AI more efficient but also aligned with how biological systems including human brains tend to prioritize and process information now all of this is pretty cool but why does this matter why is this even significant to AI research well we can now see how AI systems organize information and this has far-reaching implications for both research and practical applications this helps explain why they're so effective at various tasks by understanding the internal organization of these systems we can see how models are able to generalize identify patterns and adapt to a wide range of different challenges from language translation to problem solving this detailed organization is what gives AI its versatility and efficiency now this also gives us insight into potential improvements if we actually know how these systems organize Concepts this allows us to make targeted enhancements for example we could refine the way features are learned in order to improve the clarity of specific lobes or even develop new training methods that enhance certain capabilities now this Insight can actually also help in reducing biases optimizing computational efficiency and making AI models more interpretable which is critical for deploying AI in sensitive areas like health care and finance now it's actually pretty crazy because it parallels with human cognition now this suggests that there might be a fundamental principle about how intelligence organizes information the fact that Ai and biological brains independently arrived at similar ways of structuring knowledge implies that there could be Universal rules governing efficient information processing this has the potential to influence how we design future AI systems making them more aligned with natural intelligence and of course this could help us better understand both Ai and human cognition by studying these structures in AI we can draw parallels to the human brain and potentially learn more about how our own minds work understanding why AI develops these lobes and what functions they serve could lead to better models of human cognition offering insights into how we think and solve problems and this knowledge might help in addressing cognitive impairments designing Advanced learning tools or developing AI that collaborates even more effectively with humans now there are some limitations and caveats to any AI research this doesn't mean that AIS are like human brains the similarities are organizational not biological so while the ai's structures resemble those of a brain they are fundamentally different the AI is built using layers of mathematical functions and weights unlike biological neurons that are compressed of cells and synaptic connections the AI isn't conscious or thinking like a human it processes inputs and produces outputs based on learned patterns but there's no awareness or subjective experience involved its responses are derived purely from the data it has been trained on without any understandings or feelings now these are mathematical patterns like I said before not biological ones the structures that emerge in AI are the result of optimization processes to find the most

efficient way to represent information mathematically unlike the biological processes that evolved over millions of years these patterns are engineered to maximize efficiency in computation and task solving not to replicate the functioning of a human brain now of course this is just the beginning of understanding these structures more research is needed to fully understand the implications we're only really right now starting to understand how these structures form and what they mean for the overall functioning of AI systems there are many open questions such as how these structures change as models grow larger and whether they can be influenced actively during training to improve performance or interpretability now the field is rapidly evolving we've got many different advances happening in Ai and new techniques and discoveries emerging all the time and the study of these brain-like structures is likely to lead to even more breakthroughs not only in building better AI but also understanding fundamental aspects of intelligence itself and as more researchers explore these Concepts we might find parallels that extend beyond AI potentially offering new insights into cognitive science neuroscience and even the philosophy of the Mind