Are we confusing better word prediction with actual intelligence?#worldsciencefestival #briangreene

Were you at all surprised that sufficient data and sufficient compute in this context of neural networks that you already knew you were was limited, were you surprised at what it could do? I certainly wasn't blown away. I mean, I had been writing about this particular technology, large language models, since 2019. And it was always obvious that if you had a richer database, you'd do better. So, if you try to approximate English with what we call bigrams, so just pairs of words, you get a decent approximation of English. You can start to tell that it's English. If you do it with three words, it starts to look pretty good. If you do it with seven words, so you take the probabilities of any string of seven words, it starts to look really good, right? I mean, this is a function of the law of large numbers and having more data. Um and if you did it with like 100 words, it's going to be great. Like, that's no mystery. We've known that since at least 1965. It was obvious that as you increase the data, these models are going to get better at what they do. And what they do is they build an approximation of how people use words. It was also obvious that was not enough to get to artificial general intelligence.