Ex-OpenAI Researcher Says They're ALL Wrong About AI

an exopai researcher is funding a new startup and we have to talk about it. So there's a new type of AI that is going to be coming and this startup from this ex open this is like a senior guy at openi this is the guy that worked on you know the early forms of 01 and other significant models and this guy is developing okay his new startup and it is really crazy what he's trying to achieve. So take a look at this. So, it talks about the fact that he's trying to build a new startup that is raising between $500 million and a billion dollars in funding. Okay, that's a super significant amount because remember it was Ilas Sutska who went off to, you know, lead uh save super intelligence that I think raised money at a $32 billion valuation. Now, essentially what they're actually going to be doing is developing AI models using methods that major firms such as OpenAI and Anthropic aren't heavily focused on according to materials shown to potential investors. And they want to create AI models that can learn on the fly from realworld experience. This is the capability known as continual learning, which of course we all know that today's models can't do. Now, I think startups like this are super interesting because they're not pursuing the LLM path. They're not LLM pill. They are focusing on areas and methods of learning that have maybe not a higher chance of succeeding, but just are potentially a completely different paradigm. Not along the traditional one that we're used to with these static LLMs. Now, if you think maybe you've heard continual learning before, we have seen an interest in the past year or so where companies are starting to look at continual learning. We have to remember it was Google who did the nested learning thing. I think it was around 3 to four months ago where they introduced nested learning and essentially the idea behind this paper was where instead of seeing a neural network as one single thing you basically just think of it as many smaller learning bubbles inside each other and each bubble learns a part of a task and then updates itself at its own speed. So some parts learn quick reactions, some parts learn long-term memories and they all work together and they don't overwrite each other's knowledge which is you know the issue we have with today's current models. Now of course do remember that this while yes it's promising research it isn't like finalized. They haven't solved the entire thing. Demos has said that continual learning is something that is pretty difficult to do. So it's important to note that continual learning is something that is quite difficult to achieve. Now, of course, we do have the fact that this article continues to go on. I mean, they talk about the fact that today's most popular model development techniques seem to be unlikely to developed advanced AI that can achieve major breakthroughs in biology and medicine whilst also managing to achieve silly mistakes. So what they're talking about, right, and I think this is the fact that like most people in the industry, and I don't know if this is most people in the industry. I'm just saying this, but I'll show you guys why I believe that most people in the industry are starting to believe this is that today's AI techniques are unlikely to achieve those breakthroughs because it's mainly fundamentally generative AI. Doesn't mean that generative AI is completely adapt. Of course, not it can still do mainly, you know, those math breakthroughs and crazy stuff. But the point I'm trying to make is that like today's development techniques have severe limitations that if we want to overcome them, we're going to have to think about the problem potentially in a completely different way. Now of course you know mistakes and being the part of generative AI that isn't really useful in certain fields while it might be useful in some. So this article from the information it also talks about the fact that you know at new IPS which is the AI conference where they discuss the new AI research and all those breakthroughs. You can see someone who heads an AI research unit at Amazon said that I can guarantee that the way we're training models today will not last. That's what David Luan said. Um and I think it's a really interesting statement and apparently this was another article from the information. And they said that other researchers who attended the no IPS in San Diego last week also voiced that sentiment arguing that humanlike AI often referred to as artificial general intelligence may require a new development technique. So of course AGI is the end goal that we're all trying to achieve and it seems like trying to get there may not be through LLMs and I mean some people Yakan Land have said LM will not lead to AI. They've received backlash. He's been standing on that podium saying it by himself. But it seems now that Yan Lean is starting to of course get a bit more support from his wide peers because as more people do more and more research I think maybe certain roadblocks are starting to appear. Now I think you have to understand how today's models are trained if you want to understand why people are looking at different methods. So the problem is that like the way how we train today's models is it's not really you know that effective at what human intelligence really is. So think about it when you tell an LLM okay something new it understands it temporarily and only uses that for the conversation and then it just you know completely forgets it like it's like you

had a student who was doing brilliantly during an exam but isn't allowed to write anything down or remember it for the next time. Remember these models are static. Okay? And static is the killer because you got during training you've got the data then the model then you've got the output and then after training you've got the input and then the model and then the output. There's no learning happening. There's no feedback loop. There's no updating the brain. There's no forming new skills. There's no long-term memory. There's no improvement from experience. So think about it like this. Compared to us, if we, you know, learn chess and we lose, we adjust, we improve. But if you teach LLM a chess strategy, it performs well only because it already knew the patterns. It can't improve. Okay, these systems are static and that is the biggest problem. Now, the article talks about the fact that they're looking beyond the transformer. So, you can see here it says TOR wants to develop models that require less data and fewer servers to train. The person said it would do this by coming up with new model architectures beyond the transformers, which underlies today's most popular models according to the materials. And Trekk also wants to merge the different steps of the model training into a single process. Now this is super interesting because going beyond the transformer I think is going to I don't want to say it's going to be difficult but I mean it's pretty hard to go be to go develop something new but of course that's why he has the funding and I'm not saying it can't be done. I definitely think it can be done but it will be interesting to see what kind of methods and what kind of you know new architectures they try to achieve. And of course, I think the key here is that, you know, they're trying to, you know, get methods that are more sample efficient because I think if you can get something that's quite like humans in terms of how it's able to sample the data effectively where you don't need 10 million examples to learn something, that's going to be really, really effective. Now, of course, you can see that he says that they're going to be using, you know, large neural networks, the brain-like math that underlies most front AI models, but the company is going to rethink how some of those models are developed up to and including gradient descent, which is the standard for training neural networks. Now, gradient descent is essentially that step-by-step adjustment method that you use to be able to get the AI that we have now. But he is thinking that maybe this might not be the effective way to reach humanlike intelligence. So it's going to be pretty ambitious to you know not use the industry standard because that's like rethinking the entire you know way of doing things which is of course how you actually achieve breakthroughs because if everyone's thinking the same way then of course as they say nobody is thinking at all. Now of course I would say that this is incredibly ambitious. I mean I'm not trying to be on the AI hype train but if you read some of the things going on here it is clearly some incredibly ambitious stuff. You can see here that it says after developing the model which TORIC aims to need 100 times less data than the current state-of-the-art models, the company will develop an AI agent to automate the development of the company's products, which is insane. Okay. And take a look at what they envision the company's first products will work on. And they say industrial automation and eventually building self-replicating factories and potentially building biio machines to automatically create custom designs or even terraform planets. The material said I'm not sure if we're going too far with AI. I mean guys, it is possible to develop AI that you know does transform and terraform planets because I mean intelligence has no limits when you really think about it. But saying it out loud just sounds super sci-fi. self-replicating factories. I mean, that's just incredibly ambitious. And terraforming planets is something that is so far out of our current, you know, capabilities. I mean, it's it's just wild. Okay. So, I mean, this is clearly clearly an ambitious company. So, I'm guessing that's why they're raising so much money and their goals are clearly stated. And I mean, you know, I think it's going to be super interesting to see how this company performs. And apparently, you know, the company, what they're going to call it is they're going to say that they're working on the model named Sess, which is the name of a dwarf planet and a Roman goddess with a single algorithm. So, it's going to be super interesting to see what's going on. I mean, you know, if you guys want my opinion on this, I would say that like I'm skeptical, but this is the right kind of moonshot that you'll take if you're going to burn investor, you know, capital because when you think about the methods that they're trying to, you know, overcome, when you think about gradient descent, it just works too well. Well, it's not like it's failing. We did go from GBT3 to GBT4 to 01 and the current paradigm is still doing well. Like there's still, you know, breakthroughs. We're saying Opus 4. 5 achieve new heights. GBT 5. 2 doing math, solving maths. And the thing is as well is that like smart people have already tried like gradient descent has been around since the 1960s and people have tried to replace it. Tons of brilliant researchers have. There's a reason that we're still using this tool today. And I mean, it might work though. Transformers do have limitations. They can't learn on the fly. They are data hungry as hell. They make silly mistakes and when you think about how people win in industries, sometimes outsiders usually win. The transformer itself came from a let's try something completely different approach and attention mechanism. You know, it wasn't the obvious next step either. So, think about it. If he does manage to pull this off, that does change the entire economics of AI. Right now, you need

billions of dollars just to, you know, train a frontier model. So, it's going to be super interesting to see if this actually works.