GPT-5 Is SURPRISINGLY Good at Coding (Real-World Test)

So I'll be comparing GPT-5 against Anthropic's  Claude models when it comes to coding in   real-world production codebases. I find that  whilst many models do well on these one-off   applications, like this French Adventure  game that they made in the livestream,   that performance doesn't necessarily carry over to  coding in a real production codebase. The codebase   that I'll be editing is this application that I  made called Tensor AI, and you can download it on   the App Store using the link in the description  down below. And it basically helps stay up to   date with the latest AI news, so I have it on my  iPhone over here. And you can see all the AI news   that happened today, including GPT-5 dropping  and some other AI news. And I can listen to   a summary of this as well. And if you want to  learn how to make applications just like this,   then you can join my AI startup school, where I  teach you how to make applications from start to   finish, and get users and distribute them, and  also get sales and so forth. The change that

I'll be making is adding Japanese support to the  application, because someone left a review saying,   can you add Japanese support? And I'm going to  make it so that every article, as it's generated,   is also written in Japanese. And you can also  listen to the AI news in Japanese as well. And   then all the settings and so forth are written in  Japanese too. So I'll give this request to GPT-5   thinking, and also Claude 4 Sonnet thinking. I'll  be using Cursor in this case, and just changing   the underlying model to make sure it's a fair  comparison. So I'll use Super Whisper and say,   hey, basically I want to add Japanese support for  the EXP application. I want to make a translation   keys file that has all the translations,  everything on the application, in Japanese.    I also want the articles that are generated by  the AI when it comes to generating the AI news   in Japanese as well. I also want the daily audio  summary in Japanese too. You have to make the   database migrations for this. And I also want  a setting which automatically matches a user's   language, so they can choose device language, or  they can override the device language, and choose   either English or Japanese. Basically, download  the Revlon packages, add the support for this,   add the database migrations, and just anything  else that is required. And I'll press stop,   and then just copy over this prompt over here as  well. And this is running in two different folders   with the exact same codebase. So this is running  in a GPT folder, and press enter. This is running   in a Claude 4 Sonnet folder, and press enter. So  I'll just be speeding up this part of the video,   and then showing you what results it gives. So  it seems that Claude 4 Sonnet is already done,   so we'll wait until GPT-5 is done, compare the  solutions, and see which one performs better.

Okay, so GPT-5 is done, and it took about double  the length that Claude 4 Sonnet did. And it seems   to have changed more files as well. So firstly,  we'll apply the migrations, and then see how it   performs in the applications in both cases, and  then we'll compare the code. So I'll just write   npm install to install any missing packages, and  then I'll apply the migrations that just happened.    Finally, I'll run npm run start, and then see how  it looks inside the application itself. And yeah,   it seems to have run into a problem with a  missing semicolon somewhere. So I guess I'll   just quickly copy this, and then give it back to  Cursor, and we'll see how that performs. And I'll   try again on the application. And I think in the  settings, it should have added a language feature.    So I added language over here, device language,  English, Japanese. And it seems to have added some   Japanese to the application. So it added the theme  and the language itself, but it hasn't changed   the translations over here. It did change the  translations for theme, I think. No, it didn't add   them here. And for the rest of the application,  it just did not add translations throughout the   application, unfortunately. It is trying to load  the Japanese version of the AI News, and I think   it's failing to load that, so it says no articles  found, which is good. But it's disappointing it   didn't add the translations itself. So even when  I checked the translation keys file over here,   it only added a tiny selection of the  translations that were required for the   application. So just looking for the database  migration that it made, it looks pretty good,   but we're going to check if it works by actually  generating articles in different languages. So now   I have the aggregator running, which is fetching  all the Twitter accounts and all the relevant   news. So we'll see what happens here. So it seems  it has generated some articles about the news. So   we'll see what it's like. And it seems that there  are some English ones and some Japanese ones. If   we reload the mobile application, then we should  see some Japanese articles over here. So it's   loading the Japanese ones, but initially it loaded  the English ones. And these are all the Japanese   ones over here. This looks pretty good. But it  seems to be struggling with actually combining   the audio. And basically, it seems to have made a  mistake over here where it doesn't have a written   summary anymore. But yeah, that should be a pretty  easy fix. But yeah, reading through the code,   that's a really easy fix right over here. So  I can press keep and then save that. So yeah,   after making that tiny fix, it then generated the  audio properly. It also generated it in Japanese.    And then it also sent push notifications for every  article that was generated. So I can see on the   application. And if I press refresh over here and  then go to like the audio. And yeah, it just made   a Japanese version over here, which is like really  cool. I think the only things that it missed is it   didn't add the translation keys for everything  else on the application, such as like update   interests on the onboarding and so forth. But that  can be fixed with another prompt. And it broke the   loading thing over here. So it's always stuck on  loading. And finally, when it came to sending push   notifications, I checked through the code. It  doesn't actually check if the user's language   preference matches the like push notification  it's sending. So it should only send Japanese push   notifications to Japanese users. And it should  only send like English push notifications to   English users. So it did not consider that at all.   There's like no indication that it was trying to   do that at all. But that can be fixed in another  like prompt. But overall, I'm like very impressed   with how well it's done. I know it's done better  than other OpenAI models before. Because there's   no way that Opus 3 or like GPT 4. 1 would have been  able to do this via Cursor. Anyways, we're going

to check through the solution that Claude Sonnet  gave us. So I just started running the Sonnet   version of the application. So if I go to settings  again, I can see it says language. And it added an   interface language and also a content language. I  found it quite interesting it made a distinction   between the two. So let's go with Japanese  on both of them. And I think that's actually   very impressive. So like I did not think of doing  that. But I actually would have preferred that as   a solution. And changing the language, it changed  a few things over here. It didn't change themes,   manage notifications. It didn't add translations  to these things. It added translations. And yeah,   it didn't add translations elsewhere either. As  for the database, I tried to reset the database   to apply the migrations again. And it seems it ran  into an issue over here. So one of the migrations   or like one of the tables it's referencing does  not seem to exist. So let's quickly fix that.    So now I'll run the aggregator one more time and  press invoke over here. And then see what it comes   up with. So it's fetching the Twitter accounts.   It's like going through the articles and so forth.    So looking through the Claude 4 Sonnet version,  it did not generate any Japanese articles. So   I looked through the code base to find out why.   And basically it checks if there are any Japanese   users like on the application. And if there  aren't any users with that language preference,   then it just does not generate an application. And  what it does is it translates the English articles   into Japanese instead of just generating brand  new ones based on the content, which the OpenAI   one was doing instead. So I actually prefer the  OpenAI solution in this case rather than just   translating the articles. Because it would have  more context to write a better article that's   more natural for that language. It also makes a  Japanese system prompt as well, which is pretty   good, I guess. It didn't have to do that. It could  have just said like write in Japanese instead. So   now after updating my language preference again,  it does actually generate in Japanese. But it   only generates the content in Japanese and not the  subtitle or title, which seems slightly annoying.    So I think that solution is worse off. We'll see  if it generates the audio as well. And looking   at the database, it seems it did not add like a  language column to the articles database. So it's   how is it meant to know which article is Japanese  and which article is English. But it didn't check   the article language when it came to sending push  notifications like GPT-5 did. So it made the same   mistake or like it just did not implement the same  thing. So if I go back and then I like refresh   over here, I can see there are two articles. And  one of them is in Japanese and the other one is   in English. So honestly, the title should have  been in Japanese as well. It should have only   fetched the Japanese one, which it didn't. But the  audio summary for today is actually in Japanese,   which is good. So yeah, I actually think that  I prefer GPT-5's solution because it added the   relevant database columns, which is the language  column over here. Even though OpenAI's, I mean   Claude 4 Sonnet's, added a language column to the  audio summary, it did not add it to the articles.    And I don't like the way that it was generating  the articles where it just translated the article   to Japanese. Instead of having it generated both  in Japanese and in English, which is possible with   a single prompt. But yeah, honestly, I feel like  GPT-5 at first glance wins on this one. I'm sure

that Claude 4 Sonnet would have done better if  I used it via Claude Code or something instead.    Or it would have done better if I used Opus  instead. But Opus is very expensive. And also,   I think that GPT-5 is actually cheaper than Claude  4 Sonnet. So honestly, I feel like I might just be   using GPT-5 going forwards and seeing how that  performs. I'll still be using a mix of both so   I get a feel for like how they perform in the  long term. And make an update video about it.    So if you do want to stay tuned for that, then  do subscribe to the channel. And I guess for now,   I'll be finishing off the update using the  version that GPT-5 generated. And then I'll   update the app screenshots and description  and stuff to include Japanese as well.