Karpathy's Autoresearch On My AI Polymarket Trading Bot

Yeah, you can see this resolved as a win. And now we're just going to update the balance. Could we get up to 152 now? Yeah, we bounced up to 152. So, we made $2 so far. That's pretty good. Okay, so a few days ago, I saw Andre Party post this auto research tweet about his project, and I thought this was really cool. So, I wanted to kind of try my own version of this. he is kind of training this small nano uh GPT model on this but I wanted to see if I can kind of transfer this auto research project into like a another uh idea and the idea I had is to do this on my poly market bot. So I have created this bot that's tried to find arbitrage on kind of bitcoin up and down and it hasn't been really easy. it sometimes it finds some arbitrage here but uh I wanted to see if this training or this author research project could improve the model right so this is what I'm aiming to do so I've been running this for a while now but uh let me just explain a bit before we try it out how kind of this is set up in this loop aentic loop system okay so here you can kind of see my version of what I call the trading o research loop so basically it's just like uh extension of carpati project right so if you didn't know uh we use kind of github to kind of evolve our strategy so the idea is that the agent works inside the repo and it kind of follows the instructions in a markdown file we have called or I have called training programm so this is kind of the research playbook where we kind of defines how experiments are chosen how they are run evaluated and either if we're going to keep or discard the result, right? And we're going to take a look at my version of this later. So, uh by using these rules, the agent kind of updates the strategy code and creates new candidate experiments and we will run those on kind of our poly market bot. So, I'm going to show you a bit more about how the polyarket bot works later, but basically it's the five minute up and down Bitcoin market. And once our run is complete, the system evaluates the results. If the experiment is weak, it gets dropped. If it improves the strategy by some evaluation metric we have, it gets kept right. And if the results look usually strong, I kind of implemented this confirmation step just to try it one more time because kind of the poly market data is very noisy. Uh so yeah, I will tell you a bit more about that later. So then we kind of add this results back into the history which gives the agent more context for the next experiment and so on. Right? So basically GitHub provides kind of the part to evolve the code and the research memory. Uh the training program defines kind of the experiment logic and how uh we should do the experiments and the poly market bots just gives us kind of the live environment where the strategy is tested. So for testing I only use kind of dry mode so we don't spend any money but I plan today on this video to run like an hour or something on the poly market bot and see how it goes. So yeah, uh let's just take a look at a few things in the code before I kind of show you the setup. Okay, so here you can kind of see this is the dashboard I built for this. And you can see we are on a run now. The up time is we've been running for 37 minutes. So each run for me is 1 hour uh because we need some of that on this. We need a bit of time to actually test it out. And here are some stats. We have passed eight windows. This is times five. So 40 minutes, right? And we have done four trades and fill rate is only 50%. So that's not very good. And you can see we get a score and we have a win rate. Win rate is always 100% because it's arbitrage. So uh if you kind of look at um here is kind of the experiment history. So the I did a mistake in the beginning. So this is not 100%. But this is kind of the idea, right? So we have an experiment history. We have all the commits and then we have the score evaluator. This is kind of confirms is the project is doing better or worse or I guess the bot and you can see if it's doing better we're going to keep it right and right. So I made a few mistakes in the beginning so now it's working a bit better and then we kind of have the description and stuff like that. And we have this graph here that kind of tracks the score history but this is a bit skewed now but it's really fun right? So uh you can see the recent trades we have done. So what we are looking for is to buying uh you can see on poly market we can buy up or down. If you go to live market here you can buy up for 45 cents and you can buy down for something like 45 or 55 that's 100. So we are kind of looking to buy for let's say 49 and 50. So that's leaves us like

we only pay uh 99 but we will get paid 100 outright. So we're going to make 1 cent on that. So that is kind of the arbitrage I'm looking for. So if you go to the code of the project like I said we have this trading program or training program whatever uh how we are actually going to uh include experiments. It's a bit long but basically it's just a markdown file that has a lot of instructions on how we should do the experiments right so we have logic experiments next phase uh hypothesisdriven strategy functions we can try some asymmetry filters uh spread relative to edge filters so I had cloud code and codeex help me set up this uh experiments we can consider and yeah so it's just going to run in loop. That is the idea. So if you kind of look at the code here, you can see we are in dry mode and here are all the parameters we can adjust, right? So this is what we are testing. So it's a super fun experiment and it doesn't cost anything. But I thought for this video we can run it live for like a window and just to see how it goes. But uh before that I want to kind of record a couple of windows so you can kind of see how the experiments work and we can go to cloud code and kind of take a look at what happens when an experiment turns over. So basically we are kind of adding this to GitHub and if we go to runs here you can see we have all the commits we have done right locally. So yeah that is basically the setup. So, I think I'm just going to let this window complete and I'm going to show you kind of what happens when one uh window complete. Recall a couple of windows and then we're going to try it on the live market. Okay. So, you can see now it's just about some 1 minute left of our experiment number 18 I think or something. So since this is kind of set up in cloud code and it's like an automated process now uh this will kind of trigger off by itself. So I have choose to run this in cloud code but you don't have to. There are other ways to work around this. So this was just my idea. So I want to show you what happens now when the time passes up. Okay. So now you can see start experiment 16 uh has timed out after 1 hour. that means we're going to stop it. So we update all the uh the stats or like the trading parts and then we could do like an evaluate. We get the score and you can see uh 03 uh 07 below best kept. So frequency with crow word and we have some information here and we're going to discard this strategy. Okay. And you can see we have some interesting learnings both as symmetry experiments produce best nest blah blah blah. And now we're going to try a different logic direction spread relative to edge filter. So you can see we go into our model we update the code and now we kind of commit that to our next experiment. Okay. And then we just start again. So all of this is autonomous. And now we have started experiment 17. This filter asks is the edge real uh re uh related to how wide the books are. So we're just going to check that and this is going to run for 1 hour, right? And you can see we kind of updated our score here. So this is the ID. So I thought I can record a bit of this uh so you can kind of see the bot updates here. So yeah, that is basically what happens every hour of the experiment. N2. Okay, so that uh experiment was finished, right? You can see we cleared that up. And if you go to cloud code now, you can see uh research paused. And this uh experiment 18 was a discard. It was pretty good, but it was a discard. So, the fill rate was pretty low and stuff like that. So, now I set this on pause because I thought um I want to try this running this live. Now, of course, this is not going to be good. I'm probably going to lose money on this, but just for the video sake, I wanted to try it. So, I'm just going to say uh switch code to the best uh setup so far. So I'm going to switch the model to the best setup. I think it was something like this was the current best setup we have tested so far. So now we're just

going to write the history to the test we did. All set ready for live whenever you are. Okay. So this is kind of my live dashboard now. You can see our balance uh on the wallet 150 around that. And you can see it's stale now. So I'm just going to ask codeex here to start the bot. Uh, I'm going to ask about the package first. Set uh set package price to $5. Okay, that was done. So, now we're just going to start this and hopefully we can make some arbitrage trades. Okay, so for some reason I didn't actually even catch this. So, it looks like we actually made a successful trade here. So, you can see uh we started at 150. So, now we are at 146. And you can see uh we entered at uh so the edge is uh 00010 right. So hopefully we can get some uh so on five we only going to earn like 5 cents per trade now but uh of course you can scale that. So let's just wait until this resolve and if I see another trade I'm going to update you. So you can see now uh this is what is left of the window. So there are like 10 seconds left of the window and hopefully this will auto resolve and kind of restore our balance with hopefully what we earned on this run, right? So we kind of get our money back so we can keep trading so we don't get locked up. So you can see we are kind of now the window is complete. Hopefully we solve this one and this goes to maybe like win or something. Yeah, you can see this turned into a win, right? And you can see the P& L is here. So, let's see if we update the balance. Yeah. So, we kind of earned $1 on this trade. I'm not exactly sure what happened, but you can see it's still working. Uh, it might be that we got a better price or something. I don't It's kind of hard to follow, but let's see if we put up another trade now. Uh, but you can see our session PLA now is plus 5 cents. So this is an arbitrage of course. So we can't lose this since we are buying both sides. But uh yeah, let's see if we can make another trade here. Okay, so it looks like we got also this one. So this is looking pretty good. So we got uh another uh 99 by uh hopefully we can resolve this trade we did. And our balance now is 141 because we made a second trade. Uh this is a bit more interesting because it's uh kind of landed at the 97 mark. This means that we can make 15 cents per on this trade. Right. So I'm just going to wait and see if uh we resolve this one. Uh I did actually restart the bot to try to do two trades per window instead of just one just for the video sake. So let's just see what happens if we actually get this resolved. Uh it should have been by now but uh yeah I'll just wait it out here. So the window is over now. So let's see if we can resolve this 97 trade here and let's see if we can actually go up to should be 25 if this resolves. Okay. So we are back again. Now you can see we did resolve this uh 97. So that means we got uh 15. So we have now 25. We have a quarter in profits. Uh maybe some fees. We'll see. But uh 142 I'm still kind of missing. And we did enter a new trade with 99. And we are still missing the money from the second trade here. I might try to redeem that just to update the balance. So I'm just going to say to um Codeex here to go redeem this trade just to try to get the money back. So we kind of always top up the balance, right? Yeah, you can see we got it. So that means that we are actually uh on par now. So let's just wait for the this trade to resolve. And I think we're going to do five and that should be kind of it. If we get five out of five, I'm pretty happy. Let's see if we can do one more. So if we can get like a 100% win rate, that would be pretty cool. Yeah, you can see this resolved as a win. And now we're just going to update the balance. Could we get up to 152 now? This means that we made a couple of bucks in this period. And we also have we can do one more trade. Yeah, we bounced up to 152. So, we made $2 so far. That's pretty good. Uh not quite sure how. Okay, so we are entering a second a new trade again. We're going to do a 99. Let's see. The balance should drop a bit more. Yeah, 147. And yeah, let's just wait this out. Okay, so there is uh yeah, 5 seconds left of this window. I just want to see that we actually get this one too. And our balance should be

back up at 152 here. So yeah, I think I'm just going to call it after this one. If we do five in five, we can't really do much better than that. So let's just see if this resolves. Yeah, that's a win. And let's see if we go to 152ish here. Yeah. So we actually I'm just going to stop this now. Okay, so the takeaway is that we made $2 in like yeah about 10 minutes. So that's pretty good. Or like 20 minutes. So yeah, five out of five trades. Everything was pretty good and all was arbitrage. So super happy how this turned out. But I'm going to keep actually running the experiments just to see if we can prove it anymore. I think so. Uh, I would definitely recommend go checking out his uh, Carpatis auto um, research project. You can find it here on GitHub. But I thought it was pretty cool to actually try to adapt it to something I want to try. So, I did the arbitrage bot on Bitcoin, right? So, on Poly Market, but there's a lot of other stuff you can try it out too, and I'm definitely going to do that. So, if you enjoyed this kind of content, uh I might do a new test uh with this soon on a different uh genre or topic. So, give this video a like and subscribe if you want to see more. So, yeah, thank you for tuning in. Hope you enjoyed it. Hope this gave you some inspiration of things you want to do. So yeah, see you