# Is Bug Bounty Dead? (Ep. 173)

## Метаданные

- **Канал:** Critical Thinking - Bug Bounty Podcast
- **YouTube:** https://www.youtube.com/watch?v=ZQJ2uTJWYlk
- **Дата:** 07.05.2026
- **Длительность:** 1:01:32
- **Просмотры:** 5,190

## Описание

Episode 173: In this episode of Critical Thinking - Bug Bounty Podcast we’re talking about the negative effects that AI is having on the Bug Bounty scene as a whole. Is it over, or are we so back?

Follow us on twitter at: https://x.com/ctbbpodcast
Got any ideas and suggestions? Feel free to send us any feedback here: info@criticalthinkingpodcast.io
Shoutout to YTCracker for the awesome intro music!

====== Links ======
Follow your hosts Rhynorater, rez0 and gr3pme on X: 
https://x.com/Rhynorater
https://x.com/rez0__
https://x.com/gr3pme

Critical Research Lab:
https://lab.ctbb.show/ 

====== Ways to Support CTBBPodcast ======
Hop on the CTBB Discord at https://ctbb.show/discord!

We also do Discord subs at $25, $10, and $5 - premium subscribers get access to private masterclasses, exploits, tools, scripts, un-redacted bug reports, etc.

You can also find some hacker swag at https://ctbb.show/merch!

Today's Sponsor: Check out Zero Trust Cloud Access:
https://www.criticalthinkingpodcast.io/tl-ztca

====== Resources ======
We want your feedback on this!
https://forms.ctbb.show/future_of_bug_bounty

Evolving the Android & Chrome VRPs for the AI Era
https://bughunters.google.com/blog/evolving-the-android-chrome-vrps-for-the-ai-era

Paid Submissions?
https://x.com/d0rsky/status/2047744193976742120

Keep the Robots Out of the Gym
https://danielmiessler.com/blog/keep-the-robots-out-of-the-gym

Is my data used for model training?
https://privacy.claude.com/en/articles/10023580-is-my-data-used-for-model-training

====== Timestamps ======
(00:00:00) Introduction
(00:06:28) Network effects of Bug Bounty
(00:31:55) Hopium/Copium
(00:47:21) The Great Training Data Debate

## Содержание

### [0:00](https://www.youtube.com/watch?v=ZQJ2uTJWYlk) Introduction

the model providers. So this is OpenAI Anthropic and probably Google eventually are kind of coming for everything, right? There's this uh notion that if you build a company that's just a wrap around one of these models that the model providers are going to eventually come and like eat your company, right? They're going to come and like take over. — Best part of Viking when you can just, you know, critical think, right? Yeah, dude. — Hey, what's up guys? Before we get into the show, I wanted to mention something super quick from our friends at Threat Locker. And I actually think you are going to think it's pretty awesome because so much of Bug Bounty is often, you know, kind of quoted as like, "Yeah, but hackers will never exploit that because they can just get in via fishing. " Well, that's actually true. You know, most of the time whenever companies get breached, it's because of fishing or access to that user's account or, you know, they do something like do a whole bunch of push notification to FA and eventually a user gets so much fatigue they approve it. But they have a solution for this. Threadlocker has a thing called zero trust cloud access, right? Which prevents access to cloud resources or SAS resources based on the device you're logging in from. So if a user gets fished, right, they put in their credential, like they get fished or they get fished, they the attacker has the credentials, maybe they even have a way to get the MFA because they did some sort of SIM swap because they have a hookup at Verizon or AT& T or whatever, right? So they have the credentials, they have the MFA, they still can't get in because the zero trust cloud access like will basically straight up allow or deny people access to resources based on the device you're logging in from. So, if you're an enterprise or a company and you're concerned about the highest risk, which really is fishing, this is a way to add another like basically impenetrable layer to preventing it and securing your network. Um, yeah, back to the show. Hey, what's up guys? Before we jump in the episode, I want to mention one more thing. All of the discussion over this past weekend uh made me think it might be useful to add a little bit of clarification just because it feels like there's so much nuance being lost in the world around the debate of AI and its impact on bug bounty. So in this episode we talk about a lot of effects on bug bounty that AI is having and that it will continue to have kind of going forward. And I would say it's like 60 or 70% kind of like on the negative side of just like things that could impact the industry or us as hunters negatively. And we're here to bring you the truth. Like I'm going to do that. Like that's what we're here for. But I did want to add a little bit of nuance. When I say or when anyone on the podcast is talking about like things being over or Bug Bounty's dead, it really is uh hyperbole to bring emphasis on the problem, right? or on the potential problems that are coming. In my opinion, I think this is going to be the biggest year ever for Bug Bounty. And I even mentioned this on my blog. Um, but that's not to say that there are a lot of things kind of going downhill, which we'll mention in the episode. And so, I wanted to add this here as a little bit of context so that no one thinks I'm like, you know, quitting Bug Bounty or that I think anyone should like give up on it or that the industry is over. You know, I think those types of hyperboles are really to shout out and show um to bring awareness towards the types of things that are happening which are going to change our industry tremendously and it has the potential to go bad. And so I do think that we wanted to highlight that and get your all's opinion on how it could be done better which we'll talk about here in just a second. — Dude, what's up? — How's it going man? It's been a while since we last done an episode. — That's true. We got to cover for Justin and some medical stuff. — Yeah, absolutely. But I think it's been overdue. I'm looking forward to it. — Yeah, dude. We've got a pretty uh controversial episode. Uh I do want to say off the bat cuz there are people with uh feelings on both sides of the fence that we're going to be discussing the thoughts that either Bug Bounty is dead or we're just getting started. But uh even if that's not your cup of tea, stick around. We've got um a request for feedback basically and I'm gonna go ahead and tell it up front but uh it's forms. ctb. show futureofbugbounty and that's underscoreepparated. We really really I think need feedback and I think we're going to pass that feedback along to the platforms because I think there is a lot of different network effects on the bug bounty industry and I personally am a little bit worried about it. My wife's personally worried about it too. Um but yeah so — yeah no we we'll get into the weeds of it but it was a discussion which started on the CTS to address some of the problems we've been having and then since that discussion happened it just seems that there was like a 3-day block of just stuff dropping the entirety of this week. So, we thought it would be good to capture other people's thoughts and opinions as well outside the CTS and see what everyone thought. — Yeah. When he says CTers, there's basically a tier in the Discord for critical thinkers that we abbreviate to CT a lot of times. Um, and so that's people who have, you know, subscribed and there's a lot of really good discussions that happen there if you want to hop in. But, um, yeah, this past week has been, I feel like, a really big newssheavy week that plays a lot into this discussion. Um, did you have any other like kind of off-topic stuff or just like random news items or things that are going on in your life that you wanted to mention? — No. Um, I think — How's hunting going? — Yeah. Um, yeah, I've been spending more time on tooling. Um, a lot I'd done a three live event stint back toback. One of them over overlapped and I was like, I was fried. I spoke to you about it, but I was just like completely dead from that. It was very intense. And throughout that, I kept going, "Okay, I'll improve some tooling. I'll update this skill or I I'll make that change. " And my list just stacked up bigger than me. And now I'm slowly working through the list of backlog. And obviously, I've got company stuff, accounting, so I'm very much catching up, but I'm just coming out of the end and now doing some tooling before I jump into my next event next week. excited

### [6:28](https://www.youtube.com/watch?v=ZQJ2uTJWYlk&t=388s) Network effects of Bug Bounty

— dude. Insane. — Are you in a Are you in an event right now? — Yeah, the Lisbon event. I'm doing it remotely. Um, — nice. — It's been fine. You know, plenty of findings. I think that a lot of the discussion we're going to have, I think, play plays into it. So, actually, we can use that as a good segue. I I think they had a shorter dupe window, which I think is really smart. I think events these days need a shorter dupe window with everyone using hacking agents. I think that people are finding bugs much faster, especially that kind of I hate calling it low hanging fruit because it's more like mediumhanging fruit, right? I kind of use that phrase. And so, but that medium hanging fruit really is getting kind of like slurped up by everybody's models all at the same time. And so, I think there's like some massive dupe splits. Um, and anyways, yeah, so I think that, you know, that's kind of one of the network effects at play. But um the thing that has really triggered this discussion more than anything else were kind of two news items. We can talk about them independently before we get into the discussion about whether you know Bug Bunny's dying, but I think the first one is Google's bounty range. Did you see that? — Yeah. Um I saw it drop and the way it dropped with all the discussions we've been having to see tiers, I was just like I didn't need to see this right now. But let's get into it because the Google VRP essentially dropped their bounties, right? Um or changed some bounty ranges, which also — Yeah, I do think that it's kind of a simplification to say they dropped them. I saw a lot of people basically implying it was all bad news. does kind of seem like at least for the like really impactful bugs, which, you know, there's only a handful of them and only a few people in the world that could even find them, those bounty ranges went up, you know, but then the other ones kind of dropped like I don't I think did they say they're not paying for lows and mediums or something along those lines. — Uh yeah, it looks like it on certain scope in the VRP. Not the entire scope, just to be clear, but certain aspects in their bounty program. Android and Chrome, right, are the main ones. — Android and Chrome. Should we share this actually for people watching? — Sure. They basically have a blog post. I'll let Brandon share it, but it's called Evolving the Android and Chrome BRPs. And I mean, you can't really blame them, right? I think they're getting spammed with tons of reports. — Mhm. Yeah. And if you speak to program managers or speak to anyone's other platforms, it's very much a massive pain point right now. Some of the stats speak for themselves. And I think if you've got any open reports, you'll definitely feel this thing because man, stuff is just taking forever. Like the old life cycle of like, okay, — I probably know this is going to be like a 4 week payout maybe is just completely gone out the window for the time being. Um, whilst programs are dealing with a massive influx of reports. Um, and learning how to deal with that because I think it's only going to increase now as time goes on. Um, so it kind of makes sense. It kind of falls into that. But Google, the good, okay, let's just start off on this. The good news is there are reward increases for more impactful bugs. So that isn't good. Like up to 1. 5 mil for a zero click full chain on Pixel Titan M2 with persistence. So, it's not all bad, but there is obviously downstream effects of modifying your bounty tables with some of the quantity you're going to get in and some of the quality as well. — Yeah, definitely. Um, and then yeah, so I think that uh there's also another uh screenshot that I saw going around. It's not in this blog post, but that the um some of the vulnerabilities that you know many people would even consider mediums and highs across these um brps pay way less. Like I saw one that it's like a bug that would have paid thousands of dollars before now pays like 500 bucks. Um I don't know if you saw that one going around. — No. Have you got that? — Yeah. Let me see if I can find it really quickly. — Yeah, no worries. And yeah, here we can see some of the key changes to the VRP. They're incentivizing uh actionable reports. Um AI has obviously made report writing easier for I would say all of us in some respects. Um but it also means that they're expecting higher quality reports and more actionable reports, which I think is fair enough. Um they're phasing out some of their bonuses. Um, and basically just having a bit of a revamp of the program. So, if you hack Google or are thinking of hacking Google on some of the Android and devices, um, definitely check out this blog post before you get into it and update yourself because it is fundamentally changing the ecosystem. — Yeah, I can't find it. It's fine. If we find it, we'll wind up throwing the show notes. Um, you know, this plays into again the whole meta discussion that was really kicked off, I think, also by Anthropic rolling out their security review to more users. So, Anthropic had like a security review thing in the past. And I think people have known that Anthropic was kind of continuing to care more about the cyber security abilities of their models because they just like made such a leaprog right back when 46 dropped. And I think that this uh you know really along with everything else that we've been discussing kind of it doesn't scare me but I think it's just like we're definitely moving from a previous phase of Bug Bounty into like a completely new era. And you know we're going to go ahead and mention all of like the quote unquote bad things or all of the reasons why it's over before the reasons why we're so back. Um because I do think a lot of people were upset with my tweet. So, I quote tweeted anthropic tweet and basically said it's over when they released the security thing and a lot of people were like it's not over blah blah blah. You know, there's like lots of trolls that come out of the woodworks to argue. Um, but I don't know if they have the full picture. So, I want to paint the full picture. I think these are the network effects that um worry me and then we'll talk about the network effects like all of the upside, right? And so we can go through these one at a time and I want your like feedback on each of these and you tell me like how true you think it is. — Okay. — So the first one is — there are so many reports that these platforms and programs can't really keep up right and so this is driving some programs to shut down some programs to stop accepting lows or mediums like Google did there and then all of the programs are delayed in triage times and payment times. Yeah, I mean I don't think you can really argue with that if you're an active hunter and you've been keeping an eye on some of the notifications programs have been bringing out um how regularly you're getting paid and how things are getting triaged. I think that's completely undeniable. And one of the platforms, I can't remember which, uh, dropped a statistic of the like the trend of traffic increases, which I saw in a tweet as well, and it was something like 80% up of like compared from this time last year or something like that. — Yeah. It's insane, right? Wasn't there a thing that said like Hacker One gets like 200 reports a minute or something crazy? — Yeah. I mean I'm not sure if that is like valid reports or like total report count but it's still a crazy number. — Crazy. Okay, so here's the next one. Models. One thing that I hear all the time as like push back is like yeah but like humans are still more skilled. So even if you know these uh these agents find like the lower hanging fruit, we'll definitely always need humans for that like higher quality vulnerabilities, right? In some sense, I do think that's true, but based on the bugs that my hacking agent finds, like many of them are so skilled, I wouldn't have found them. And many of them are exactly like the things I find. And we're like in this situation where, you know, I always bring up Cyberbench because it's like the only full like kind of production level eval, but you know, the Cyberbench score went from like 30 something% to 60 something% with 46 and 47. And do we think that's going to like get worse? It's not right. These models are going to continue to improve. — I don't want to take us on a side path now. I'm wondering if we should get into your whole debate — about what you think around like privacy. — Well, so here just here's the thing on that though. Models will continue to improve I think overall. But — yeah, — to contradict that Opus 47, what on earth? Like I really was not a fan of that. Did you switch over or have you used it? What did you think? — As far as I can tell, I have some indications that it's slightly worse and other indications that it's about the same. It's really hard to tell for me personally. I know a lot of people have disliked it, but — yeah. And it's there's so many nuances here as well because like we also had the debate a couple of weeks back when I was like, dude, is your clawude like severely nerfed? Because mine is just it's making more work for me right now. So I think although actual models at their base will become better and the latest round of the open source models that we had dropped last week definitely proves that. Like if you look at some of the benchmarks from these smaller 27 billion models and 30 whatever billion models insane. They pack a punch for their size. But the taxonomy around it is model providers obviously have to scale traffic properly. If they're doing new R& D or releasing a new model or whatever, they have to ensure that they have enough compute to do that. So as a consumer, we're in this spot where we just randomly get throttled and we're sort of at the mercy of that which I really don't like. So overall, yes, I would say models are getting better, but there's obviously a lot of nuances around the economy of that as well. Yeah. So I think um well like actually let's get into we're going to table that. So we're going to argue after this discussion on whether it's over or we're you know we're still early. Um me and Brandon are going to have a little debate around um model providers and training data and this sort of thing. Uh — Mhm. — because I want to like clearly lay out all these network effects. So I'm also going to summarize. So the first one is that the network's effects on whether bug bounty is over is one. There's so many reports that the programs platforms can't keep up. That's point one, right? And under that is that they're some are closing, some are like reducing what they accept and they're all delayed. The next one is these models keep getting better, right? The other thing I would say is that the model providers, so this is open AI and anthropic and probably Google eventually are kind of coming for everything, right? there's this uh notion that if you build a company that's just a wraparound one of these models that the model providers are going to eventually come and like eat your company, right? They're going to come and like take over. And I think that is um sometimes overblown and other times I think it's like extremely relevant. And in this case, I think it's extremely relevant because security is so important. And you know, in the past, it has been hard, but now, dare I say it's easy. Like I I really think the effort I have to put in to find bugs is definitely far less. Right? I'm not necessarily saying it's easy. I do think that you have to be still be highly skilled to vet false positives from true positives. Actually, let's add that to theopium copium section down here below. Um, still lots of false positives. So, we're going to get down to that in a second, but I think that the model providers are coming for security because one, they need to make sure that their networks infrastructure is secure and there's some level of money in it. It's not actually that big relative to some other industries like shopping or whatever, but uh or advertising, right? Like the security um industry at large is not as big as some others that they could go after. But I do think that it's also important to them and then humanity, right, that we like continue to keep things secure. Um, and so I think that they're coming after that. The next thing is we have so many startups that are basically coming out to do automation pin like pin testing. Like of course there are the expose of the world that have been around as their main gig, but every other company's getting into it. Like Sneak is building one. Whiz is building one. — I'm pretty sure the bug bounty programs are building them. like the bug bounty platforms. — Yeah. And it's I feel like this is a continuation but a buffed continuation of like the cycles we see in security. Like if you go back I don't know maybe 5 years ago it was ASM ASM new ASM platform attack surface management attack surface management. Now the next thing of that is okay agentic pen testing and you actually uh tweeted I think it was this week uh quote tweeting someone's like prices for a pen test and you're like it's a race to the bottom now. — Yeah. — And I feel like that's so true because — it to your earlier point it hasn't ever been easier to do this. It's very powerful now that we can make a tool do some form of test. I'm not going to say if it anyone and say yeah um yeah I'm not going to do that but — it's now easier than before to do that wrap it up in the service and sell it. I think the thing is understanding if it's actually effective and actually good or not but to your point it does kind of feel like a race to the bottom in that sense because it's become so common. Everyone's doing it. Oh, lower prices. Get a full pen test for $100 by our agentic agent. And you're like, okay, sounds great, but like what are the actual results of that? What's how effective is it? — Yeah. And I think like you and no one would probably argue that it's going to be better than your kind of mom and pop pentest shop that does a twoe engagement with one pentester who doesn't keep up with the industry and has no incentivization to work hard because they're mostly going for a compliance check mark, right? Like even if it's a lower quality agentic pin test, I bet it's still going to be better than those old ones, you know, like the the for compliance just for the check mark like sock 2 pin test and so things are still going to get more secure even bad agentic pin tests are still better than like check mark pin tests from the past and so I think things are going to get more secure and I think that's really good for humanity but I do think that all of these layers of like Yeah. So let me go ahead and mention two other points uh so we can like list all the things that are over that like make me feel like it's over. One is internal companies now internal red teams can now run hack bots on their production software before it goes external facing right — and on top of that these agents are like really good at open source code review and so they can run that internally. So they're able to run like white box and blackbox hack bots or security review on production apps before they come out, right? Then they can also get automated pentest from third parties, whether it's from the bug money platforms or whether it's from companies like Expo or whatever. So by the time it gets to us, the amount of bugs that are there is going to be so low. And I'm not and I, you know, I still think that we'll be able to find vulnerabilities. I think that, you know, top bug talent is still the most talented group of individuals for finding vulnerabilities. And now we are all even more leveled up. But my point is that I think that the total industry and the money that went into it and like my earnings and your earnings like it's a mix of low hanging fruit and then like really complex bugs, right? — And a lot of people made their living or at least made um like they you know made way more money because of a lot of that low hanging fruit, right? you get a new private program invite, you go in there, you farm it, you feel great about yourself, — and then, you know, then you work really hard during live hacking events on these hardened targets to find like these deep bugs. And it's a mix of all of that. And I think that like the former is going to be going away, whereas the latter might last a little bit longer. — Yeah. And it feeds into our earlier point of I think the economy of bug bounty as we know it is definitely like dead or slashch changing in terms of — it's going through a transition phase with all these toolings and things like that and having spoke to some program managers and some other programs in the industry pe they are taking their reports or taking report this is programs by the way, I'm not talking about platforms to be specific programs. Some programs are using our reports as expected to develop their own tooling and AI tooling to help detect some of these problems that they keep seeing and keep arising. So that's definitely happening — for some programs I've spoken. — When we submit it, it becomes their intellectual property. So they can use it the programs, not the platforms. Yeah, — exactly. — So it the TLDDR is like the bars raising, right? like we've leveled up, but also it's works both ways. — Yeah. My final point here before I recap all of them for why it's over is that top hackers like me and like you are now much more effective, right? And so yeah, that bodess well for me and you, you know, maybe today, maybe for six months, maybe for a year, but I think long term that fact that the highest skill people can now scale their knowledge and their expertise across many more um scope items. I didn't know whether to say programs or subdomains or whatever, but can now scale their knowledge across many more applications means that there's going to be less for everyone else, right? And so, um, I'll recap now. So, here's my claim for why it's over. There's so many reports the companies are beginning programs are beginning to close or not accept low and mediums or we have super delayed times and triage and payments which um adversely affects full-time hunters like you and I, right? Because it's like how do we know that we're going to be able to meet our bills next month or what have you, right? model providers are offering uh are now offering like coderview based services and I think eventually we'll get to offering hack bots. The models will keep getting better and so that delta between like the human plus AI skill gap and the AI only will continue to remove because it's getting closer and closer to our skill level. Startups are now doing pentest for cheaper and cheaper. So companies can now like, you know, buy a $500 pin test or $1,500 pin test before it goes into their bug bounty scope or they can just, you know, stop doing bug bounty. Internal devs at those companies and internal red teams can use their agents to test both their white box and blackbox reviews or um code. The platforms are eventually going to be doing agentic content test on it before it gets our scope. And then if you get past all that, now it's in scope. You're competing with top hackers like me and like you who are going to be running a bunch of highskll, highly developed AI hack bots at scale all the time. — Mhm. Yeah. I mean when you wrap it up like that, it doesn't sound great, does it? — And I only say this because like I feel like there are some things like one, I think that there is also some or copium that we'll mention in a minute. I also think there are things that we can do to do better. I think there are also things that platforms can do to do better. And I want to talk about all that, but we also want your ideas. I know I mentioned this at the beginning, I want to mention it again. We're gonna have a forum up. That's forums. ctb. show future_of_bug_bounty. And I we would I would love to hear more ideas or thoughts about this like why am I wrong or how can we fix this iterate on this? Um so yeah. — So I guess let's dive a little deeper in there. So I guess the overall problem is that there is an increased amount of like quantity of reports right now in the CTS there were initial ideas and there was actually a tweet from I think it was a CTO from hacking proof which is a web free security like bug bounty platform um and they implemented like a I don't know what you'd call it like a submission fee a pledger I don't know um for each report that you put in. Let me share my screen because it's quite an interesting tweet. Can you see that? — Yes, sir. — Um, cool. So, they're implementing a fee per report that basically says, look, if you want to report, you have to pay this fee, but you'll get it back at some point. Providing a report is valid. So, what's interesting is $1 wasn't enough skin in the game to like properly implement it. people just didn't really care. Sprayed and prayed. $5 — 80% drop in AI slop and low quality submissions at $10 a max. And I thought it was quite an interesting tweet because — programs and platforms have a very real problem that they need to address and that is now filtering the noise and actually getting those valuable findings that some people are producing in front of their triage teams and their internal team so they can fix it. I thought this was a really interesting concept. Um, I don't know how I feel about it is what I would say. I think it's interesting. — I think it's genius. I mean, I think that you once get in the game um because it means that the bug hunters are going to be much more skeptical of what they're about to report and it will save a ton of — of time on the back end. I think this is like maybe something we need to apply at scale. I'm sure you saw the GitHub hack/GitHub uptime issues. Basically, GhostTy, uh, the maker of GhostTy, the really nice terminal, uh, is coming off is like moving all their code off GitHub because of GitHub's downtime and stuff. But if you look at like the number of projects on GitHub, it's like a straight line up because of Vibe Coding. Every everyone's releasing stuff and people who weren't devs before, like their agents are telling them to like get stuff on GitHub. And so it's like it kind of makes sense. They've had a hard time keeping up. I mean, agents can push like — constant commits like uh mine and JD's bug bounty uh hackbot — uh it pushes every finding to GitHub, right? And so it's like and every lead and every note as a separate commit. And so it's like you can imagine why they're struggling to keep up with this volume. And so like what if they charged you — I don't know a dollar a month for a like per repo, — you know? It's like — Yeah. No, it would definitely be effective to combat because it would fundamentally change user behavior, right? If you are just — create, oh, I just create this, push it up, and forget about just in case I need it, you'll probably go, actually, — I just got a USB laying around. I could just do that instead. Um, but some of the points raised in the channel, I can't remember who raised them, was the point around the economy of that, right? Obviously different regions say I don't know for like Netflix subs for example they scale that appropriately — so that would have to come into consideration I think. — Yeah I think they would have to do that. Yeah I mean cuz like five bucks to somebody in India who's getting started in bug bounty they like probably would never want to do that. — Exactly. But it's very interesting. But at five, like I say I'm conflicted because there's two sides to it. But overall, if I had to pay $5 per bug, which meant faster triage, it got handled properly and it reduced the amount of like slop, which meant your overall triage experience would be better. I would take that for sure. — I just don't know how it would scale across the community and things like that. I'd be interested to hear people's thoughts. — Yeah, hack and proof has a really big benefit of being on chain — cuz like on submission people can literally just like add $5 in gas from it like of Ethereum, right? And so it's like super easy for them to manage that or whatever. Whereas it's like — on Hacker One, it's going to be kind of weird like is it going to come out of a future payment? Are you going to like PayPal them or Venmo? It's like it feels like a lot harder to implement, right? — Yeah. And there was a concept as well that someone else mentioned to like put the monetary things aside of having a concept of tokens which essentially acted as money. So say if you get x amount of valid reports that are triaged you get a certain amount of tokens you can report with and then if you get to a certain signal or rep that gets entirely removed and then you can report and things like that. But again, I don't like I'm saying this because there are ideas that hold weight, but I don't know. I can't figure out what the best approach would be to scale across platforms, which would lead to the best outcome for hackers and for the program and for the platform as well. — Yeah. The biggest issue I think with uh anything like that is like kind of the same issue that programs have with like banning people. It's like so are these people who can't afford it or who are banned like what are they supposed to do with the vulnerabilities they find? Just yolo them on the internet, right? Or proxy them through other people or

### [31:55](https://www.youtube.com/watch?v=ZQJ2uTJWYlk&t=1915s) Hopium/Copium

whatever. It's like there are issues there. But let's jump on this uh copium train where we're talking about the opposite instead of it being over. I think here's like the reasons to be optimistic about it. So before we move on, do we cover the data policy usage before we get to that or not? — No, it's more negative stuff. Let's go positive. — Okay. All right. Go. And I feel like it can be, you know, people are still going to like hearken it back and understand like they're going to place I think our debate in a second into context of that first half. But yeah, now let's talk about the good stuff. So I saw who was it? Hackluke. you know, and it's a lot of other people have said it before, but there's just no better time to learn something than today. Um, and JD mentioned that too. I think like, you know, excess doctor wanted to like learn has been like wanting to learn like to go deeper in his like front-end client side attacks and he'll just literally tell Claude code like, "Hey, spin me up a lab where I can exploit this. No, it's not working. Do this right. " And now teach it to me, you know, and like kind of like walk yourself through. I don't know how many people have that much discipline. It feels like probably pretty few. Um cuz you and I, you know, as bug hunters, we always have this pull on our time where it's like we could just be doing more bug hunting or improving our automation or I don't know, spending time that doing something that we actually like love, you know, something leisurely. So I think it is like hard to u make the time to do that. But you know, if I was early in my career, of course, and I was learning, I feel like there is this huge upside to learning with LLMs, right? — Uh 100%. I actually gave a talk at my old school um a couple of months back where I was trying to drill this concept into the kids because I was like look AI can give you the answers but use it as a sparring partner. Don't ask for the answer directly like use it to learn and you'll be much more effective and it'll pay dividends further down the line. Um, and also Daniel Measler drops a blog post a while back which I just love the concept because it captures it so perfectly and it's don't take your robots to the gym. And it's like exactly the same where — be disciplined enough to understand when and when to apply because obviously you go to the gym for the benefits of going to the gym. There's no point of like offloading that to someone else. So yeah, I really argument and I've definitely used it to improve and study a little bit on my side as well back some of the mobile stuff I done towards the end of last year after an event as well. — Yeah, I saw a quote tweet by Andre Kaparthy uh from Yine. Um he says, "You can't outsource or sorry, you can outsource your thinking but you cannot outsource your understanding. " Basically, — if you want to outsource your thinking, but don't outsource your understanding, right? I think this is something that Justin's really, really passionate about that he's saying all the time. It's like, you know, it's fine to automate bits of the hacking thing, but you still need to understand what's happening and how to do it. So, — 100%. — So, yeah. So, I think that we're early in that regard. I mean, um, we humans have never been more empowered to go and do anything in the world. Just before we hopped on the call, I just shared like three side projects I've built with you in the last like two weeks. because of the power of these coding agents, right? And um and I think the second big thing is that the scope is actually big enough across these big anchor programs that no one could actually even test it all. — Like let's imagine our doom scenario, right? a company like Yahoo is trying to test stuff internally and then it gets out and they hire or like before they push it to the new scope they like have the bug bounty platform that they're hosted on do like an agentic red teaming of like you know that asset or that scope or whatever and then it goes live and then you know whatever they hire expo to do it. I think that like the amount of work to test like tens of thousands or hundreds of thousands of subdomains deeply is actually too much for even the combination of the internal red team and the platform agentic part and a third party, right? And there so there's still going to be plenty of vulnerabilities I think that kind of like come out due to the large scale of large companies. — Yeah. And it feeds into the next thing we discussed as well is the fact that you combine that and then you combine a team which has adopted AI for development. You combine those two and the rate that people can push things out, new functionality, new features, it's just like it's insanely quick now. And I've actually spoken to I can't say names but programs which have said like yeah on one side we have got the AI tool and it helps us and things like that but the scale of which we're pushing things out now and the requirement that is built that we can develop so fast um in terms of our development life cycles have shortened that functionality is being made quicker and pushed out quicker before we actually have time to properly vet and fix things. So I would almost definitely say like yeah the scopes there and also to combine with that the functionality being developed on top of that scope is a multiplier as well. — Yeah and I think that rate is going to continue to increase obviously like we stay on top of tech where like we're implementing that and like using coding agents a lot. I think you know when you look at the stats of like the amount of people that have been like leveled up by using coding agents at work especially like these large enterprises I think it's something like 50% are using AI but like a lot of that usage is like one off or like asking about the code not actually implementing it. I think that like the way in which AI has allowed me to you know vibe code at like 10 or 100x the amount of output that I used to be able to put out. I think that when these enterprise companies are then moving at that same rate, like imagine 10,000 software engineers now being like putting out 50 times more code, right? Like just the amount the volume of required testing is then like gigantic, right? It's like unfathomably large. And so I think that really is some hopeium. — Yeah. Cuz like imagine just imagine your scenario right there and then working in the apps team having to deal — try to review that. — Oh man. You'll just be like, "Claude, find all VS and please report back. " — Yeah. All right. And I'm gonna mention this and I'm gonna give a shout out to Joel um the former podcast host of this podcast with Justin help helped found it. technogeeek. He mentioned to me I was, you know, saying it's over, talking about some Hackbot stuff like maybe two months ago or something and I've already mentioned this on the pod once, but and he kind and like he said something like, you know, like what percentage of the bugs that come out of your Hackbot system are valid and I was like, ah, man, it's such a it's such an astute observation, right? because it feels to me like I'm getting a bunch of these free bugs, but I'm like just choosing not to submit a bunch of things that aren't actually real findings, whether they're not critical enough or whether the um the actual like uh data that is exposed is like not is it supposed to be public or there's like a misunderstanding or it's like a gadget and not a full bug. So I think like um when going through this discussion, it may be easier than ever before to find stuff, but the people who are finding stuff might not actually be able to differentiate between real bugs and things that are not impactful. And I think they're very quickly going to be kind of like uh they're going to very quickly maybe give up or be demotivated or feel like that AI can only produce slop because like they literally don't have the skills to validate whether it's true or not. And so then they end up, you know, stopping hunting or stop submitting or getting kicked out of programs because they're submitting so much junk, right? — Yeah. Well, and I think this is exactly what's happening from everything we just spoke about, right? The stats, the amount of submissions now that platforms and programs having to deal with. I think that's very real for sure. Uh, and what was the I've completely lost track of where we are in this list. What was the last We had one more Hopian point, didn't we? — Oh, we do. I think that's it. I think that there are, you know, maybe Oh, I was going to say this. Yeah. Sorry. There is one big hoping point. One is that like with the power of these current like kind of hacking agents, current hunters have this massive advantage that they can apply those bugs at scale and apply their expertise at scale more than ever before. So even given all of that stuff that like you know makes me makes us think that it's so over I think that um there's like a balancing factor here of we're all going to be much more effective — both at scale because you can like when you find something you say hey go hunt for these other places but then in efficacy as well. So like let's say you find some like hidden gem of scope that like other people haven't got access to. You are not going to get in there and find three bugs like you used to. You're going to get in there and find 10 bugs and some of them are going to be more critical than ever, right? Because now you are I think a lot of times we're all allocating our time and effort, right? In the past you get access to some scope, you have like six or seven hours, then the next day you get a new invite or a buddy messages you or whatever. You pivot off of that, right? So what you got done in that seven hours might be all that you do on that scope, right? Because you forget to come back to it or you get busy or you feel like you kind of have like squeezed the towel as dry as you could at that point, right? And so I think that AI allows us to squeeze more out of that and get done more in less time. So I think that like basically the little nuggets or niches or scope that we find or the bugs that we oh and there are so many things that I found in the past that I would have not been able to then finish. like I would have found leads in the past and not been able to finish them that I can now finish because of AI too. So I think that um we're all kind of becoming superheroes, you know, in some sense and that will offset all of the negative things that we just mentioned. — Yeah, 100%. And like you do have to appreciate as well, build somewhat of a different skill set to the one that you're used to building. Before it was very much okay just get the fundamentals or look at whatever bug class you're looking at and get really good at that. Now it's a mix of managing context, learning how to build a good skill file, learning how to manage the insane amounts of like output that's generated from some of these things. And then also learning how to get that in a format where you can look at it and go, "Okay, yeah, that's valid. That's a gadget. That isn't. That's not. " So, I agree, but it's definitely you have to learn how to use it properly, otherwise you'll just be in the pit and cycle of lots of noise, lots of junk, not being effective. — Yeah. And so, um, you mentioned that hacken proof thing as like a way to potentially kind of fix this with, you know, maybe like payon submission or token submission required in order to like cut down on the noise. Another really nice way, well actually sorry, there's two things here. One, I'll let you mention what a couple companies have done already around like screenshots and videos. — Yeah. So there's been some programs especially in some live events as well basically asking you to like start your submission with a screenshot or video to help them ensure that what you are submitting is actually valid and is not AI slop because you have the proof or the P or whatever it might be that you actually have done the work and it is a valid finding which I think is quite an interesting and effective thing to do. — Yeah. And this actually whenever you mentioned that made me really think about what Justin said. Justin mentioned uh to me like maybe in like a chat or something or maybe he posted on the critical thinkers like uh a video first or video only kind of like bug bounty program. I don't love that uh at large because I think that there's a lot of times that I just would not do that for a lot of bugs that I'm currently submitting. But I do think that it should be like a fast track um or you know like some sort of way it should play a factor in whether a bug gets accepted and triage and like in the order it gets triaged in and all of that like I think that very often a screenshot some screenshots and like a video massively increase the validity of a submission right because the triager isn't necessarily guessing you know of course you can forge screenshots or forge videos but that's a lot of work and then yeah I assume the platform would kick you off if they caught you doing that, like intercepting requests or something to like fake a video. — And so I think um I think that programs before they close shop, if they did want to reduce the volume, they could do something like that, like video required, right? The same way you could do like dollar value required or whatever. — I feel like that's a fair point. And in all honesty, if I and I do this on most of my submissions, but obviously sometimes you get lazy. If you did make a video pinging like for every submission, you would be protected start to finish for every finding for when because here's the other thing as well which is like frustrating but understandable — is that there's now some cases where the point from like triage to validation to it being passed in internal teams the bug is no longer reproducible or just doesn't exist for one reason or another. Yeah. — And it's like I didn't make it up. You validated it at the time. Like it was definitely real and you confirmed that as well. But I feel like it would also help that because that has gotten me out of this exact scenario a few times where they go, "Okay, fair enough. You've got a video PC like we'll honor it type thing. " So initially it would be a pain, but once you get used to that flow and you actually see a return on the benefit of doing that, I feel like it would be quite a good idea. Yeah, I think that's actually great feedback for the platforms. I was thinking this whole time I think this entire episode would be great for platform uh for program owners and platform um you know customer success managers and product people to listen to. But I think that's actually a really great point that and I think it maybe until the validation and triage times come down. I wish that the platforms would add that as a standard. Like basically if the bug hunter invested time in your program to find a vulnerability and they found videoed it and submitted it, like so what if you fixed it in the meantime? Like that's not their fault because then like it literally incentivizes waiting longer in hopes that just like some new feature fixes it or that your internal team finds it, you know, like um it would be really great if that was a standard where it's like, you know, it still needs to be paid if it's videoed. And like you said, a lot of hunters do this anyway just to protect themselves for any like, you know, higher critical bugs to make sure they get it. — Yeah. I mean, it's only benefits. It's just extra work really, which — to be honest with the current like economy and state of things. I don't actually mind if that becomes a thing really. I like I think that's fair enough. It's um it helps both sides. So

### [47:21](https://www.youtube.com/watch?v=ZQJ2uTJWYlk&t=2841s) The Great Training Data Debate

please do that platforms. — All right, dude. Let's have this uh training data debate. — Yeah. Okay, cool. So, one thing that I'm conscious of and not a fan of right now is there's obviously you've got like um what's it like hacker one brain MCP? What what's the name of that project when it can you can hook in hacker one? — Yeah, it's H1 brain, right? It's by Yeah, — that's the one. Um, and you can hand it over and a lot of hunters are actively doing this to convert all their reports into usable skills so they can get going um, and get hunting straight away with skills already created from previous work they've done. Fair enough. One thing I'm not in love with and one thing I'm conscious of is that when you're doing that, you're train you're passing over all of your reports to model providers which then gets thrown into like the big pot of training data which I feel like is kind of like you're shooting yourself in the foot long term. I feel like because especially with the Claude uh security release as well, I just feel like okay well if I put what four 500 whatever it is reports into Claude and that then gets categorized and uses training data then the model provider also has all of my stuff and all my research or whatever as well as the platform. So I'm not a fan of that but you don't really seem to — yeah mind multiple things. one, it's over anyways, you know, like it's already over, but like let's say you don't think it's over and I would just push back on a couple of things. One, do you have training data disabled? Like do you have you disabled the optin? — So, I've actually done two things here on my personal one, I did disable it. And second of all, — if you have an organization on these platforms, like you actually buy an organization and have your licenses assigned for an organization, — you're not used for training data — that you're out of that. — Y — so I have an organization which I use to help prevent that as well, — right? So I that kind of like puts away the worries, right? But I think that I've heard that in claude code that is like not as valid. Did you research that? — What what do you mean — that basically if you're using cloud code you have to like go in and opt out again basically like kind of like maybe even per machine like there's like some sort of settings file. So like if you are like dynamically installing cloud code on like a new on like VPS or you're spinning up or you're scaling or you're running on multiple places, you actually have to go and disable the training data across all of those. — All right. No, I didn't know that. But I — I think that's the case. Um but anyways, let me push back a little more. One is I think there are two things. Not everyone is going to buy an organization. — Mhm. — Right. — Agreed. — And like I'm sorry by not everyone I mean not all bug hunters. Not all top one. Let's assume that all the information from the bottom like two like the bottom like if you're not in the top 10,00 you've got no secret skills. Even the top 10,000 bug hunters are not going to all remember or go in or disable that or buy an organization, right? So they're already probably slurping up lots of extremely high quality data. And it doesn't take an absurd number of samples. Like all it takes is like a couple users. the I this is probably why the tokens are so subsidized, but with one Cloud Max subscription, I mean, you can generate hundreds of millions of tokens every week or every month, right? And if they have high quality skills and they're running it on like hardened targets like we are for Bug Bounty, that those session logs are extremely high quality training data because the models today are so good. So basically even if they didn't generate synthetic data internally which I think they are it would still be plenty of training data just from the people who forget to buy an or forget to turn it off. But then furthermore, I think that personally the there are so many extremely intelligent people at these companies and they have so much money to throw around with buying stuff from third parties that I think it's extremely likely they have some of the best cyber evaluations even for appsac you know that look exactly like bug bounty hunting and they're probably running it on their own infrastructure internally as well that my intuition is that even if they had no user data they would be able to basically heel climb which they do anyways for all evals right and right now not only do they have cyberbench as like a base eval expo has published you know eval open source lots of other companies have I think they can just develop their own evals internally but furthermore when I was at um I don't remember the name of the conference there was a San Diego AI oh offensive AI con it's like oicon it was in San Diego like last year or something Um the there were people from all the top companies Meta, Google, Anthropic, OpenAI and there was many there was a couple talks that all implored the audience. So these are all people who are experts in security and AI and I mean like actual experts not like armchair experts. They implored them to basically create more cyber data sets and more cyber evals. Many of them did. They went on to do that. And so I think it's very likely that one these companies did internally. two, they're buying these data sets and they're buying these evals from these thirdparty people who are providing it or selling it. And we're just going to see continue massive improvement for cyber capabilities in these models. And so even if they had no user sessions, I still think the models would continue to improve. My point behind all that, Brandon, is that I don't think you personally putting in your reports changes the outcome of the improvement of these models at all or their ability to release a product that will put us out of business. No, I do agree with what you're saying, but get it's almost like the old like privacy argument of like, oh, why do you care about privacy if you got nothing to hide? Blah blah. It's more the principle of you are putting actively putting in your reports and handing it over to another third party — which can be you like and — here's the thing as well like — I guess it would be more applicable for like actual exploit researchers and things like that but regardless — we're in this middle phase where right now it's sort of like a free-for-all. You can do that if you want. What's interesting is that there's no been no like policies put out by any of the platforms to say like you can or can't do this or things like that because technically you are handing that over to a third party right which is outside of you the platform and the program manager. So I don't know if there's going to be changes there. — My point is and I'm coming at this I'll be honest if I could self-host everything I would like everything but there's just too much work in that. I would be much more comfortable in not doing that or at least having some protections around that because it's just like you just feel like you're adding to that inevitability of like just do you see what I'm saying? Like — so I agree with — I'm contributing to this. — Yeah, I agree with you on the privacy front. It feels like that to me. I think the analogy you're looking for is also voting. Do you all vote in England or is this just a USA thing? What — you all are a monarchy, right? Oh yeah, yeah. We vote of course. — Yeah, I'm just joking. Yeah, I So, you know, in obviously my single vote doesn't matter at all, — right, to any election. There's never been an election decided by my vote, but the act of voting is very important, right? And like taking it to heart and taking it seriously and exercising it in that way is important. And I think that's what you're saying, right? It's like your individual reports don't matter, but at large, you don't want to contribute to the problem. And so I think that what I would say is that — there's a time to stand on principles and um do like a riskreward analysis. U well actually sorry you should always be doing both those things. You should be deciding what you want to stand on principles and you should also be always doing a riskreward analysis. And I think for the vast majority of people and for me personally, the risk versus reward of, you know, using my reports as um improvements to my Hackbot system. Actually, I I've actually never I don't think I've ever used my reports to improve my system actually. But anyways, I think if you want to do that, the riskreward there is heavily on the reward side because I think the risk of it changing the outcomes that are going to happen are near zero. And I think the rewards are that you find more bugs. And if you found like one $5,000 bug or 10,000 bug on that, I think that that's probably more net value than you would get from holding back and trying to not contribute to the problem. Um, but I think every person has to live by their own standard of what they basically every person needs to live by the uh live to the standard that they believe in and that they want to like hold themselves to. So if people don't want to, I think that's totally reasonable. Yeah. And it the thing is as well like as I said we're in this gray area where policy and programs aren't covering this yet. I don't know maybe I foresee some changes around that happening. Um because it kind of makes sense right you are technically handing that over into another — third party which is completely it's its own thing. It hasn't signed or agreed to anything. Um, but I would say I'm very much on the side of if I could self-host everything I would, but I can't obviously. — Um, and I very much am on the side of caution like around those things. Is it — Let me give you another point in your favor. — Go. — I think that everyone in security knows this, but there are plenty of people that don't. And so maybe there's some listeners that have never thought about this. They probably have, but at every company there are administrators that have access to all the data. There is someone at Google that can just straight up log in. They can just log into the to the uh to the server, straight up dump the DB of all of your emails, right? Like we know this exists because all of us have worked at companies and there are administrators with access to everything, right? Like that's true at Meta and I'm sure it's true at Anthropic, right? There's somebody Anthropic that has the keys of the kingdom that can read all the sessions. And to be honest, I trust Google's isolation and um permissioning because they've had to iterate on this and they've held sensitive data for decades. And I'm sure that they are very anal and uh have very extreme logging and um a lot of checks and balances for who has access to that data and when they access it, where's their audit log, blah blah. Whereas these startups like open anthropics sometimes feel like they're flying by the seat of their pants. So, is there a China or North Korean plant at these companies that is just saving off all the session data that goes in or whatever? It's like it's not out of the realm of possibility. I don't think it's very likely, but I don't I do think that as a security person who needs to be skeptical and needs to think about these things, it is reasonable. It's at least rational to be like mildly worried that somebody will have access to everything that you put into these AI systems. — Yeah, exactly. just I'm I would obviously I'm more on the cautious side of it because that's just where I stand on but like it's the same thing if you paste an API key into like one of these chats it tells you like nah like you need to rotate this straight away now on some model providers so it's the same argument there but I know as you said there's a lot of hunters that have done it made skills like speedrun that initial revision loop and they're killing it. So, — yep. — I guess it's up to the individual. — Yeah. All right. Well, I we both been on here for a while. We started late cuz we were discussing it. Um, so yeah, for anyone who stuck around again, if you have any feedback, forums. ctb. show/futurebugbounty. All of those are underscored. So, future_of_bug_bounty. Give us some thoughts. Uh if you have any ideas, if you do think this would be super beneficial for programs or platforms, feel free to you know tell them to listen. I think um you know the kind of TLDDR of all of this is that to me it feels like there's a lot of network effects that make it impossible to predict, but there are a lot of negative ones. And so I think people should, you know, personally I think this year's kind of a cash grab and Bug Bunny is going to get really hard by sometime next year. But I think there is a lot of copium around being able to scale, being able to find more than ever, being able to um learn faster than ever and do more than ever. Um it does kind of pain me that it makes Bug Bounty more of a lifestyle and less of a hobby, you know? I think to really thrive, you have to eat and breathe it right now. And that kind of sucks for people who just like want to dabble in it. Like it's going to be really hard to be a dabbler. Yeah, I agree. And I think that's as well again going to change like that funnel of people that you have coming into bug bounty. The people that stay, the people that drop out as well. So it is exactly as you say going to have more downstream effects than we can possibly predict or threat model on this call. — All right. Well, you got anything else, Brandon? — No, that's it. We've covered the doom and gloom and theopium. I think we've done we've ticked everything off on the list. — Perfect. Thanks guys for tuning in. See y'all. Peace. — And that's a wrap on this episode of Critical Thinking. Thanks so much for watching to the end, y'all. If you want more critical thinking content, uh, or if you want to support the show, head over to ctv. show/isord. You can hop in the community. There's lots of great highlevel hacking discussion happening there on top of master classes, hackalongs, exclusive content, and a full-time hunter guild if you're a full-time hunter. It's a great time. Trust me. All right, I'll see you there.

---
*Источник: https://ekstraktznaniy.ru/video/50246*