Real Bugs vs. AI Testing — Can It Handle Broken Apps? KaneAI Live App Demo

13:21

Real Bugs vs. AI Testing — Can It Handle Broken Apps? KaneAI Live App Demo

Python Simplified 17.05.2026 3 126 просмотров 198 лайков

Machine-readable: Markdown · JSON API · Site index

Смотреть на YouTube

Поделиться Telegram VK Бот

Транскрипт Скачать .md

Анализ с AI

Описание видео

What if AI could help test your apps using plain English instead of massive Selenium scripts? 🤯 In this video, we’re going to run an AI testing agent on a LIVE intentionally broken application — complete with hidden logic flaws, inventory bugs, and looping failures. 💀🔥 Instead of manually writing every browser test ourselves, we’ll use natural language to tell an AI agent named KaneAI what to test — and then watch it control the browser, generate assertions, validate APIs, and even create its own testing scenarios automatically (make sure you stay until 10:39 for this one! 😉) ⭐ Try KaneAI for FREE using my link: https://www.testmuai.com/kane-ai/?utm_source=youtube&utm_medium=mariya_ifm&utm_campaign=kane_ai&utm_content=dv&via=vibetest 📚 What you'll learn: ✅ How AI-powered browser testing actually works ✅ How KaneAI compares to traditional Selenium & Unittest workflows ✅ Supervising AI-generated testing scenarios ✅ Assertions vs “fail test immediately” conditions ✅ Manual interaction testing ✅ API testing with automated validation ✅ Debugging AI-generated testing workflows ✅ Test version history and rollback ✅ How AI can generate Selenium-style test code automatically ✅ How modern AI-assisted QA workflows actually look And yes... the AI actually starts looping and breaking at one point — forcing us to step in and fix the testing objective manually 🙃 We’ll start with a simple beginner-friendly example using newsletter subscription testing, then move into a live warehouse ordering system with a real backend API and embedded bugs. 🧾 What we'll test: ❌ Negative inventory quantities ❌ Orders larger than available stock ❌ Inventory that refuses to reach zero ❌ Broken conditional logic ❌ Edge cases AI gets stuck on 🛠 Technologies & Tools Used: • KaneAI • Selenium • Python • FastAPI • Ngrok • WSL 🌐 Live App Used In The Video: https://evident-expulsion-suction.ngrok-free.dev/ 🌐 API Endpoint: http://72.60.178.132:8000/warehouse 💻 GitHub Repository for Live App Code: https://github.com/MariyaSha/warehouse_api_demo And honestly… software testing starts feeling less like traditional QA automation and more like supervising junior AI testers that work in the background while you do other things 😅 ⏰ Time Stamps ⏰ 00:00 Intro - Can AI Actually Test Broken Apps? 01:13 Quick KaneAI Demo 01:56 How AI Browser Testing Works 03:16 Testing a Real Broken Warehouse App 05:38 AI Gets Stuck in a Loop 05:49 Fixing the Test Objective 07:14 Manual Interaction Testing 07:43 Testing Negative Inventory Bug 08:23 API Testing with KaneAI 09:43 Version History & Rollback 10:39 AI-Generated Testing Scenarios 12:46 Final Thoughts 📚 Related Tutorials: ⭐ Build a Simple API with FastAPI: https://youtu.be/dglcDIUTSsY ⭐ Selenium Tutorial For Beginners: https://youtu.be/aSeqMYNhEHo ⭐ Python unittest Tutorial: https://youtu.be/EqJWhlC1H6k 💬 Discuss in the comments: • What do you think about AI-powered software testing and KaneAI? • What applications would you test with AI agents? • What is the most dangerous type of bug? #Python #SoftwareTesting #AI #Selenium #Automation #QA #FastAPI #WebTesting #APITesting #PythonProgramming

Оглавление (12 сегментов)

Intro - Can AI Actually Test Broken Apps?

Software testing is the only reason your favorite apps don't randomly crash anymore. Today, one broken page or one tiny login bug is a death sentence for your product or your resume. So, we test. But, writing tests is excruciating. We check every element, we validate input, we test edge cases, and that's before we even know if the whole thing works. So, if it sounds like mountains of work for a tiny app, imagine testing something the size of Amazon, Instagram, or ChatGPT. Who has time for that? So, today, we're going to see if AI can actually take some of this work off our plate. We'll use an agent called Kain AI, and we'll walk through a few real testing scenarios. Starting with a simple example, then moving on to a real application with actual bugs in the API, and then finally, letting AI generate its own test scenarios. And trust me, you want to stay until the very end for this one. So, if you're curious to see how AI fits into modern testing workflows, let's roll.

Quick KaneAI Demo

Okay, so let's start with a quick demo. We will head over to Kain AI. You can find the link in the description or the pinned comment. You can, of course, start with a free trial and then upgrade later if you find it useful. Now, the main idea here is pretty simple. We are testing a live application using natural language. And first, we will use the quick author mode, saving the generated scenarios one for later because this one is extra impressive. So, instead of writing code, we'll just describe what we would like to test. For example, I will type subscribe to the newsletter at pythonsimplified. org. And then, Kain AI will generate a test case, planning a sequence of steps, and it will actually run those steps in the

How AI Browser Testing Works

browser just like we would with Selenium. In our case, it created five steps. First, it opens the website. Then, it finds the email input and types a value. Next, it clicks the subscribe button. After that, it checks if some kind of a confirmation appears, and then finally, it verifies that this element is actually visible, which is the part we are really testing. When we assert something, we're basically checking if something is true. Everything before that is just set up an interaction. The assertion is what actually determines whether the test passes or fails. And this is where things get interesting. If we expand some of these steps, you will see a fail test immediately flag. These aren't assertions. They're more like guard conditions. They can stop the test before we even reach the final check. So, if we recreate the same flow with Selenium and unit tests, I have great tutorials for both, the structure looks very similar. Each step depends on the previous one. So, if we cannot type an input, there is no point in clicking the subscribe button. And these flags, they stop the script from running as soon as something breaks. Perfect. So, now that we understand the basics, let's move on with something a bit more realistic.

Testing a Real Broken Warehouse App

Let's say we're running a simple warehouse ordering system. There is a backend API that manages the inventory and a public interface where users can place orders. We actually built this system together in a previous API tutorial, but I slightly enhanced it for this demo. The important part is this is a live system. If you type this IP address in your browser, you'll be able to interact with it yourself. And I highly recommend following along. Now, here's where things get even more interesting. I have intentionally embedded small logic flaws into the system, so you can try to catch them. First, we are allowed to order negative quantities. And second, the inventory never actually reaches zero, okay? It is fine with keeping one box or bottle or loaf, but it will not let you order it. So, let's quickly reset the inventory. Okay, I will do this from my server. And then back in K and AI, we will ask it for several things. First, open the app and test the tomato inventory. Then, make sure we cannot order more tomatoes than what's available. Next, check that valid quantities work as expected, and then finally, try to reduce the inventory all the way to zero, which we already know is going to be a problem. Now, let's hit enter. And perfect. K and AI broke it down into logical steps, and since they all look good to me, I will hit approve, and then the browser automation will begin. Now, since there were a few shenanigans involved in securing my app, we are initially dealing with a warning page, but once K and AI figures it out, we can proceed with the tests. And this has to do with ngrok, so if you're curious to find out more about it, I explained it in the pinned comment right below. This is my shenanigan. So, when we look at the automation, we see that it checks the inventory. Great. It tries to order more than available, but it gets blocked. Amazing. Then, it places a valid order. Also good. And then finally, it tries to reach zero, which is exactly where we expect things to

AI Gets Stuck in a Loop

break. But, instead of failing the test, it actually gets stuck in a loop, and it repeats the same steps over and over again without making any progress. So, this is where we step in. We will pause

Fixing the Test Objective

the execution, we will stop that last test, and we will click on the more options button editing the objective. To clarify our instructions, we will go ahead and type check tomato inventory. Test that I am allowed to order the remaining quantity. If I am not allowed to, fail the test. Now, we will go ahead and apply that change, and we will run the test from this point. There's no need to rerun everything from scratch. And now, this time, instead of looping, we get a clean conditional flow that behaves the way we expect. Now, once everything is done, we can go ahead and save the test, as well as validate the generated code. And now that the session is done, and we're no longer racing the clock, we can go ahead and inspect what Kain AI actually produced. We can have a quick peek at the code, and even though it is still being validated on my end, we can already download it and open it on our system. And perfect, we see familiar Selenium imports and a full test script generated directly from our prompt. Yay! But sometimes, it's easier to just click around and enter values instead of trying to describe it. And for this, we

Manual Interaction Testing

have the manual interaction feature. So, if we go back to our test summary, we can click on the edit steps button, and we will add another check to our logic. We will replay the existing test, and once we reach the very end, we will switch to the manual interaction mode at the top right of the automation browser. Then, we will go back to the main page, and we'll try to order a quantity of

Testing Negative Inventory Bug

minus 20 tomatoes, and we'll hit send. And okay, we see that instead of getting an error, we are actually adding 20 boxes of tomatoes to our inventory, which is a very big no-no. So, what we will do is we will turn off the manual interaction. We will go to the last step in our workflow, we will expand it, and click on the three dots. Then, we will set a failure condition of fail test immediately. Meaning, if somebody tries to order a negative quantity, they should not be allowed to hit send. And this is how we add steps manually. Additionally, we can

API Testing with KaneAI

test the API that serves all these inventory values, and if we're already talking about tomatoes, we might as well focus on the tomatoes endpoint. Specifically, since it accepts an order quantity parameter, we can test how this request is processed. So, let's go ahead and copy the URL, and we'll head back to our test environment. We will then type {slash}, and we will choose add an API. We will then, of course, paste the URL right over here, and then under query parameters, we will add a key of order quantity. Also, giving it a value of 10. Then, we will hit send, and in our new API step, we can double-check that by clicking on show API, and then inspecting the response. We can see that we ordered 10 boxes of tomatoes, and that the remaining quantity dropped from 1,000 to 990, which tells us that the back-end logic is working here, not just the UI. We do have issues elsewhere, but this specific API request is fine. Perfect. So, now we can go ahead and save everything once again, but it doesn't mean that our previous version is lost. We can access

Version History & Rollback

it by clicking on the version history tab where we store both the current version of our tests and all the previous ones. So, if you mess something up and you want to go back, you can just select one of the earliest versions, only one in our case, and then we click on revert. I am not going to do that on my end, but the option is here. Now, what we've seen so far was mainly a demo of the core features, and we didn't follow the step-by-step testing process that I've shown you in previous videos. Normally, we will go through a proper checklist, testing things like existence, typical inputs, outputs, and edge cases, and so on. But, the cool part is that AI can actually generate and adapt this checklist based on the app you're building. So, instead of using the same approach every single time, these scenarios are tailored to your specific app and your specific software stack. So, now the crème de la

AI-Generated Testing Scenarios

crème, which I've saved for last, is the generate scenarios mode. Instead of following your instructions, Kain AI will analyze your app and generate its own tests. It is designed to check for scenarios that you probably wouldn't even think of. So, let's switch to generate scenarios, and let's give it a prompt to analyze our warehouse app. And I gave it both the Angular URL as well as my servers' IP. Now, after a few moments, we get a set of generated scenarios with different priorities. Some are must-haves, and others are should-haves. And this is actually important, because if you are using the free trial, like I am, you're limited to 10 scenarios, while Kain AI is suggesting 15 of them. So, you can just go in, you can expand these scenarios and you can manually disable some of the tests. And that way you control how many sessions you use. But on my end, I'm just going to click on create and automate and let's see what happens. And okay, it asks for a URL again, so I will just paste it. And once I confirm I get a warning about the session limit, which is fair. But even with this warning, as you can see, we immediately start executing our first test. And at the same time, okay, we now have a full queue of tests waiting to run. And these should match all the scenarios that we just approved. Now, the idea behind this feature is that the test can run in the background while you are doing some other work. But if you want to jump in and just watch one of them, you can just click on view session. And then it will take you back to the same interface we used earlier, automating the app in the browser. Otherwise, when each test finishes, the next one starts immediately and the queue gets gradually smaller. And ideally, instead of running these tests one by one, you would run them in parallel using multiple agents depending on what you have available. I believe we

Final Thoughts

have two of them available in the demo. So, if you found this workflow interesting, make sure to check out Kainos using my link from the description. And don't forget to tell me in the comments what you think about it and what kind of projects you would like to use it for. And thank you so much for watching. If you found this video helpful, please share it with the world. And don't forget to leave it a huge thumbs up and all kinds of comments. Now, if you'd like to see more videos of this kind, you can always subscribe to my channel and turn on the notification bell. I'll see you soon in an awesome tutorial and in the meantime, bye-bye.

Другие видео автора — Python Simplified

Ctrl+V

Экстракт Знаний в Telegram

Экстракты и дистилляты из лучших YouTube-каналов — сразу после публикации.

Подписаться

Лучшие методички за неделю — каждый понедельник