# Automating My Bill Payments with Python and Selenium

## Метаданные

- **Канал:** Corey Schafer
- **YouTube:** https://www.youtube.com/watch?v=jNoTEiI2cVU
- **Дата:** 01.06.2024
- **Длительность:** 1:18:29
- **Просмотры:** 59,940
- **Источник:** https://ekstraktznaniy.ru/video/11728

## Описание

In this Python Programming video, we will be learning how to automate bill payments using Selenium. We'll cover how to set up Selenium, fill out forms, interact with page elements, and send notifications to ourselves via email or text. This tutorial is based on a real-world example that I use myself, which many of you will be able to apply to similar problems in your daily workflow. By the end of this tutorial, we'll have a robust script that can handle bill payments efficiently and securely. Let's get started...

The code from this video can be found at:
https://gist.github.com/38f71858dad289f0ab95c67d15244892

DotEnv Tutorial - https://youtu.be/PYuTzLswn_Y
Name == Main Tutorial - https://youtu.be/sugvnHA7ElY
Error Handling Tutorial - https://youtu.be/NIWwJbo-9_8
Logging Tutorial - https://youtu.be/-ARI4Cz-awo
Email Tutorial - https://youtu.be/JRCJ6RtE3xU
Crontab Tutorial - https://youtu.be/QZJ1drMQz1A
Unit Testing Tutorial - https://youtu.be/6tNS--WetLI
Email-to-Text Gateways - https

## Транскрипт

### Segment 1 (00:00 - 05:00) []

hey there how's it going everybody in this video we're going to be doing something a little different from my normal tutorials we're going to be seeing a real world example of a problem that I solved using a python script that saves me a bit of time and hassle now I like showing these real world examples because it takes a lot of the topics that I've done separate videos on and combines them all into a single script to accomplish something uh for me that's useful so in this script you'll see me using uh an automated web browser dot files for secret credentials logging uh sending email and text notifications and a few other things that we've learned separately in other videos now unlike my other tutorials I'm not going to go super in-depth on every single step uh that's what those individual videos on each of those topics is for instead this is more about showing how to take a variety of different skills to accomplish a single goal also this script is going to contain private information that's specific to myself uh so you're not really going to be able to follow along by writing the code yourself since I'm going to hide a lot of my private credentials uh it's more meant to just show you how it's done rather than following along step by step uh in the code yourself so with that said let's discuss the problem that I'm trying to solve and see the solution that I came up with so I'm the type of person that hates manually paying bills uh I have just about every single Bill set up to use autopay and most companies these days offer that capability but one of my local utilities doesn't have an autopay solution so I have to remember to manually go in to this site each month just to pay a single Le bill so instead of doing this manually each month I'm going to write a script that does this for me automatically so it's going to go out to the local utilities website that I have pulled up here it's going to fill in my account information uh my last name navigate through a few different pages and finally submit the payment information now I'm going to be keeping logs of the activities and send an email and a text message to myself each time that this script is run so that I know when the bill was paid and how much it was so with that said let's go ahead and get started so first things first we need to navigate a web browser using python so to do this I'm going to be using selenium now selenium allows us to navigate different browsers using Python and also has a headless mode that allows our script to navigate websites without actually opening a browser window for us visually and since headless mode allows us to work within browsers without an actual window being open uh that allows us to run this in the background or even where a guey might not be available like on a server uh so while I'm writing the script I'm going to have selenium actually opening my browser so that we can see what's going on and debug any problems uh but once we're finished I'm going to set this to headless mode so that it can do all of this in the background without opening any windows so the first thing that we're going to need to do is install selenium and we can just do that with Pip so that's pip install and this is s e l e n i u m and now that we have selenium installed and needs to know how to work with our browser so each browser is different and also each browser has different versions that can change how selenium needs to interact with it so to do this we're going to be installing a driver uh for the browser inversion that I plan to use so this will act as a bridge between selenium and our web browser so I'm going to be using Chrome on a Mac operating system so I'm going to download the driver that works with my version of Chrome and my operating system but if you're using a different browser or operating system then you're going to need to do a web search to find the browser that or the driver that is compatible with your setup so for Chrome we can grab the drivers at this website and I already have it pulled up here and like I said I'll have links to all of these in the description section below now on this page right here at the top it says if you're using Chrome version 115 or newer uh then to go to this dashboard here I'm using version 125 currently and uh so I'm going to go ahead and click on this and we can see here that at the top it says that the latest stable version is 1250 uh 6 42278 I just updated my Chrome so that should be what mine is so I'm going to click here that I want the latest stable one and now we have the drivers here now these are uh Chrome here this is actually a browser to test with uh we're going to want the Chrome drivers now like I said I am on a Mac and I have the app Apple silicon chips so I'm going to use the Mac arm 64 here um but if you have an Intel processor you'll use x x64 uh and things like that so just be sure to get the uh browser and the operating system that you're using so I'm going to copy that URL and now I'm going to paste that in and it should go ahead and do my download for me we can see that finished I'm going to open this up now I already have a uh project

### Segment 2 (05:00 - 10:00) [5:00]

created here on my desktop so I'm just going to unzip this and then I will place this within my project directory and let me go ahead and clean this up a bit I'll go ahead and sort this by name uh now these drivers you can have located anywhere that you'd like but they're pretty lightweight so I usually just put them within my project directory anytime I need a new one so we can see here that it's in this chrome driver uh Mac arm64 folder so I'm going to open this and we can see the Chrome driver within here now on Mac sometimes you can't run applications that you've downloaded online unless you give it explicit permission now there are a couple ways to do this but uh one way is to just rightclick and go to open with terminal and usually this works for me when I do this it says are you sure that you want to open this if I say open then it will essentially give it the permissions that it needs now I have forgot to do that step before uh and like I said if you're on a Mac you'll likely run into permission issues whenever you go to uh execute your code so if you are having permission issues uh then that could be what it was so now I'm going to close that down okay and uh open up VSS code here and now we should have everything that we need in place in order to automate our browser interactions so let's test this by opening a website so first we need uh to import selenium and specifically we're going to want to import web driver so this is from selenium import web driver and now we're going to connect selenium to our driver so I have the driver located in my project directory here we can see on the far left what the directory structure looks like now I'm going to show you uh how to dynamically get this path location in order to tell selenium where to find this web driver so we're going to do this with the path lib module so I will say from pathlib import path and sorry if you see these Imports jumping around sometimes here I have it uh set up to automatically sort these whenever I save it um and we're just getting some warnings here because I haven't used these Imports yet now I haven't done a tutorial on the path lib module yet uh but basically this is an object-oriented way of working with paths instead of using os. paath like we've seen in previous tutorials and I will have a video on path lib coming out in the near future uh so we can see how to work with this in more detail but to give you a quick example of what we can do we can uh get an absolute path to our project directory um right from our script so I will just call this script directory and this is going to be equal to path and we want the path of the current file and then if we do a DOT resolve then that will give us an absolute path to that file and now we want the project directory so now we have the absolute path of that file to get the directory I can just call its parent and that will give me the project directory and since we know that the driver is within that directory that we just saved a second ago uh we can get that by saying driver path is equal to now there are a couple ways we can do this uh with the pathlib module you can actually do uh this uh kind of syntax here where we do a forge slash so I could say script directory Forge slash Chrome driver Mac arm64 and that kind of makes it look like uh how you would see it in you know a file system or something like that I need to put this here within quotes and we called this uh Chrome driver so I can say Chrome driver so that's one way to do it uh now if you're just changing over to using pathlib uh then you can still you might be used to uh os. path. jooin um the pathlib module has something like that uh we could simply just say script directory and instead of join we can say join path and then pass in the directory file name and then I'll get rid of that on the end and here so whichever one you're most familiar with I'll just go ahead and keep that as join path and let me expand this out here a little bit to make sure that I got this right actually I missed a Ford slash here or just a regular slash sorry so I'll add that in and now let's test and make sure uh that we have all of our correct path locations here so now to use this driver we need to import this service object from selenium so to do this I will come up here and say from

### Segment 3 (10:00 - 15:00) [10:00]

selenium web driver. chrome. service import service and now down here we can simply say service is equal to use that service object and then pass in our driver path and now to tie all this together uh let's go ahead and create a driver that will navigate our browser so I can say driver is equal to web driver. Chrome and this is going to be service is equal to that service object that we just created and now we have our web driver here so let's navigate to a URL so for now let's just do something like Google so I'll say https uhfor www. google. com and when we're done doing everything that we want to do with our driver uh it's a good practice to always close this connection so to do this we can just say driver. quit and save that so now to make sure that we have all of this uh correct so far let's go ahead and run this and see if it navigates to google. com so I'll go ahead and run this okay it looks like we have an error here um if I scroll up okay so it looks like I made a typo here uh this is Chrome driver so let me save that and rerun it and now we can see there that it opened up our web browser very quickly and then closed it back down and that's just because it didn't have anything to do it opened up the website and then closed it immediately now for debugging purposes I'm going to want to keep the browser open so I can see what's going on and inspect some elements so to do this I'm going to add some Chrome options that will keep the browser open and also set a debug variable that we can change from True to false in order to uh switch back and forth between that debug mode so let's clean up the code here a little bit and put all of this stuff together and now below our Imports here I'll just say debugging is equal to true for now and now I'm going to want to import those Chrome options so to do this I can say from uh selenium and I'm actually just going to copy and paste this here so from selenium web driver. chrome. op options import options so now before we create our driver here this is where I'm going to create some options so I'll just say Chrome options is equal to options and we won't add anything there just yet uh because I'm going to want to test if I am debugging so I'll say if debugging then I want to say Chrome options. addore experimental underscore option and then we're going to pass in detach set this equal to True uh now I know this might look a little bit weird here uh but this is one way to keep your browser open if you're using Chrome and I'm going to be using this because this is kind of the official way that I saw to do it online but if you're using another browser and don't have that available or you just want to keep it simple then one thing that I've seen people use is simple input statements so we could just say input and then say you know press enter and what that'll do is it'll just pause your script in a certain spot after your uh driver opens up a website and um then it'll keep your uh browser open for you okay so if we are debugging then this is what will keep our browser open uh if we are not debugging then I'm going to add some more option here I'll say Chrome options. addore argument and we're going to add this argument of-- headless and let me spell that correctly and that will basically tell Chrome that we want to run this in headless mode like I was talking about before uh where it doesn't open up a window at all and now we need to actually pass in these options whenever we create our driver and we can do that just by passing this in as an argument when we create that object so I'll say options is equal to Chrome options that we created there and also let's go ahead and check uh if we are debugging or not to determine whether we want the driver to quit at the end of our script so I'm going to say if not debugging then driver. quit uh but if we are debugging um then it will not hit that statement so now if I run this then we can see that now it opens up our browser and

### Segment 4 (15:00 - 20:00) [15:00]

it stays open for us so now we can see that it navigated to google. com so keeping the browser open uh allows us to investigate the pages and find the elements that we need to find um so now let's La navigate to the login of the utility page um where I pay my bills so that I can show you how we can automatically fill out those forms and submit the payment now since this is a real world example of a script that I actually use I'm going to uh use the real local utility website and my actual credentials but for obvious reasons I'm going to uh hide a lot of this information so that people don't have my login and payment information now I've done videos in the past on keeping sensitive information within environment variables and how that's useful in this video I'm going to be using uh the python. package to help with this and it'll make it super simple for hiding sensitive information and loading it into your scripts and I actually just released a video on this recently um so let's see how this works so first we need to install the package so I'm going to close my browser here for now uh open back my terminal and this is PIP install and this is python d. v let's install that and like I said I just released a video on this and I'll be sure to leave a link to that video in the description section below so now that we have that installed uh we want to create a DOT file Within our project that will hold our sensitive information and this file will be called EnV now I've actually already created this file for myself and it's just an empty file here and within this file we can put the information that we don't want to be visible from within our script so let me write in a couple of values here so that we can see how this looks uh so for example I know the form will ask me for an account number and my last name so if I were to add those to this EnV file it might look something like um account number and I meant to make this uppercase and for now let's just say this is equal to one 12 3 4 5 6 um and then I can say last name is equal to Schaffer so you can see that there's no spaces and also no quotes around the numbers and strings or anything like that um so now I'm going to fill in uh all of the information that I know that I'm going to need for this script and then we'll see how to load it into uh our script using the EnV module so I'm going to fast forward this a bit just as I insert my login and payment information and keep those private but I'll pick right back up after I get all of those values typed in okay so I added all of those credentials to the EnV file that I'm going to need for this script and now let's see how we can use the EnV module to load those in so first to import uh what we need here uh we need to import load CV from the EnV module so I'll say from Dov import load. EnV and now all we need to do is run this uh method at the top of our script so here underneath the bugging I'll just go ahead and run that load. EnV function and what that's going to do is it's going to take everything that's in ourv file and turn it into an environment variable that we can access now to access an environment variable let's use the OS module so here at the top I just import OS so for example if I wanted to grab my account number then what I could do is say account number is equal to os. GV and then whatever we called this in thatv file and I called this uh the same thing here I just called it account number and that's how we're going to be able to fetch all of that private information um let me close down thatv file so I don't accidentally open it up now I have a bunch of these that I'm going to need for this script uh so instead of having you watch me type all these in manually let me just skip ahead a bit after I grab all of these okay so I've got all of the information that I added to myv file uh we can see here that I have the login page uh the account number the last name then I have my card information here uh my phone number and then an app password now the app password is something that I'm going to use uh to send an email later now one thing that I want to point out is that I'm actually currently using an incorrect credit card number and I'm doing that on purpose because I'd rather the payment fail on purpose while I'm testing and debug and debugging instead of accidentally submitting a payment uh that could be incorrect so once I feel comfortable with the script and I think that it everything should run smoothly then I will change this to my actual credit card number okay so you can see

### Segment 5 (20:00 - 25:00) [20:00]

that one of the things that I listed in my credentials here uh is the URL for the login page to the utility website and again since I'm using a real world example uh I'm going to keep the URL to this website hidden because a lot of small local utility websites like this are a bit outdated and aren't set up to handle a lot of traffic uh if anyone wanted to look up this site then I'm sure that plenty of you could easily find it uh but I'm just going to leave it out of the code so that they don't get hammered too hard with a bunch of Python scripts from this tutorial um but with that said let's replace google. com down here uh with my login to my uh local utility and see the information that I need to provide in order to automate this process so I'm going to uh copy this login page here and we will open that um instead of Google now I'll save that and run it and it opens up the website here and I'm just going to scroll in a bit okay and here we can see the web page and the information that I need to fill in order to find my account and how much I owe now this will be different for every website and form that you fill in uh but you'll need to find a way that we can find and select certain elements within a web page so that we can fill them out automatically so to do this we can inspect the elements within the browser and find if they have any identifying attributes uh the best case scenario is if the element that we want to find has an ID since those are supposed to be unique on a web page uh so let's go ahead and do this for the account number and the last name and we'll see if these have IDs so I'm going to go to inspect to where we can look at the HTML for this now I know that this is a little bit small here let me see if I can make this larger I can and now scroll down and here is that account information and we can see that does have an ID of account uh and down here is the name and we can see that has an ID of name so that's good those web elements having those IDs makes it a lot easier for us uh to find and select those within our script so now let's close this down and go back to our script and now let's uh see how we can use selenium to fill out the form values with these IDs so first let's see how we can get selenium to actually find these elements so to do this we can use the driver's find element method so above where we are uh testing if we debugging for the quit there I'm just going to say account input is equal to driver. findor element and there are different ways that we can use this find element method to find elements on the page first we're going to need to tell it what we're looking for in this case we're looking for an ID so to do this we are going to need to import the buy object from selenium so towards the top I'll import this so we can say from selenium do web driver. common. by import by and now down here at the bottom within this find element method here we can say driver. find element and use that bu object that we imported do ID and now pass in the ID of what we want to find and that was account so again just a quick look at this is saying account input is equal to driver. findelement by ID account ID now if I were to run this now then we wouldn't really notice anything because right now it's only finding that element to test if it's found it let's put in a conditional and then add some text to that field so I can say if account input and that will tell us whether or not it found it then we can now send some keys to that text field so I can say sendor keys and for now let's just say 1 2 3 4 5 so now let me run this code and if I run this we can see that it opens up our website here and within the account number it automatically put 1 2 3 4 5 there for us so now we actually have this doing some useful Automation in our web browser so now if I simply plug in my actual account number into the code instead of this sample account number then it should fill that in correctly but we can also see that this information isn't blurred out within the browser uh now bear with me here I'm writing a bit um of additional code that I wouldn't be writing if I wasn't sharing this tutorial online for

### Segment 6 (25:00 - 30:00) [25:00]

everybody to see uh but some of these videos or some of these fields I mean I don't want to be visible when shown in the browser and I could just blur these out uh later on while editing the video but just in case I miss anything uh I'm going to go ahead and write a function here that blurs out uh the information here within this field so what I'm going to do is set this to a password type and whenever you instead of being able to see the information that I've typed in here uh 1 2 3 4 5 it'll just be a bunch of dots um now this might actually be useful for some of you if some of you uh need to be able to run some JavaScript while automating web browsing or something like that so that's another reason that I wanted to add this in is because changing the attributes of some elements with selenium uh might be something that a lot of us find useful so to do this what I'm going to do is I'm going to write a function that EX some JavaScript code that gives uh these fields a password type and that password uh will black out the text like I said now I'm going to be using JavaScript because selenium itself is designed more for interacting with web elements based on their existing state uh rather than altering the HTML directly but I can have it execute some JavaScript for me that will do this so let's create this function and we'll see how this is done so I'm going to close this down and now underneath uh debugging here at the top let's create a function actually let me put this below uh our constants here so I'm going to create a new function here and let's call this um set field to password and the arguments that this function is going to take is going to be driver and then the element ID uh that we want to change so now within here all we have to do to execute this dver uh JavaScript is to say driver. execute uncore script let me spell that right execute unor script and now the script that we want to execute so this is going to be some very short and sweet JavaScript here we're just going to say document. element by ID and then let's pass in um that element ID from our parameters there and now once it gets that then we can set the type of that equal to password whoops and this is an F string here I forgot to include this within braces so let me put that in and now we're not getting that warning anymore okay so again this is super simple we just have this python function here takes in our driver our element ID uh driver. execute script run some JavaScript here to grab uh that element with that ID and set the type equal to a password so now let's run this uh before uh we set that uh account or before we send those keys and see if this hides those keys and actually I'll do this here within uh the conditional so that we're sure that it finds that um so I'll say set field to password pass in our driver and the ID that we want is account so now if I save this and run it then we can see now um that it actually did set this to a password field so we can't actually see that 1 2 3 4 5 uh that's all just blurred out there so now that'll help me in um keeping my information uh private and also uh hopefully it helped you learn something more about selenium and how we can uh change around some of these fields or run a script now normally I would uh clean this up a bit and do some error handling to make sure that there were no errors in the JavaScript code but I just decided to add this one last second in order to keep that information hidden so I'm just going to keep that nice and simple and keep it like it is um but we'll see what it looks like to catch some errors here in just a second okay so now that we are setting that to a password now let me put my real account number in there so I'm going to uh grab this from my constants up here and instead of sending in these dummy Keys we'll actually send in that account number and now if you remember uh all we need to do now is have selenium fill in my last name and then submit the form in order def find my account so for the last name this is going to be the same thing that we did for the account number uh except using the name ID so I'm just going to copy this line here and instead of count input this is going to be name input

### Segment 7 (30:00 - 35:00) [30:00]

and instead of account ID this is going to be Name ID and now since we're going to need both of those uh in order to submit the information I'll just add that to my conditional so I'm going to say if account input and name input because we're going to need both of those and now I'll go ahead and set both of those to password fields and we will send the keys to both of those as well and this is going to be name input. send keys and what we're going to want to send is my last name which isn't much of a secret I have it in mymv file anyway uh but everybody already knows that if you're watching this video okay so now let's run that and make sure that everything is still looking good okay so we can see that it has the account number filled in our last name filled in here and now in order to submit this form all we would need to do here is click this search button and what this search button does is it actually submits that form uh to find my account so we're going to want to be able to click on this but first we're going to want to see how a selenium can find this before it can click on it now in this case I want to show you how you can find something with selenium that doesn't have an ID like we saw before uh for example let's inspect that element with our HTML and we can see what this looks like so let me in insect this and let me make this larger so everyone can hopefully see I'll make the entire thing larger here and blow this up okay so we can see here that this is an input type uh equal to submit uh the name is equal to submit and the value is search but there is no ID here um this other one has an ID of reset but for some reason this one doesn't have an ID so we don't have an ID to go off of so if we want to do more advanced searches with selenium we can use something called X paath now I don't know XPath very well myself but it's a language used for navigating through XML which is also applicable to HTML so let me type in what I need to find uh the submit button here using XPath and then I'll explain step by step uh exactly what it's doing so I'm going to go back to my script here and again first let me type this out and then I will explain what it's doing so I'm going to say submit button is equal to driver. findor element and now instead of buy ID it's going to be by X path and now for the X path here I will type this out and then um explain exactly what this is doing okay so let me explain what this x path code is doing here um so this Forge SL Forge slash here at the beginning uh tells it to look across the entire HTML document and input right here whoops I didn't mean to move that around input says that the element being searched for is an input element now this entire part here um at type equals submit is a condition that the input element must have an attrib type with the value of submit and this typically identifies a submit button in a form and then we have another conditional here um with uh a name equal to submit so now that should be all of the information that we are able to gather when we inspected that HTML so now it should be able to find that submit button now if it does then let's have selenium click on that button for us so to do this I can come down here and say if submit button then submit button doclick it should be as simple as that and by clicking on that button if we typed in the correct credentials then it should take us to the next page where we can enter our payment information so let's see if this works so far so I'm going to save that and run it we can see it feels in our information and then immediately submits that form and now we are here at the payment processing page so we can see that worked we were taken to the next page with my account information and where I can submit a payment so now that we're navigating to a new page uh this actually brings up something that you might run into when using selenium and uh something that we're going to want to check for uh so when we go to new pages take time to load and depending on how long they take to load the elements that we're trying to find might not be loaded by the time our script searches for those elements which could cause problems so instead let's refactor our code here a

### Segment 8 (35:00 - 40:00) [35:00]

bit and add some selenium code that will wait for elements to load before we search for them so to do this I'm going to close down our browser here and go back to our script so to do this we're going to need to make a couple of imports so I'm going to come up here to the top and we are going to say uh from selenium do web driver. support import expected underscore conditions that's expected underscore uh conditions and we'll just import that as EC so that we don't have to type that in every time and now underneath here I'll say from selenium web driver. support. UI import web driver weight oops and we can see that we have an error here because I didn't spell conditions correctly let me correct that okay and lastly we're also going to uh import a something to check for a timeout eror here so I'll say from selenium do common. exceptions import and that's going to be timeout exception and again sorry that those jump around a bit whenever I save those uh it's just um sorting those Imports but we can see that we have the uh timeout exception imported uh right there so now I'm going to create a new function that will wait for an element to be present before we try to locate it and if it times out then that means that the element never loaded on the page so we'll call this function wait for element so I'm going to go down below our set field the password function that we created here and I will call this one uh wait4 element and we're going to want to pass in several different arguments here so we're going to want the driver and then uh buy conditional which is the um whether it's an ID or uh something like that then we're going to want an element identifier and what this will be the name or how we're going to identify uh what's out there so if it's by ID then this would be account or last name or something like that uh and now let's set a timeout equal to we'll just say five for now um now if all of this is a bit confusing right now uh just hold on and this will all make sense uh once we uh look through this so I'm going to put in a try accept block here and the accept is going to be that uh timeout exception and for now I'll just put in a uh simple print statement where we say uh make this an FST string timed out uh waiting for and then we will pass in that element identifier and just return none okay so now within our triblock here uh we're going to use those expected conditions that we imported from selenium to check for the presence of our element and we can do this by saying um I'll just call this element present is equal to and expected conditions which we imported as EC and the name of this method is presence of element and underscore located so presence of element located and all we need to pass in here is that bu and then our element identifier so that will check for the presence of an analment uh so now we just want our driver to wait until that presence is detected before we try to find it uh and do anything with it so to do this oops and real quick we can see that we're getting a an error here um let me scroll over this and see oh this is actually takes in a tupal um and not those as individual elements there okay so that's good okay so like I was saying uh now we just want to wait for our driver uh to wait until the presence is detected before we try to do anything with it so to do this I can simply just say uh web driver weight that we imported just a second ago pass in the driver and pass in the timeout dot until Elementor present and that is what we just created there so everything here within this Tri block will wait until the presence of an element is detected uh if it times out then it will throw a timeout exception and returns none um now if it doesn't throw an exception uh then I'm just going to return what we had before where we just grab that element using the find element method so

### Segment 9 (40:00 - 45:00) [40:00]

down here under the tri accept block I'm just going to return and that was driver. uh find element and we'll do buy and then the element identifier which we've already done uh down here that's the same thing here by ID account um except here we are using our parameters of the function instead so now everywhere that we were simply just finding that element before I'm going to replace this uh with the weight for element function so let me come down here to where we were just finding these elements and this is going to be wait for element now we also passed in uh the driver here so let me add that and for name input we will do the same so we can see here it's waiting for element passing in the driver that buy conditional and then the element identifier and let's see what else do we have there nothing we just had a timeout that we can set but I'm just going to leave that at 5 oh and let's not forget our submit button down here so where we were finding that we can do the same thing we can wait for that submit button going to pass in the driver as well buy X path and then that X paath um text there and lastly since we're already refactoring our code a bit here let's also split what we currently have into functions where each page that we navigate is a different function uh that way if we decide to write unit test for this later then it will allow us to pinpoint exactly where problems are occurring um so right here where we are going to the login page I'm just going to put all of this within a function so I'll call this um login in to account and we will take in a driver as an argument and now all of this is going to remain the same but we're just going to indent this to where it is within that function and now let's create a main function that handles the overall flow of this script and we'll add some exception handling here as well um So within our main function so I'll just go down here and create a main function and within here this is where I will um add all of these things here starting from let's say the service so now within our main function here we'll say We'll create that service uh create our Chrome options create our driver uh we will put our uh debugging statement in here as well and actually I'm going to put these uh Chrome options uh up here towards the top with our um debugging here sorry just need to move a couple of things around we had one more thing here yep script directory and the driver path uh we'll put up here okay so just a little justification what I'm doing here and an explanation um uh this is kind of personal preference but I like to keep you know my debugging here and loading the environment variables and um the constants script directory things like that all the options uh outside and towards the top uh and then keep the main flow of the ACT application down here uh where we can have our functions and then our main function here so now let's add in uh just a little bit of exception handling and this is going to be pretty generic so all we're going to do here is we are going to uh try to use that um log account uh function here and we'll pass in that driver and then we'll have an exception and this accept is just going to be web driver exception as e and let me spell that correctly and now I'll just print out that this was a uh general error so make this an FC screen and say General web driver error and then we will uh just pass in that error there now we always want this to be checked so we will um put this in our finally block and we'll say finally if not debugging driver. quit and lastly let's do our if double uncore name is equal to double uncore main then we want to run that main function now we can see that this is getting underlined here and that is because we have not imported this journ web driver exception so I'll come up here to where we were uh importing these

### Segment 10 (45:00 - 50:00) [45:00]

exceptions and instead of just timed out exceptions we also want that one as well uh so now we shouldn't have any errors in our script okay so I know that we just did a lot here um but like I said at the beginning of the video the concept of this video is basically showing how we can take a lot of topics that we've learned about in other videos and apply it to a real world situation and using a script that I actually use in my day-to-day life uh but what I will do is for anything that I have done a video on in the past I will leave a description to those videos in the description section below or I'm sorry a link uh so that you can view those if you don't uh understand anything that we're doing here so for example I have a video on exception handling if name equals Main and what that does uh so I'm just kind of combining all of this into one real World application here uh so that we can see how all this comes together okay so we made a lot of changes there just doing a little bit of refactoring so now let me save this and run it and make sure that we're still getting to that payment page and we are that all looks good so I'm going to close this down for now oh and actually let me go ahead and open that back up and I'll show what we have left to do so now that we're on this payment page we're just about finished up here so my utility company automatically puts the amount that I owe each month into uh this payment amount field here now all we have to do is do what we did before where we search the page source for the IDS on uh of the field elements that we want selenium to fill out for the credit card information and first since we're on a new page um I'll split this up into a new function called submit payment and we'll do all of that logic there so over here underneath login to account since we're on a new page here I'm going to make a new function called submit payment and let me go ahead and close down our Google Chrome here you can see that we actually have uh several of these so I'm going to close all these down I might fast forward the video just a second okay so now that we are on that new page all we need to do is submit my payment information uh now I'm going to go ahead and type uh everything out and just skip forward a little bit because this is not going to be anything new here I'll go ahead and uh type one out uh just so we can see it um but this is basically going to be the same things that we've seen before um so we're going to wait for an element pass in the driver and I've already written down these IDs uh from The Source uh just so that we can speed this up a bit so the ID for the card number field is Buy ID and this was going to be card number was the ID for that one and there's also an input field for um the card month the card year uh the CVV on the back the payment amount and things like that so really quick I'm going to go ahead and put in all this information I'm going to skip forward just a bit okay so I've fast fored here to where I've uh inserted all of this information uh this looks like a lot but it's nothing that we haven't seen before so all I'm doing here is I'm waiting for these elements to load on the new page uh we're waiting for the card number and then we're getting the month input the year input the CVV input the payment amount the confirm checkbox and the submit button we're getting all of those and then we're checking if all of those uh loaded and exist so numb input month year CVV confirm submit you can see that we left out the payment amount because that's for something else and if all those exist there then we want to send keys of the card number the card month the card year and the CVV so basically this is just a lot of code here to fill out uh to find the elements of that field for us and then to fill out now like I said you can also see that I uh grabbed the payment amount here now I want to check this amount to make sure that uh there's actually a payment due and I'll also add in a check to make sure that the payment isn't unusually high so I've had an issue uh with this utility before where I was charged for something that I wasn't supposed to be charged for and it was a very large amount uh so if the amount is over a certain threshold then I'll just have the script not submit the payment and send myself a message uh to check with them manually so to do this I'm just going to uh put in another conditional here and say uh if that payment amount exists then let let's try to uh get that as a string so I'll say

### Segment 11 (50:00 - 55:00) [50:00]

uh payment amount string is equal to payment amount and we can use this get attribute method and the attribute that we want is the value of that field because it's already um already filled out with the amount that I owe so now I'm going to try to convert that to a float um and if it doesn't convert to a float then I'll have it uh HD an exception so I'll say try and we'll just create a new value here called payment amount is equal to and we're going to try to get a float of that payment amount string and now we'll also check here if this float if it is less than or equal to zero because if it's then there is no payment due so I'll just return false also I want to see if that amount due is excessively High um so I will also say let's see here L if payment amount is greater than let's say something like $300 um I could also do some analysis here to where I see if it's um above a certain standard deviation from the Rolling average or something like that I'm just going to keep it simple and say if it's over $300 which I consider a large amount for this utility then I will also return false and now we will throw an exception here uh if it can't do any of this within here we'll throw a value error and we'll just return false there as well so all of this here will return false and break out of our function if anything seems off with the payment amount but if it makes it through that code and everything looks good then we can continue on with submitting our payment so now that we've filled in all of these forms here and sent in our keys for all the card numbers all that's left to do is to uh click this confirm check boox and uh click the submit button so we can come down here oh and I actually want to uh be inside of this conditional here uh where it check to make sure that those actually um exist and all I want to do here is there's nothing fancy that we haven't seen here yet I just want to um run the click method on both the confirmed checkbox and the submit button and like we saw before when I looked up my account information clicking the submit button will take us to a new page and in this case it's either going to take us to a page that our payment was successful or that the payment failed now when you're writing scripts like this you're going to tailor this to your own needs um we don't always get a nice tidy way to know if something was a success or not um sometimes we might have to Simply check the page for a uh specific text to know whether our payment went through or not now that's what we're going to need to do in this situation so for example I know that when a payment is successful whenever I submit a payment to this website it sends me to a page where the text thank you for your payment is visible um if you do something like this then you'll need to either uh know what to look for to determine a success or determine whether something is a failure but like I said in this specific case I know that it takes me to a page where it just has the text thank you for your payment so what I'm going to do after submitting my payment is use that same web driver weit function that we saw before to just wait a few seconds to see if that thank you for your payment text appears on the next page so to do this we don't actually have to uh navigate to that page because it's going to do it on its own so I will just throw in a try here and we're going to use that uh web driver weight that we saw before and pass in that driver and pass in a Timeout and we're going to wait until and let me make this a couple of lines here so we'll use that expected condition again that we imported as EC and this has a method uh this expected condition of text to be present underscore in element it's a long one so that's text to be present in element and the text that we are going to be looking for here first we have to pass in a tuple to find the element that we are uh looking for um so we're going to have a by conditional and again I'm just going to use x path here and for

### Segment 12 (55:00 - 60:00) [55:00]

this x path uh I'm just going to look inside the entire body of the HTML and within the if I see that text thank you for your payment then I know that payment was successful so if that is successful then I'm just going to come down here and return true and if it's not successful then we're going to uh then that's just going if it doesn't see that text thank you for your payment in the body of that HTML then what it's going to do is it's going to throw this timeout exception um so we can handle this timeout exception here and we'll just say uh print payment failed and then instead of returning true we're going to want to return false so now I'll save that so now that I have all this place it should work if we enter the correct uh credit card information like I said way earlier in the video I currently have the incorrect card information in my credentials for now so that we can check to make sure that this fails first so let's run this submit payment function uh and then run the script to see what we get so in order to run this function let's see that was called submit payment so down here in our main function first what we do here is log into the account and then we can submit payment again we need to pass in that driver now remember we're returning true or false whether this payment was successful so I'll just say payment success is equal to the return value of that function if it returns true then pay payment success will be true um if it's false then it'll be false um so I can say if payment success then I'll just print out here um successful payment and if it's not a success then we'll have an else and I will just copy that paste it in and say failed payment okay so it's been a while since we've tested this here um hopefully I don't have any uh typos or anything but in order to know let's just run this with my fake credit card credentials and we'll see what it does so it logs me in tries to submit that payment uh we can see here that the website's still open it says we're sorry but we're currently unable to process your payment if I go back here to the output then we can see payment failed payment so that is failed payment there and that is payment failed from the so the first one here is from the timeout exception uh the second one here is from our conditional if the payment was successful okay so that is working as expected uh even though it was a failure um because I have the incorrect uh credit card information um so we're getting extremely close here so everything that we've done so far uh you know we've navigated defined my account we have automated uh the credit card information and then we've submitted the payment now everything has worked uh automatically so far so we're doing this automation correctly and it looks like if I was to set my credit card credentials uh to the correct credentials then that would have worked but one thing that I want to do before entering my correct credentials is I want to have a way to message myself each month and tell myself that this payment was attempted uh if the payment was successful and how much the amount was for so I'm going to have this automatically send me an email and a text message with this information so first of all let's add some logging to our application so that we're keeping track of all the project of our script as we go and then I can include that in my messages so first to do this I'm going to uh import the logging module now instead of actually keeping a log file for this script uh I'm just going to have the log sent to me in an email so instead of saving it to a file I'm just going to have it right to a text stream uh to do that we're going to use the string IO class from the io module so I'm going to import that as well so import IO and now I'm going to format my logs here towards the top of the script um so we'll do that uh right here below uh debugging so I'm going to create a log stream like I said instead of having this in a file so I'm just going to use a string IO text string and now I'm going to set the log format and the log format I'm going to have as and I'll explain this uh right after I get it typed out here so in formatting logs

### Segment 13 (60:00 - 65:00) [1:00:00]

it's still common to use Python's old style string formatting uh the percent sign uh message is a placeholder for the log message uh so here I'm saying that I uh want the log message and no other information so I don't want the time or the logging level or anything like that I just want the messages here um so the way that I have it set up here is almost most have it uh behave more like a print statement than anything else uh except it'll keep all of them in one place and allow me to send them all at once um now after that uh the S here uh indicates that the log message should be formatted as a string so now using both of those I can say logging do basic config to set up our log here and I'm going to set this up as a stream and we will have that stream equal to log stream I will set the level equal to logging doino and for the format I'll set this equal to that log format that we created before okay so now I'm going to go through the script here and I'm going to add some logging statements and I also want to replace any print statements that I currently have with logging now since I'm mainly just replacing print statements here I'll show you one replacement and then I'll skip uh skip forward after I've replaced all the others uh just to save us some time so let's find one here so we can see here uh where we have a print statement so instead of having this as a print statement let's just turn this to a logging statement so I'll say uh loging doino and we'll have that as a logging statement instead of a print statement and that's basically all we have to do is add these logging dotinfo uh anywhere that we want to save that information and now I'm just going to go through the script and I'm going to place these in a few other locations um but I'm going to fast forward just a bit as I place some of these around okay so I went through and uh changed out my uh print statements with these logging doino statements uh there are a couple of other additions uh that I made only with logging I didn't add anything else um but just little things here and there um like up here I'm logging the payment amount uh whenever we get to the payment screen um if the payment is less than zero then I'm just saying exactly why this failed here so uh logging that no payment was due or that the payment amount was high or that there was an invalid payment amount like it was a string or something weird like that um but those are the only additions I made it was just adding some logging statements to different parts of our script okay so down here within our uh payment success if it's true or false this is where I'm going to have it send me an email and a text message uh telling me that it was either successful or it failed so now that we have the logging set up let's see exactly how we can do this uh by adding that functionality that will send me some emails and send me some text messages of these results so that I can stay informed uh when this script runs and if there were any issues now again I've done a detailed video in the past on how to send emails using Python and I'll be sure to put a link to that video in the description section below uh so that you can see the details of what I'm doing here uh but I'm just going to do this fairly quickly here so to do this I'm going to make a couple of imports so up here at the top uh I'm going to say from email. message we're going to want to import email message and now I'm going to create a separate function to send this email so I'll just go down here uh to the very bottom here uh create a new function and we'll call this sendor message uh since it's going to do emails and text messages um so first we're going to do a subject here and then we're also going to want to send in a receiver so for the sender I'm just going to uh send this from myself so this is going to be Corey M shaer gmail. com is my email address and now let's create our message so first of all we're going to set this equal to that email message object that we imported and now we're going to set a couple of things for this message so first we're going to have a from value we're going to set that equal to the sender then we are going to have a two value we're going to send that equal to set that equal to the receiver then we're going to have a subject and that subject we will set

### Segment 14 (65:00 - 70:00) [1:05:00]

equal to the subject here that we're passing into the function and now message. setor content oops and this needs to be message spelled that incorrectly so this is going to be the content of the message and what I'm going to do here is I'm just going to dump everything from our log stream so I'm just going to say logstream doget value and since I've added all this logging throughout the script that should be all the information that I will need to know exactly what happened um or at least enough to go off of in order to investigate uh whenever I get the email message okay so now let's go ahead and send this so I'll say with SMTP lib oh and actually as I'm typing that out I'm realizing that I didn't import that so let's be sure to import SMTP lib okay now let's go back down here and where are we at here we are so with SMTP lib do SMTP SSL again I have videos on exactly what we're doing here uh but right now we're just uh connecting to uh the server that we can use to send messages and the ports and all of that we're going to have that as an SMTP connection and now I'm going to uh log in so I'm going to log in with my email address corium shaer gmail. com and that is just the sender and I'm also going to need my app password now remember uh this is uh pulled in from myv file that we saw here at the very beginning so I'm using my login information here for uh my email you would have to do um different okay so now we can just use SMTP do sendor message with the message that we created oops and I accidentally have a colon there that I don't want and there we go and let's see I'm getting an error there did I not spell that correctly nope I did not smt TP lib and now if I go back down here then that is good so now that we have this send message function this is what I'm going to use if the payment either fails or succeeds so I'm going to go down to our main function and add this in So within our payment success here uh we can send a message actually let me just create a message here equal to successful payment and I will pass this in as the logging info I will also pass that in as the uh subject for the email and the receiver of this email I'm sending it from myself but I also want myself to receive it and we're going to do the same thing here if the payment was a failure but instead of a successful payment we will do failed payment pass in that message as our logging message and then we will send that message to my email okay so that should send me an email with the results of this script but since this involves a payment I kind of also want this texted to me uh when the script is run now a lot of people don't know this but a lot of phone numbers actually have an email address associated with them that will send a text message to the number when you send a message to that email address now this depends on your cell phone provider but most of the major ones have email gateways available for the numbers so for example let me open up my browser here I actually have this open up here in the browser so for example I have AT& T if I wanted to send a text message to myself I could just put in my phone number um at mms. att. net and in the description section below I'll add a link to this page that has these gateways listed for a lot of the providers now I don't know if this works internationally I know that works here in the United States but this is just something that I use a lot because it's an easy free version to send myself text notifications through python but if you're at all interested then I can also show uh some paid versions in the near future you know using things like twio and services like that to send text messages so let me know if that's something that you would be interested in but for this video I'm going to use this simple easy paid route

### Segment 15 (70:00 - 75:00) [1:10:00]

of just uh sending this uh to my phone number email address and that's actually going to text me and this is why I like this method so much because it makes it so easy so in order to do this all I have to do is send another message here but instead of gmail. com uh like I said this ismms. att. net for my specific Network for AT& T and what I need to do here is all this is going to be an FST string all I need to do is come up here and put in my phone number in front of that email address and what that will do is it will send me a text message so I will save that now I'm also going to have this s me an email if it runs into this uh General uh web driver exception here uh so I'm just going to save these and down here for the message I'm just going to have let's say failed to run script this is a bit more General here that would probably mean that you know something is wrong with the web driver or something like that okay so very last thing here down here at the bottom if I have debugging turned on then it likely means that I'm sitting here at my computer working on this script uh so I'm going to change this so that it prints out our log stream so that I can see it there if we're debugging so I'm going to change this to say if debugging then what I want to do here is I just want to print out my logstream so that was logstream doget Val value and then we'll say else driver. quit Okay so that's it I'm not going to write any more code I promise I think we're done here um but uh let's make sure all this works so we added a lot of different code here so first of all I'm going to run this script one more time with my incorrect credentials to make sure that this is working as expected and that I get my text message and my email so I'm going to run this and it goes through everything here let me close this down okay so we can see here that uh since we have debugging turned on that it printed out this log stream here of get value um and so it's saying the payment amount of 4338 failed payment now tutorial is getting a little long already so I'm not going to show myself logging in uh to my email and showing you that those came through but I'm looking at my phone right now and I can confirm that I did just receive that email that said that the payment failed and I also received a text message that the payment was attempted and that it failed with the incorrect credentials so now to wrap all this up uh let me put in the correct credentials and then I'll turn off the buing so that it runs the browser in headless mode which means that it doesn't even actually open the browser user interface to navigate to do everything that it needs to do uh it just does it all in the background so let me make those changes so first I'm going to set uh debugging to false so that was up here at the top set that equal to false and really quick I'm going to open up myv file here and set this to my correct credit card information and I'll just fast forward to when that's done okay so I just changed all of that around to where it should be my correct credit card information uh so now the moment of truth so I should be able to run this script and have it pay my bill automatically hopefully I didn't make any typing mistakes or anything like that and I hope it all works uh but if it doesn't then I should get an email and a text telling me exactly what happened but let me go ahead and run this and again it's going to run in headless mode so I'm not going to get any output here I'm only going to get a text message and an email okay so I did just get those and I'll go ahead and put the email that I received on the screen uh now so that we all have some sense of accomplishment instead of just uh staring here at a blank screen um so you can see that it actually did go through um and I also paused the video and took a second to uh check the utility company website and I did see that the payment went through in the amount that they expected as well so that all did work um exactly how we wanted it to work so now the script is working as intended uh all there is to do now is to schedule this uh in some way to run however often that we want this to run so I'm going to set

### Segment 16 (75:00 - 78:00) [1:15:00]

this up as a Cron job on my system to run once a month um I still need to uh check with the utility company to figure out exactly when they put the payment due amounts on the website but once I get that information I'll just have this uh run automatically uh once a month to pay my bills for me and if you want to see how to run chome jobs on a certain schedule then I do have a video on that as well that's for mac and Linux uh I know that Windows has the ability to run a bash shell now but I'm not sure if Chron tab is included in that or not if you know let me know in the comments below uh but if you're on a Windows machine and that's not available to you then I'm sure that they have uh their own schedulers as well I just don't know much about uh windows schedulers for running scripts on a certain time frame but if anyone's interested then let me know and I can look more into that uh to show exactly how to uh run schedule jobs on Windows and I can put something together now as for this script I think it would be a great script to use in an advanced unit testing video uh that I'll be putting together in the near future because this script has a lot of more advanced concepts that can be difficult for people who are new to unit testing to know exactly how to test so for example since we're going out to websites and sending emails and things like that those are things that we want to mock in our unit tests so that we're not relying on things outside of our control that can break our script so showing how to mock these calls and test this appropriately would be a good addition to the unit testing tutorial that we already have out there but with that said I think that's going to do it for this video I know that this was a long one but hopefully you found this walkthrough helpful uh I know I didn't go as in depth as I normally do when focusing on a certain topic but I wanted to show how we can take a lot of these individual Concepts that we've Ted talked about before combine them all into something useful that perhaps you can use to solve a similar problem that you may have uh like I said I hate paying bills manually so having it do this for me automatically and send me updates is a huge help for me so I'd like to continue uh doing videos of scripts like this uh that I've created for myself in real world situations one that I'm almost done with is a financial Hub that I've created for myself where I parse my PDF bank statements and I created a tter goey uh to categorize those transactions and then use pandas to visualize and analyze my spending habits um that's a pretty large project uh that I've been working on and once it's all done I'll release several videos covering how to do everything that I did for that so be on the lookout for that uh but if anyone has any questions about what we covered in this video then feel free to ask in the comment section below and I'll do my best to answer those and if you enjoy these tutorials and would like to support them then there are several ways you can do that the easiest way is to Simply like the video and give it a thumbs up and also it's a huge help to share these videos with anyone who you think would find them useful and if you have the means you can contribute through patreon or YouTube and there are links to those pages in the description section below be sure to subscribe for future videos and thank you all for watching