Matplotlib Tutorial (Part 9): Plotting Live Data in Real-Time

hey there how's it going everybody in this video we're going to be plotting data that's continuously being generated in real time so these realtime plots would be great for plotting data that is changing frequently that you want to monitor so for example I see a lot of examples on certain sites that will you know plot things like realtime YouTube subscriber counts or maybe you're reading in data from a sensor and want that immediate feedback uh no matter what you're doing you're likely going to find a use for something like this so first we're going to look at a basic example where I plot some data that's directly in my python script so that we can wrap our heads around how this is working and then we'll look at a real world example where we're actually monitoring a CSV file now if you're pulling data from a real-time API or a sensor of some kind then it's pretty common to write those results to a CSV file so we'll monitor that for changes and make updates to our plot when there's new data available now I would like to mention that we do have a sponsor for this series of videos and that is brilliant. org so I really want to thank brilliant for sponsoring this series and it would be great if you all could check them out using the link in the description section below and support our sponsors and I'll talk more about their services in just a bit so with that said let's go ahead and get started okay so I've got some sample code pulled up here in my script so first we'll look at these real- time plots using this list of data directly in my script and then we'll look at a real world example with data that I'll load in from a CSV file now if you've been following along with the series then you'll likely recognize the other map PL plot lib code that I have here at the moment but if not let me go over this real quick just in case okay so I have a few Imports here at the top I'm importing the random module uh the count function from iter tools uh we'll go over how we're using those here in just a second I'm importing pandis here we're importing pip plot for mat plot lib we are setting our plot styles to use the 538 Style just so these look a little nicer uh now this entire section here is just the data that we're going to be plotting just to get our heads wrapped around how this is working right now uh we'll look at that here in a second and finally here at the bottom we are doing PLT do tight layout to add some automatic padding to our plots and PLT doow will just show us our plots and as usual I'm going to have a link in the description section below to all of this code here if you would like to download that or copy and paste it into your editor so that you can follow along with this uh video okay so first off uh we're used to using static plot that we've already seen in this series before so for example I have an X and A Y list

here with values and I'm plotting this with the PLT do plot method which is just going to make a basic line chart so if I run that then we can see that works so that is pretty simple but now let me

delete the current data that we have in these lists and I'll uncomment out this function that I have right here and explain what our desired outcome is so let me remove uh PLT do plot and for our X and Y values I'm just going to set these to be empty list and now let me uncommon out our index and our function here so for anyone who has never used the count function from the hter tools Library basically by default it just counts up one number at a time and each time we get the next value so I have a function here called animate and with this within this function we are appending to our X values list and also our yv values list now X values is just going to append a value that count up by one so it's just going to be sequential and Y values is appending a random number here between one and five or between zero and five so let's say that we wanted to run this function every second and plot these values that are getting appended to our list uh well how would we do that so to do that we can use the funk animation class from the map plot lib animation module so let me import that and we'll take a look at how this is done so below our other Imports here I'm going to say from matplot lib do animation import Funk animation and now down here below our function let me make sure I spelled all that correctly uh yep so down here below our animate

function let's tell matplot lib that we want to run that function on a specific interval so we can plot that new data so to do this I can simply say uh create a variable here called Annie and I'll say Annie is equal to Funk animation and now we need to pass in a few arguments so first the figure that we want to animate now we'll learn more about figures in the next tutorial when we learn about subply but basically we just need to pass in the figure from our current plot and to do that we can just say PLT do GCF which is get current figure okay and next we need to pass in the function that we want to run for our animation so that was the animate function and also the

interval for how often we want to run this function the interval is in milliseconds so if you want 1 second then you pass in 1,000 and I think 1 second would be good for us in this video but you can change that depending on your data so I'm going to pass in an interval equal to 1,000 which is 1 second okay so now Matt plot lib is going to run this animate function every second but currently we aren't plotting the data So within the animate function we're going to plot our X and our y lists so I'm going to say PLT do plot and we will plot out those X values and we will also plot out those y values now

if we run it like this then it will somewhat work but there's going to be one big problem so let me run this and I'll show you what that problem is so if we run this we can see that we get an updating chart but you can see here that this is weird we're getting all kinds uh of different colors here um so let me close this and I'll explain what's going on here so the reason it's doing

that is because our plot method is actually plotting a brand new line every time but it's not clearing out uh the old lines so there are actually multiple lines getting stacked on top of each other there but they're just being covered up uh so you can't really tell so one way we could solve this is to Simply clear out our axis and if we uh do that then every time it plots that new line from scratch uh we won't have that issue with the different colored lines because it's always going to plot the same color so to clear the axis we can simply run the CLA method which I'm assuming stands for Clear axis so I'm going to put that right above our plot so I'm going to say PLT do c and now if I run this then we can see that this looks better it's updating our plot with those uh random Val values each time our animate function is Run Okay so that's nice but using random data directly within our animate function doesn't really give us an idea of how this would help us plot real world data that's coming from an outside source so let's take a look at how we would do that so to do this I'm going to

plot data that's from a CSV and this CSV is going to be constantly updated by an outside Source now this Source could be data that you pulled down from an online API and put it into a CSV file uh it could be data that you're saving from a sensor anything like that it's very common uh to save data from sources like that to a CSV file now the source for my CSV data is going to be a simple python script that's going to be continuously adding values uh but it doesn't really have much to do this with this video so I'm not going to go into a lot of detail uh how that's adding that data but if you'd like to see how I'm doing this then I can open this script up really quick and just give a quick overview so

let me go over this really quick and show how I'm writing random data to a CSV file uh in real time so I'm importing the CSV module random and time uh we're setting some starting values here of zero and then total one and total two equal to a th000 and then we're just using the built-in CSV module for this stuff so the field names here are going to be the headers for the CSV file uh we are opening that up and writing those headers and then we're saying while true so this is just going to continuously run uh we are opening up that data in a pin mode so it's going to keep appending to the CSV and we are creating a dictionary writer and we are writing out this info here and the info is all of the data for those headers uh so we're writing that row and then we're also printing this out to the console so that we can get some real-time feedback in the console as well and then lastly we're updating

the values so I'm just incrementing my X values by one every time now your X values could be anything they could be dates they could be uh you know anything um and now for total one we are saying total one is equal to its old value uh plus a random integer between -6 and 8 uh so I just wanted some variation there it can either go down or it can go up but it's got a better chance of going up uh total 2 is equal to Total 2's value at uh plus or minus uh -5 to6 so it can also go down or up uh but it's more likely to go up as well but there's also not as much uh variance in these random datas here so that'll just kind of mix up the data for those two totals there so that's how I'm writing in real time to our data. csv file so that's the live data that we're going to be monitoring but again that could be live data from any Source it doesn't matter that it's coming from a python script it could be anything uh so now let's see how to do this so I've got some code commented out down here at the bottom and this just reads in all of the data from that CSV file that we're going to be monitoring and if you've been following along with this series then we've done this several times uh but just in case let me uncommon out this and I will uh show how to do this so first I'm going to put this into our animate function and then

I will describe what's going on here so I'm just going to replace these old values that we were using uh just to do some testing there and now our new values here we are reading in data. csv and that's going to be the data. csv file that is getting live data written to it on a continuous basis uh for our x value we are grabbing the x value row uh so that's going to be all of the data for the X values in that CSV same for y1 we are just grabbing the total one for Y2 so we're going to have two lines on this plot that we are monitoring that is going to be the total two column so now since this is going to run that animate function every second that means that it's also reading in the data from that CSV every second now we can simply plot that out and when we plot this out let's also give these labels so that we know which one is which using a legend so let's pretend that these are you know YouTube subscriber counts and we're plotting the live counts of two different channels or something like that um so below our clear axis there

I'm going to plot out first our y1 so pass in X for our x-axis y1 for our y- axis then I'm going to give this a label and I'll just call this channel one and now I'm going to copy this and also plot out a line for Y2 and I will call this Channel 2 now when we clear our AIS it's also going to clear our Legend so we're going to need to recreate that and I will just do that down here at the bottom by saying PLT do Legend and make sure you actually have that uh indented here within your animate function now with live data coming in I think it's usually best to specify an exact location for the legend uh I've had it uh you know switch spots only several times because uh the data is being updated and it's bit distracting because this tries to pick the best location by default but if the data is constantly being updated then it could be switching around on you so I'm just going to set a location here and I'm going to set this equal to the upper left of our plots and you can change that based on your data and also I usually put my tight layout method in here as well uh so that each time it plots uh it takes that into consideration so I'm also going to put that here within my animate function as well so I'll do tight layout and run that okay and that should be all we need to monitor our CSV file uh now this file isn't created yet so let me open up my terminal and I'm going to run that script that's going to generate data for that CSV file so I've got my terminal pulled up here so now I'm going to uh say Python and that script was called datag gen dopy so I'm going to run that and when I run that it creates that CSV file and now this is writing data to that CSV file and it's going to continue as long as we don't kill our operation here so if we go back to our script now that CSV is being updated with live data so let's go to the script and see if this is picking up those real time changes so I'm going to run our code here and when I run this you can see that that's already been running for some time and that these uh already taken off uh now so we can see that it's actually updating as that data is being written to that CSV file now let me uh close this down for a sec and pull this back up in the terminal and stop this and let me start this back over because uh it was already kind of far along whenever we checked that last time so now let me run this again and now it's basically just started so we can see that it just started writing data out to this CSV file so it's picking up those live changes from that CSV file and plotting that data as it comes out and this can be updated on any interval that you'd like so if your data is only updated every 5 seconds or something like that then you can change your interval so that it's only updating when you expect new data that way you don't have to check every second so I was thinking that in the near future maybe I'll put together a YouTube API tutorial using python and maybe we can actually see how to build a working version of something like this with real YouTube data uh that would be a cool little project to do uh so we can see here I think uh I gave my channel one a better chance of having better random values and it was behind for the longest time and now it's just constantly been going up okay so we can close this down but it's kind of fun to watch these uh live plots uh take off like this so this was

a pretty basic animation that we put together here using this live data but there's a lot more that you can do depending on your needs so for example if you needed to run an initialize function for your animation uh that sets things up one time before your animation uh first runs and then um to do that the funk animation class has an init Funk argument that you could pass in to do that uh or if you needed to pass in additional arguments to our animate function uh then our uh f or Funk animation class also has an F args argument to do that so there's a lot more that you can do with this and some of these animations can get pretty complex I've seen some people use these for uh drawings and math simulations and gravity simulations and all kinds of neat stuff now there is a way that we could have updated our plot uh without totally clearing out the axis uh but it takes a bit more code since we'd have to also manually check our X and Y limits and things like that uh but it is possible to not clear out our axis here and just update the lines on the axis with the live data so there's more than one way to do things like this um but I thought that this was you know good enough for this video I think that this works well with the amount of data that we were looking at now if anyone wants to see the other more manual way then I can also try to include that in the Snippets for uh this video when I upload those to GitHub so you can probably find those there if I don't forget okay so that is the basics of running these cool little live plots and we can see that now when I run this uh since we haven't cleared that out for a while that uh that data. csv file has just been continuously getting updated so every time I run this it's just um updating with the newest ones but we still have over let's see I think that updates that every second so we have over 200 seconds of data here already Okay so we're just about finished up here but before we end I would like to mention the sponsor of this video and that is brilliant. org so in this series we've been learning about matplot lib and how to plot data in Python and Brilliant would be an excellent way to supplement what you learn here with their Hands-On courses they have some excellent courses covering the fundamentals of statistics and these lessons do a deep dive on how to think about and analyze data correctly they even use python in their statistics courses and will quiz you on how to correctly analyze the data within the language their guided lessons will challenge you but you also have the ability to get hints or even Solutions if you need them it's really tailored towards understanding the material they've also recently released a programming with python course and they even have a coding environment built into their website so that you can run code directly in the browser and that is a great way to complement watching my tutorials because you can apply what you've learned in their active problem solving environment and that helps to solidify that knowledge so to support my channel and learn more about brilliant you can go to brilliant. org CMS to sign up for free and also the first 200 people that go to that link will get 20% off the annual premium subscription and you can find that link in the description section below again that's brilliant. org CMS okay so I think that is going to do it for this video I hope you feel like you got a good idea for how these animations work and how you can use that to plot real-time data I definitely think this is one of the coolest things that we can do with matplot lib it opens up a lot of possibilities in terms of how we can plot data from apis or sensors and get immediate feedback like this now in the next video we're going to be going over subplots so far in this series we've simply been creating our plots with the p plot object and that works great for what we've done so far but if you want additional plots or subplots then it's best to create your plots with the subplots method so definitely be sure to check out that video but if anyone has any questions about what we covered in this video then feel free to ask in the comment section below and I'll do my best to answer those and if you enjoy these tutorials and would like to support them then there are several ways you can do that the easiest ways to Simply like the video and give it a thumbs up and also it's a huge help to share these videos with anyone who you think would find them useful and if you have the means you can contribute through patreon and there's a link to that page in the description section below be sure to subscribe for future videos and thank you all for watching