[ML Coding Tips] Separate Computation & Plotting using locals

hi there so today I just wanted to bring you a quick coding tip that I often encounter in my daily machine learning researcher life that might not be super common in let's say traditional software engineering or

elsewhere so often I have a bunch of let's say of a bunch of models right and I use these ipython notebooks or colapse to analyze my data plot things and so on so I have a bunch of models let's say they're called M one and M two and I usually run my big jobs on a cluster so let's say I have run some jobs for one for each model and have some logs that I want to analyze so I'll go through my models and I here I you know load a bunch of logs and I'll also compute some stuff some statistics and some just things right that I wanna what I have computed and maybe these things are called a and let's this and B right so now I've computed these things now I want to analyze them and let's short cut and say printing is plotting so think of these this might be numbers right and now I want to plot them here out let's just print them which I do this now every time I want to you know change something here in my printing maybe I am I want the separator to be that I'll have to load all the logs and compute all the stuff right each time I run this cell which is not super cool right so I usually would like to factor out the plotting and stuff like this from the computation so I could extract one

of them into like a function but then at the point of these notebooks is that I can run each of the cells and they'll run right there right so functions aren't really cool in these notebooks so what I'll usually end up doing is you'll have some second loop in here right and let's see you'll have some data up here and you here at the end you'll something say something like data for this model is a and B or something like this right and then down here the first thing I do is I get my data and then I'll unpack again so da DB either a long pack or I'll just address them in dictionary notation like this and then I can do my plotting right some people use tuples here right they could just go a and B but then you'll have to do this unpacking the problem is now if I want to add something here I write I compute something new I now need to add something here right I need to remember to store it in the data array and then I need to here remember to unpack it in the same order right and then I need to produce to put it in the plotting

right so this is very cumbersome this line here and this line here they're very because you kind of duplicate your own variable names all over the place just because you want to compute them here and use them here a software engineer would usually tell you let's do something like a data class or in its most simplest form is say a class and you know there are multiple ways of achieving this but let's just do a and B here this is more probably the most verbose you can also do name two posts outer classes data classes and so on but ultimately you produce a class like this and then here you say this is a data class a and B and then here down here you can at least address them like this right and you don't have to do the data dictionary notation or remember the the order but now again if you want to add the C no you know not only do you have to add it here but up here and you have to add it here and beware if there's a doc string and then you can use it here right this is just I'd is too cumbersome so here it is a trick and please only use these in like notebooks like this will lead to so much memory problems and everything and if you work with a software engineer you might have to get them chocolate for doing this but alright so what you

can do is you can save your local variables in the in this dictionary right so this is a built-in that saves the local variables I'm sorry that gives you a dict of the local variables here are ABC m and all a bunch of other things right it's not clearing these ipython notebooks they have a bunch of stuff around them such that the meaning is slightly different depending where locals is but in essence you'll get the a B and C what you want now the locals dict refers in order for this to go through the local stick refers of course is the same objects not newly constructed so we're safe here to make a copy of that right like this and then down here we simply retrieve these local variables and update the current local variables using the local variables we stored and voila we can use a B and C without ever having to find them right if this doesn't work that

means that the locals here are is a copy basically and this is a python optimization and i read i don't have to use it because in these ipython notebooks it appears to work but you might want to have an empty exact statement here such that the Python interpreter can never be sure which variables are created in here and therefore can't optimize away so that is a horrible trick again don't anything serious upon this but it is very duper super easy you can just add any variable here and then use it down here so easy right and so wrong alright this was it I hope you enjoyed this I want to bring these kind of tips every now and then as an intermix into the research papers I hope you enjoy this bye