10 Python Features You’re Not Using (But Really Should)

Here's a CSV file with millions of sales records. And here I have a function that reads the file and computes the total sales amount. Now, when I run this, you see that every time I read this file, if I need it in my code somewhere, it's going to take time. I mean, it's not that bad. This file actually reads like 10 million records. So, I'm kind of surprised it works this fast. But every time you do it, you actually have to wait for the dis IO. This is what happens when I add caching. As you can see, the second read happens instantly. And what's really cool is that this is really easy to add in Python. In today's video, I'll walk through 10 features exactly like this. Really powerful and really easy to add in Python. Now, if you want to learn more about how to design a piece of software from scratch, I have a free guide for you. You can get this at iron. codes/design guide. contains seven steps I take when I design new software and hopefully it helps you avoid some of the mistakes that I made in the past. Guide the link is in the video

description. There's lots of features in Python that help you reduce code size, improve clarity, eliminate entire classes of bugs, and speed up expensive operations. And unfortunately, most developers don't know they exist or don't know when to apply them. And because they're part of the standard library, you actually don't need extra dependencies of frameworks. Python is really a batteries included language.

Now the first thing I already kind of showed you is caching. So in the before version of the code, we simply had this total from file function. There's nothing special here. Just uses pandas to read a CSV file. But if you want to cache this function, that's actually really easy to do. So in this case you simply import from funk tools the cache decorator and then you just write cache on top of the total from file function. If I now run the code then you see that this is what happens. So the second read happens instantly. You also see that it doesn't print reading the file anymore because this is actually cached. So it doesn't execute the function twice. So this is really perfect for expensive file parsing, large configuration loading, uh calculations that are expensive rendering templates or anything deterministic. You eliminate duplicated work and it's going to make your system much more responsive. It's great fit for uh CLI tools, ETL processes, any repeated batch operations. The second feature that I

use a lot is protocols. So if you take a look at this piece of the script. So what this does is that it takes a sale object that's actually a very simple class that just has an amount and a currency and a converted value and then it uses a rate fetcher to convert that sale amount into a different currency. Now the code contains these types of uh type annotations which is fine. Actually the Python interpreter ignores these at runtime. So they're mainly there for you as a developer to better understand the code. The issue here is that the value that's injected here is of type static rate fetcher and that directly ties this function to this particular class. So if you wanted to replace this with some other rate fetching method maybe you want to take it from a database or you want to use an external API. This is uh annoying because then uh the type doesn't match. You can't simply create another class that has exactly this structure and then put it in here. Well, you can actually your Python code will run, but you run into all sorts of uh type issues while you're working on the code. So, that's not so nice. What's nice about protocols is that it allows you to specify a sort of contract of what an object is supposed to be like when you use it. So what you can do instead of directly putting in this static rate fetch is actually create a protocol and that I'm going to import from typing and then what we'll do is I'll create a class that's called let's say rate fetcher and this is a protocol class and this is going to have a single method called get rate and we don't provide an implementation because this is a protocol. And now I can change the type here to a rate fetcher instead of a static rate fetcher. As you can see, there's no type errors here. It's all very clean. This works just as well. And now I could create another class and simply inject it here if I wanted to. For example, let's say I have a unit rate thatcher which simply always returns one. This can be something you might want to do uh in a test or something. And now what I can do is in the code where I actually call this. So that's happens right here. I can now input a unit rate fetcher. And that also works as expected. So let's run this and see what happens. So of course here the converted value is the same as the original value because we're applying a conversion rate of one. So in short this protocol defines a structural interface and any object that implements the methods in this case get rate is accepted regardless of inheritance. And this is really nice for plug-in systems uh test fakes dependency inversion and strategy patterns. You get a lot of flexibility without tightly coupling classes through inheritance chains and type checkers can now warn you if an object doesn't satisfy the interface. The third feature

I want to show you is replace from the data classes module. If you look at what happens in this convert sale function, you see that actually it gets the rate and then it stores the converted amount in the sale object. So it mutates the sale object that you pass as an argument and then it returns that as a value. Now, it would be cleaner if you would get a new sale object instead of mutating the existing one. Now, you can manually extract the data from the sale object and create a new one. But if you use a data class, there's actually a much cleaner way of setting that up. So, from data classes, I'm going to import data class and replace, which is something that we'll use later on. So first I'm going to convert this sale class into a data class. So this is going to have an amount which is a float. It's going to have a currency which is a string. And I of course need to type that correctly. And it's also going to have a converted value which is a float or a non value. And default this is going to be none. There we go. And now what we can do is make this a frozen data class so that we can't modify objects once we have created them. And of course now we have an issue here because in the convert sale we now try to assign a value to the converted uh instance variable of the sale object which is not allowed. But instead what we can now do is use replace. And this is actually really easy. I can just create a new sale object and that's going to be a call to replace. We pass the original sale object as an argument and then we assign the converted value to sale amount times rate like so. So this gives me a new sale object and then I can remove this line that's no longer needed and I can return that as a value. Obviously, you can also directly return uh this call. So that would make it even shorter. Would look like so. Replace produces a new object with some of the fields changed. In this case, there's just a single one. And this is much safer, easier to reason about, and makes undo and redo patterns also very easy to implement. Also fits naturally with event sourcing, business workflows, and pure function. So this is a really nice feature of data classes in my opinion. The next feature I want to

mention is pair-wise from iter tools. So here I have a function that computes sales deltas. So it takes a sequence of numbers and then for each of the pair of numbers in the sequence it computes the delta and then returns a list containing all of these deltas. So in this case I just implemented a manual uh for loop that goes through the entire list of numbers and computes that. But actually there's a much simpler way to do this. Namely by using the pairwise function from its. So I'm going to import this and then instead of doing all of this going to delete all of this code and I'm going to return a list comprehension of b minus a for a and b in pairwise of the numbers. And that's it. It's really easy. So this handles these type of adjacency algorithms very elegantly. It reduces mental load, removes uh off by one mistakes and works great if you're dealing with time series data logs signals other types of deltas that you might need. The next feature I want

to talk about is assignment expressions or the wall rust operator. It's kind of infamous. Uh some people like it, some people don't like it at all. I don't use it all that much, but when I use it, it can be really useful. So, here's an example of where you could typically use this. So, I'm reading some kind of large file in this function. And because I use a y loop in this case, I have to uh write this code twice. Unfortunately, Python doesn't have a do loop where you can kind of invert the uh the condition. So, in essence, I have to duplicate the read operation code in this case. What's interesting about the walrus operator is that it turns the assignment into an expression. So instead of doing it like this, I can delete this line. And then I can write it like so. And now I don't need to do this. Still need to add a column. And then that's all it takes. The warus operator simplifies code that reads, parses or streams. It can sometimes be really helpful in reducing repetitive code in loops. Now, like I said, I'm not using this all that often, but when I'm in need of this type of thing, it's very useful to have it as a feature in Python. Next up, and that's more of a

library than a feature, is path flip. Pathflip is actually really helpful. So, if you take a look at this function that loads all the JSON files in a particular directory. So, I'm using os. list there and then using that to uh read the file, constructing the path, etc., etc. So there's a lot of manual work that happens here and with path lip this is actually much easier to set up. So the first thing that I'm going to do is from path lip I'm going to import path and then what I'm doing here is replace this entire for loop by something that's much simpler. So I turn the directory that's passed as an argument into a path. You can also directly accept a path argument here if you wanted to, but I'll leave it like this for now. So turn it into a directory, but then you can call glob on this and search for all the files ending with JSON. And now I can simply load the data by calling JSON. loads path. ext. And path itself is also a very helpful object. So for example I could do results and then I can do path stem equals the data and now I don't need all of this like so and then I can simply return the result. It's much simpler. So as opposed to OS path lib gives you highle path objects with lots of these type of intuitive methods. It avoids string based path entirely which reduces bugs. files, directories, globs, writes, read, all work consistently and overall it makes your file handling code feel more Pythonic and modern. The next thing

that I want to show you is an easier way to suppress exceptions. So let's say we have a function that removes temp files. And here's what the body of that function looks like. So it just takes a temporary file and it unlinks it. And in this case, I'm also using path li and then the unlink method. And here I don't really care if this fails. If the temp file doesn't exist, I don't need to do anything. So I simply want to ignore that error. So to do that, I just put it in a try accept block, which is totally fine. But actually what you can do is use suppress, which makes this a bit shorter. And that is part of context lip. And what this gives you is a context manager that suppresses a particular type of exception. It's really easy. You just type with suppress and then the error that you want to suppress. And now I don't need this except part with the pass. So this is a much shorter way to write the code. And on top of that, I also find the intent clearer in that well we are explicit about that. We don't really care about this error. This is a great feature for tear down steps. optional files, retries and graceful fallbacks. The next feature that I want

to talk about is context vars. So if you take a look at this example, we have a process user function. This is all asynchronous. So this gets a user ID and a request ID. And here I have another function that's called handle request that also gets a user ID and a request ID. And then I'm calling these using asyno. gather. So these are launched basically at the same time and handled concurrently. Now this is a relatively simple example but if you have like a whole chain of things where you have to pass through the request ID through all sorts of different layers and functions then you add lots of arguments to the functions even though the function itself may not necessarily need it. So instead what you can do is import context vars which gives you capability of doing this slightly differently. So what you can do then is create a request ID. And I'm doing this here as a global variable. You can also define it elsewhere, but we're going to define a context var. And actually I'm going to do it like this. It's a bit easier. And then I can simply specify a name request ID like so. And let's say default this is going to be unknown like so. And now what you can do in your handle request function. So that's let's say the top layer where you get the request ID for the first time is create a token. And let me also rename this so that we don't get any conflicts. And then once we're done with processing the user, I can reset the context like so. and process user. I'm simply not going to pass the request ID anymore like so. And then here inside I can access the context var using the get method. And this then gives me the request ID which I can then use in my process user function like so. Now you might wonder why do I need to create a context var here and can't I just use a global string if I wanted this? Well no that doesn't work because we have multiple concurrent function calls. So that's going to lead to all sorts of conflicts and that's exactly what context file solves. So each of these async tasks gets its own isolated logical context and this is really helpful for logging tracing uh metrics or user session info that you want to keep track of and this is of course especially powerful in asynchronous or concurrent programs. The

next feature that I want to show you is match with guards. Now if you take a look at this function, this has like a bunch of uh if else statements. So if the amount is less than zero, we're going to return invalid. If it's zero, we'll return zero. If it's larger than 10,000, we return large. Otherwise, we return normal. So this is fine. You can imagine maybe if there are more cases like this, the if else statement becomes a bit more complicated. But what you can actually also do is use structural pattern matching to write this slightly differently. And what I like about this is that currently in the if else statement, we have to repeat the variable all the time. And you can actually shorten this a bit. So what you can do with pattern matching is that you write match the amount. Let me indent this. And then we're going to handle the cases. So in this case you can instead of writing this you can write something like this. So if it's less than zero we're going to return invalid. And then we can also do the other cases. So if it's zero we simply return zero. If it's larger than 10,000 we return that default we return normal. So in my opinion this is a really nice way to express logic like this. In essence, pattern matching gives you a declarative way to describe these cases and this makes it easier to write validation rules, classification, parsing logic, things like that. And as your logic grows in complexity, in my opinion, this is a bit easier to manage than all of these if else chains. The final feature

I want to show you is exit stack from context lip. So if take a look at this function that reads files, it gets a list of paths and then it's going to read them and it will return a list of strings. And as you can see, what I'm doing here is sort of manual uh opening and closing of the files trying to figure out if it exists or not. This is not really a great way to write this. But from context lib, we can import exit stack. And this allows us to do this more cleanly. So what I'll do here is create a context manager like so. And then files is going to be stack dot enter context. And then I can call open for n paths if pexists. And then what I can do is simply return f read for f in files. And then I can remove all of this. like so. So here I'm using opening files as context but actually exit stack lets you manage an arbitrary number of context managers dynamically and it can be different things. It can be a context manager for accessing a database or for doing other things as well. And this is really nice for uh when you don't know in advance which resources you're going to need. This simply collects all the entered context and closes them automatically. So, if you have some optional resources, maybe a plug-in architecture, a conditional setup, some multi-resource operation, this can be really helpful. I hope you enjoyed this

video covering some lesser known Python features. Give this video a like if you did and subscribe to the channel to watch more of my videos covering Python and software design. And I'd like to know, were you aware of all these features? Uh, which one are you going to use in your next project? And by the way, if you want to take this a lot further, make sure you check out my video on writing Pythonic code. This brings together many of the principles behind these features. Link is right here. Thanks for watching and see you next