The Hidden Mechanism Behind Clean Python APIs (Descriptor Deep Dive)

If you've used something like add property in Python, you have used descriptors. Today, I'll dive into what descriptors are. I'll show you a couple of cool things you can do with them. And you'll also find out why sometimes assigning to the dunder dict of an object changes an attribute and sometimes it doesn't. I have a mostly empty main file here with a class called thing. There is a main function where I create an instance of that class and then I simply print it. And when I run this then this is what I get as a result. And now I'm going to add another class called demo prescriptor. And this is going to have two methods, a donor get and a donor set method. I'll explain in a minute what these do. So we have a get and let's say this method returns an integer and we're simply going to print something here and I'm going to return the magic number 42, of course, without a dot. I'm also going to create a set method. And let's say this also gets an integer as a value. It's not going to return anything. Going to copy over this print call. And that's the only thing I'm going to do here. Then in my thing class I'm going to create an attribute X which is a demo descriptor instance. And now let's look what happens if I print t dox like so. Let's run this again. Now you see this is being printed and we get back as a result 42. We can also do this. Let's try that again. And now, as you can see, set is called with value 10. Now, of course, it still prints 42 because we're not uh storing any information here in the demo descriptor. But of course, what we could do is add an initializer where self dot value equals 42. And then here we're going to return self value. And in the set part we're going to do self do value equals value like so. And now when we run this again, you see that now it actually prints 10 as expected. You can even do this and when we run it like so you see we now also get the value 10 which is a bit weird. Now what is actually happening here? A descriptor is any object [clears throat] that implements one of these methods. Get, set, or there's also delete. And that is then stored as a class attribute like what you see right here. And if you look at what is actually passed here, we get the instance and we get the owner. And if you see what it prints, you see that the instance is the thing object that is actually uh this object that we created right here. and the owner that's actually going to be the type. So as you can see this prints the class name. Now set when you call that this also has arguments. So there is also an instance object and in this case a value. Well that's the same thing. So instance is the uh object that the thing belongs to. If you access the attribute like this then actually the instance is going to be none. And that's also what you see here. So we calling get with instance none. The owner is still the thing class. Now like I mentioned it's a bit weird that you get back this value of 10 cuz that's you know something that you would expect to be part of an instance. So what you will typically see and implementations of these descriptors is that there's a check here whether instance is some object or whether it is none. And if the instance is none then we're going to return self which is the uh descriptor itself. So if we do this, so then of course this needs to be another type as well because this can also return uh demo descriptor like so. And then when we run this again, so now you see that when we call get with instance none, in other words, we call thing. x, we're actually getting back the descriptor itself. And this is actually the behavior that we want because if we have class level access like uh this then that's something that you can use for introspection and frameworks actually depends on this. It also prevents trying to compute a value with

no instance because we're handling the special case at the start and also this makes sense from a Python point of view. Basically if there is no instance there's no associated value. So we simply directly return the object that it refers to which is the demo descriptor instance that we have here. And what I find most interesting is not just how a feature like this works but how you can use it to improve the design of your software. Like too often developers learn some particular feature but they don't learn the deeper design thinking that's behind it. And that's exactly why I am building something new called software design mastery. This is much more than just an online course. Goes very deep into software design and architecture far beyond what I can cover here on YouTube. It's the most comprehensive thing that I've ever created. Now, if you want to be the first to know when the doors open, join the free waiting list at iron. co/mastery. Link is in the description. Now, let's go to a slightly more complex example. I'm going to build my own property decorator. So I have a user class here which has an initializer first and last name that I'm simply storing and I have a full name method that returns this. So I have my user here that I create in the main function and then I call that method. So when I run this then this is what we get. Now what I want to do is I'd like full name to be a property. Now of course you can use the built-in property decorator. That's not the point. For fun, let's build this ourselves and see how it actually works. So I'm going to create a class here called let's say simple property and this is going to have an initializer and this will receive a getter function. So that's a callable. I don't really care about the types of the callable, so I'll just write any. And it's going to return none. And of course, I need to import that like so. And the only thing we're going to do here is store the getter function like so. And in essence, what we're doing here is creating a sort of decorator. So I can now write simple property on top of this method name and it's going to get as part of the argument here uh this particular method which it will then store as an instance attribute. Now with the descriptor we can control what happens when we read from this and that is by implementing the get dumber method. So like before this is going to get an instance which is an object or it's none and we're going to get the owner and this will return let's say any we don't really care about it. You can probably make something a bit more neat with generics. I'll leave that up to you. Now first thing we need to do is if instance is null we're going to return self. So we just return the descriptor if there is no instance just like before and if not we're going to return self. fget and we're going to pass the instance. So this makes sure that when we call get it will actually call the property and return the result of that call. And now what we can do is instead of calling it like this I can call it like this. And if we run that then you see we get exactly the result that we want. And of course just like before I can also print user full name. And if we do that let's run that again. Then you see now we get this simple property object. So we simply get the descriptor back. Now I want to go back to what I mentioned in the beginning of the video related to the dict object. Now this object typically contains the attributes of an object. So we have here as an experiment I have a class called non-data which is a descriptor. So it has getter and uh the only thing it does is it returns a string values just for uh printing what is happening here. And as you can see I have my class a which has an attribute x of type non data and it contains the descriptor instance like so. So you would expect if I call a. x X it's simply going to return this right and that is exactly what I'm doing here in the main function but then what I do is I assign another value to the dict object with the same key of X and I'm going to override it with that and then what happens if I print a dox it's going to be shadowed by that assignment basically the descriptor will be replaced by this string value directly and let me comment

this out because we'll address that later on. But if I run this particular file, you see that the first thing that it prints is from descriptor. And that's because I create the instance and then I simply print a. x. But then if I replace it, well, this overrides the descriptor. So the descriptor is then no longer used. But there's a second example and this is the interesting part. So I have another class here which has a get and a set. So the get is the same as the example we had before. But the main difference here is that descriptor also has a set dollar method. Now what happens if I create a B object which has an instance of this particular descriptor and then I do this particular assignment. Actually it doesn't overwrite the descriptor. As you can see it simply says from descriptor. Again, when a descriptor has a set dumber method, it's a data descriptor and it takes precedence over the instance dictionary. So, as you can see, descriptors are a nice feature, but there's a couple of things like that to be aware of. Now, finally, before I end this video, I want to show you a few cool designs with descriptors, and you may even recognize these from existing packages or libraries. So, here is one example, which is uh validated fields. So let's say you want some kind of reusable validation across many classes without copy pasting properties or stuffing validation into a dunder init method everywhere. So here's how you could set that up. So I have a validator type which is a cable that gets a string and anything else and returns none and the string is actually in this case the name of the field and any is the value that needs to be validated. And I have a couple of functions here. So this function checks whether a value is non-mpy. So the first thing I do is check if it's a string because we expect this to be a string and then I check when I strip any whites space characters that this is not empty and if it is empty I raise a value error. Here I have another function that checks the minimum value. So I get an integer and this is actually a higher order function. So uh with minimum value you provide the actual minimum that it needs to be and then it returns a validator function with a field and a value. It just checks if the value is less than that minimum then you get a value error. So what you can then do is create a descriptor and this is a validated field class. It's a generic class. So it validates a field of a particular type t. It has an initializer where it gets some data like a type to optionally cast to and a tuple of validator functions. That's the type that I defined above. Then we have a couple of smart thunder method. The first is set name which I'm using to kind of learn the attribute name automatically. So I don't have to set it manually. And the way that it works is that it stores this in storage name field. Then we have the actual descriptive part. So get actually simply uh gets the attribute uh belonging to that particular name that we stored and set actually does the cast to the particular type and then goes through this tupole of validators and then calls each of these functions and then in the end it's going to set the value that it got back. So what you can then do is have a customer class with a name and an age. Uh these are then validated fields for a string and an integer. And there we're going to pass validators. So the name should be non- empty. And I want the age to be minimum 18 years. And now what we can do is create a customer with particular settings. This is just a random age. Don't pay any attention to that. And then we can try a few things like setting the name to some empty space which is not allowed or setting the age to 17 which unfortunately is also not allowed. And then when I run this we see that well the first one that works fine but then we get an error. Name should be non- empty string and age should be 18 years or more. So this is a cool example of how you could use a descriptor in a system with validation functions. You can add any list of validation functions here in the tool. It's really neat. Here's another example of something that you could do. And this is a lazy property. Again, also something that's already present in Python, but it's still cool that you could actually build something like this yourself. Uh, in this case, let's say you have something expensive you need to compute. You don't want to compute it every time you access it. So then you could have this lazy property class which builds on the simple property that I showed you before. So this has a getter and the only thing it does is that it stores a computed value. So that's what's happening right here and it just checks before like hey if uh there is already a value like that then we're simply going to return that directly instead of computing it. So it won't call the

function and then what you do it's just a simple property then you use it as a decorator just like before and then uh you can uh compute something that you want right so here uh compute some revenue by country it doesn't really matter but then uh I have some data I create a report and then I call that property twice and as you'll see when I run this print statement is only done once because the second time the value is cached There we go. By the way, if you're enjoying this video, give it a like. It really helps me out. And if you want to learn more about Python software design and how to write great code or direct your AI code monkey to write great code, subscribe to the channel or let your AI code monkey subscribe to the channel. That's also acceptable. Like I've shown you today, descriptors are the mechanism behind the property decorator and a lot of declarative reusable field logic like validators. I think it's a really cool feature of Python and it's behind a lot of the magic in Python frameworks and libraries. But now I'd like to hear from you. Are there other places where you've seen this type of descriptor mechanism being used? Would you ever build a descriptor yourself? Let me know in the comments. I talked about properties quite a bit today, but when should you use a property actually and when should you just use a method? I did a video explaining how I make that decision and I think you're going to find it very helpful. Check it out right here. Thanks for watching and see you next