Catching up with async Rust

This video is sponsored by Brilliant. Hi everyone, I'm back with a video that is also an article that is available on my blog right now if you're a patron 10€/month or above, and if you're not, then you get to either watch this video right now for free or wait six months to get access to the article. [feisty music] [with like, percussions and stuff] In December 2023, a minor miracle happened. "async fn in trait" shipped. As of Rust 1. 39, We already had freestanding functions like this, and we had async functions in impl blocks like this, but we did not have async functions in traits. If you tried it up until Rust 1. 74, you had an error message. For the longest time, the async-trait crate was the recommended way to have async fn in traits. And it worked, but it changed the trait definition and any implementations to return to pinned, boxed futures. Boxed futures are futures that have been allocated on the heap because you see: futures, the value that async functions return, the things that can be polled, can be of different sizes. The future returned by this function is smaller than that function because bar simply has more things going on, more state to keep track of. That array there is not deallocated while we sleep asynchronously. It's all stored in the future. And that's a problem because typically when we call a function, we want to know how much space we should reserve for the return value. We say that the return value is sized. And by we, I really mean the compiler. Reserving stack space for locals is one of the first things a function does when it's called. Here, the compiler reserves space on the stack for step 1, _foo, and step2. As seen in the disassembly, that's a third party subcommand of cargo. We see the very first thing that happens in the function is sub sp, #256, that reserves 256 bytes on the stack. Why sub? Because the stack grows down on ARM64. Makes sense. Although there's nothing forcing the compiler to do so, all the locals in this code are laid out next to each other. And so we can predict what it's going to print, the distance on the stack between step1 and step2. And you might think it's going to be 128 because that was the size of the future returned by foo, but that would be a fence post error. We have to take into account the size of step1 And so it actually prints 136. Our stack looks a little something like this. Back to this AsyncRead trait. If we somehow got a hold of an &dyn AsyncRead, a reference to a trait object of type AsyncRead and called read on it, how would we know how much space to reserve for the local foo? Well, it depends on what the actual concrete type is behind the trait object. So you could imagine that you could make the size of the future part of the trait. You just call the first method and it lets you know how big the future is going to be. Then you reserve that on the stack and then you call the actual read method and then you pass it the address of the member you just reserved. And I'm pretty sure that's how unsized locals work. roughly what the longer term plan is. But for the time being, you cannot do that. The only way to hold an arbitrary future is to box it. Even though in this code sample, these are two different features of different sizes, as we've seen, the locals are the same size because it's just a pointer to the actual future. And a pointer on 64-bit is just 8 bytes. But of course, this actually prints 16 bytes. Why? Well, let's use a debugger and find out. If we set a breakpoint right before the end of main, we stop on the second println. LLDB has limited support for Rust, but we can still print the local foo and see what's in there. And we see two pointers, one to pointer, that's the data, and one to vtable. And if we examine (x), bi"g" values as hexadecimal, x slash 8 gx, that's borrowed from GDB, The fourth one is actually the address of our async function. foo, or rather it's the address of the closure in our async function, but that's just an implementation detail of how the compiler de-sugars our async code into machine code. Other fields in the vtable point to other functions. Typically one of those will be the Drop implementation for a given type. It's important to know how to free a value we hold, even if we don't know statically what type it is. And I just want to note that not all boxes are two pointers, only boxed trait objects. If you take a string and then you put it into a box, so it's like double heap-allocated, the box is only 8 bytes, it's just one pointer because we statically know what the type is. But if you convert it to a Box then that box could hold anything that implements display. So you have to point to the data, to the string itself, and then point to the vtable that has a list of functions, including one that knows how to display the string. And that is kind of what the async-trait crate does. It takes a trait definition like this one, and it turns it into this abomination, which is much easier to read in article form. Again, if you're a patron, 10€ or above, you can go read it on the blog right now. The important part of this is the return type of read, it's a Pin

But let's forget the async-trait crate for a moment, because as of Rust 1. 75, there is native support for async functions in traits like this. We can implement that trait on any type, since we've defined the trait ourselves, so we can implement it on the empty tuple, for example, and just put garbage in there just to make the future a little big. And we can notice that this time the futures aren't boxed, as opposed to the async-trait crate, because the local "fut" in this example code is actually not 16 bytes, it's much bigger. So that's it, right? Everything is solved. Well, no, not everything, because do you know what this prints? if you have an `r` of type Box and then you call r. read on it, what's the size of the future it returns? It's not 224 bytes, as we've just seen, because that's just for the empty tuple, that's for one specific implementation of the async read trait, not for all of them. Our parameter could be any type, any type that implements async read, so we have the unsized locals problem again, and this is actually a compile error, you cannot box that trait. In the future, maybe the size of futures for async methods will be part of the vtable for trait objects, and then async fn in traits will be dyn-compatible, something that used to be called object safe, but was renamed, because object safety was a terrible term for that. But for now, async fn in traits are not dyn-compatible. There are many other things that are not dyn-compatible, taking self by value, for example, is not dyn-compatible, because you would have to know the size of self statically, but you can take Box, because a Box is going to be the same size, no matter what self actually is, no matter what the concrete type is in the box. That's just to say, I see a lot of complaints about async Rust being like a second language within Rust, but no, this is a problem that you have in regular Rust. When you want something to be dyn- compatible, you have a bunch of constraints, and some of them might be lifted in the future, but for now, you just have to play by the rules to achieve dynamic dispatch. And just because our trait isn't dyn-compatible doesn't mean it's impossible to work with. We can take an `impl AsyncRead` for example, that's fine, because it's really just shorthand for having a generic type parameter on the function. We can also take a reference to an impl AsyncRead, or an immutable AsyncRead but we cannot take a reference to a dyn AsyncRead, we cannot make a trait object, because something in our trait violates the current restrictions of dyn- compatibility, and that thing is that it has an implicit associated type. Because when we have an async fn in trait, we're really saying that we have an fn that returns something that implements Future. And we've seen before that impl trait in argument position translates to a generic type parameter like this, and when it's in return position, it translates to an associated type parameter like that. In fact, on Rust nightly, we can implement that trait pretty easily, as-is, this full program compiles and runs. Again, this is easier to see in article form, you can just copy and run it. This pattern should be familiar to those of you who have spent quality time with the tower crates, via the hyper crate for example, their Service trait has a `Future` associated type. You can see the full trait definition here. And when implementing Service, you pretty much have three choices. One, implement future by hand to avoid heap allocations. Two, set the associated type future to a boxed future, or three, require Rust nightly and opt into the impl trait in assoc type feature. As of Rust 1. 75, one could imagine a simpler version of this trait, like so, just making the call function async. Implementing that trait would be comparatively trivial, here's a no-op service, you don't have to worry about the type of the future, you just have an empty function, returning Ok(()). And here's a Service that logs any incoming request, it requires that the Request type implements Debug, it just prints it, and calls the inner service and it waits. None of those futures are boxed, and this all works on Rust 1. 75 stable. But this currently comes with several limitations, and we're going to get into them. To start with, we can no longer name the return type of Service:: call Some tower services rely on this, for example, the built-in `Either` service that can call either the left service or the right service. And this scenario, to be able to define the associated future for Either, you have to be able to name the future type for A and B, and you just compose those together. But this scenario is not actually an issue for our simplified Service trait, because we don't need to name the parent's Future trait either, so we just don't need to care, and we get this much simpler and more readable code, it almost even fits in a video. This is honestly a win for native async, fn in traits. Things get complicated when it comes to specifying additional bounds, like lifetimes or Send. You know what's not complicated though? Today's sponsor, Brilliant. This is a book. You can turn pages forward, back, you can use a bookmark if

you want to get fancy. But you know what it doesn't have? Interactive exercises. Aww yeah. When's the last time you saw a book about unlocking rental value in AirBnB? I mean, there probably is such a book, I've just never seen one. Brilliant is honestly one of my favorite platforms. I just love all these interactive thingamajigs. They fit my brain better than textbooks or regular lectures. What you're looking at right now is the data path, because knowing is half the battle. The other half is developing a daily learning habit, which I hate to admit, but those mobile apps are actually very good at doing. There's so many little dead moments in your day where all you have is a phone and some time, so why not do yourself a favor and go to brilliant. org/fasterthanlime to get a free 30-day trial of everything Brilliant has to offer. You'll also get 20% off an annual premium subscription. Let's have a quick refresher on lifetimes. Rust famously differs from other languages in that it has you annotate the lifetime values. A function can return something that borrows from the input, but you have to say so. Like this. In this time, the lifetime annotations could be elided, we could just not name it because it's just one of them, but I'm writing it that way so that it's very clear how the data flows. In this example, t borrows from s. You can tell by looking at the range of addresses they both refer to, you can see the nice little overlap. Because we get the compiler, that information can prevent us from doing something dangerous, like using t after we freed s. That would normally be a "use-after-free", but in Rust, that's just a compile error, which is fantastic. You can also return something that doesn't borrow from the input, we just have to say so. Here the return value is owned, and our use after free is no longer a use after free, we can call t function and then free s and then use t. That's fine, because t now points to its own memory, which again if we print the address ranges, we can see there's no overlap this time, they are pointing at separate areas of memory. Async functions add another layer of complication, because they can be in a partially executed state. Sort of. Earlier, we equated this with this, our simplified Service trait. But they're not actually equivalent. See, with a classic tower Service trait, the returned future cannot borrow from self. This implementation of it for the type i32, which accepts Request of type i32 and adds itself to it, doesn't compile. We cannot have the returned future borrow from self, we need the returned future to be static, to be owned, to have potentially infinite life. There is a way to make this work, and it's to make the associated type generic over a lifetime. So now type Future has a lifetime parameter, it also has the "self must outlive 'a" constraint, that's just a limitation of generic associated types, GATs, which were introduced in Rust 1. 65, and explains why they're not used by tower's Service types, they just didn't exist back then. If the Service type did use GATs, as just shown, then we could borrow from self, like so. But we can't, because that's not how the tower Service trait is defined, it wants a future that does not borrow from self, and so we have to do whatever we need to do with self before returning a future. In this weird contrived case, this means that we need to dereference self before the async block. And again, this isn't stable, we're using feature impl_trait_in_assoc_type. By comparison, our simplified Service type works on Rust 1. 75 stable and allows borrowing from self, no problem. But this is a breaking change, being able to do service. call several times in a row, obtaining several separate owned futures and spawning them on an executor is a feature of the tower Service trait. With our simplified service trait, we cannot have multiple concurrent requests, which is a bummer for an HTTP server for example. And that's exactly why currently Rust warns you if you have an async fn in a public trait, because with that syntax, you're not able to specify additional bounds. Say we wanted to make our simplified service trait closer to the original tower trait. We could see that the return future ought to be 'static, it's not allowed to borrow from its arguments. But then our implementation breaks, and from the diagnostic you can see here, it's not exactly clear why. I don't believe it's possible to solve that issue whilst sticking with the async fn syntax, we have to switch to returning an impl Future via an async* block in the implementation as well. With this change though, we're able to have several requests in flight concurrently, which again is kind of the point of tower and hyper. It even looks like we're able to spawn them on the tokio runtime, but this doesn't make sense because tokio:: spawn requires send. It has multiple worker threads that just steal from the queue some work to be done, and so futures just bounce between different threads and they have to implement the `Send` trait. That's what indicates that it's fine to send them between different threads. And there's nothing currently in the definition of `Service` that requires that the return future implement `Send`. You know what's happening here is that the compiler from this sample code can actually see through the interface, you can see the concrete type, which is just i32, it's definitely safe to send an i32 from one thread to the other, but if we add

something, if we capture something from that async block that isn't `Send`, like an Rc, then we get the compile error that like the captured value is not `Send`, so the future you cannot pass it to tokio:: spawn. We also could have seen this problem if we hid the concrete type from the compiler. If we use the generic function that accepts any service and then called call from that function, then we would have gotten the same error. It would be like "there's nothing in the signature of "`do_the_spawning` that" "says that the future returned by the" "service is send, but tokio::" "spawn requires that, so i can't" "prove that this is fine, so here's" "a compile error. " We can add bounds, there's not enough bounds, you can see that the Response itself needs to be Send and Debug and 'static, and the Error also but unlike the tower Service call, there's nothing we can do here outside the trait definition to say that we want the future to be Send. We have to change it in the trait definition itself, we have to change the return type to be impl Future blah blah + Send, that's the only place where we can actually have it. Our trait is strictly less flexible, less versatile than the original tower Service trait. [also bird chirps] So this is pretty much where things currently stand in terms of async Rust, as I'm making this video, so depending on when you watch it, when you read the article, it might not be up to date. I encourage you to go check the Rust blog, or go check areweasyncyet. rs The async work group has also been releasing a couple of crates to try and give you a taste of what could happen in the future. Ultimately, the big thing I'm waiting for is "dyn async traits", because that will actually allow us to do all that without boxing everything. For those of you who don't know, I have a podcast. I've had a podcast for months now. I'm doing a podcast with James. Each week we take turns giving presentations about something that we're interested in. James is more doing embedded hardware stuff and I'm more doing web... devops stuff. James: "If you can segment your" "product in a way where" "a little bit of lag on the user interface" "isn't going to cause" "problems with your brakes," "then it's way easier to not do everything" "in the super safety" "critical world. So exactly" "this kind of thing. Yeah. " Amos: "And this is why" "I like that you're my" "co-host, James. Because" "people get scared. Oh, they hear about" "Kubernetes in the cars or" "whatever. " Huge thanks to our producer, Amanda, who's making all this possible. Go listen to the podcast. If you've been missing my voice because the video output has not been regular, listen to that. That's it for today. Thanks for watching and I'll see you next time. Bye. [piano music] [that's me playing btw] [and sherlock on my arm] Have yourself a merry little christmas Pop that champagne cork Next year we may all be living in New York No good times... [fade-out]