Async or native thread? | Learn Rust part 64

When should you use threads and async programming in Rust? In this video, I'll explain when you should use one over the other and the limitation of threads. Let's start with threads. When should you use it and what is the limitation of using native threads? Native threads are good for CPUbound computations. If your programming is running slow because there are some computation that uses the CPU for a long time, then you might be able to make the program run faster by spawning native threads. Maybe your program spends a lot of time using the CPU to calculate some stuff. If this calculation can be done in parallel, then spawning a native thread is a good choice. The limitation with native threads is that it uses more memory than if you were to spawn a thread using the async programming model. The other limitation is that there might be a OS thread limit. The number of native threads that you can spawn is limited by some setting inside the operating system. These limitations with a native thread might make you consider using the async programming model. When you spawn a thread inside async programming, it doesn't spawn a native thread. It spawns a virtual thread that is managed by the async runtime. These are called green threads. Because these green threads are not native threads, they're not limited by the operating systems thread limits. Okay. So, what is async programming useful for? Async programming are useful when you need to spawn many threads. For example, thousands of threads. AC programming is useful when the bottleneck of the program is IO bound. For example, here IO computation means reading or writing from a file or sending a HTTP request and waiting for a response. In this case, your program is slow because it needs to wait for some data to arrive. It's not a CPU inensive computation that is slowing down your program. In this case, async programming is a great choice. So now let's look at examples of spawning native ths and then switching over to async programming. What I'm going to show you is that when we spawn thousands of native threads, your program will eventually crash. However, by switching over to the async programming, your program will be able to execute spawning thousands of threads without crashing. So, let's start by trying to spawn a million native threads. I'll begin by creating a for loop or in zero to a million and then we'll spawn a native thread using the standard thread module. For this example, we'll spawn a thread that will sleep for a few milliseconds and then print this index i. Let's say that this thread is going to sleep for 100 milliseconds. And then we'll print this value for ike. Now, when we spawn a thread, it returns a join handle that we need to call join on so that the main function will wait for the thread to finish execution. So, let's say that handle is equal to get the join handle. And now let's put this into a vector. Let's say that mute handles is a vector. And then let's store this into our handles vector. Okay. And then next we're going to run a for loop to call join on all of these handles. So notice that here we're running four loops. The first for loop spawns the threads. And in the next for loop we're waiting for the results of the thread to finish. So you might be asking why did I not write the code like this? just call join over here? Well, if you do this and if you didn't have the second for loop, then the code will mean a different thing from what I want to show you here. It's saying that spawn a thread and wait for this thread to finish execution. Once it is done, then spawn another thread and then wait for that thread to finish execution. But what I want to show you here is that I want to try to spawn a million threads concurrently and then start executing all of these code. And then inside the next for loop, I want to wait for all of these threads that we spawn to finish execution. So the difference is if we call join here, we're only spawning one thread at a time. However, by putting the join handle inside a vector and then calling join over here, we're trying to spawn a million threads concurrently and then wait for all of the threads to finish execution. Okay, let's execute the code. So what you're seeing is that the native threads being spawned and executing the code. And at some point you'll see a error that looks something like this. This is basically saying that I spawned too many threads and it hit the limit for the number of threads that I can spawn inside my operating system. So this is an example of what can go wrong when we spawn many native threads. The number of native threads that you can spawn is limited by the operating system. But this limitation doesn't exist when we convert this code into async programming. So to do this, I'm going to first copy this code, paste it here, and then I'll comment out the code that spawns native threads. We're going to convert this code into code that spawns async threads. First, replace this code that spawns native threads into a code that creates a future. So let's say let future is equal to async move. And since this is not a closure, we'll remove the vertical bar. It's simply creating a future. Then change the print statement to async thread. And

now instead of calling sleep on the native thread, you'll need to change this to calling sleep on the async thread which is provided by Tokyo runtime. Tokyo time sleep. This will return a future. So we'll need to await on it for this code to actually wait for 100 milliseconds. Now currently this code simply creates a future. Remember to execute a future we need to call a wait on it. Now if we call future await here then the code means to run the for loop 1 million times and for each iteration create a future that will execute this code. Wait for this future to finish execution. But what we want to do here is spawn 1 million async threads concurrently. Execute all of them at once and then wait for all of the results to complete. So instead of calling future a weight, what we need to do is call spawn on this future. We'll do this by calling Tokyo task spawn and then passing in our future. Calling the function spawn will execute the future immediately. So here we don't need to put a wait. Now similar to the case for native threads, the main function will need to wait for this spawn future to finish execution. To do that, we need to do something similar to calling the function join on the handle. When we call the function spawn, it's going to return a handle. That handle. Let's also store this in a handles vector which I declared over here. It's an empty vector and for each iteration we get a handle and then store that handle into our vector. And then inside the next for loop we'll take each handle and then wait for these handles to finish execution. We'll do that by calling a weight. A wait will return a result either okay with some data inside or error. We'll simply unwrap it. make sure that it didn't return any errors. Okay, so now we've converted a code that uses native threads to spawn 1 million native threads, execute them concurrently into doing something similar using async programming. Let's now execute the code. Okay, the code executed to completion without any errors. This was an example of spawning 1 million threads using async programming. And finally, let's go over the key points. Which one should you use? Should you use native threads or should you use the async programming? The answer depends on your use case. If the bottleneck of your program is a CPU computation and you only need to spawn a few threads, then using native threads is a good choice. On the other hand, if the bottleneck of your program is a IO computation or you need to spawn millions of threads, then the async programming is a better choice.