Implementing Command Pipelining | Redis Internals

so radius has a very interesting feature called pipelining a typical way to fire commands to ready server is that first your client connects to the ready server right it's a TCP connection then your client fires the command server computes the response it basically does the needful and it sends back the response right so first you do set K comma B then your address response okay then a client sends get K and then it sends V right so here if I'm just setting a particular key and then getting the value for that particular key I'm doing a lot of round trips because a lot of packets needs to be sent across the network from your client to your server from your server response back to client then the second request then the response to that so here if it's within the same machine you would not see a difference because it's just local connections just localhost but as it goes in an infrastructure over the Internet maybe this round trip times become extremely critical right because this is the major uh basically cost that would be involved when your client is firing a lot of requests to reduce and your server is responding back so a lot of time goes into this round trip so can we optimize this is what pipelining actually does so the idea of pipelining is pretty simple that hey yo instead of just sending one command over the TCP Network what if I send multiple commands in one request so for example if I am firing three commands right so instead of sending First Command then getting the response then fire second command then get a response then for third instead of that what if I fire three commands back to back in one request so in one round trip I am able to send command one command two and command three and your server can independently execute command one then command two then command three and then in one time response it sends me back the result so for example if I do ping set KV and get K in response I'll get pong OK and V here one key thing to note it's a it is not a transaction it is just command pipelining in which you are multiplexing a lot of command in one request sending it to the server will execute them one after another and it would send you end responses so if you send n commands you would be getting n responses out of it so it is just about clubbing the request right it is not about transaction at all right so a server is independently executing those three like those n commands into pipeline it would compute n responses and would send you back and responses multiplexed in one response right so overall you just do one get or one request and one response in that one request you are multiplexing a lot of commands right so this is the idea of pipelining now with pi where you could say why do we want to do that but before we do that if you think about it when your client is sending multiple requests to the server now your server instead of evaluating one and then immediately sending back the response what it has to do is it has to buffer because your server can only send one response in that one response it has to send everything right so in request if I send three commands in response I have to send three responses right so which means that your server has to buffer the responses so if I'm sending three commands then a server is validating one output of which is buffered in memory then evaluate second output is buffered in the memory evaluate's third and then once everything is evaluated it is sending back three responses so here what would happen is it would lead to a higher memory consumption so if you are doing a lot of pipelining agreed you will get great throughput but it would have an impact on memory consumption so you have to be wary of that fact right but obviously my planning helps you improve throughput because now the number of operations that you can run are much larger otherwise a lot of time of your ID server goes into waiting for getting the request and then sending the response so a lot of time goes into that because system calls are blocking so overall commands per second that you are executing is substantially higher when you use pipelating right so for redis when and why so because when for redis because all the operations that you are doing they are in memory which means that Computing them would not take much time when it receives a command for it execution literally takes micro seconds if not nanoseconds right it would be much faster a lot of time of ready server also goes into making the network call sending back the response or receiving the response from that right which is

where your power of pipelining comes in with pipelining your redis has to do fewer contexts which from your in-memory processing than to network i o because now imagine if your pipeline hundred commands right so here instead of like get one command compute it or basically make the necessary changes and send it as a response so IO you are doing Network i o then doing some CPU processing or some basically memory uh input output and then you are responding back and then again the cycle come back for hundreds of requests instead if you are pipelining hundreds of requests in one request it gets it can quickly fire 100 it can quickly evaluate those 100 commands without having to context switch to network i o and then immediately send back this is the power this is why it betters the throughput right now that we understand what pipelining is let's quickly take a look at it how do we implement it in our implementation the implementation seems pretty straightforward but just few hunches here and there that would come our way right but before we do that it's always better to see how pipelining indeed works right so let me quickly do this where I'll show you like always top right is where our normal ready server is running and what I'm doing is I'll be putting in commands I'll show you how basically commands are fired how pipelining is done so that we know how to implement it right so here I am connecting to normal redis client not here but here bottom left as you can see that is CLI let me yeah okay ready CLI and I'm doing minus P Port 6378 because I'm connecting to normal ready server now if I connect to that now let me fire a command so here I'm sending ping and I in response I get pong right this is normal this normal command that I'm setting but with pipelining what happens is you have to send multiple commands in one shot so what we would do is we would be firing a simple netcat through netcat we would be sending the request or the normal bytes to the server so here out here how it would look like so in pipelining what we would do is we would be just doing this so you hear what I'm doing is I'm doing a printf it's a shell printf in which I'm passing command 1 command to command 3 pipe with netcat localhost 6379 so 6379 is our normal radius server right so here I'm literally putting command 1 after command to after command 3 and then sending this encoding this in bytes and sending it as a request to 6379 right this is what the pipelining is all about so in one request in one network packet or another Network packet but in one request in one i o call I'm sending this entire request in which it has multiple commands literal concatenated if I fire this I'll get some response because it is not proper but let me do this right It Centers some but it did not get any response because this was invalid let me put a proper command here so if I do this it would look something like this so here don't be scared of this it's normal that we have always written the resp protocol it's the exact byte that we are sending star 1 implies it's an array like you like remember when we send commands through ready CLI it is sent as an array of strings so let's say if I'm sending ping set K comma V and get K right if I'm sending that here you can see we are doing star one slash r slash n dollar four ping slash r slash n this is your ping command that you are sending right it's a bulk string encoding right this is how we are sending ping command then the second command that we would send is set KB right so set KV is three arrays or sorry the three element array so your first thing becomes star 3 right array of it three elements slash r slash n dollar three slash ration set slash R slasher so set and then dollar one slash r slash n k then it's slash are slashing then dollar one slash r slash n v slash r slash n this became your KV right set k v and then the third command is dollar uh star two slash r slash and dollar three g e t slash r slash n dollar one slash r slash n k slash r slash n right and I'm passing this to 637a here you see the response right so we passed in three commands back to back right so just one request that has went in and now we are getting in one

response we are getting output of three commands right so here output of First Command is per set K comma V is okay and then what we are and then when we fire the next request is get k which means we are because this is a normal Network protocol that is where you see dollar one then V it's basically dollar one slash r slash n v and which is why it is printed like this right so we send three commands back to back and in response we got three command or the output of those three commands right this is what pipelining is all about and this is what we have to make changes in our code base to implement this right okay now let's move to our code base and this is what our code base actually looks like up until now so what we would do is we would start somewhere we would start the fundamental thing is changing like earlier we wrote a code such that it was accepting in one network i o a lot one request and then you are sending one response to that right let's just quickly take a look at what would have changed so I'm in my async underscore TCP file and here if you look at nothing has changed up until now but here something has changed so our function read command which was reading one command because our radius here I was sending one command and we're getting one response right that's what we implemented that now changes to read commands right so we are accepting that same communication object and instead of getting one command I am getting multiple commands and in response I am passing instead of passing one command I'm passing multiple commands that is the fundamental change we are doing so first let's take a look at read commands so read command was written in sync underscore TCB file the first ever implementation that we had now here instead of sending one we are sending many so if you look at this I have created a new type called Ready cmds which is just an array of star radius command so the normal radius command that we had now changes to array of register so that's why I'm just naming it right it's just multiple commands that we are passing the idea here is pretty simple earlier we used to do decode one right instead of just doing decode one what we are doing is we are continuously decoding it the idea here is that we would want to accept multiple commands through that so which means that like we just saw when we sent a request we just got multiple commands back to back literal concatenated so earlier we were when we got a body right earlier where we got the value from the redis client or anywhere we just passed one object out of it we never looked forward now we have to right so that is where what we are doing is we are literally iterating through this so we are decoding we get values and values we are iterating through this and each value would be I'm converting it into array string that gives us token and then I'm appending it and I'm creating normal radius command out of it which we already did right we already created this and now these two aristic is doing nothing but accepting an error of interface and converting it into string nothing fancy here right so the idea here is I'm decoding everything that I have like here if I look at this the main function decode now changes earlier decode function used to decode that one value but now instead of returning an interface it is returning and slice of interface this function is that main thing that got changed so now what we are doing is the idea is pretty simple a decode function instead of returning one object it will be returning an array of objects right so for example if I am passing one because each command that we are sending from ready CLI to this is an area of string right and we have multiple search so it would be array of Springs right but how do we get that the idea here is pretty simple so here I'm just creating a values like a decode can iterate through instead of just decoding one value we are iterating through multiple such concatenated values until the data ends so here I'm creating values having some index decode one and decode 1 returns object the Delta like the number of bytes read right and error if any right remember we used it during encoding and decoding right third fourth video in which when we were understanding resp that same thing will be utilizing over here Delta becomes our friend so now what we are doing is let's say I am passing in three values back to back concatenated so I'm reading first value and then my Delta would be the number of bytes that I've written over here so I'll start from here and then read the second value then I'll start from here read the third value so my decode function instead of returning one object it will be returning multiple objects each decoded back to back and value is appending over here this is the main change that we are

doing right so decode was returning one object now it is returning an array of objects and then each object if we are doing pipelining this would be an array of strikes right so that's the idea behind it okay now that we have sorted out our decode function now instead of having one command we are getting array of commands and now this area of commands can now very well be sent for response this is where I've read the commands and in respond we'll do that exact same thing so respond instead of accepting one ready CMD it is expected it is now accepting multiple radius commands even if we pass one it would be just array of strings right so now we are doing this and I'm passing it to eval and respond now instead of getting one radius command it is getting multiple radius commands because so instead of having direct switch case we are putting it within the for Loop right so our main logic does not change it's just that the way we are consuming we were consuming the input now it changes we're just expecting one command now we can have array of commands right so now we are iterating through every single command evaluating it the way we used to but now instead of directly returning it we are adding it to a buffer so earlier the code that we wrote used to in the evaluation we used to pass i o reader writer i o read writer and then used to write to it directly right but now instead of that the eval functions that we have they would be returning the response output and this would be added to the buffer and at the end of this evalu and respond we would be sending it over the socket that's the idea now so the peripheral has changed everything else has remained the same right so now let's just take a look at how set would work right so just taking example of let me just go through all of them so I just created some helper constants that were there so that we don't have to re-implement it again and just copy like just use this variable instead of writing this complex part and rewriting it every time right now eval ping instead of returning and it used to return an error but now it is just returning a slice of bytes which is the actual response so now in case there is an error we are encoding the error and returning it over here otherwise everything else Remains the Same and output is bytes similarly for set error we are encoding the earlier we used to directly send error but now we are encoding the error and sending it then we come over here again encoding the error and sending it encoding the error and sending it but at the end we are just creating this new object and returning response okay right so here instead of doing socket i o we are just returning this slice of byte which will then be sent over the socket by eval and respond right that's the idea similarly all the changes have been made at every other level right instead of returning error we are returning encoded version of error which is a slice of byte encoded in resp format like this if it's an error we are doing minus percentages slash r slash n that we all learned right that same thing we are doing over here and for evil TTR also exact same thing so ins so the idea here is instead of accepting so now we are making our code generic enough that instead of just being restrictive that will get one command and we have to compute and respond what we are doing is we are accepting concatenated commands understanding it interpreting it evaluating multiple commands and buffering the output and sending it back so the eval and respond command that we have this is where we are buffering all the output because here we are doing buff dot right which will write to this buffer the output of ping command will be returned to this buffer then we are doing set output offset get output of get will be returned to this buffer and when we pass in this three command evaluation is done then we do this C dot write Bob dot bytes right so the entire thing would be sent over the socket to my reddish client right and this is how you would be implementing the pipelining logic we have covered every single file that we altered everything is right here nothing has changed much it's just that now we have made our code generic enough that we are supporting pipeline right initially even when redis was built they never thought that they would have to do that they also started something similar except one command responded back and then the new feature request came in where they have to make their code AS generic as possible and this is how you would be implementing pipelining right we covered all the files that we had to and this is how you implement pythoning let's quickly take a look at uh the implementation or on when we are doing it on our own server go run main. go

and now here I am passing that exact same thing but instead of passing it to 6379 I'm passing it to 7379 I get pong I get plus pong plus okay with a simple string that in this response that we send and then dollar One V slash r slash n right so we just implemented we are just we have just started supporting pipelining in our own radius implementation right fascinating isn't it so start small and then eventually build complex features on top of it keep code AS generic as possible right for us to make this change we didn't have to change a lot of stuff it's only the input output part that we had chosen because every other thing every other function was encapsulated right and this is how you should be structuring your code so that your life becomes simpler when adoc requirements come in and that is it for this one I hope you found it amusing I hope you understood how pipelining like what pipelining is how it is useful and how useful it is rather and how we implemented it in our own radius implementation right the idea would remain the same keeping the code generic accepting multiple concatenated commands buffering the response and sending it in one shot right earlier we thought that we would be streaming the response that would be much helpful but not at all pipelining change that flow we had to keep now we have to buffer the output and send it in one shot right great that is it for this one I hope you like this I hope you learned something interesting I would really really appreciate you to implement I would really urge you to implement this the source code of this again github. com you can go through this particular commit to see how it is implemented the name of the commit messages are self-explanatory I would highly encourage you to check out till this commit and see the changes that we made right so that is it for this one I'll see you in the next one thanks