Introduction to gRPC | Tutorial from Melkey

In this video, I'm going to talk to you guys all about GRPC, specifically what is it and when should we use it. So, this video is a two-part series where today we're going to be covering all about the history, the theory, and just what is the concept of GRPC. And in the following video, I'm going to show you guys an hands-on actual practical example of using GRPC in your backend systems. Now, if you don't know who I am, my name is Mel Keydev. I'm one of the instructors here on Frontend Masters. I have a few courses related about Go and building with Go and systems at scale on Go. can check those out if you're interested in more. But in today, I'm going to focus all about teaching you guys about GRPC. All right, so what is GRPC? Well, first of all, it was created by Google and GRPC comes in a bunch of different languages. And as it says here on grpc. io, it is a high-performance open-source universal RPC framework. Now, we can read here on why GRPC. GRPC is a modern open-source high-performance remote procedure call, that's what RPC stands for, framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking, and authentication. It is also applicable in last mile of distributed computing to connect devices, mobile applications, and browsers to backend services. Now, the G in GRPC stands for a bunch of different things. A lot of people think it stands for Google, so Google remote procedure call. Uh a lot of actual sources say it stands for GRPC, so it's GRPC RPC. Uh but the official kind of documentation Google says that G always changes. Every time there's a release, a major release for GRPC, the G changes to something like that starts with a G. Could be green, could be gopher, could be whatever it is. But this is kind of like an ongoing gag within the Google community of what the G stands for in GRPC. Now, we can't really talk about GRPC without first talking about what RPC, what the remote procedure call is. Now, remote procedure calls are very similar to local function that a lot of developers are already aware of. When you write a script in any kind of programming language, when you write functions and you call them, they're pretty similar in that sense. However, the key difference with RPC is that a remote procedure call enables one machine or service to invoke another machine or service as if the function was actually local to the call calling machine. So, this is why a lot of times you hear RPC or GRPC is very popular for microservices to interact with another because with RPC, you're allowed to invoke the functions of the other machine, the other service, as if they were local to the calling machine. And you can see this little demonstration that we have these two microservices communicating with basically invocation of microservice one to microservice two denoted by the kind of green arrows in the small box of one machine calling the function of another machine. And so, GRPC is an implementation of RPC that connects a large amount of microservices running within like a data center or across a team's organization that allows you to communicate with one another pretty seamlessly. It allows us to develop production-ready systems. But most importantly, it allows us to have type-safe APIs at scale. Now, how does GRPC allow us to do this? Because microservices can be written in a bunch of different languages. And the key here is something called protocol buffers or protobuffs for short. Now, these are language-agnostic mechanisms that allow us to encode data. And these can be used to send or receive structured data over the wire when we communicate between these microservices. Now, if we take our example above and give it more detail, you can see here we have these two microservices, my Go service and my React service. My Go service will be my backend and my React client will be my frontend. And these can communicate pretty seamlessly even though they're written in two complete two different languages, one was Go, the other one is entirely in JavaScript, because with GRPC, my Go service can communicate to other languages, other microservices if they have the correct GRPC stubs. This concept is super powerful and one of the main reasons why GRPC is so popular, especially within microservices, because one proto file, which is the definitions of your GRPC server, which I'll show in one example, can be used to generate stubs for all of our GRPC clients. So, I could use this stub here. I could literally copy and paste it. Go down here. Have another client, let's say this is PHP for whatever or PHP service. If it has the stubs, this can also communicate with the GRPC service without a problem. And I'll make these green to be specific. So, an example of what your profile could look like is you could have this message search request field and it has these three parameters. And notice how these are all strongly typed, which is one of the pluses for using GRPC. You can see we have two three fields. One's called query, page number, and results per page. And the query is going to be a type string, where page number and results per page are both going to be type of int32. And none of these are optional. You can see here there are number for the first value, second

value, and third value in the search request message field. Now, extending that proto file, you can see here we have that same search request and now we even have the search response. So, the struct or the message type of a search response is going to be a string that is repeated, so it can be multiple of these results. And then we have the service definition, which defines the actual RPC service for the search, which takes in the search request, which is defined up here, and it's going to return or expect a return type of the search response. And this is kind of the signature definition of the function for search service. So, when we create our protocol buffers and our stubs, we can expect that the caller and the stubs will have these definitions for the languages for when we want to kind of implement them in different systems like I have written out here. And you can see in this diagram I've created in that search. proto file, which we have defined up here, we can create multiple different stubs for our Go service, so the actual GRPC server, and even the client, so the GRPC stub. And you can create these across all your different microservices, which is really why GRPC is such a strong mechanism for scaling out to your different services, because it doesn't matter how many you have or whatever language they are written in. This could be Java, PHP, this could be Go, this could be Python. Doesn't really matter. If you have the single source of truth for your proto files, you can create a bunch of different stubs that will work in any sort of system. And these communicate via HTTP2 exchanging the different payloads. Now, GRPC is actually also very performant due to protobuffs being faster than JSON. And secondly, it's due to the streaming capabilities of communicating over HTTP2. And this actually allows GRPC to handle many concurrent RPC calls over a very small number of TCP connections. It's really powerful because here I'm primarily talking about GRPC-Go, which is used for the Go programming language. But if you go back here, you can click it for C++ as a quick start guide. If you go back, there's even for Java. You can go back for Python. And there's a bunch of different languages here that are all open-source. You can even see for Ruby. If you click this, there's a quick start. There is the repo that you can click into. You can see here GRPC repo. And this is it. This is for C++, Python, Ruby, Objective-C, PHP. You can scroll down here and you have all the different libraries here. So, you can go Go, Java, Kotlin, PHP. And it will all open up to the different languages that GRPC is able to use, which again just shows you how rich the ecosystem is for this framework. So, why is GRPC so good? It's basically because of the protobuffs are faster than JSON for encoding and decoding data. It has a rich ecosystem. It's type-safe. It can scale to other microservices. And it's built on top of HTTP2. And you can see here in this little more detailed diagram, we have the React client and the Go service just for example. And you can see our two stubs here for our GRPC client, our GRPC server, and kind of what happens through the workflow. So, we have our stub makes the request. It goes through the encoding process of creating the actual protobuff through our stub and then transports it via HTTP2 to our other microservice, so in this case our Go microservice. Takes the transport, decodes it from the decoding stub, and into processing to our GRPC server. And we actually send this info back and forth between the two microservices. And we can expand this to include other microservices that also can communicate over the wire. So, in this part of the tutorial, we're actually going to be looking at a hands-on example of spinning up your own GRPC server using Go, as well as doing the protobuff command to generate your protobuff files. Now, this isn't meant to be an in-depth example of how to write complex GRPC code. This is more coupling the first portion of this tutorial, which is more of a theoretical breakdown, the who, what, how, and you know, what is GRPC in the first place, and kind of just coupling this with a very bare-bones example of how you can set a GRPC server at home, how to create your own protobuff files, and how to test it from your computer. So, we're going to actually start this by showing the commands you're going to need to install on your machine. The first one being protoc or protoc, and you can install this if you're on Mac with Brew, so Brew install protoc. And this will actually generate the protobuff Go code by running a command I'm going to show you in just a little bit. And then the other thing we're going to need is GRPCurl. So, this is a command that allows us to actually test with GRPC, cuz you know, you can't do like a regular curl cuz it's on a different protocol. But this will allow us to essentially kind of curl our own local server. Okay, so I'm going to start by making two directories. I'm going to make one called let's just say server. I'm going to make another one called proto. So, let's actually go ahead and first start in the proto directory and we'll make a file and define our protobuf file here. So we

can just do greetings. dot proto like so. And here the syntax for our protobuf file all follows the same so it's not go, it's not anything like that. It's like its own definition. So we can do proto 3, that's the version we're going to be using the protobuf syntax for this version. We're going to call this package greeting. And I should actually rename this. I don't want to name it greetings. I'm going to rename it to greeting. Here we're going to pass the options. So this kind of the definition of this go package or the protobuf file of how this go package will be called. And we can say grpc_tutorial/proto. All right, so this is going to define that in our go. mod we have our grpc module. And then here this is going to be the same definition and I actually noticed that it's not an underscore. Oops. But a dash. So the first thing let's go ahead and create our request and response. So we're going to do message hello request. All right. And here we're going to define it's going to be a string. We're going to have the name of the field which is going to be name and we are going to give it this definition that's going to be the first value in our hello request. And the second value can be this int32. We can just do age is going to be two. So what this is telling us is in our service when we have a hello request definition it's going to expect two fields, name and age, both of which have a strong type definition. Next let's go ahead and do hello response. And it's going to be technically the same thing so we can do string message. Do one here. And timestamp. Like so. Okay. So now the response is going to have a message and a timestamp. Okay. And so now we can actually describe our service. Right? So we can do a service. Let's just call this greeter. All right. And then we can define the functions under our greeter service. So we can use RPC, this is a keyword to define something RPC protocol. We can say the name of the function we're going to say is say hello. Then the request type so we know that's going to be the hello request struct that we have just a few lines above and it's going to return our hello response type. Okay. And then we can close the function definition. And then you can add more functions here as you wish. RPC you know, another function. But basically you can see how it works that every function could have a pair of requests and response structs or they can even reuse them. So I can have another function that says RPC say hello again with the same function definitions of hello request and hello response as the signatures in the actual RPC function. So if you've written go before typically we'd have our imports defined here, maybe some other functions and then followed by the func main here. Now this is all great, this is all pretty standard stuff. But we need to be able to import all the definitions we have from our green. proto, right? Like I mean we want the request, we want the response. But since this is in go how can we actually get these definitions in a way that our go programming language, the server, can understand and import them and not just use the dot proto? Well this is where we actually going to use the protoc command line to generate those files, the protobuf files, in a format or a stub that our go language server can use and import into our functions. So the command is kind of long but basically we have this protoc and we're going to find the output. So this command is going to generate two files for us. One is going to be just the greeting. pb protobuf. go file and the other one here as defined with this grpc_out flag it's going to be the greeting_grpc. pb. go. And the most important part is to actually give this command the path to our greeting. proto file which we defined in this proto directory. So that is this file right here. So if we run enter you can see we have a let's see. Oh, I forgot to put this. Let's go ahead and save. And now if we go back we should see our protobuf file here and then two additional files that were generated. You can see code generated by the protoc-gen-go. It says do not edit and the same thing here with the greeting_grpc. pb command. You can go through this. It has all the definitions for everything we want. So you can see our say hello function and all that is defined in this generated file here. You can see our hello request with the name definitions to make it acceptable and easy to import in our go server because now it is written in go. So let's go ahead and start with our imports. I'm going to kind of move faster. I might copy and paste some code but just pause the screen see and we'll go through these one by one.

So we'll need the context package format, log, the net package time we'll need the specific Google product imports. And reflections. We'll need these two files here. You can see we can't import them. So if we go mod tidy it's going to download everything and it should if I go back get rid There you go. So everything's just unused. And now we actually want to bring in the definitions from our greetings. proto file that was generated. So we can do PB and this is going to be using the same module path that we defined here. So this grpc-tutorial/proto. If we go here grpc-tutorial/proto we can see imported as PB and not used. So everything is being found correctly. Okay, now let's define a quick struct. We'll say our server. Okay. PB I'm going to put in the Oops. It's going to be PB. All right. Now let's do our struct to hold our different functions. We can do server struct. And PB and it's going to be unimplemented greeter. So this kind of provides a very bare bone implementation of all the things that we need for our grpc server. You can see now this should be embedded by value instead of pointer to avoid a nil pointer dereference when methods are called. So you can see we have a bunch of different things here. Our hello response, hello request, the structs, the interfaces. We're going to do an unimplemented greeter server just to hold the definitions for this — quick example. And then from that we're going to do S is going to be a pointer to our server struct. And here we're going to just say hello, the function name. We're going to pass it the context and the request as well as the response and potential error handling. So we're going to do ctx. context like so. And then here we're going to do our request. It's going to be a pointer to PB. hellorequest. So we already know the definition here and it's going to respond with PB. helloresponse and a potential error. Okay. We can add some quick logging here. I just did that really quickly. And then here we can actually define the return type. So we're going to do return and reference to PB. helloresponse. We can open up this struct and put in the values we need. So we have our message field which is a string and so is our timestamp. So we're going to do format. sprintf like so. We can do hello string. Right? You are digit years old. We can do our request. name and our request. age like so. And then underneath we also have the timestamp. So timestamp and we can do and we This is just something we can put the RFC 3339. So just a constant here. And we can put nil like this. So we have our say hello function defined and this kind of what's going to accept our grpc requests from and expect them to be in the hello request format and then respond with the correct expected hello response struct. — Okay. So now let's do our func main and quickly do a spin up a quick server and I'm going to show you how you can curl some examples using the grpcurl command. We can do listen err. We can use the net package so nt we can do just listen. All right. So this is going to spin up a TCP network that we can listen to. So TCP and we can put a port. We can do 551. 51 if err does not equal to nil log. fatal f failed to listen. And put the err like so. Okay, so we can do our error handling. Okay. And then here we can define our function our service. We're going to do grpc. new server like so. Okay. And then here we're going to do our PB. We're going to do register our greeter server. We're going to pass in our grpc server and pass in our server struct that we have defined up here which is going to have all of the methods that we defined for the say hello function and you know, all the different things that's required in a grpc server. This kind of helper function allows us to bind them both together. And then one more thing I'm going to use the reflection package from go just to allow us to see all the definitions using that grpcurl command I showed you earlier. And now all we need to do is log the print line that the server is in fact running. So, we're going to log print line like so. We can just say something gRPC server is running.

And if error, we can define it as S serve and then pass in that listen TCP server like this. So, we can do error if it's not nil, then we can do another log. fatal and say failed to serve error. Like so. — [clears throat] — So, with all that being said, we can go back into the root of our project. We already know how to define and generate our protobuf files. Now, we can actually go and go run our server main. go. So, you can see here our gRPC server is running. So, we're going to open a new uh terminal. And this is where we're going to showcase the gRPC curl commands. So, we're going to gRPC curl like so. And then you can do {dash} plain text for the response type that you want. You can do localhost:50051 and list. And so, this is going to give us all the different things that the server our gRPC server has. So, it has a greeting server and it has the different um functions that come from using the reflection package from gRPC. Then we can describe the service. We can do gRPC curl. Let's do plain text again. The localhost:50051. And then we can do describe and we can put in that greeting. dot greeter which is the response to the first command here from our list command. You can see here it's a service with the actual function definition. So, we have an RPC function called say hello with the request type and the response type expected in this service that we have defined from our protobuf files. And you can see it was correctly picked up in our main. go. Everything's binded correctly and it's working. So, now we can actually find the list of services. Go ahead and make our own request. We're going to do our gRPC curl. The same plain text. This time I'm going to pass {dash}d to describe a payload. And here we're going to just kind of describe it the same we would uh regular JSON. So, here we know that our hello request and just a reminder here, hello request requires two uh fields. One is a string name and the other one is an int32 of called age. So, here the first will be name. We can just say something like Melky. Okay? And then the second field is going to be age. It's going to be type int32. So, we can just say — uh 50. Okay? Then we're going to specify the location. So, localhost:50051. And then here we're going to do greeting. greeter/sayhello. All right? So, now if I run this. So, as you can see, we've actually successfully curled the endpoint with greeter say hello with the proper request type. You can see hello Melky, you're 50 years old. And so, I hope this was very uh informative, very basic implementation of how we can go from just a random protobuf file generated to our programming language of choice which was go. But again, as we said, you can do it in Ruby, PHP, any language. There's really a bunch of different uh ways that it's supported by Google and other open source maintainers. And how we can easily bring this in to our main. go file. This should look fairly similar to how you'd run a HTTP server. They're all pretty aligned, but a whole different protocol. And there's lots of really cool things you can use with gRPC and protobuf files. All right. So, I hope this video explained what gRPC is super helpful for you. Let me know if what videos you want to see next. If you're interested in the Go programming language like I said earlier, I have two courses here on Frontend Master Kit that you should definitely check out. One is building Go application that scale on AWS. So, we explore different types like Dynamo, uh gateways, and a bunch of different other stuff like Lambdas to host and run our application. And we have a second uh course which is the full breakdown, the complete Go for Professionals where we introduce going from absolute zero from scratch to developing an entire application with database layers, testing layers, API layers, everything you need to start building solid HTTP service using the Go programming language. So, you can check that out. If you did enjoy this, let us know and we'll definitely try to spin up a function or a course in the future or maybe more of these tutorials focusing on Go and the gRPC topic that we covered today. Hope you guys had a great time and let me know. I'll see you later.